This question is for testing:-
There is a email website https://www.sharklasers.com where we can create temporary emails. I'm using this site to send email and get some verification code. As of now i'm using selenium which takes some time.
So, I decided to use RestSharp(C#) to read the email.
General Steps:
Step 1: Navigate to https://www.sharklasers.com
Step 2: Edit and set an email id (creates email id with just one click)
Step 3: Sending email to the newly created temp email(from some mail like gmail..)
Step 4: Click on the received email.(Here will get the unique URL for the email)
That's the steps.
This I tried with POSTMAN which is working properly.
POSTMAN Steps: (Captured the traffic using charles proxy)
1.Create Email - POST Request :
https://www.sharklasers.com/ajax.php?f=set_email_user&email_user=Emailidhere&lang=en&site=sharklasers.com&in=+Set+cancel
2.Get list of emails - GET Request : https://www.sharklasers.com/ajax.php?f=get_email_list&offset=0&site=sharklasers.com&in=Emailidhere&_=1547991487
I'm having issue with the 2nd step.
Below observations may help you to help me.
Observation in browser:
- Entering the 2nd URL directly to the browser - getting some default information about the sharklasers website.
- First navigate to the sharklasers website , set the email which is created , then in new tab , enter the 2nd URL - getting the expected response.
Observation in POSTMAN:
- Entering the 2nd URL directly to the POSTMAN - getting some default information about the sharklasers website.
- First sending the set the email Request JSON file here, then sending Request to the 2nd URL JSON file here, getting the expected response.
Observation in Visual Studio (RestSharp C#)
- Entering the 2nd URL directly to the POSTMAN - getting some default information about the sharklasers website.
- First sending the set the email Request , then sending Request to the 2nd URL - getting the same default information about the sharklasers website. NOT Getting the expected response
Set Email RestSharp code taken from POSTMAN:
var client = new RestClient("https://www.sharklasers.com/ajax.php?f=set_email_user&email_user=testemailstackoverflow&lang=en&site=sharklasers.com&in=+Set+cancel");
var request = new RestRequest(Method.POST);
request.AddHeader("Postman-Token", "ddae1b83-b974-4e9b-a013-940c2e91d50a");
request.AddHeader("cache-control", "no-cache");
IRestResponse response = client.Execute(request);
Console.WriteLine(response.Content);
Get Email RestSharp taken from POSTMAN:
var client = new RestClient("https://www.sharklasers.com/ajax.php?f=get_email_list&offset=0&site=sharklasers.com&in=testemailstackoverflow&_=1547991487");
var request = new RestRequest(Method.GET);
request.AddHeader("Postman-Token", "7625df90-9ee9-46e6-b9ca-568dde93618e");
request.AddHeader("cache-control", "no-cache");
IRestResponse response = client.Execute(request);
Console.WriteLine(response.Content);
Now i want to get the same response in Visual studio(Restsharp C#) which is observed in POSTMAN and browser.
This question may be little confusing. If any questions regarding the question , please ask me.
Thanks in advance.
Related
I am getting to the stage of hair pulling with this one, I'm hoping someone can see if I'm doing anything wrong.
I'm trying to POST some form data to website using Azure data factory web activity however whilst I get a response (I get the page and some headers) it is different to the response I get if I make the exact same request using C# and HttpClient code. I've used fiddler to view the request being post'd using my C# script and according to the request information given in data factory they are exactly the same - so same headers, same content format etc...
This POST request is to login to a website which has a custom login mechanism, so no OAuth or anything like that unfortunately. It is supposed to return a cookie, which it does if I use my C# script, but if I make the same POST request using data factory web activity then I get different html sent back (it just returns the same login screen) and also different set of response headers in the "ADFWebActivityResponseHeaders" part of the activity output!?! See below for what is returned in the web activity output response headers:-
"ADFWebActivityResponseHeaders": {
"Pragma": "no-cache",
"Vary": "Accept-Encoding",
"X-Frame-Options": "DENY",
"Cache-Control": "no-store, must-revalidate, no-cache, post-check=0, pre-check=0",
"Date": "Wed, 09 Sep 2020 08:09:30 GMT",
"Server": "Microsoft-IIS/8.5"
}
If I do this via C# I also get a 'Set-Cookie' as well (strangely if I make a 'GET' request for the homepage of this site I do get a 'Set-Cookie' in the response!!!), but never when doing this via data factory. I'm struggling to see how this is possible unless data factory is modifying my request in some fashion? Below is my C# code, pretty simple/standard:-
var handler = new HttpClientHandler();
handler.CookieContainer = new CookieContainer();
handler.UseCookies = true;
handler.UseDefaultCredentials = false;
// Create our http client which will perform our web requests
var HttpClient = new HttpClient(handler);
HttpClient.BaseAddress = new Uri("**REMOVED**");
// Some of the extracts take a LONG time, so set the timeout for default of 30mins
HttpClient.Timeout = TimeSpan.FromMinutes(30);
// Set the 'form' parameters we're going to POST to the server in the request
var parameters = new Dictionary<string, string>
{
{ "username", "**REMOVED**" },
{ "password", "**REMOVED**" }
};
// URL encode the parameters
var content = new FormUrlEncodedContent(parameters);
// Submit our POST with the parameters
var response = await HttpClient.PostAsync("**REMOVED**", content);
Running this code and using fiddler I see the following request with headers, these are the only headers:-
Content-Length: 80
Content-Type: application/x-www-form-urlencoded
username=REMOVED&password=REMOVED
and in the 'input' side of the web activity is the details of the request, I've added the headers in the web activity and these are correct:-
"method": "POST",
"headers": {
"Content-Type": "application/x-www-form-urlencoded",
"Content-Length": 80
},
"body": "username=REMOVED&password=REMOVED"
Note that in the data factory I'm using a self hosted integration runtime as this website blocks addresses that do not come from the specific IP addresses used externally by our on-prem network/firewall. I know that is not the problem as I'm getting a response with the normal login page from the site (if I use the Azure integration runtime I get a denied response).
Here is a screen shot of the web activity in data factory:-
Really hope someone out there can see what I'm missing or whatever...
Turns out this does work and will list the cookies in the JSON output from the activity as shown below (note this is to be found in the output of the ADF activity, so you would pick up the cookie from the output a bit like... #activity('Login and get cookie').output.ADFWebActivityResponseHeaders["Set-Cookie"] )
However, in my case the url I was POSTing to was responding with a 302 (moved temporarily) but the 'Location' header which should be there is not in the ADFWebActivityResponseHeaders - which is why I missed it. I tried using Chrome with the developer tools and looked at the response directly which is where I found the 302 response code. After that, I just used the new URL given in the response headers (i.e. the url in the 'Location') that I found when using the browser dev tools.
Unfortunately at the time of writing, the Azure data factory HTTP activity does not follow redirects (and doesn't list all the response headers either!) so if anyone encounters the same problem they will need to manually find out and get the url's for any redirects. In other words, try using a tool like browser/postman and look at the response if it doesn't work in ADF... you might find there is a redirect going on :-)
There is a feature request logged for this here, be sure to add your vote :)
edited to update the Azure feedback change of URL after MS decided to change things on the feedback site!?!
I am following the Yahoo official documentation (https://developer.yahoo.com/oauth2/guide/openid_connect/getting_started.html). I can successfully get an authorization code after user logins with Yahoo. I am now at step 3, trying to exchange the authorization code for a token, but Yahoo keeps returning an http error 500.
To exchange the authorization code for the access token from Yahoo, I am using the following RestSharp syntax:
var client = new RestClient(provider.TokenUrl);
RestRequest request = new RestRequest() { Method = Method.POST };
request.AddParameter("client_id", codeModel.clientId, ParameterType.GetOrPost);
request.AddParameter("client_secret", provider.Secret, ParameterType.GetOrPost);
request.AddParameter("code", codeModel.code, ParameterType.GetOrPost);
request.AddParameter("grant_type", "authorization_code", ParameterType.GetOrPost);
request.AddParameter("redirect_uri", codeModel.redirectUri, ParameterType.GetOrPost);
request.AddHeader("Content-Type", "application/x-www-form-urlencoded");
var response = client.Execute<TokenResponseModel>(request);
responde.data returns the following:
content: {"error":"ACCESS_TOKEN_GENERATION_FAILED","error_description":"Access token generation failed"}
StatusCode: InternalServerError
The official documentations states: "The request parameters below are transmitted using HTTP POST in the request body. You can, however, also send the parameters client_id and client_secret in the HTTP Headers instead".
I have tried both methods (clientid and secret as part of the body and as an Basic Authorization Header) and both return the same result.
When sending the clientid and secret as part of the Basic Authorization header, both parameters above are replaced by the following:
client.Authenticator = new RestSharp.Authenticators.HttpBasicAuthenticator(codeModel.clientId, provider.Secret);
As stated before, the only message returned by Yahoo is "internal server error".
Is there something wrong with the RestSharp syntax that could be causing this? Any other ideas will be greatly appreciated.
Needless to say, all parameters of the request contain the data they need.
Thanks
When you create your application profile at YDN you must make sure to select at least one API permission. For example try "Profiles (Social Directory) Read Public".
If your application has no API permissions then token generation will fail just the way you described.
If you already created an application with no permissions then you will have to delete it and create it again.
I have a loop that will loop through records in my DB, pulling information i need and then creating 3 folders & upload a file.
This works OK for like 40 records but then it starts erroring out with the below response back from sharepoint: <!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01//EN\"\"http://www.w3.org/TR/html4/strict.dtd\">\r\n<HTML><HEAD><TITLE>Bad Request</TITLE>\r\n<META HTTP-EQUIV=\"Content-Type\" Content=\"text/html; charset=us-ascii\"></HEAD>\r\n<BODY><h2>Bad Request - Header Field Too Long</h2>\r\n<hr><p>HTTP Error 400. A request header field is too long.</p>\r\n</BODY></HTML>
I am not sure whats going on, i read online its todo with cookies but i am using HTTPClient to send the request so i dont know how that would effect it? I also seen onlne about changing the kestrel?
Can anybody shed some light on this for me? Provide me with an easy but working solution? I dont use CSOM for integrating to sharepoint online, i use HTTP Requests, below is a sample of how i interact with sharepoint.
It seems as if i get blocked or banned temporarily cause if i wait a good bit, i can then make the same request that failed previously, and it will work! So strange.
Sample code (Used to create a resource at Sharepoint):
//Set Endpoint
var sharePointEndpoint = $"https://{hostname}/sites/{site}/_api/web/folders";
//Set default headers
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", sharePointToken); //Set token
client.DefaultRequestHeaders.Add("Accept", "application/json;odata=verbose");
//Pre-Body data setup
var metaData = new MetaDataModel();
metaData.type = "SP.Folder";
//Body data setup
var bodyModel = new ExpandoObject() as IDictionary<string, object>;
bodyModel.Add("__metadata", metaData);
bodyModel.Add("ServerRelativeUrl", location + "/" + directoryName + "/");
//Set content headers
HttpContent strContent = new StringContent(JsonConvert.SerializeObject(bodyModel));
strContent.Headers.ContentType = new MediaTypeHeaderValue("application/json");
strContent.Headers.ContentType.Parameters.Add(new NameValueHeaderValue("odata", "verbose"));
// Send request, grab response
var response = await client.PostAsync(sharePointEndpoint, strContent);
//Return response message
return response;
It turns out I needed to use Content-Length header when sending the request, once done I was able to successfully communicate with sharepoint without encountering this error.
More information here: https://social.technet.microsoft.com/Forums/en-US/26459f1c-945d-4112-9200-69c5a33a37ff/sharepoint-online-rest-api-returns-http-error-400-a-request-header-field-is-too-long?forum=sharepointdevelopment
Thanks.
I'm trying to access a website that requires login via a form.
I used the Postman HTTP client.
I tried to do the normally http post request but didn't seem to work, I get a successful status code (200 OK) but it doesn't log in, eventually did work with a GET request with BODY parameters (I hadn't seen GET request with body parameters).
Well, I tried to simulate this request in C# code with no luck, I even tried the generated code that Postman offers with no luck again.
Down below is the Postman request and the C# code snippet based on auto-generated Postman code. Does anyone know if is there to make this request with any library or if there is something that I miss?
Thank you in advance.
var client = new RestClient("https://thessalia-3.teilar.gr/login.asp");
var request = new RestRequest(Method.GET);
request.AddHeader("cache-control", "no-cache");
request.AddHeader("Content-Type", "application/x-www-form-urlencoded");
request.AddHeader("Referer", "https://thessalia-3.teilar.gr/login.asp");
var parameters = new Dictionary<string, string>();
parameters["userName"] = JsonConvert.SerializeObject("myusername");
parameters["pwd"] = JsonConvert.SerializeObject("mypass");
parameters["loginTrue"] = JsonConvert.SerializeObject("extravalue");
var content = new FormUrlEncodedContent(parameters);
request.AddParameter("application/x-www-form-urlencoded", content);
IRestResponse response = client.Execute(request);
Console.WriteLine(response.Content);
Console.WriteLine(response.StatusCode);
Postman Request Photo
Edit:
Postman Request Body Parameters
I've also tried to run this but also not logged in.
Auto-generated code form Postman
If the request was successful (200) and you got the HTML page for "Invalid Credentials", then your code that's making the request should be fine and the issue is with the credentials. Like I said in my first comment, don't serialize the parameters to JSON, URL-encode them instead:
parameters["userName"] = HttpUtility.UrlEncode("myusername");
parameters["pwd"] = HttpUtility.UrlEncode("mypass");
parameters["loginTrue"] = HttpUtility.UrlEncode("extravalue");
This is the standard way and it works with writing the parameters directly to the request stream, or with a utility class like StringContent. However, since you're using the utility class FormUrlEncodedContent, it URL-encode them for you, so you don't have to. In that case, simply assign them directly as string:
parameters["userName"] = "myusername";
parameters["pwd"] = "mypass";
parameters["loginTrue"] = "extravalue";
I have an API for getting data that is created in Swagger. I Only have an base URL and an username, password and a token for that. When I go to the URL it will go to a login page and after login, We can access a list of APIs and get data from that.
Now I need that to be done in C# using restsharp. So that I can get the result in JSON and can update the values to DB.
This is my code which I used in C#
var restClient = new RestClient("https://v3.fusesport.com/api/events/")
{
Authenticator = new HttpBasicAuthenticator("xxxxx", "xxxxx")
};
var request = new RestRequest(Method.GET);
request.AddHeader("cache-control", "no-cache");
request.AddHeader("Token", "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx");
request.AddHeader("content-type", "application/json");
IRestResponse response = client.Execute(request);
This is getting an connection closed error.
I tried the API in postman app in chrome, it is getting the below error.
{
"detail": "Authentication credentials were not provided."
}
This is the screenshot of postman call with token
Postman with token
This is the screenshot of postman call with basic authentication
enter image description here
Can you help me what I am doing wrong. I think the API is using session based authentication.
Thanks in Advance.
I think your headers are incorrect. Instead of
request.AddHeader("Token", "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx");
try adding
request.AddHeader("Authorization", "Token 9944b09199c62bcf9418ad846dd0e4bbdfc6ee4b");