This is me publicly documenting my mistake so that if I or anyone does it again, they don't have to spend 3 hours tearing their hair out trying to fix such a simple thing.
Context
I was sending an HttpRequest from one C# MVC ASP.NET application to another.
The applications require an HTTPS connection, and we are using URLRewrite to redirect an HTTP request to an HTTPS url.
One application was sending a POST request with some JSON data in the body, pretty standard stuff. The other application was set up to receive this data with an MVC controller class (CollectionAction and Insert methods for GET and POST respectively).
Symptoms of the problem
The receiving application was running the GET method (CollectionAction) instead of the POST action (ItemAction). The reason for this was that the request coming in to the application was in fact a GET request, and to top it off the JSON data was missing too.
I sent the header "x-http-method" to override the request method from GET to POST (I was already setting the request httpmethod to POST but this was being ignored). This worked but still I had no data being sent.
So now I am stuck pulling my hair out, because I can see a POST request with content-length and data being sent out and I have a GET request with no data or content-length coming in (but the headers were preserved)
Turns out I was using UriBuilder to take a base URL and apply a resource path to it. For example I would have "google.com" in my web.config and then the UriBuilder would take a resource like Pages and construct the url "google.com/Pages". Unfortunately, I was not initializing the UriBuilder with the base URL, and instead was using a second UriBuilder to extract the host and add that to the path like so:
public Uri GetResourceUri(string resourceName)
{
var domain = new UriBuilder(GetBaseUrl());
var uribuilder = new UriBuilder()
{
Path = domain.Path.TrimEnd('/') + "/" + resourceName.TrimStart('/'),
Host = domain.Host
};
var resourceUri = uribuilder.Uri;
return resourceUri;
}
The problem with this code is that the scheme is ignored (HTTP:// vs HTTPS://) and it defaults to HTTP. So my client was sending out the request to an HTTP url instead of the required HTTPS url. This is the interesting part, URLRewrite was kicking in and saying that we needed to go to an HTTPS url instead so it redirected us there. But in doing so, it ignored the Http-Method and the POST data, which just got set to defaults GET and null. This is what the 2nd application could see at the receiving end.
So the function had to be rewritten to this which fixed the problem:
public Uri GetResourceUri(string resourceName)
{
var baseUrl = GetBaseUrl();
var domain = new UriBuilder(baseUrl);
var uribuilder = new UriBuilder(baseUrl)
{
Path = domain.Path.TrimEnd('/') + "/" + resourceName.TrimStart('/'),
};
var resourceUri = uribuilder.Uri;
return resourceUri;
}
Related
I am trying to implement a Xamarin app that works with the Asana API.
I have successfully implemented the OAuth as documented in the Asana documentation here... at least I assume it is successful. I get an access token from the token endpoint in an HTTPResponse with HTTP Status "OK".
But then when I turn around and try to make an API call with that same access token, I get a 403 Forbidden error. I tried the same API call in my browser (after logging in to Asana), and it works fine, which leads me to believe that I do have access to the resource, I must have an issue with authorizing the request on my end.
The API call in question is (documented here): https://app.asana.com/api/1.0/workspaces.
My C# code is as follows (abbreviated to relevant parts, and assume that ACCESS_TOKEN contains the access token I got from the token exchange endpoint):
HttpClient client = new HttpClient();
client.BaseAddress = "https://app.asana.com/api/1.0";
client.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", ACCESS_TOKEN);
client.DefaultRequestHeaders.Add("Accept", "application/json");
And then I use this HttpClient (named client) in the following function:
// Returns a list of the Asana workspace names for the logged in user.
private async Task<List<string>> GetWorkspacesAsync()
{
List<string> namesList = new List<string>();
// Send the HTTP Request and get a response.
this.UpdateToken(); // Refreshes the token if needed using the refresh token.
using (HttpResponseMessage response = await client.GetAsync("/workspaces"))
{
// Handle a bad (not ok) response.
if (response.StatusCode != HttpStatusCode.OK)
{
// !!!THIS KEEPS TRIGGERING WITH response.StatusCode AS 403 Forbidden!!!
// Set up a stream reader to read the response.
// This is for TESTING ONLY
using (StreamReader reader = new StreamReader(await response.Content.ReadAsStreamAsync()))
{
// Extract the json object from the response.
string content = reader.ReadToEnd();
Debug.WriteLine(content);
}
throw new HttpRequestException("Bad HTTP Response was returned.");
}
// If execution reaches this point, the Http Response returned with code OK.
// Set up a stream reader to read the response.
using (StreamReader reader = new StreamReader(await response.Content.ReadAsStreamAsync()))
{
// Extract the json object from the response.
string content = reader.ReadToEnd();
JsonValue responseJson = JsonValue.Parse(content);
foreach (JsonValue workspaceJson in responseJson["data"])
{
string workspaceName = workspaceJson["name"];
Debug.WriteLine("Workspace Name: " + workspaceName);
namesList.Add(workspaceName);
}
}
}
// I have other awaited interactions with app storage in here, hence the need for the function to be async.
return namesList;
}
Finally found the answer. It looks like I was using HttpClient incorrectly; a subtle thing that should be equivalent, but is not due to the way it is implemented.
The answer
I needed to place the final slash at the end of the BaseAddress property of HttpClient, and NOT at the start of the relative address for the specific request. This answered question explains this.
To fix my code
I needed to change the setting up of the BaseAddress:
HttpClient client = new HttpClient();
client.BaseAddress = "https://app.asana.com/api/1.0/"; // FINAL SLASH NEEDED HERE
And remove the slash from the request's relative address:
// DO NOT put slash before relative address "workspaces" here
using (HttpResponseMessage response = await client.GetAsync("workspaces"))
Why I got the original error
When HttpClient combined the BaseAddress with the relative URI I specified in GetAsync(), it dropped off some of the base address, since the final slash was not included. The resulting address from combining the BaseAddress with the relative URI was a valid URL, but not a valid page/API call in Asana. Asana thus did an automatic redirect to a login page, which, of course, the rest of the API call would be forbidden from there.
How I discovered this
In debugging, I grabbed the access token returned during my app's authorization with Asana. I then recreated the request to the "/workspaces" API myself in Postman, and the request worked as expected. This confirmed that my authorization worked fine, and the issue must be with the specific request rather than the authorization. In debugging I then looked into the HttpResponseMessage, which has a property called RequestMessage, that includes the actual URL the GetAsync() made the request against. I observed the Login URL from Asana, rather than the BaseAddress I specified... which led me to the question/
answer linked above.
Hope this explanation helps anyone who comes across a similar error!
I've been tasked with building a service which pulls information from a 3rd party API into our internal datawarehouse. They have a get request to pull the data I want where you specify the parameters you want via query strings. E.g.
http://www.api.com?parameter=firstname¶meter=surname
In my code the length of the URL is over 3600 characters long as the requirement is for 116 parameters.
My web request is generated using this code:
private HttpWebRequest GetWebRequest(string url, string type, int timeout)
{
var httpWebRequest = (HttpWebRequest) WebRequest.Create(_baseUrl + url);
httpWebRequest.Method = type;
httpWebRequest.Timeout = timeout;
httpWebRequest.ContentType = "application/json";
httpWebRequest.Headers.Add("Authorization", "Bearer " + _token.access_token);
httpWebRequest.ContentLength = 0;
return httpWebRequest;
}
When I run the code I am getting back a web exception with the message "Unable to connect to the remote server" with an internal exception message of "No connection could be made because the target machine actively refused it IP Address"
I have not included the entire URL in this post as I have found that if I copy and paste the url into Postman and run the request I get the response I expect so I know that the URL is formatted correctly. I have also discovered that if I cut down the length of the url to around 2100 characters the request then works.
I have been searching but have not found any definitive documentation to suggest that there is a limit to the length of the URL, but I can not explain why the whole url works in Postman but not in a c# web request and that if I cut the length of the URL it then works in the web request!
If anyone has any ideas about this I'd be greatfull.
An old post suggests that depending on the server and client the maximum request length is somewhere between 2 - 4 and 8 KB, which is consistent with your observations.
Postman is 'another' client, so it is well possible that it works there while it doesn't in your program. Bottom-line is: you should not GET such long requests. You should POST them instead. If you have control over the service, you could change it so it supports POST too, if not already (documented or not).
I am trying to call Google's OAuth2 authentication service as per these instructions: https://developers.google.com/accounts/docs/OAuth2ForDevices
I put all of the required parameters into the query string and sent the request. This worked for the "Obtaining a user code" section but not for the "Obtaining Access and Refresh Tokens" section.
After much playing around and getting 400 Bad Request errors, I found that, instead of putting the data in the query string, you can create a request with a FormUrlEncodedContent and send the data through as content with application\x-www-form-urlencoded Content-Type.
Here is the code before:
var requestMessage = new HttpRequestMessage();
requestMessage.Method = "POST";
requestMessage.RequestUri = new Uri(fullUrl);
Where fullUrl is something like:
https://accounts.google.com/o/oauth2/device/code?client_id=812741506391-h38jh0j4fv0ce1krdkiq0hfvt6n5amrf.apps.googleusercontent.com&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.profile
And the new code is:
var requestMessage = new HttpRequestMessage();
requestMessage.Method = "POST";
requestMessage.RequestUri = new Uri(url);
requestMessage.Content = new FormUrlEncodedContent(CreateDictionary(queryStringNames, queryStringValues));
Where url is:
https://accounts.google.com/o/oauth2/device/code
and queryStringNames and queryStringValues are string arrays of the names and values of the required parameters.
What is the difference between these two methods? Is it safe to assume that all POST calls can use the URL Encoded Content requests instead of putting the data in the query string?
In general, POST requests do not need query string but it is still subjected to Server's logic implementation. In case of OAuth which is quite known standard and they do follow good practice, it is safe to use form encoded data unless mentioned explicitly in API to send Parameter as query string.
Query String & Post data are two different set of parameters. If server is expecting Query string then you must send query string only. It all depends on how server side logic is implemented. You can not use them alternatively. Most API documentation specify clearly what are they expecting.
I have an ASP.NET MVC web site. One of my routes is a URL that takes 5 parameters. For the sake of illustration, these parameters are named parameter1, parameter2, parameter3, parameter4, and parameter5. Currently, I'm constructing a URL in some C# code that will POST to the mvc action via a WebClient. that code looks like this:
WebClient myWebClient = new WebClient();
myWebClient.UploadStringCompleted += myWebClient_UploadStringCompleted;
string url = "http://www.example.com/customer/" + parameter1 + "/orders/" + parameter2 + "/" + parameter3 + "/" + parameter4 + "/" + parameter5;
myWebClient.UploadStringAsync(new Uri(url, UriKind.Absolute));
I'm confident that the UploadString method does a POST. I need to do a POST, because my parameter values can be very long. In fact, I estimate that occasionally, the total url length may be 20000 characters long. Regardless, I get a 400 error when I attempt to post my data. In an effort to debug this, I'm trying to figure out how to simulate a POST in Fiddler.
Assuming that I am passing values via a query string as shown above, what values do I enter into Fiddler? From the Composer tab, I'm not sure what to enter for the Request Headers area. I'm also not entirely sure what to enter for the url. I'm not sure if I put the entire URL in there, including the parameter values, or if those belong in the Request Headers.
What I need to enter into Fiddler, so that I can debug my issue?
Basically all your parameters are a part of the URL, and this is the root of your problem. Here is what is going on: you are hitting the URL length limitation, and receiving a "400 Bad request" error. In real world most web browsers do not work with URLs more than 2000 characters long.
To resolve this problem I would suggest doing a bit of refactoring, so that request is posted to the URL http://www.example.com/customer/parameter1/orders or even http://www.example.com/customer/orders with parameters send in request body. Here is how test such request in Fiddler:
On Composer tab choose POST request verb
Specify the URL as
http://www.example.com/customer/parameter1/orders
or
http://www.example.com/customer/orders
In Request Headers section you can set content type header like
Content-Type: application/x-www-form-urlencoded
or any other header you might require. Or just leave it blank which will work in your case.
Finally in Request Body field list your parameters in query string form
parameter1name=parameter1value¶meter2name=parameter2value
In this new case here is how you can send such a request using WebClient:
WebClient myWebClient = new WebClient();
myWebClient.UploadStringCompleted += myWebClient_UploadStringCompleted;
string url = "http://www.example.com/customer/orders";
string data = "parameter1name=parameter1value¶meter2name=parameter2value";
myWebClient.UploadStringAsync(new Uri(url, UriKind.Absolute), data);
I simply mimic the exact request that was sent.
This is how I do it:
Open Fiddler
Go to the page that I want to re-issue the command i.e. repeat the bug step but watch for the request in the list
Select it from the list and right-click, go to replay > reissue and edit
This build a replicated request but hits a break point before it is sent (You will see the red bar on the right)
Above this you can edit the values that were sent by double-clicking on any of them in Headers, QueryString etc
Then hit Run to Complete
I'm trying to call a web service from a c# application, with sessionID.
In order to do this I need to set the "Domain" header in a cookie.
In Fiddler it looks like - "ASP.NET_SessionId=izdtd4tbzczsa3nlt5ujrbf5" (no domain is specified in the cookie).
The web service is at - "http://[some ip goes here]:8989/MyAPI.asmx".
I've tried:
http://[ip] ,
http://[ip]:8989 ,
http://[ip]:8989/MyAPI.asmx
All of these cause runtime error.
I've also tried the ip alone (i.e. 100.10.10.10) , which doesn't cause a runtime error, and sets the cookie, but the cookie is never sent when I invoke a web method.
Here's my code for setting the domain:
if (!string.IsNullOrEmpty(currentSessionID))
{
req.CookieContainer=new CookieContainer();
Cookie cookie = new Cookie("ASP.NET_SessionId", currentSessionID);
cookie.Domain = GetCookieUrl(); //<- What should this be?
req.CookieContainer.Add(cookie);
}
So what should the domain be?
Thanks.
I believe it should simply be [ip]. Drop the http:// part of what you've tried.
According to this page on MSDN, your code should be
cookie.Domain = "100.10.10.10";
Next, exactly what error are you getting? Also, are you confusing a Compile error with a Runtime error? I find it hard to believe you are getting a compilation error as Domain is a String property which means you can put pretty much anything into it.
Finally, why are you sending a cookie to a web service? The normal way is to pass everything in the form post or on the query string.
Update
BTW, if you absolutely must add a cookie to the header in order to pass it to a web service, the way you do this is (taken from here):
byte[] buffer = Encoding.ASCII.GetBytes("fareId=123456"); //the data you want to send to the web service
HttpWebRequest WebReq = (HttpWebRequest)WebRequest.Create(url);
WebReq.Method = "POST";
WebReq.ContentType = "application/x-www-form-urlencoded";
WebReq.ContentLength = buffer.Length;
WebReq.Headers["Cookie"] = "ASP.NET_SessionId=izdtd4tbzczsa3nlt5ujrbf5"
Stream PostData = WebReq.GetRequestStream();
Note that this sets the header inline with the request without instantiating a "cookie" object. The Domain property of a cookie is to help ensure the cookie is only sent to the domain listed. However, if you are initiating the request and trying to append a cookie to it, then the best way is to just add it as a string to the request headers.
The reason the cookie was not sent is that the request's content length should be set after adding the cookie, and not before.
The domain is the ip alone.
// Simple function to get cookie domain
private string GetCookieDomain(string uri)
{
Uri req_uri = new Uri(uri);
return req_uri.GetComponents(UriComponents.Host, UriFormat.Unescaped);
}