With .NET Core 3.1 DI, I use an HttpClient instance inside an Azure Function to send data to an external API.
var clientRequest = new HttpRequestMessage(HttpMethod.Post, clientConfig.BaseURL + RequestParam);
string payloadBody = GenerateLoginPayload(clientConfig.Username, clientConfig.Password);
clientRequest.Content = new StringContent(payloadBody, Encoding.UTF8, "application/json");
clientRequest.Headers.Clear();
var response = await _api.SendAsync(clientRequest);
As you can see, I even cleared the clientRequest Headers. However, checking on Fiddler, it is still sending extra Cookie Headers. This is messing up our requests, as to the API, no past Cookies must be sent.
Cookie: KEY_PHPSESSID=HehRVwcsmiUmoxO4zYaOpO1B9hm%2BfAK9igvuRw4YjgA%3D%3AJ7Bgip94EQjfJ%2FSt5RcnFTY3jcZ%2FlChCoyK54v%2Fu%2FJI%3D
Accept: application/json
Content-Type: application/json; charset=utf-8
Content-Length: 87
My question is, what is adding that extra Cookie header value, and how can I clear or remove it?
I also checked this SO question about extra cookies being sent which is similar to my situation, but t does not seem to apply to Azure Functions and .NET Core, as I do not have any reference to Windows.Web.Http.HttpClient
This is also how I registered my HttpClient, and the _api.SendAsync() is reused multiple times across multiple endpoints, and it's possible that the Cookie originated from a previous call. How do I ensure the Cookies don't bleed to other Requests?
builder.Services.AddHttpClient<IApiCaller, ApiCaller>()
.ConfigurePrimaryHttpMessageHandler(() => new HttpClientHandler() { UseCookies = false })
.AddPolicyHandler(HttpRetryPolicies.RetryWithJitter());
Maybe it's a DefaultRequestHeader of HttpClient itself.
You could try _api.DefaultRequestHeaders.Clear();
Put a breakpoint before the request and check the contents of
clientRequest.DefaultRequestHeaders.GetCookies()
If the cookie is in there, it might get set in the implementation of your ApiCaller. If there are no cookies there, it might be set by a setting/proxy in Azure
Related
I have this code:
var handler = new HttpClientHandler
{
AllowAutoRedirect = false,
CookieContainer = new CookieContainer(),
UseCookies = true
};
return new HttpClient(handler)
{
Timeout = Timeout.InfiniteTimeSpan
};
I'm using OpenID Connect with ASP.Net Core which returns a correlation cookie like this:
cookieHeader = "correlation=ABCDEFG; path=/signin; secure; HttpOnly";
I make my call to the site:
using (HttpResponseMessage msg = client.SendAsync(request).Result)
And get redirected to the OpenID server. However, HttpClient's cookie container doesn't contain any cookies. If I manually add an identical cookie to the response modifying only path to equal / e.g.
cookieHeader = "correlation=ABCDEFG; path=/; secure; HttpOnly";
Then the cookie appears in HttpClient's CookieContainer. I don't want to have to modify ASP.Net Core base OpenID functionality to change the cookie paths so HttpClient will pick up the cookies and I can authenticate. Is there a way to make HttpClient save off cookies with paths specified?
It looks like this is a by-design silent failure. Trying to add these cookies manually yields a CookieException. It looks like ASP.Net Core's OpenID Connect implementation relies on an RFC that supersedes the one .NET Framework currently implements. In RFC 2109 (.NET Framework), you can't set a cookie with a path that is not part of the current request while .NET Core (per RFC 6265) does permit and even rely on this behavior. I guess it'll be manual cookie storage for now.
Source
I have the following code in .Net Core:
using (HttpClient client = new HttpClient())
{
client.DefaultRequestHeaders.Clear();
client.DefaultRequestHeaders.TryAddWithoutValidation("Authorization", "Bearer " + _accessToken);
client.DefaultRequestHeaders.TryAddWithoutValidation("Dropbox-API-Arg", GenerateJsonFromObject(new {path = filePath}));
var request = new HttpRequestMessage(HttpMethod.Post, "https://content.dropboxapi.com/2/files/download");
var result = client.SendAsync(request).Result;
}
As you can see it's quite simple piece of code and it works correctly, but only on Windows.
When I'm running the same code on my Linux VPS (Ubuntu 16.04 server with .Net Core 1.0.4) I'm receiving the following error message from Dropbox API:
Error in call to API function "files/download": You provided a non-empty HTTP "Content-Type" header ("application/x-www-form-urlencoded"). This API function requires that the header be missing or empty.
So API requires empty or null Content-Type Header, but somehow in .Net Core this header is automatically added to my request.
The result of printing request headers is the same on Windows and on Linux:
Method: POST, RequestUri: 'https://content.dropboxapi.com/2/files/download', Version: 1.1, Content: <null>, Headers:
{
Authorization: Bearer XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Dropbox-API-Arg: {"path":"/tracklogs/night_ride.gpx"}
}
So my question is:
How to avoid adding Content-Type header automatically by .Net Core?
What I've tried:
1 Set an empty Content-Type header:
client.DefaultRequestHeaders.TryAddWithoutValidation("Content-Type", "");
Won't work, even if I set value of header to "application/json" it's automatically changed to application/x-www-form-urlencoded :(
2 Remove whole header:
request.Headers.Remove("Content-Type");
or:
client.DefaultRequestHeaders.Remove("Content-Type");
In both cases I'm receiving the following exception:
Misused header name. Make sure request headers are used with HttpRequestMessage, response headers with HttpResponseMessage, and content headers with HttpContent objects.'
You are doing a POST. This adds a default content-type for the normally required body.
It's not clear if the specification mandates a body but this the point of a POST over a GET, and is sometimes implemented as required (although of course it might be a blank). In view of this, I suspect .net always adds a content-type header to give that body meaning. https://www.rfc-editor.org/rfc/rfc9110.html#name-method-definitions
Possibly this request should be a GET which doesn't require a body. There may also be alternative ways to make the request which give more control over the construction of the request.
I'm using the HttpClient class to send some data to specific host. I just want to send a pure header without any additional lines in it like ("Host: http"). So this line is the last to be removed from the header, but I don't know how.
The code:
HttpRequestMessage msg = new HttpRequestMessage(HttpMethod.Post, aUrl);
msg.Headers.Clear();
msg.Headers.Remove("Host");
msg.Headers.ExpectContinue = false;
Encoding encoding = ConfiguratorASUST.Instance.Encoding ?? Encoding.GetEncoding(ConfiguratorASUST.ENCODING_DEFAULT);
msg.Content = new StringContent(aStr, encoding);
_client.SendAsync(msg);
The result header in Fiddler:
POST http://http//localhost.fiddler:60001/POS/POSTELESPIS HTTP/1.1
Content-Type: text/plain; charset=windows-1251
Host: http
This line Host: http needs to be removed from the message's header. But how on earth can I do that?! I tried the following:
msg.Headers.Clear();
msg.Headers.Remove("Host");
To no avail. Actually I also see the header Proxy-Connection: Keep-Alive being added.
If you carefully inspect your URL, it looks like your it is wrong anyway: http://http// - is your host really named http, and do you really need two slashes after it? Anyway if you fix that, the Host header will carry localhost.fiddler:60001.
By removing the Host header, you're essentially downgrading your request to HTTP/1.0.
You can set the HTTP version in the HttpRequestMessage as explained in Set HTTP protocol version in HttpClient:
msg.Version = HttpVersion.Version10;
But when using Fiddler, it acts as a proxy, and forwards your request as an HTTP/1.1 request - including the host header again. You can also alter the request in Fiddler. This is explained in How do I prevent fiddler from insering "Host" HTTP header?, but note the bold text, emphasis mine:
Per the RFC, as a HTTP/1.1 proxy, Fiddler is required to add a Host header.
It's not clear why this is problematic-- any server that has a problem with this is, by definition, buggy and should be fixed.
You can remove the header if you'd like (although doing so can cause problems elsewhere). Click Rules > Customize Rules. Scroll to OnBeforeRequest and add the following:
if (oSession.oRequest.headers.HTTPVersion == "HTTP/1.0")
{
oSession["x-overridehost"] = oSession.host;
oSession.oRequest.headers.Remove("Host");
}
I have rather strange problem. With all that questions over internet how to add and get cookies, I want the opposite ;)
When I try to send request via HttpHandler it is adding it's own Cookie header. I have to get rid of it. Without digging into details - when it is added, server I am trying to request is giving me wrong answer. It works without this cookie (tried in fiddler).
But back to the problem, code:
string domain = "someMysteriousDomain";
var handler = new HttpClientHandler();
handler.UseDefaultCredentials = false;
handler.AllowAutoRedirect = true;
handler.ClientCertificateOptions = ClientCertificateOption.Manual;
handler.UseCookies = false;
var httpClient = new HttpClient(handler);
var request = new HttpRequestMessage(HttpMethod.Get, domain);
request.Headers.UserAgent.Add(new ProductInfoHeaderValue("Mozilla", "5.0"));
request.Headers.AcceptEncoding.Add(new StringWithQualityHeaderValue("gzip"));
var response = await httpClient.SendAsync(request);
Raw request seen in fiddler:
GET https://domain HTTP/1.1
Accept-Encoding: gzip
User-Agent: Mozilla/5.0
Host: domain
Connection: Keep-Alive
Cookie: cadata477E7C1824F44800AF0077724F65345="51595d316-0286-44bb-bc6f-ffb1fd311a92SqJA36rA69YW7aBg+iHXYi9LAcBLN6DBWE8a3MLejd2VCluO/UQ5eF6F6T4NWh4NhdRcv4rea15Hs0e2q6GatMac59UVbljhREYdH6PRbzZC/2qn8QHtpc6go5B56R"; mobile=0
I don't want to add that cookie! How to delete/clear/whatever it?
I am using Visual Studio Community 2015, with Windows Universal Project.
What is interesting, after rebooting my pc after few hours, I was able to make 2 or 3 requests without this cookie (using THE SAME code) and then mysterios cookie returned.
What it is about? How to get rid of it?
Thank you for reporting this issue - this is a known issue with the System.Net.Http.HttpClientHandler API implementation on Windows 10 and we are working on fixing it in an upcoming release.
In the meanwhile, a possible workaround is to use the Windows.Web.Http.HttpClient API with the underlying HttpBaseProtocolFilter class. This class has a property called CookieManager that stores all the cookies for each URI. You can write a method to delete the cookies from the CookieManager for the destination URI before sending a request. This will ensure that no cookies get sent. You can see this sample for how to delete cookies from the CookieManager: https://github.com/Microsoft/Windows-universal-samples/tree/master/httpclient
Thanks
Sidharth [MSFT]
I'm writing two small pieces of C# code. The first is for a client-side Portable Class Library. All it does is send messages to an Azure Service Bus topic via the Azure Service Bus REST API, using HttpClient.
I populate the BrokerProperties header on the REST call with valid JSON, and I expect that on the server side, when I receive the message through a subscription, that I'll get my instance of BrokeredMessage.Properties populated with the values I sent from the client.
The one problem I've had on this side is that the documentation says to set Content-Type to application/atom+xml;type=entry;charset=utf-8, but even when I do I get application/json; charset=utf-8, so I'm just using application/json.
With that aside, as far as I can tell, this does what it's supposed to do. It creates the client and the request message, sets the headers, and sends the message. I get a 201 Created every time. Here's all of it:
private async static void SendServiceBusMessage(Command command)
{
// Create the HttpClient and HttpRequestMessage objects
HttpClient client = new HttpClient();
HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Post, topicUri);
// Add the authorization header (CreateAuthToken does the SHA256 stuff)
request.Headers.Add("Authorization", CreateAuthToken(topicUri, authSasKeyName, authSasKey));
// Add the content (command is a normal POCO)
// I've tried application/atom+xml;type=entry;charset=utf-8, always see application/json in the request
request.Content = new StringContent(JsonConvert.SerializeObject(command), Encoding.UTF8, "application/json");
// Add the command name and SessionId as BrokeredMessage properties
var brokeredMessageProperties = new Dictionary<string, string>();
brokeredMessageProperties.Add("CommandName", command.GetType().Name);
brokeredMessageProperties.Add("SessionId", Guid.NewGuid().ToString());
// Add the BrokerProperties header to the request
request.Content.Headers.Add("BrokerProperties", JsonConvert.SerializeObject(brokeredMessageProperties));
// I've also tried adding it directly to the request, nothing seems different
// request.Headers.Add("BrokerProperties", JsonConvert.SerializeObject(brokeredMessageProperties));
// Send it
var response = await client.SendAsync(request);
if (!response.IsSuccessStatusCode)
{
// Do some error-handling
}
}
and here's an example of the HTTP request it sends. Compare it to the example at the bottom of Send Message documentation... aside from the Content-Type, it looks (functionally) identical to me.
POST https://myawesomesbnamespace.servicebus.windows.net/commands/messages HTTP/1.1
Authorization: SharedAccessSignature sr=https%3A%2F%2Fmyawesomesbnamespace.servicebus.windows.net%2Fcommands%2Fmessages&sig=SomeValidAuthStuffHere
Content-Type: application/json; charset=utf-8
BrokerProperties: {"CommandName":"CreateJob_V1","SessionId":"94932660-54e9-4867-a020-883a9bb79fa1"}
Host: myawesomesbnamespace.servicebus.windows.net
Content-Length: 133
Expect: 100-continue
Connection: Keep-Alive
{"JobId":"6b76e7e6-9499-4809-b762-54c03856d5a3","Name":"Awesome New Job Name","CorrelationId":"47fc77d9-9470-4d65-aa7d-690b65a7dc4f"}
However, when I receive the message on the server, the .Properties are empty. This is annoying.
The server code looks like this. It just gets a batch of messages and does a foreach loop.
private async Task ProcessCommandMessages()
{
List<BrokeredMessage> commandMessages = (await commandsSubscriptionClient.ReceiveBatchAsync(serviceBusMessageBatchSize, TimeSpan.FromMilliseconds(waitTime_ms))).ToList();
foreach (BrokeredMessage commandMessage in commandMessages)
{
// commandMessage.Properties should have CommandName and SessionId,
// like I sent from the client, but it's empty
// that's not good
if (commandMessage.Properties.ContainsKey("CommandName"))
{
string commandName = commandMessage.Properties["CommandName"] as string;
// Do some stuff
}
else
{
// This is bad, log an error
}
}
}
So, I'm a bit stuck. Can anyone spot something I'm doing wrong here? Maybe it's the Content-Type problem and there's a way around it?
Thanks!
Scott
Seattle, WA, USA
OK, finally getting back to this. What I misunderstood (and I'd argue the documentation isn't clear about) is that arbitrary properties cannot be passed through the BrokerProperties header. Only named properties from the BrokeredMessage class (like SessionId, Label, etc.) will come through Service Bus to the server.
For properties to show up in BrokeredMessage.Properties, they have to be passed as custom headers on the request. So, in my case,
request.Headers.Add("CommandName", command.GetType().Name);
gets the CommandName property to show up on the server after the message is passed through Service Bus.
And to pass the SessionId value, I'll still want to pass it through BrokerProperties header.