I'm using the HttpClient class to send some data to specific host. I just want to send a pure header without any additional lines in it like ("Host: http"). So this line is the last to be removed from the header, but I don't know how.
The code:
HttpRequestMessage msg = new HttpRequestMessage(HttpMethod.Post, aUrl);
msg.Headers.Clear();
msg.Headers.Remove("Host");
msg.Headers.ExpectContinue = false;
Encoding encoding = ConfiguratorASUST.Instance.Encoding ?? Encoding.GetEncoding(ConfiguratorASUST.ENCODING_DEFAULT);
msg.Content = new StringContent(aStr, encoding);
_client.SendAsync(msg);
The result header in Fiddler:
POST http://http//localhost.fiddler:60001/POS/POSTELESPIS HTTP/1.1
Content-Type: text/plain; charset=windows-1251
Host: http
This line Host: http needs to be removed from the message's header. But how on earth can I do that?! I tried the following:
msg.Headers.Clear();
msg.Headers.Remove("Host");
To no avail. Actually I also see the header Proxy-Connection: Keep-Alive being added.
If you carefully inspect your URL, it looks like your it is wrong anyway: http://http// - is your host really named http, and do you really need two slashes after it? Anyway if you fix that, the Host header will carry localhost.fiddler:60001.
By removing the Host header, you're essentially downgrading your request to HTTP/1.0.
You can set the HTTP version in the HttpRequestMessage as explained in Set HTTP protocol version in HttpClient:
msg.Version = HttpVersion.Version10;
But when using Fiddler, it acts as a proxy, and forwards your request as an HTTP/1.1 request - including the host header again. You can also alter the request in Fiddler. This is explained in How do I prevent fiddler from insering "Host" HTTP header?, but note the bold text, emphasis mine:
Per the RFC, as a HTTP/1.1 proxy, Fiddler is required to add a Host header.
It's not clear why this is problematic-- any server that has a problem with this is, by definition, buggy and should be fixed.
You can remove the header if you'd like (although doing so can cause problems elsewhere). Click Rules > Customize Rules. Scroll to OnBeforeRequest and add the following:
if (oSession.oRequest.headers.HTTPVersion == "HTTP/1.0")
{
oSession["x-overridehost"] = oSession.host;
oSession.oRequest.headers.Remove("Host");
}
Related
With .NET Core 3.1 DI, I use an HttpClient instance inside an Azure Function to send data to an external API.
var clientRequest = new HttpRequestMessage(HttpMethod.Post, clientConfig.BaseURL + RequestParam);
string payloadBody = GenerateLoginPayload(clientConfig.Username, clientConfig.Password);
clientRequest.Content = new StringContent(payloadBody, Encoding.UTF8, "application/json");
clientRequest.Headers.Clear();
var response = await _api.SendAsync(clientRequest);
As you can see, I even cleared the clientRequest Headers. However, checking on Fiddler, it is still sending extra Cookie Headers. This is messing up our requests, as to the API, no past Cookies must be sent.
Cookie: KEY_PHPSESSID=HehRVwcsmiUmoxO4zYaOpO1B9hm%2BfAK9igvuRw4YjgA%3D%3AJ7Bgip94EQjfJ%2FSt5RcnFTY3jcZ%2FlChCoyK54v%2Fu%2FJI%3D
Accept: application/json
Content-Type: application/json; charset=utf-8
Content-Length: 87
My question is, what is adding that extra Cookie header value, and how can I clear or remove it?
I also checked this SO question about extra cookies being sent which is similar to my situation, but t does not seem to apply to Azure Functions and .NET Core, as I do not have any reference to Windows.Web.Http.HttpClient
This is also how I registered my HttpClient, and the _api.SendAsync() is reused multiple times across multiple endpoints, and it's possible that the Cookie originated from a previous call. How do I ensure the Cookies don't bleed to other Requests?
builder.Services.AddHttpClient<IApiCaller, ApiCaller>()
.ConfigurePrimaryHttpMessageHandler(() => new HttpClientHandler() { UseCookies = false })
.AddPolicyHandler(HttpRetryPolicies.RetryWithJitter());
Maybe it's a DefaultRequestHeader of HttpClient itself.
You could try _api.DefaultRequestHeaders.Clear();
Put a breakpoint before the request and check the contents of
clientRequest.DefaultRequestHeaders.GetCookies()
If the cookie is in there, it might get set in the implementation of your ApiCaller. If there are no cookies there, it might be set by a setting/proxy in Azure
I have the following code in .Net Core:
using (HttpClient client = new HttpClient())
{
client.DefaultRequestHeaders.Clear();
client.DefaultRequestHeaders.TryAddWithoutValidation("Authorization", "Bearer " + _accessToken);
client.DefaultRequestHeaders.TryAddWithoutValidation("Dropbox-API-Arg", GenerateJsonFromObject(new {path = filePath}));
var request = new HttpRequestMessage(HttpMethod.Post, "https://content.dropboxapi.com/2/files/download");
var result = client.SendAsync(request).Result;
}
As you can see it's quite simple piece of code and it works correctly, but only on Windows.
When I'm running the same code on my Linux VPS (Ubuntu 16.04 server with .Net Core 1.0.4) I'm receiving the following error message from Dropbox API:
Error in call to API function "files/download": You provided a non-empty HTTP "Content-Type" header ("application/x-www-form-urlencoded"). This API function requires that the header be missing or empty.
So API requires empty or null Content-Type Header, but somehow in .Net Core this header is automatically added to my request.
The result of printing request headers is the same on Windows and on Linux:
Method: POST, RequestUri: 'https://content.dropboxapi.com/2/files/download', Version: 1.1, Content: <null>, Headers:
{
Authorization: Bearer XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Dropbox-API-Arg: {"path":"/tracklogs/night_ride.gpx"}
}
So my question is:
How to avoid adding Content-Type header automatically by .Net Core?
What I've tried:
1 Set an empty Content-Type header:
client.DefaultRequestHeaders.TryAddWithoutValidation("Content-Type", "");
Won't work, even if I set value of header to "application/json" it's automatically changed to application/x-www-form-urlencoded :(
2 Remove whole header:
request.Headers.Remove("Content-Type");
or:
client.DefaultRequestHeaders.Remove("Content-Type");
In both cases I'm receiving the following exception:
Misused header name. Make sure request headers are used with HttpRequestMessage, response headers with HttpResponseMessage, and content headers with HttpContent objects.'
You are doing a POST. This adds a default content-type for the normally required body.
It's not clear if the specification mandates a body but this the point of a POST over a GET, and is sometimes implemented as required (although of course it might be a blank). In view of this, I suspect .net always adds a content-type header to give that body meaning. https://www.rfc-editor.org/rfc/rfc9110.html#name-method-definitions
Possibly this request should be a GET which doesn't require a body. There may also be alternative ways to make the request which give more control over the construction of the request.
I'm using RestSharp to make a call to REST service. My call looks something like this:
var request = new RestRequest("/foo", Method.POST);
request.JsonSerializer.ContentType = "application/json; charset=utf-8";
request.AddJsonBody(new string[] { "param1", "param2" });
var response = this._client.Execute<Foo>(request);
For most other calls this works fine. I'm running into issues when the response is compressed. The headers in the response look (mostly) like this:
HTTP/1.1 200 OK
Uncompressed-Size: 35000
Content-Length: 3019
Content-Encoding: deflate
Content-Type: application/json
The issue is when I call this method with RestSharp I keep getting the error:
Error: Block length does not match with its complement.
I've tried setting the Accept-Encoding header in the request but it still produces the error. I also tried using a custom deserializer but the error is occurring before deserialization. From what I can tell, RestSharp should automatically handle deflation if the Content-Encoding header says deflate (which it does).
How can I get RestSharp to handle the deflation properly?
UPDATE
In the end I was able to have the service changed to look for an Accept-Encoding header in the request with a value of identity. If found, the service was changed to return the data uncompressed.
This is unfortunately not really a solution to the original issue but it does resolve the problem for me. If a better solution is posted I will try it.
According to this post, you should be able to handle it if you won't pass charset=utf-8 in content type.
Please refer to this:
RestSharp compress request while making rest call to server
I'm writing two small pieces of C# code. The first is for a client-side Portable Class Library. All it does is send messages to an Azure Service Bus topic via the Azure Service Bus REST API, using HttpClient.
I populate the BrokerProperties header on the REST call with valid JSON, and I expect that on the server side, when I receive the message through a subscription, that I'll get my instance of BrokeredMessage.Properties populated with the values I sent from the client.
The one problem I've had on this side is that the documentation says to set Content-Type to application/atom+xml;type=entry;charset=utf-8, but even when I do I get application/json; charset=utf-8, so I'm just using application/json.
With that aside, as far as I can tell, this does what it's supposed to do. It creates the client and the request message, sets the headers, and sends the message. I get a 201 Created every time. Here's all of it:
private async static void SendServiceBusMessage(Command command)
{
// Create the HttpClient and HttpRequestMessage objects
HttpClient client = new HttpClient();
HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Post, topicUri);
// Add the authorization header (CreateAuthToken does the SHA256 stuff)
request.Headers.Add("Authorization", CreateAuthToken(topicUri, authSasKeyName, authSasKey));
// Add the content (command is a normal POCO)
// I've tried application/atom+xml;type=entry;charset=utf-8, always see application/json in the request
request.Content = new StringContent(JsonConvert.SerializeObject(command), Encoding.UTF8, "application/json");
// Add the command name and SessionId as BrokeredMessage properties
var brokeredMessageProperties = new Dictionary<string, string>();
brokeredMessageProperties.Add("CommandName", command.GetType().Name);
brokeredMessageProperties.Add("SessionId", Guid.NewGuid().ToString());
// Add the BrokerProperties header to the request
request.Content.Headers.Add("BrokerProperties", JsonConvert.SerializeObject(brokeredMessageProperties));
// I've also tried adding it directly to the request, nothing seems different
// request.Headers.Add("BrokerProperties", JsonConvert.SerializeObject(brokeredMessageProperties));
// Send it
var response = await client.SendAsync(request);
if (!response.IsSuccessStatusCode)
{
// Do some error-handling
}
}
and here's an example of the HTTP request it sends. Compare it to the example at the bottom of Send Message documentation... aside from the Content-Type, it looks (functionally) identical to me.
POST https://myawesomesbnamespace.servicebus.windows.net/commands/messages HTTP/1.1
Authorization: SharedAccessSignature sr=https%3A%2F%2Fmyawesomesbnamespace.servicebus.windows.net%2Fcommands%2Fmessages&sig=SomeValidAuthStuffHere
Content-Type: application/json; charset=utf-8
BrokerProperties: {"CommandName":"CreateJob_V1","SessionId":"94932660-54e9-4867-a020-883a9bb79fa1"}
Host: myawesomesbnamespace.servicebus.windows.net
Content-Length: 133
Expect: 100-continue
Connection: Keep-Alive
{"JobId":"6b76e7e6-9499-4809-b762-54c03856d5a3","Name":"Awesome New Job Name","CorrelationId":"47fc77d9-9470-4d65-aa7d-690b65a7dc4f"}
However, when I receive the message on the server, the .Properties are empty. This is annoying.
The server code looks like this. It just gets a batch of messages and does a foreach loop.
private async Task ProcessCommandMessages()
{
List<BrokeredMessage> commandMessages = (await commandsSubscriptionClient.ReceiveBatchAsync(serviceBusMessageBatchSize, TimeSpan.FromMilliseconds(waitTime_ms))).ToList();
foreach (BrokeredMessage commandMessage in commandMessages)
{
// commandMessage.Properties should have CommandName and SessionId,
// like I sent from the client, but it's empty
// that's not good
if (commandMessage.Properties.ContainsKey("CommandName"))
{
string commandName = commandMessage.Properties["CommandName"] as string;
// Do some stuff
}
else
{
// This is bad, log an error
}
}
}
So, I'm a bit stuck. Can anyone spot something I'm doing wrong here? Maybe it's the Content-Type problem and there's a way around it?
Thanks!
Scott
Seattle, WA, USA
OK, finally getting back to this. What I misunderstood (and I'd argue the documentation isn't clear about) is that arbitrary properties cannot be passed through the BrokerProperties header. Only named properties from the BrokeredMessage class (like SessionId, Label, etc.) will come through Service Bus to the server.
For properties to show up in BrokeredMessage.Properties, they have to be passed as custom headers on the request. So, in my case,
request.Headers.Add("CommandName", command.GetType().Name);
gets the CommandName property to show up on the server after the message is passed through Service Bus.
And to pass the SessionId value, I'll still want to pass it through BrokerProperties header.
When adding an Accept header to a HTTPClient it seems to automatically add spaces, which seems to be a problem for the web service that I am using. They expect a Content-Length of no more than 120. But, because of the spaces being added it's making it 131.
This is my current code:
client.DefaultRequestHeaders.TryAddWithoutValidation("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");
It seems to add spaces as can be seen in Fiddler:
text/html, application/xhtml+xml, application/xml; q=0.9, image/webp, */*; q=0.8
I know in normal cases this shouldn't be a problem for the server, but in my case it is. when the request is sent from a browser, there are no spaces.
Edit: It also does the same for Accept-Language and Accept-Encoding
I had the same problem and this worked for me :
httpRequestMessage.Headers.TryAddWithoutValidation("Accept","text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");