Develop Reference

C# Add Accept header to HttpClient

What is the difference between these two calls? My end goal is to have
Accept: application/json sent over the wire, not to append to some default set of other MIME types.
HttpClient client = new HttpClient();
client.DefaultRequestHeaders.Add("Accept", "application/json");
vs.
client.DefaultRequestHeaders
.Accept
.Add(new MediaTypeWithQualityHeaderValue("application/json"));
My CLR is .NET Core 2.0.
Sniffing the wire reveals no difference:
# just .Add("Accept"...
~ % nc -l 8000
GET / HTTP/1.1
Connection: Keep-Alive
Accept: application/json
[...]
# with MediaTypeWithQualityHeaderValue
~ % nc -l 8000
GET / HTTP/1.1
Connection: Keep-Alive
Accept: application/json
[...]
So, outside the bizarre naming of that type, nothing else to gain here right?

There is no difference.
DefaultRequestHeaders.Accept is a collection of string type, where you can add your header to accept using the new instance of MediaTypeWithQualityHeaderValue.
client.DefaultRequestHeaders is a dictionary that accepts key for and value for the request header and matches the results according to them.
DefaultRequestHeaders
has overloads.
The only thing that differs between them, is the fact that DefaultRequestHeaders.Accept will require you to initialize a new instance of MediaTypeWithQualityHeaderValue class, resulting in another reference type in the heap, while client.DefaultRequestHeaders will add the data to the dictionary, removing the cost of resources and the need to initialize a new instance.
It is really up to the user as to how and what to use.

There's no difference in the end result, as long as the names and values are correct.
The HTTP standard specifies that certain headers have a quality factor, hence the name MediaTypeWithQualityHeaderValue. It's a MediaType header value that can have a Quality factor. You can pass the quality factor if you use the MediaTypeWithQualityHeaderValue Constructor (String, Double) constructor
The Accept header section in the standard shows several examples that use the quality factor. For example,
The example
Accept: audio/*; q=0.2, audio/basic
SHOULD be interpreted as "I prefer audio/basic, but send me any audio type if it is the best available after an 80% mark-down in quality."
You could write that with
var requestAccepts=client.DefaultRequestHeaders.Accept;
requestAccepts.Add(new MediaTypeWithQualityHeaderValue("audio/*",0.2));
requestAccepts.Add(new MediaTypeWithQualityHeaderValue("audio/basic"));
Or you can enter the raw header value with :
client.DefaultRequestHeaders.Add("Accept", "audio/*; q=0.2, audio/basic");

I don't think there is a difference it's just an easier way to add standard headers through properties like Accept, also convenient to add custom headers through the Add method.

Cookies Max-Age in Web Forms

Is it possible to set Max Age for Cookies in Web Forms Application? I know, that it's okey to set Expire, but is there a way to set Max Age?

Asp.Net doesn't specifically provide this property on HttpCookie, probably because they are very Microsoft-centric, and IE doesn't support max-age (as least, as of IE11)
However, you can still do it. Here's some code demonstrating the proper and invalid ways to set this cookie with max-age:
// doesn't work:
var mytestcookie = new HttpCookie("regular_httpcookie", "01");
mytestcookie.Values.Add("max-age", "300");
Response.Cookies.Add(mytestcookie);
// *does* work:
Response.Headers.Add("set-cookie", "testingmaxage=01;max-age=300; path=/");
And it renders like this in the HTTP response:
Set-Cookie testingmaxage=01;max-age=300; path=/
X-AspNet-Version 4.0.30319
Set-Cookie regular_httpcookie=01&max-age=300; expires=Fri, 10-Jun-2016 15:02:15 GMT; path=/
As you can see above, if you are also setting cookies using HttpCookie, this will create a second "set-cookie" header on the response , but the browser won't mind, it will just add it to the list of cookies.
I tested on IE11 and Chrome and this is a non-issue - the cookies will all go in, as long as they have differing names. If the cookie name conflicts with one already set in HttpCookies, the last one in wins. Check out the text of your HTTP response to see which one goes in last. (Best to simply make sure they don't conflict though)
As I mentioned at the beginning, when testing on IE11, I noted that it's ignoring the max-age property of the cookie. Here's a link to a way to settle that issue:
Set-Cookie: Expire property, clock skew and Internet Explorer issue

Remove Host header in HttpClient request

I'm using the HttpClient class to send some data to specific host. I just want to send a pure header without any additional lines in it like ("Host: http"). So this line is the last to be removed from the header, but I don't know how.
The code:
HttpRequestMessage msg = new HttpRequestMessage(HttpMethod.Post, aUrl);
msg.Headers.Clear();
msg.Headers.Remove("Host");
msg.Headers.ExpectContinue = false;
Encoding encoding = ConfiguratorASUST.Instance.Encoding ?? Encoding.GetEncoding(ConfiguratorASUST.ENCODING_DEFAULT);
msg.Content = new StringContent(aStr, encoding);
_client.SendAsync(msg);
The result header in Fiddler:
POST http://http//localhost.fiddler:60001/POS/POSTELESPIS HTTP/1.1
Content-Type: text/plain; charset=windows-1251
Host: http
This line Host: http needs to be removed from the message's header. But how on earth can I do that?! I tried the following:
msg.Headers.Clear();
msg.Headers.Remove("Host");
To no avail. Actually I also see the header Proxy-Connection: Keep-Alive being added.

If you carefully inspect your URL, it looks like your it is wrong anyway: http://http// - is your host really named http, and do you really need two slashes after it? Anyway if you fix that, the Host header will carry localhost.fiddler:60001.
By removing the Host header, you're essentially downgrading your request to HTTP/1.0.
You can set the HTTP version in the HttpRequestMessage as explained in Set HTTP protocol version in HttpClient:
msg.Version = HttpVersion.Version10;
But when using Fiddler, it acts as a proxy, and forwards your request as an HTTP/1.1 request - including the host header again. You can also alter the request in Fiddler. This is explained in How do I prevent fiddler from insering "Host" HTTP header?, but note the bold text, emphasis mine:
Per the RFC, as a HTTP/1.1 proxy, Fiddler is required to add a Host header.
It's not clear why this is problematic-- any server that has a problem with this is, by definition, buggy and should be fixed.
You can remove the header if you'd like (although doing so can cause problems elsewhere). Click Rules > Customize Rules. Scroll to OnBeforeRequest and add the following:
if (oSession.oRequest.headers.HTTPVersion == "HTTP/1.0")
{
oSession["x-overridehost"] = oSession.host;
oSession.oRequest.headers.Remove("Host");
}

How to get RestSharp to properly deflate compressed HTTP response?

I'm using RestSharp to make a call to REST service. My call looks something like this:
var request = new RestRequest("/foo", Method.POST);
request.JsonSerializer.ContentType = "application/json; charset=utf-8";
request.AddJsonBody(new string[] { "param1", "param2" });
var response = this._client.Execute<Foo>(request);
For most other calls this works fine. I'm running into issues when the response is compressed. The headers in the response look (mostly) like this:
HTTP/1.1 200 OK
Uncompressed-Size: 35000
Content-Length: 3019
Content-Encoding: deflate
Content-Type: application/json
The issue is when I call this method with RestSharp I keep getting the error:
Error: Block length does not match with its complement.
I've tried setting the Accept-Encoding header in the request but it still produces the error. I also tried using a custom deserializer but the error is occurring before deserialization. From what I can tell, RestSharp should automatically handle deflation if the Content-Encoding header says deflate (which it does).
How can I get RestSharp to handle the deflation properly?
UPDATE
In the end I was able to have the service changed to look for an Accept-Encoding header in the request with a value of identity. If found, the service was changed to return the data uncompressed.
This is unfortunately not really a solution to the original issue but it does resolve the problem for me. If a better solution is posted I will try it.

According to this post, you should be able to handle it if you won't pass charset=utf-8 in content type.
Please refer to this:
RestSharp compress request while making rest call to server

Help With .NET CookieContainer

I've recently run into some problems with the CookieContainer. Either I'm doing something seriously wrong or there is some kind of bug w/ the CookieContainer object. It doesn't seem to update the cookie collection with certain Set-Cookie headers.
This might be a lengthy post and I appologize, but I want to be as thurough as possible so I'm going to list my HTTP sniffing logs as well as my actual implementation code.
public bool SendRequest(HttpWebRequest request, IDictionary<string, string> data, int retries)
{
// copy request in case request instance already failed
HttpWebRequest newRequest = (HttpWebRequest)HttpWebRequest.Create(request.RequestUri);
newRequest.Method = request.Method;
// if POST data was provided, write it to the stream
if (data != null && data.Count != 0)
{
StreamWriter writer = new StreamWriter(newRequest.GetRequestStream());
writer.Write(createPostString(data));
writer.Close();
}
// set request with global cookie container
newRequest.CookieContainer = this.cookieJar;
try
{
using (HttpWebResponse resp = (HttpWebResponse)newRequest.GetResponse())
{
//CookieCollection newCooks = getCookies(resp.Headers);
//updateCookies(newCooks);
this.cookieJar = newRequest.CookieContainer;
this.Html = getResponseString(resp);
/* remainder snipped */
So there is the code, here are two request to responses I sniffed in Fiddler:
Request 1
POST /login/ HTTP/1.1
Host: www.site.com
Content-Length: 47
Expect: 100-continue
Connection: Keep-Alive
Response 1
HTTP/1.1 200 OK
Date: Wed, 02 Dec 2009 17:03:35 GMT
Server: Apache
Set-Cookie: tcc=one; path=/
Set-Cookie: cust_id=2702585226; domain=.site.com; path=/; expires=Mon, 01-Jan-2011 00:00:00 GMT
Set-Cookie: cust_session=12%2F2%2F2009%20%2012%3A3%3A35; domain=.site.com; path=/; expires=Wed 2-Dec-2009 17:33:35
Set-Cookie: refer_id_persistent=0000; domain=.site.com; path=/; expires=Fri 2-Dec-2011 17:3:35
Set-Cookie: refer_id=0000; domain=.site.com; path=/
Set-Cookie: private_browsing_mode=off; domain=.site.com; path=/; expires=Fri, 01-Jan-2010 17:03:35 GMT
Set-Cookie: member_session=UmFuZG9tSVYL%5BS%5D%5BP%5DfhH77bYaVoS9j9Yd8ySRkyHHz%5BS%5Dk0S8MVsQ6AyraNlcdcCRC0RkB%5BP%5DfBYVM4vn6JQ3HlJxT3GlJi1RZiMGQaITg7HN9dpu9oRbZgMjhJlXXa%5BP%5D7pFSjqDIZWRr3LAfnhh3btv4E3rvVH42CeOP%5BS%5Dx6kDyvrokQEHyIHPGi7zswZbuHrUdx2XKEKKJzw1unDWfw0LZWjoehAs0QgSOz6Nzp8P4Hp8hqrULdIMch6acPT%5BS%5DbKV8zwugBIcjr5dI3rVR%5BP%5Dv42rsTtQB7dyb%5BP%5DRKb8Y83cGqhHM33hP%5BP%5DUtmbDC1PPfr%5BS%5DPC23lAO%5BS%5DmQ3mOy9x4pgQSOfp40XSfzgVg3EavITaxHBeI5nO3%5BP%5D%5BS%5D2rSDthDfuEm4sT9i6UF3sYd1vlOL0IC9ZsVatV1yhhpQ%5BE%5D%5BE%5D; domain=.site.com; path=/; expires=Fri, 01-Jan-2010 17:03:35 GMT
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8
Request 2
GET /test?search=jjkjf HTTP/1.1
Host: www.site.com
Cookie: tcc=one; cust_id=2702585226; private_browsing_mode=off; member_session=UmFuZG9tSVYL%5BS%5D%5BP%5DfhH77bYaVoS9j9Yd8ySRkyHHz%5BS%5Dk0S8MVsQ6AyraNlcdcCRC0RkB%5BP%5DfBYVM4vn6JQ3HlJxT3GlJi1RZiMGQaITg7HN9dpu9oRbZgMjhJlXXa%5BP%5D7pFSjqDIZWRr3LAfnhh3btv4E3rvVH42CeOP%5BS%5Dx6kDyvrokQEHyIHPGi7zswZbuHrUdx2XKEKKJzw1unDWfw0LZWjoehAs0QgSOz6Nzp8P4Hp8hqrULdIMch6acPT%5BS%5DbKV8zwugBIcjr5dI3rVR%5BP%5Dv42rsTtQB7dyb%5BP%5DRKb8Y83cGqhHM33hP%5BP%5DUtmbDC1PPfr%5BS%5DPC23lAO%5BS%5DmQ3mOy9x4pgQSOfp40XSfzgVg3EavITaxHBeI5nO3%5BP%5D%5BS%5D2rSDthDfuEm4sT9i6UF3sYd1vlOL0IC9ZsVatV1yhhpQ%5BE%5D%5BE%5D
So as you can see, the CookieContainer (this.cookieJar) which is used for every request is not picking up the Set-Cookie header for refer_id, cust_session, refer_id_persistent. However it does pick up cust_id, private_browsing_mode, tcc, and member_session... Any ideas why this might be?

Just wanted to update this post in case someone else came across this. Issue is that .NET complies with the RFC specification for cookie tags, but not all sites do. So, ultimately, the issue is not Microsoft, or .NET for the matter. (Although, IE, manages the cookies fine so it would be better to rewrite their .NET cookie parsing methods using the same parsing methods) The issue is the sites that do not follow RFC specifications.
Nonetheless, an issue I've often encountered is that sites will use commas in the expiration dates in their cookies. .NET interprets these as separators between different cookie fields and strips the ending and everything there after off of the cookie.
RFC spec: "Cookie:, followed by a comma-separated list of one or more cookies." An easy solution to this problem would be for the web server to enclose values with commas in quotation marks, per the RFC document. However, there is no RFC police, so we can only hope that people follow the rules.

MSDN SetCookies:
SetCookies pulls all the HTTP cookies out of the HTTP cookie header, builds a Cookie for each one, and then adds each Cookie to the internal CookieCollection that is associated with the URI. The HTTP cookies in the cookieHeader string must be delimited by commas.
MSDN GetCookieHeader
GetCookieHeader returns a string that holds the HTTP cookie header for the Cookie instances specified by uri. The HTTP header is built by adding a string representation of each Cookie associated with uri. Note that the exact format of the string depends on the RFC that the Cookie conforms to. The strings for all the Cookie instances that are associated with uri are combined and delimited by semicolons.
This string is not in the correct format for use as the second parameter of the SetCookies method.

This is only a quick scan through you code but it seems that you are sending post data before you send the cookies in the request.
if (data != null && data.Count != 0)
{
StreamWriter writer = new StreamWriter(newRequest.GetRequestStream());
writer.Write(createPostString(data));
writer.Close();
}
// set request with global cookie container
newRequest.CookieContainer = this.cookieJar;
What might be happening is when you write your post data to the stream this is sent to the remote server. However for a cookies to be set they must be sent to the server before any postdata. The simple solution is to swap this around like so:
// set request with global cookie container
newRequest.CookieContainer = this.cookieJar;
if (data != null && data.Count != 0)
{
StreamWriter writer = new StreamWriter(newRequest.GetRequestStream());
writer.Write(createPostString(data));
writer.Close();
}

CookieContainer has 2 major issues that I have come across, whether it is by design or bug I don't know.
1) Cookies set on a 302 post are not picked up.
Example
Post to site
302 redirect response
Load New page which sets cookie
Solution
Set autoredirect to false and manually follow the redirects and set the cookies yourself
2) .Net is VERY fussy about incorectly form cookie strings that have a comma in the string. This is actually correct, but occasioally cookies have the date set that include a comma, which stops all cookies being set.
Solution
Manually parse cookie strings and add yourself. A horrible task. I have a sprawing mess of a hack function, that loops and ifs but the end result is it works for all cases I have thrown at it so far. IT isn't pretty but it gets job done
Not sure the above is your issue, but maybe. If not some food for thought anyway

My solution: replace " UTC" with " GMT".

Try using CookieContainer.GetCookieHeader and CookieContainer.SetCookies
YourCookieContainer.GetCookieHeader(new Uri("your url"));
YourCookieContainer.SetCookies(new Uri("your url"), "string from GetCookieHeader");