Develop Reference

Return Unauthorized (401) only when calling from code (c#)

UPDATE
As #Alexandru Clonțea suggested, I checked the fiddler log and found:
In both success or fail cases, there are actually 2 requests being sent. The first request are mostly the same for both cases, it's something like:
GET http://myservice.com/handler?param1=something&param2=somethingelse HTTP/1.1
Authorization: Basic xxxxxx
Accept: application/json, application/xml, text/json, text/x-json,
text/javascript, text/xml
User-Agent: RestSharp/100.0.0.0
Host: myservice.com
Accept-Encoding: gzip, deflate
Connection: Keep-Alive
The response for them are the same, which is:
HTTP/1.1 301 Moved Permanently
Content-Type: text/html; charset=utf-8
Location: /handler/?param1=something&param2=somethingelse
Date: Sat, 08 Sep 2018 01:50:16 GMT
Content-Length: 115
Moved Permanently.
I have noticed that it always try to redirect the call to /handler/?param1=something&param2=somethingelse, and that's because of the setup of the server code. it's actually working as expected. The difference is in the second request. The second request of the failure case (which is the c# code) doesn't have the authorization header and that's why it failed. Now, my question will be, why does the second request miss the authorization header? How can I fix it? Below is an example of the failed request:
GET http://myservice.com/handler/?param1=something&param2=somethingelse HTTP/1.1
Accept: application/json, application/xml, text/json, text/x-json,
text/javascript, text/xml
User-Agent: RestSharp/100.0.0.0
Accept-Encoding: gzip, deflate
Host: myservice.com
Backgroud:
I have a service written in GO deployed on a server. It requires a basic authentication. For example, I can call it successfully with the following request:
GET /handler/?
param1=something&param2=somethingelse HTTP/1.1
> Host: myservice.com
> Authorization: Basic xxxxxx
> User-Agent: RestClient/5.16.6
> Accept: */*
The request above is made by a rest api client tool (like postman), and it's working fine. It's also working fine if I call it from a browser.
Problem:
Now, I try to make the same call to the same service using c# code, and I have it as:
// pass cert validation
ServicePointManager.ServerCertificateValidationCallback += (sender, cert, chain, sslPolicyErrors) => true;
ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3 | SecurityProtocolType.Tls | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls12;
HttpClient client = new HttpClient();
var byteArray = Encoding.ASCII.GetBytes(username + ":" + password);
var auth = new System.Net.Http.Headers.AuthenticationHeaderValue("Basic", Convert.ToBase64String(byteArray));
HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Get, url);
request.Headers.Authorization = auth;
var response = client.SendAsync(request).Result; // don't need async
But in this case, I am getting Unauthorized (401) back. I have checked into the actually request that was sent by the code, it had exactly the same authorization header as the one shows above (Authorization: Basic xxxxxx, and the xxxxxx is the same as above) and same uri as well. Actually, everything it sent looks the same as when I used the rest api client tool, but it was just failed in code.
when I check the log on the server side, I see the log below when it returns 401:
[GIN-debug] redirecting request 301: /handler --> /hanlder/?param1=something&param2=somethingelse
but I don't see this log when the call is from the rest api client tool (or browser)
As you may know from the log, the server-side code is using the go gin framework. But since it works fine in other cases, I don't think it's a problem with the server-side code.
Back to the C# code, I have tried to use the HttpWebRequest with NetworkCredential instead of the HttpClient, and I also try to use client.DefaultRequestHeaders.Authorization = auth, but I was still getting the same error.
I am wondering if someone has seen this before or could help? It will be really appreciated.

As a workaround, I can modify the request to be http://myservice.com/handler/?param1=something&param2=somethingelse so that no redirection is needed. Thus, it will be correctly authorized.
But still, haven't figure out how to make the second request to be sent with the authorize header yet.

DotNetOpenOAuth Content-type

I am tearing my the remainings of my hair off trying to solve an issue with DotNetOpenOAuth. Below are the details of the problem.
Consumer = new DesktopConsumer(xeroProviderDescription, _tokenManager);
var endPoint = new MessageReceivingEndpoint(apiEndPoint, HttpDeliveryMethods.PostRequest | HttpDeliveryMethods.AuthorizationHeaderRequest);
var request = Consumer.PrepareAuthorizedRequest(endPoint, accessToken);
var content = Encoding.UTF8.GetBytes(payload);
request.ContentType = "application/json";
request.ContentLength = content.Length;
This works perfectly fine, the server accepts the request and replies. Now, I want to switch to using xml to talk to the remote service (this is their prefered format over json) So I changed all the code into sending XML data instead of json and changed the content-type to application/x-www-form-urlencoded as per the web service provider documentation. The problem now is that the service is returning 401 unauthorized request - Reason: invalid signature whenever application/x-www-form-urlencoded is used
Questions:
Is the Content-Type taken into account when generating the oauth signature (I looked into the DotNetOpenOAuth source code with no luck)?
Anyone has ever encountered this issue and resolved it?

Remove Host header in HttpClient request

I'm using the HttpClient class to send some data to specific host. I just want to send a pure header without any additional lines in it like ("Host: http"). So this line is the last to be removed from the header, but I don't know how.
The code:
HttpRequestMessage msg = new HttpRequestMessage(HttpMethod.Post, aUrl);
msg.Headers.Clear();
msg.Headers.Remove("Host");
msg.Headers.ExpectContinue = false;
Encoding encoding = ConfiguratorASUST.Instance.Encoding ?? Encoding.GetEncoding(ConfiguratorASUST.ENCODING_DEFAULT);
msg.Content = new StringContent(aStr, encoding);
_client.SendAsync(msg);
The result header in Fiddler:
POST http://http//localhost.fiddler:60001/POS/POSTELESPIS HTTP/1.1
Content-Type: text/plain; charset=windows-1251
Host: http
This line Host: http needs to be removed from the message's header. But how on earth can I do that?! I tried the following:
msg.Headers.Clear();
msg.Headers.Remove("Host");
To no avail. Actually I also see the header Proxy-Connection: Keep-Alive being added.

If you carefully inspect your URL, it looks like your it is wrong anyway: http://http// - is your host really named http, and do you really need two slashes after it? Anyway if you fix that, the Host header will carry localhost.fiddler:60001.
By removing the Host header, you're essentially downgrading your request to HTTP/1.0.
You can set the HTTP version in the HttpRequestMessage as explained in Set HTTP protocol version in HttpClient:
msg.Version = HttpVersion.Version10;
But when using Fiddler, it acts as a proxy, and forwards your request as an HTTP/1.1 request - including the host header again. You can also alter the request in Fiddler. This is explained in How do I prevent fiddler from insering "Host" HTTP header?, but note the bold text, emphasis mine:
Per the RFC, as a HTTP/1.1 proxy, Fiddler is required to add a Host header.
It's not clear why this is problematic-- any server that has a problem with this is, by definition, buggy and should be fixed.
You can remove the header if you'd like (although doing so can cause problems elsewhere). Click Rules > Customize Rules. Scroll to OnBeforeRequest and add the following:
if (oSession.oRequest.headers.HTTPVersion == "HTTP/1.0")
{
oSession["x-overridehost"] = oSession.host;
oSession.oRequest.headers.Remove("Host");
}

How to get RestSharp to properly deflate compressed HTTP response?

I'm using RestSharp to make a call to REST service. My call looks something like this:
var request = new RestRequest("/foo", Method.POST);
request.JsonSerializer.ContentType = "application/json; charset=utf-8";
request.AddJsonBody(new string[] { "param1", "param2" });
var response = this._client.Execute<Foo>(request);
For most other calls this works fine. I'm running into issues when the response is compressed. The headers in the response look (mostly) like this:
HTTP/1.1 200 OK
Uncompressed-Size: 35000
Content-Length: 3019
Content-Encoding: deflate
Content-Type: application/json
The issue is when I call this method with RestSharp I keep getting the error:
Error: Block length does not match with its complement.
I've tried setting the Accept-Encoding header in the request but it still produces the error. I also tried using a custom deserializer but the error is occurring before deserialization. From what I can tell, RestSharp should automatically handle deflation if the Content-Encoding header says deflate (which it does).
How can I get RestSharp to handle the deflation properly?
UPDATE
In the end I was able to have the service changed to look for an Accept-Encoding header in the request with a value of identity. If found, the service was changed to return the data uncompressed.
This is unfortunately not really a solution to the original issue but it does resolve the problem for me. If a better solution is posted I will try it.

According to this post, you should be able to handle it if you won't pass charset=utf-8 in content type.
Please refer to this:
RestSharp compress request while making rest call to server

How to get content type of a web address?

I want to get type of a web address. For example this is a Html page and its page type is text/html but the type of this is text/xml. this page's type seems to be image/png but it's text/html.
I want to know how can I detect the content type of a web address like this?

it should be something like this
var request = HttpWebRequest.Create("http://www.google.com") as HttpWebRequest;
if (request != null)
{
var response = request.GetResponse() as HttpWebResponse;
string contentType = "";
if (response != null)
contentType = response.ContentType;
}

HTTP Response header: content-type
For a more detailed response, please provide a more detailed question.

using (MyClient client = new MyClient())
{
client.HeadOnly = true;
string uri = "http://www.google.com";
byte[] body = client.DownloadData(uri); // note should be 0-length
string type = client.ResponseHeaders["content-type"];
client.HeadOnly = false;
// check 'tis not binary... we'll use text/, but could
// check for text/html
if (type.StartsWith(#"text/"))
{
string text = client.DownloadString(uri);
Console.WriteLine(text);
}
}
Will get you the mime type from the headers without downloading the page. Just look for the content-type in the response headers.

You can detect the Content-Type by the Http header of the response,for http://bayanbox.ir/user/ahmadalli/images/div.png ,the header is
Connection:keep-alive
Content-Encoding:gzip
Content-Type:text/html; charset=utf-8
Date:Tue, 14 Aug 2012 03:01:41 GMT
Server:bws
Transfer-Encoding:chunked
Vary:Accept-Encoding

Read up on HTTP headers.
HTTP headers will tell you the content type. For example:
content-type: application/xml.
There are two ways to determining the content-type
the file extension invoked by the URL
the http header content-type
The first one was somewhat promoted by microsoft during to old days and is not a good practice anymore.
If the client has display constraints accepting only certain content-type, it would request the server with the headers like
accept: application/json
accept: text/html
accept: application/xml
And then if the server could supply one of those and chooses XML it would return the content with the header
content-type: application/xml.
However, some services include further information like
content-type: application/xml; charset=utf-8
rather than using a header of its own for the character encoding.