Develop Reference

C# HttpWebRequest send xhr request - 400 bad request

I have to send ajax request from C#. In browser the request looks like:
Request URL:https://sts-service.mycompany.com/UPNFromUserName
Request method:POST
Remote address:xxxx
Status code:
200
Version:HTTP/2.0
Referrer Policy:strict-origin-when-cross-origin
Headers:
Host: sts-service.mycompany.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: https://sts.mycompany.com/
Content-type: application/x-www-form-urlencoded
Content-Length: 17
Origin: https://sts.mycompany.com
DNT: 1
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Params:
Form data:
MS.Aution
Cookies: no cookie
And in C# my request:
WebRequest webRequest = WebRequest.Create("https://sts-service.mycompany.com/UPNFromUserName");
((HttpWebRequest)webRequest).Referer = "https://sts.mycompany.com/";
((HttpWebRequest)webRequest).Host = "sts-service.mycompany.com";
((HttpWebRequest)webRequest).KeepAlive = true;
((HttpWebRequest)webRequest).AllowAutoRedirect = true;
((HttpWebRequest)webRequest).UseDefaultCredentials = true;
((HttpWebRequest)webRequest).UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0";
webRequest.ContentType = "application/x-www-form-urlencoded";
((HttpWebRequest)webRequest).Accept = "*/*";
((HttpWebRequest)webRequest).Headers.Add("Origin", "https://sts.mycompany.com");
((HttpWebRequest)webRequest).Headers.Add("Accept-Encoding", "gzip, deflate, br");
((HttpWebRequest)webRequest).Headers.Add("Accept-Language", "en-US,en;q=0.5");
((HttpWebRequest)webRequest).Headers.Add("Upgrade-Insecure-Requests", #"1");
((HttpWebRequest)webRequest).Headers.Add("DNT", #"1");
((HttpWebRequest)webRequest).AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
webRequest.Method = HttpRequestType.POST.ToString();
string msg = "MSCnE.Automation";
webRequest.ContentLength = msg.Length;
Stream reqStream = webRequest.GetRequestStream();
byte[] msgb = System.Text.Encoding.UTF8.GetBytes(msg);
reqStream.Write(msgb, 0, msgb.Length);
reqStream.Close();
var response = (HttpWebResponse)webRequest.GetResponse();
StreamReader sr = new StreamReader(response.GetResponseStream());
string Result = sr.ReadToEnd();
response.Close();
I get error:
The remote server returned an error: (400) Bad Request.
In browser in Network tab the request looks like:
Type is json but in Headers Content Type is application/x-www-form-urlencoded
Maybe this is the reason?

something along these lines (not tested):
public async Task<string> SendPOST()
{
var dict = new Dictionary<string, string>();
dict.Add("DNT", "1");
dict.Add("someformdata","MSCnE.Automation");
using (var formdata = new System.Net.Http.FormUrlEncodedContent(dict))
{
//do not use using HttpClient() - https://learn.microsoft.com/en-us/dotnet/api/system.net.http.httpclient
using (System.Net.Http.HttpClient httpClient = new System.Net.Http.HttpClient())
{
formdata.Headers.ContentType = System.Net.Http.Headers.MediaTypeHeaderValue.Parse("application/x-www-form-urlencoded");
formdata.Headers.ContentType.CharSet = "UTF-8";
httpClient.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0");
using (var response = await httpClient.PostAsync("https://sts-service.mycompany.com/UPNFromUserName", formdata))
{
if (response.IsSuccessStatusCode)
{
var postresult = await response.Content.ReadAsStringAsync();
return postresult;
}
else
{
string errorresult = await response.Content.ReadAsStringAsync();
return errorresult;
}
}
}
}
}

HttpClient: "HttpRequestException: An error occurred while sending the request"

I'm trying to get a image using HttpClient and I'm getting this error:
HttpRequestException: An error occurred while sending the request
Using WebClient with DownloadData method, works fine.
var cookieContainer = new CookieContainer();
using (var handler = new HttpClientHandler() { CookieContainer = cookieContainer })
using (var client = new HttpClient(handler))
{
client.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36");
client.DefaultRequestHeaders.Add("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8");
client.DefaultRequestHeaders.Add("Accept-Encoding", "gzip, deflate");
client.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36");
client.DefaultRequestHeaders.Add("Accept-Language", "pt-BR,pt;q=0.8,en-US;q=0.6,en;q=0.4");
client.DefaultRequestHeaders.Add("Upgrade-Insecure-Requests", "1");
client.GetAsync("cookieGenerateUrl").Wait();
client.DefaultRequestHeaders.Remove("Accept");
client.DefaultRequestHeaders.Add("Accept", "image/webp,image/apng,image/*,*/*;q=0.8");
var imagem = client.GetByteArrayAsync(imageUrl).Result;
}
What is the equivalent to WebClient.DownloadData on HttpClient class?

Probably the problem here is that you are calling client.GetByteArrayAsync with no proper handling of the awaitable.
the fix is:
var res = await client.GetByteArrayAsync(url);
or
byte[] arr;
client.GetByteArrayAsync(url).ContinueWith((x) => arr = x.Result);

Getting some invalid response(some random characters) in c#

I am doing a get request in c# but i am getting some invalid response than the original content
code
using System;
using System.Net;
using System.Text;
using System.IO;
public class Test
{
// Specify the URL to receive the request.
public static void Main(string[] args)
{
HttpWebRequest myHttpWebRequest = (HttpWebRequest)WebRequest.Create("http://www1.bloomingdales.com/api/store/v2/stores/367,363,6113,364,4946?upcNumber=849262004629");
// Set some reasonable limits on resources used by this request
myHttpWebRequest.MaximumAutomaticRedirections = 4;
myHttpWebRequest.MaximumResponseHeadersLength = 4;
// Set credentials to use for this request.
myHttpWebRequest.Credentials = CredentialCache.DefaultCredentials;
myHttpWebRequest.Method = "GET";
myHttpWebRequest.AllowAutoRedirect = false;
myHttpWebRequest.ContentLength = 0;
myHttpWebRequest.Accept = "application/json, text/javascript, */*; q=0.01";
myHttpWebRequest.Headers.Add("Cookie", "ShippingCountry=US;");
myHttpWebRequest.UserAgent = "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/49.0.2623.108 Chrome/49.0.2623.108 Safari/537.36";
myHttpWebRequest.Headers.Add("Accept-Encoding", "gzip, deflate, sdch");
myHttpWebRequest.Headers.Add("Accept-Language", "en-US,en;q=0.8");
myHttpWebRequest.Headers.Add("X-Macys-ClientId", "NavApp");
var response = (HttpWebResponse)myHttpWebRequest.GetResponse();
var rmyResponseHeaders = response.Headers;
Console.WriteLine("Content length is {0}", response.ContentLength);
Console.WriteLine("Content type is {0}", response.ContentType);
// Get the stream associated with the response.
Stream receiveStream = response.GetResponseStream();
// Pipes the stream to a higher level stream reader with the required encoding format.
StreamReader readStream = new StreamReader(receiveStream, Encoding.UTF8);
Console.WriteLine("Response stream received.");
Console.WriteLine(readStream.ReadToEnd());
Console.ReadLine();
response.Close();
readStream.Close();
}
}
The above getting invalid character like ???????? and some other characters
You can check original response here
curl 'http://www1.bloomingdales.com/api/store/v2/stores/367,363,6113,364,4946?upcNumber=849262004629' -H 'Cookie:shippingCountry=US;' -H 'Accept-Encoding: gzip, deflate, sdch' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/49.0.2623.108 Chrome/49.0.2623.108 Safari/537.36' -H 'Accept-Language: en-US,en;q=0.8' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'X-Macys-ClientId: NavApp' --compressed
How to fix this and get exact response(orginal response)?

You're not decompressing the response stream. Set the Accept-Encoding header to empty string until you're ready to work out which compression was used and decompress it.

Error 500 with authorization while consuming OAuth2 RESTful service through C#

My current job is to consume a RESTful API with OAuth2. Currently I worked out how to get the access token and it is working ok while I use the chrome extension Rest Console, but when I try to do it from my application I always get the error that I am sending an invalid OAuth request. Below you can see three of the ways I tried to consume the API, but to no success. The page always returns error 500. Any help will be appreciated, if I had missed something crucial.
var auth = "Bearer " + item.access_token;
/* First Attempt */
var client = new RestClient("http://<link>");
var request = new RestRequest("sample", Method.GET);
request.AddHeader("Authorization", auth);
request.AddHeader("Content-Type", "application/json;charset=UTF-8");
request.AddHeader("Pragma", "no-cache");
request.AddHeader("User-Agent", "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36");
request.AddHeader("Accept", "application/json");
request.RequestFormat = DataFormat.Json;
var response = client.Execute(request);
var content = response.Content;
/* Second Attempt */
string sURL = "http://<link>/sample";
string result = "";
using (WebClient client = new WebClient())
{
client.Headers["Authorization"] = auth;
client.Headers["Content-Type"] = "application/json;charset=UTF-8";
client.Headers["Pragma"] = "no-cache";
client.Headers["User-Agent"] = "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36";
client.Headers["Accept"] = "application/json";
byte[] byteArray = Encoding.UTF8.GetBytes(parameters);
var result1 = client.DownloadString(sURL);
}
/* Third Attempt */
var request = (HttpWebRequest)WebRequest.Create(sURL);
request.Method = "GET";
request.ContentType = "application/json;charset=UTF-8";
request.Accept = "application/json";
request.Headers["Authorization"] = auth;
request.UserAgent = "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36";
string content;
HttpStatusCode statusCode;
using (var response = request.GetResponse())
using (var stream = response.GetResponseStream())
{
var contentType = response.ContentType;
Encoding encoding = null;
if (contentType != null)
{
var match = Regex.Match(contentType, #"(?<=charset\=).*");
if (match.Success)
encoding = Encoding.GetEncoding(match.ToString());
}
encoding = encoding ?? Encoding.UTF8;
statusCode = ((HttpWebResponse)response).StatusCode;
using (var reader = new StreamReader(stream, encoding))
content = reader.ReadToEnd();
}
--------EDIT--------
For the first attempt I also tried to add the authentication to the client variable client.Authenticator = Authenticate; where OAuth2AuthorizationRequestHeaderAuthenticator Authenticate = new OAuth2AuthorizationRequestHeaderAuthenticator(item.access_token, item.token_type);

The code seems right. The fail attempts you did suggest that the issue is with the token and not the code. Bearer tokens have expiration time. So semms like your token expired between the first time you got it using chrome REST Console extension and when you wrote your code. But the strange situation here is the 500 error code you got. 401 is standard response when you token expired or not exist. 500 error code always mean server error.

PUT request to https site keeps returning status: 400 Bad request

I'm trying to make a PUT request to plug.dj with a httpWebRequest in c#, the problem i'm having is that i keep getting error 400(bad Request) but the request i made looks the same as the original one, am i misisng something?
These are the request headers that i got from chrome when i go to the network tab:
PUT /_/booth/lock HTTP/1.1
Host: plug.dj
Connection: keep-alive
Content-Length: 39
Accept: application/json, text/javascript, */*; q=0.01
Origin: https://plug.dj
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36
Content-Type: application/json
Referer: https://plug.dj/ao3
Accept-Encoding: gzip, deflate, sdch
Accept-Language: nl-NL,nl;q=0.8,en-US;q=0.6,en;q=0.4
Cookie: {Left out becouse it's used to authenticate}
The request payload and the request Url:
{"isLocked":true,"removeAllDJs":false}
https://plug.dj/_/booth/lock
And this is the one i created in c# using httpWebRequest(the authentication is correct so that's not the error):
var httpWebRequest = (HttpWebRequest)WebRequest.Create(requestUrl);
httpWebRequest.ContentType = "application/json";
httpWebRequest.Method = "PUT";
httpWebRequest.Accept = "application/json, text/javascript, */*; q=0.01";
httpWebRequest.Headers["Accept-Encoding"] = "gzip, deflate, sdch";
httpWebRequest.Headers["Accept-Language"] = "nl-NL,nl;q=0.8,en-US;q=0.6,en;q=0.4";
httpWebRequest.ContentLength = data.Length;
httpWebRequest.Headers["Cookie"] = GetCookies();
httpWebRequest.Host = "plug.dj";
httpWebRequest.Headers["Origin"] = "https://plug.dj";
httpWebRequest.Referer = "https://plug.dj/" + _room;
httpWebRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36";
httpWebRequest.Headers["X-Requested-With"] = "XMLHttpRequest";
using (var streamWriter = new StreamWriter(httpWebRequest.GetRequestStream()))
{
streamWriter.Write(data);
}
var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
{
return streamReader.ReadToEnd();
}