I have to send ajax request from C#. In browser the request looks like:
Request URL:https://sts-service.mycompany.com/UPNFromUserName
Request method:POST
Remote address:xxxx
Status code:
200
Version:HTTP/2.0
Referrer Policy:strict-origin-when-cross-origin
Headers:
Host: sts-service.mycompany.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: https://sts.mycompany.com/
Content-type: application/x-www-form-urlencoded
Content-Length: 17
Origin: https://sts.mycompany.com
DNT: 1
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Params:
Form data:
MS.Aution
Cookies: no cookie
And in C# my request:
WebRequest webRequest = WebRequest.Create("https://sts-service.mycompany.com/UPNFromUserName");
((HttpWebRequest)webRequest).Referer = "https://sts.mycompany.com/";
((HttpWebRequest)webRequest).Host = "sts-service.mycompany.com";
((HttpWebRequest)webRequest).KeepAlive = true;
((HttpWebRequest)webRequest).AllowAutoRedirect = true;
((HttpWebRequest)webRequest).UseDefaultCredentials = true;
((HttpWebRequest)webRequest).UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0";
webRequest.ContentType = "application/x-www-form-urlencoded";
((HttpWebRequest)webRequest).Accept = "*/*";
((HttpWebRequest)webRequest).Headers.Add("Origin", "https://sts.mycompany.com");
((HttpWebRequest)webRequest).Headers.Add("Accept-Encoding", "gzip, deflate, br");
((HttpWebRequest)webRequest).Headers.Add("Accept-Language", "en-US,en;q=0.5");
((HttpWebRequest)webRequest).Headers.Add("Upgrade-Insecure-Requests", #"1");
((HttpWebRequest)webRequest).Headers.Add("DNT", #"1");
((HttpWebRequest)webRequest).AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
webRequest.Method = HttpRequestType.POST.ToString();
string msg = "MSCnE.Automation";
webRequest.ContentLength = msg.Length;
Stream reqStream = webRequest.GetRequestStream();
byte[] msgb = System.Text.Encoding.UTF8.GetBytes(msg);
reqStream.Write(msgb, 0, msgb.Length);
reqStream.Close();
var response = (HttpWebResponse)webRequest.GetResponse();
StreamReader sr = new StreamReader(response.GetResponseStream());
string Result = sr.ReadToEnd();
response.Close();
I get error:
The remote server returned an error: (400) Bad Request.
In browser in Network tab the request looks like:
Type is json but in Headers Content Type is application/x-www-form-urlencoded
Maybe this is the reason?
something along these lines (not tested):
public async Task<string> SendPOST()
{
var dict = new Dictionary<string, string>();
dict.Add("DNT", "1");
dict.Add("someformdata","MSCnE.Automation");
using (var formdata = new System.Net.Http.FormUrlEncodedContent(dict))
{
//do not use using HttpClient() - https://learn.microsoft.com/en-us/dotnet/api/system.net.http.httpclient
using (System.Net.Http.HttpClient httpClient = new System.Net.Http.HttpClient())
{
formdata.Headers.ContentType = System.Net.Http.Headers.MediaTypeHeaderValue.Parse("application/x-www-form-urlencoded");
formdata.Headers.ContentType.CharSet = "UTF-8";
httpClient.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0");
using (var response = await httpClient.PostAsync("https://sts-service.mycompany.com/UPNFromUserName", formdata))
{
if (response.IsSuccessStatusCode)
{
var postresult = await response.Content.ReadAsStringAsync();
return postresult;
}
else
{
string errorresult = await response.Content.ReadAsStringAsync();
return errorresult;
}
}
}
}
}
Related
I made http post request with content type application/x-www-form-urlencoded UTF-8 using Httpclient.PostRequest. I need to convert application/x-www-form-urlencoded request result to json string. How can I application/x-www-form-urlencoded to json in C#?
I did next post request.
HttpResponseMessage response;
using (var request = new HttpRequestMessage(new HttpMethod("POST"), "https://sv.ki.court.gov.ua/new.php"))
{
request.Headers.TryAddWithoutValidation("Connection", "keep-alive");
request.Headers.TryAddWithoutValidation("Accept", "application/json, text/javascript, */*; q=0.01");
request.Headers.TryAddWithoutValidation("Accept-Encoding", "gzip, deflate, br");
request.Headers.TryAddWithoutValidation("Accept-Language", "en-US,en;q=0.9");
request.Headers.TryAddWithoutValidation("Referer", "https://sv.ki.court.gov.ua/sud2608/gromadyanam/csz");
request.Headers.TryAddWithoutValidation("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36");
request.Headers.TryAddWithoutValidation("Origin", "https://sv.ki.court.gov.ua");
request.Headers.TryAddWithoutValidation("X-Requested-With", "XMLHttpRequest");
request.Headers.TryAddWithoutValidation("Cookie", "_ga=GA1.3.1754947237.1562858299; PHPSESSID=1nosm5dhdljsv8tpsu97bl8dn5; cookiesession1=257F9822DPOYFGLLUDVTJCMDYYR98AB6; _gid=GA1.3.1599643655.1563891940; _gat=1");
request.Content = new StringContent("q_court_id=2608", Encoding.UTF8, "application/x-www-form-urlencoded");
response = await _client.SendAsync(request);
}
Encoding.RegisterProvider(CodePagesEncodingProvider.Instance);
var json = await response.Content.ReadAsStringAsync();
As StringContent type I used application/x-www-form-urlencoded. application/json doesn't work. Post request return result string "\u001f<....". I need to convert response result application/x-www-form-urlencoded to application/json
This should give you some idea. In the example below inside PostAsync you pass in a new instance of StringContent where you set the type of content i.e. in this case Json.
public static async Task MainAsync()
{
using (var client = new HttpClient())
{
client.BaseAddress = new Uri("http://localhost:1234");
var content = new FormUrlEncodedContent(new[]
{
new KeyValuePair<string, string>("", "Joe Bloggs")
});
var result = await client.PostAsync("/api/Customer/exists", new StringContent(content.ToString(), Encoding.UTF8, "application/json"));
string resultContent = await result.Content.ReadAsStringAsync();
Console.WriteLine(resultContent);
}
}
I'd like to get phone number from https://sprzedajemy.pl/doskonale-dla-pary-planujacej-poszerzenie-rodziny-sprawdz-warszawa-2-1b8e55-nr57347155
Phone number is "protect" and I have to click "show number" to get request with phone. Before I send request I have to get from source data-id="805c74a74f3ea9fe6db5da90d722" from button "show number" and send POST with this token as _rp_offerID.
Correct answer is:
<span><strong> 516 000 551</strong></span>
My answer is:
?
My complete code:
HttpWebRequest getRequest = (HttpWebRequest)WebRequest.Create("https://sprzedajemy.pl/oferta-dane.telefon");
getRequest.Method = "POST";
getRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36";
getRequest.ContentType = "application/x-www-form-urlencoded";
getRequest.Host = "sprzedajemy.pl";
getRequest.Referer = url;
getRequest.Headers.Add("accept-encoding", "gzip, deflate, br");
getRequest.Headers.Add("accept-language", "pl,en-US;q=0.9,en;q=0.8,ru;q=0.7");
getRequest.Headers.Add("origin", "https://sprzedajemy.pl");
getRequest.Headers.Add("X-Requested-With", "XMLHttpRequest");
var postData = "_rp_offerID=" + itemId;
var data = Encoding.ASCII.GetBytes(postData);
getRequest.ContentLength = data.Length;
using (var stream = getRequest.GetRequestStream())
{
stream.Write(data, 0, data.Length);
}
var httpResponseP = (HttpWebResponse)getRequest.GetResponse();
var streamReaderP = new StreamReader(httpResponseP.GetResponseStream());
string strPhone = streamReaderP.ReadToEnd();
Console.WriteLine(strPhone);
I don't know what is wrong with my code...
If I use REST client for Chrome with:
POST https://sprzedajemy.pl/oferta-dane.telefon
User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36
Content-Type: application/x-www-form-urlencoded
Referer: http://sprzedajemy.pl/doskonale-dla-pary-planujacej-poszerzenie-rodziny-sprawdz-warszawa-2-1b8e55-nr57347155
accept-encoding: gzip, deflate, br
accept-language: pl,en-US;q=0.9,en;q=0.8,ru;q=0.7
origin: https://sprzedajemy.pl
X-Requested-With: XMLHttpRequest
Host: sprzedajemy.pl
Content-Length: 48
Body Form Data:
_rp_offerID=80e158b0281e04a2102fd7bce6eba0cd3833
Answer is correct
Why don't you use HttpClient ?
it's much easier!
check the example below:
using System;
using System.Net.Http;
using System.Text;
namespace httpClient
{
class Program
{
static void Main(string[] args)
{
using (var client = new HttpClient() {BaseAddress = new Uri("https://sprzedajemy.pl")})
{
client.DefaultRequestHeaders.Add("accept-encoding", "gzip, deflate, br");
client.DefaultRequestHeaders.Add("accept-language", "pl,en-US;q=0.9,en;q=0.8,ru;q=0.7");
client.DefaultRequestHeaders.Add("origin", "https://sprzedajemy.pl");
client.DefaultRequestHeaders.Add("X-Requested-With", "XMLHttpRequest");
var postData = "_rp_offerID=80e158b0281e04a2102fd7bce6eba0cd3833";
var stringContent = new StringContent(postData, Encoding.Default, "application/x-www-form-urlencoded");
var result = client.PostAsync("oferta-dane.telefon", stringContent).GetAwaiter().GetResult();
}
}
}
}
i`ve tested this code and the return was 200
I am trying to extract data from a government site which is rendered on display of a pop up. I checked the network console and got the POST request URL and able to replicate the request-response on Postman. Now I am trying to make the call programmatically. I tried using the default code generated by Postman but it did not work.
I am writing the code in C# and I am able to get the response but I am not able to get the correct encoding to extract the response.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://saubhagya.gov.in/dashboard/data/dashboard_saubhagya");
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.Accept = "application/json, text/javascript, */*; q=0.01";
request.Host = "saubhagya.gov.in";
request.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36";
request.Headers.Add("X-Requested-With", "XMLHttpRequest");
request.Headers.Add("Accept-Encoding: gzip, deflate");
request.Headers.Add("Accept-Language: en-US,en;q=0.9");
request.Headers.Add("Cache-Control: no-cache");
request.Headers.Add("Pragma: no-cache");
request.Headers.Add("Origin: http://saubhagya.gov.in");
request.Referer = "http://saubhagya.gov.in/";
String CookieStr = "_ga=GA1.3.590075225.1533991967; _gid=GA1.3.790472263.1533991967; saubhagyasession=BHxX%2FFfttUfxM7JhoIruGVzdq0m%2F4sGeTn95c%2BUB%2BGvJok9PkS3g9pR8vLVfeEJ1XB8UULGNThvbAeN5HfAu%2FE6qt%2F5X3qL8Yla4my0qmxSmz6Q9ztpLztCD0PyY17uWDnJgkSjSt%2BSF0B5Xh32SUsxBXHH%2BeFGwtIXdAnzSLcxC0MO8KZSiE2io4ksZO6AZ31YSxnGei6CluQzg4fCFgXvVwR4%2F00%2FKAbf0MnhLwaTtXxD0jngmDv3Rjy8enD87c20vwObHGTgcLC3KQoh2lw5L1WRF1lVLlpjzLrUoeJV3cD8o0c15bT5SA%2FV1Y8OqFPhqhpr0%2BzzG%2FbAVs6OKMmLiokl7hHrPx5NECDsmY3KzmCkNHka%2B1ueEWTv%2FTOUqH2hll2A8485gFhqFgnrh%2FKkhOb6I8lChI2QQoyHr%2B9U%3D92add88ce105d8b3ec1dd72efa1dd7ec9b9f1e52";
CookieContainer cookiecontainer = new CookieContainer();
string[] cookies = CookieStr.Split(';');
foreach (string cookie in cookies)
cookiecontainer.SetCookies(new Uri("http://saubhagya.gov.in/dashboard/data/dashboard_saubhagya"), cookie);
request.CookieContainer = cookiecontainer;
using (var streamWriter = new StreamWriter(request.GetRequestStream()))
{
string json = "ci_csrf_token=&state=35&district=638&village=645115&vtype=&discom=&search_text=&uuid=&maptype=states&kyroargs=&drilldownkey=&kyroclickid=&kyrorefreshid=&page=dashboard_saubhagya";
streamWriter.Write(json);
streamWriter.Flush();
streamWriter.Close();
}
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
using (StreamReader sr = new StreamReader(response.GetResponseStream(), System.Text.Encoding.GetEncoding(response.CharacterSet)))
{
var result = sr.ReadToEnd();
}
I get an encoded/junk string as output.
Requesting for help!
In the header you state that you accept gzip, but the response is not decompressed on your end, so just add:
request.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
Then you can remove this line because headers will be added automatically:
//request.Headers.Add("Accept-Encoding: gzip, deflate");
My current job is to consume a RESTful API with OAuth2. Currently I worked out how to get the access token and it is working ok while I use the chrome extension Rest Console, but when I try to do it from my application I always get the error that I am sending an invalid OAuth request. Below you can see three of the ways I tried to consume the API, but to no success. The page always returns error 500. Any help will be appreciated, if I had missed something crucial.
var auth = "Bearer " + item.access_token;
/* First Attempt */
var client = new RestClient("http://<link>");
var request = new RestRequest("sample", Method.GET);
request.AddHeader("Authorization", auth);
request.AddHeader("Content-Type", "application/json;charset=UTF-8");
request.AddHeader("Pragma", "no-cache");
request.AddHeader("User-Agent", "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36");
request.AddHeader("Accept", "application/json");
request.RequestFormat = DataFormat.Json;
var response = client.Execute(request);
var content = response.Content;
/* Second Attempt */
string sURL = "http://<link>/sample";
string result = "";
using (WebClient client = new WebClient())
{
client.Headers["Authorization"] = auth;
client.Headers["Content-Type"] = "application/json;charset=UTF-8";
client.Headers["Pragma"] = "no-cache";
client.Headers["User-Agent"] = "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36";
client.Headers["Accept"] = "application/json";
byte[] byteArray = Encoding.UTF8.GetBytes(parameters);
var result1 = client.DownloadString(sURL);
}
/* Third Attempt */
var request = (HttpWebRequest)WebRequest.Create(sURL);
request.Method = "GET";
request.ContentType = "application/json;charset=UTF-8";
request.Accept = "application/json";
request.Headers["Authorization"] = auth;
request.UserAgent = "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36";
string content;
HttpStatusCode statusCode;
using (var response = request.GetResponse())
using (var stream = response.GetResponseStream())
{
var contentType = response.ContentType;
Encoding encoding = null;
if (contentType != null)
{
var match = Regex.Match(contentType, #"(?<=charset\=).*");
if (match.Success)
encoding = Encoding.GetEncoding(match.ToString());
}
encoding = encoding ?? Encoding.UTF8;
statusCode = ((HttpWebResponse)response).StatusCode;
using (var reader = new StreamReader(stream, encoding))
content = reader.ReadToEnd();
}
--------EDIT--------
For the first attempt I also tried to add the authentication to the client variable client.Authenticator = Authenticate; where OAuth2AuthorizationRequestHeaderAuthenticator Authenticate = new OAuth2AuthorizationRequestHeaderAuthenticator(item.access_token, item.token_type);
The code seems right. The fail attempts you did suggest that the issue is with the token and not the code. Bearer tokens have expiration time. So semms like your token expired between the first time you got it using chrome REST Console extension and when you wrote your code. But the strange situation here is the 500 error code you got. 401 is standard response when you token expired or not exist. 500 error code always mean server error.
I'm trying to make a PUT request to plug.dj with a httpWebRequest in c#, the problem i'm having is that i keep getting error 400(bad Request) but the request i made looks the same as the original one, am i misisng something?
These are the request headers that i got from chrome when i go to the network tab:
PUT /_/booth/lock HTTP/1.1
Host: plug.dj
Connection: keep-alive
Content-Length: 39
Accept: application/json, text/javascript, */*; q=0.01
Origin: https://plug.dj
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36
Content-Type: application/json
Referer: https://plug.dj/ao3
Accept-Encoding: gzip, deflate, sdch
Accept-Language: nl-NL,nl;q=0.8,en-US;q=0.6,en;q=0.4
Cookie: {Left out becouse it's used to authenticate}
The request payload and the request Url:
{"isLocked":true,"removeAllDJs":false}
https://plug.dj/_/booth/lock
And this is the one i created in c# using httpWebRequest(the authentication is correct so that's not the error):
var httpWebRequest = (HttpWebRequest)WebRequest.Create(requestUrl);
httpWebRequest.ContentType = "application/json";
httpWebRequest.Method = "PUT";
httpWebRequest.Accept = "application/json, text/javascript, */*; q=0.01";
httpWebRequest.Headers["Accept-Encoding"] = "gzip, deflate, sdch";
httpWebRequest.Headers["Accept-Language"] = "nl-NL,nl;q=0.8,en-US;q=0.6,en;q=0.4";
httpWebRequest.ContentLength = data.Length;
httpWebRequest.Headers["Cookie"] = GetCookies();
httpWebRequest.Host = "plug.dj";
httpWebRequest.Headers["Origin"] = "https://plug.dj";
httpWebRequest.Referer = "https://plug.dj/" + _room;
httpWebRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36";
httpWebRequest.Headers["X-Requested-With"] = "XMLHttpRequest";
using (var streamWriter = new StreamWriter(httpWebRequest.GetRequestStream()))
{
streamWriter.Write(data);
}
var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
{
return streamReader.ReadToEnd();
}