I have this code that creates a request and reads the data but it is always empty
static string uri = "http://yiimp.ccminer.org/api/wallet?address=DshDF3zmCX9PUhafTAzxyQidwgdfLYJkBrd";
static void Main(string[] args)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
// Get the stream associated with the response.
Stream receiveStream = response.GetResponseStream();
// Pipes the stream to a higher level stream reader with the required encoding format.
StreamReader readStream = new StreamReader(receiveStream, Encoding.UTF8);
Console.WriteLine("Response stream received.");
Console.WriteLine(readStream.ReadToEnd());
response.Close();
readStream.Close();
}
When i try to access the link from browser i get this json:
{"currency": "DCR", "unsold": 0.030825917365192, "balance": 0.02007306, "unpaid": 0.05089898, "paid24h": 0.05796425, "total": 0.10886323}
what am I missing?
When you perform the request from a browser there are a lot of headers that get sent to the web service. Apparently this web service validates the UserAgent. This is a decision on the part of the web service implementation, they might not want you to programmatically access it.
var client = (HttpWebRequest)HttpWebRequest.Create(new Uri("http://yiimp.ccminer.org/api/wallet?address=DshDF3zmCX9PUhafTAzxyQidwgdfLYJkBrd"));
client.AutomaticDecompression = DecompressionMethods.GZip;
client.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36 Edge/15.15063";
client.Headers[HttpRequestHeader.AcceptEncoding] = "gzip, deflate";
client.Host = "yiimp.ccminer.org";
client.KeepAlive = true;
using (var s = client.GetResponse().GetResponseStream())
using (var sw = new StreamReader(s))
{
var ss = sw.ReadToEnd();
Console.WriteLine(ss);
}
Sending the headers seems to make it work.
Related
Hello I'm trying to simulate a browser and this specific URL is not working well:
https://www.btgpactualdigital.com/fundos-de-investimento/spx-nimitz-fic-fim-access
It returns always the same response code... I'm using for test purpose this simple code:
using (var client = new WebClient())
{
client.Headers[HttpRequestHeader.UserAgent] = "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0";
client.Headers[HttpRequestHeader.Accept] = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8";
client.Headers[HttpRequestHeader.AcceptLanguage] = "en-us,en;q=0.5";
var responseStream = new GZipStream(client.OpenRead(url), CompressionMode.Decompress);
var reader = new StreamReader(responseStream);
textResponse = reader.ReadToEnd();
}
But it is always returning a different code from I can see on browser... I got all Headers from browser to try simulate same scenario.
Any suggestion ? Thanks !
I am trying to extract data from a government site which is rendered on display of a pop up. I checked the network console and got the POST request URL and able to replicate the request-response on Postman. Now I am trying to make the call programmatically. I tried using the default code generated by Postman but it did not work.
I am writing the code in C# and I am able to get the response but I am not able to get the correct encoding to extract the response.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://saubhagya.gov.in/dashboard/data/dashboard_saubhagya");
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.Accept = "application/json, text/javascript, */*; q=0.01";
request.Host = "saubhagya.gov.in";
request.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36";
request.Headers.Add("X-Requested-With", "XMLHttpRequest");
request.Headers.Add("Accept-Encoding: gzip, deflate");
request.Headers.Add("Accept-Language: en-US,en;q=0.9");
request.Headers.Add("Cache-Control: no-cache");
request.Headers.Add("Pragma: no-cache");
request.Headers.Add("Origin: http://saubhagya.gov.in");
request.Referer = "http://saubhagya.gov.in/";
String CookieStr = "_ga=GA1.3.590075225.1533991967; _gid=GA1.3.790472263.1533991967; saubhagyasession=BHxX%2FFfttUfxM7JhoIruGVzdq0m%2F4sGeTn95c%2BUB%2BGvJok9PkS3g9pR8vLVfeEJ1XB8UULGNThvbAeN5HfAu%2FE6qt%2F5X3qL8Yla4my0qmxSmz6Q9ztpLztCD0PyY17uWDnJgkSjSt%2BSF0B5Xh32SUsxBXHH%2BeFGwtIXdAnzSLcxC0MO8KZSiE2io4ksZO6AZ31YSxnGei6CluQzg4fCFgXvVwR4%2F00%2FKAbf0MnhLwaTtXxD0jngmDv3Rjy8enD87c20vwObHGTgcLC3KQoh2lw5L1WRF1lVLlpjzLrUoeJV3cD8o0c15bT5SA%2FV1Y8OqFPhqhpr0%2BzzG%2FbAVs6OKMmLiokl7hHrPx5NECDsmY3KzmCkNHka%2B1ueEWTv%2FTOUqH2hll2A8485gFhqFgnrh%2FKkhOb6I8lChI2QQoyHr%2B9U%3D92add88ce105d8b3ec1dd72efa1dd7ec9b9f1e52";
CookieContainer cookiecontainer = new CookieContainer();
string[] cookies = CookieStr.Split(';');
foreach (string cookie in cookies)
cookiecontainer.SetCookies(new Uri("http://saubhagya.gov.in/dashboard/data/dashboard_saubhagya"), cookie);
request.CookieContainer = cookiecontainer;
using (var streamWriter = new StreamWriter(request.GetRequestStream()))
{
string json = "ci_csrf_token=&state=35&district=638&village=645115&vtype=&discom=&search_text=&uuid=&maptype=states&kyroargs=&drilldownkey=&kyroclickid=&kyrorefreshid=&page=dashboard_saubhagya";
streamWriter.Write(json);
streamWriter.Flush();
streamWriter.Close();
}
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
using (StreamReader sr = new StreamReader(response.GetResponseStream(), System.Text.Encoding.GetEncoding(response.CharacterSet)))
{
var result = sr.ReadToEnd();
}
I get an encoded/junk string as output.
Requesting for help!
In the header you state that you accept gzip, but the response is not decompressed on your end, so just add:
request.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
Then you can remove this line because headers will be added automatically:
//request.Headers.Add("Accept-Encoding: gzip, deflate");
I want to login to Rockstar Social Club page https://pl.socialclub.rockstargames.com
I have this script
public static void Login()
{
string firstUrl = "https://pl.socialclub.rockstargames.com/profile/signin";
string formParams = string.Format("login-field={0}&password-field={1}", "mynickname", "mypassword");
string cookieHeader;
WebRequest req = WebRequest.Create(firstUrl);
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
byte[] bytes = Encoding.ASCII.GetBytes(formParams);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
WebResponse resp = req.GetResponse();
cookieHeader = resp.Headers["Set-cookie"];
string pageSource;
string getUrl = "https://pl.socialclub.rockstargames.com/games/gtav/pc/career/overview/gtaonline";
WebRequest getRequest = WebRequest.Create(getUrl);
getRequest.Headers.Add("Cookie", cookieHeader);
WebResponse getResponse = getRequest.GetResponse(); //Here returns me this error: System.Net.WebException: 'An error occurred while sending the request"
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream()))
{
pageSource = sr.ReadToEnd();
}
}
Error occures in WebResponse getResponse = getRequest.GetResponse();
System.Net.WebException: 'An error occurred while sending the request'
I don't know how to repair this, and succesfully login to this website.
I have accomplished what you are attempting to do, but on a different website.
Basically - a few years ago, I wanted to create a website that would track my Guild/Company details on Final Fantasy XIV.
They didn't have an API, so I made one.
In order to get the information I required, I needed to use a mix of HtmlAgilityPack along with the C# WebBrowser control.
In order to pass the verification token stage above, you need to run the page source in a Web Browser control.
This will allow dynamic fields and data to be generated.
You then need to take that data, and submit it with your post data.
This is to fool it into thinking the request is coming from the page.
Be warned, when doing your posts - you may need to allow for redirects and you may need to mirror the referrer and host fields to match the website you are emulating.
The specific process I followed was:
Navigate to login page in WebBrowser control
Get page source
Load into HtmlAgilityPack HtmlDocument class
Use XPath to scrape the login form.
Take _verification tokens, csrf tokens etc make note of them.
Post a web-request with the necessary data to the form target destination url.
Read the response
Be aware - sometimes the response will actually be html code that tells it to do a Javascript redirect - in my case with Final Fantasy XIV - it was loading up another form and performing an autopost on page load.
You will also want to use
LoggedInCookies = new CookieContainer();
In your first HttpWebRequest
followed by:
request.CookieContainer = LoggedInCookies;
for each subsequent request.
The cookie container will trap and persist the authentication related cookies, while the WebBrowser control and HtmlAgilityPack will allow you to scrape the fields from the web forms that you need to break through.
Adding some code from wayback when I solved this for Final Fantasy XIV's lodestone website.
This code is very old and may not work anymore, but the process it follows could be adapted for sites that do not use Javascript as part of the login process.
Pay attention to the areas where it allows the request to be redirected, this is because the Server endpoint you are calling may do Action redirects etc
If your request does not allow those redirects, then it will not be emulating the login process.
class LoggedInClient
{
public static CookieContainer LoginCookie(string user, string pass)
{
string sStored = "";
string url = "http://eu.finalfantasyxiv.com/lodestone/account/login/";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
CookieContainer cookies = new CookieContainer();
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36";
request.CookieContainer = cookies;
HttpWebResponse response1 = (HttpWebResponse)request.GetResponse();
Console.WriteLine(cookies.Count.ToString());
string sPage = "";
using (var vPage = new StreamReader(response1.GetResponseStream()))
{
sPage = vPage.ReadToEnd();
}
HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(sPage);
sStored = doc.DocumentNode.SelectSingleNode("//input[#type='hidden' and #name='_STORED_']").Attributes["value"].Value;
string param = "sqexid="+user+"8&password="+pass+"&_STORED_=" + sStored;
string postURL = doc.DocumentNode.SelectSingleNode("//form[#name='mainForm']").Attributes["action"].Value;
//Console.WriteLine(sStored);
postURL = "https://secure.square-enix.com/oauth/oa/" + postURL;
request.Method = "POST";
byte[] paramAsBytes = Encoding.Default.GetBytes(param);
request = (HttpWebRequest)WebRequest.Create(postURL);
request.ContentType = "application/x-www-form-urlencoded";
request.Method = "POST";
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36";
request.CookieContainer = cookies;
request.AllowAutoRedirect = false;
try
{
using (Stream stream = request.GetRequestStream())
{
stream.Write(paramAsBytes, 0, paramAsBytes.Length);
}
}
catch (Exception ee)
{
Console.WriteLine(ee.ToString());
}
string sGETPage = "";
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (var vPage = new StreamReader(response.GetResponseStream()))
{
sPage = vPage.ReadToEnd();
sGETPage = response.Headers["Location"];
}
}
// Console.WriteLine(sPage);
request = (HttpWebRequest)WebRequest.Create(sGETPage);
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36";
request.CookieContainer = cookies;
HttpWebResponse response2 = (HttpWebResponse)request.GetResponse();
Console.WriteLine(cookies.Count.ToString());
sPage = "";
using (var vPage = new StreamReader(response2.GetResponseStream()))
{
sPage = vPage.ReadToEnd();
}
// Console.WriteLine(sPage);
doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(sPage);
string _c = doc.DocumentNode.SelectSingleNode("//input[#type='hidden' and #name='_c']").Attributes["value"].Value;
string cis_sessid = doc.DocumentNode.SelectSingleNode("//input[#type='hidden' and #name='cis_sessid']").Attributes["value"].Value;
string action = doc.DocumentNode.SelectSingleNode("//form[#name='mainForm']").Attributes["action"].Value;
string sParams = "_c=" + _c + "&cis_sessid=" + cis_sessid;
byte[] bData = Encoding.Default.GetBytes(sParams);
// Console.WriteLine(sStored);
request = (HttpWebRequest)WebRequest.Create(action);
request.ContentType = "application/x-www-form-urlencoded";
request.Method = "POST";
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36";
request.CookieContainer = cookies;
request.AllowAutoRedirect = true;
try
{
using (Stream stream = request.GetRequestStream())
{
stream.Write(bData, 0, bData.Length);
}
}
catch (Exception ee)
{
Console.WriteLine(ee.ToString());
}
string nextPage = "";
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (var vPage = new StreamReader(response.GetResponseStream()))
{
nextPage = vPage.ReadToEnd();
}
}
// Console.WriteLine(nextPage);
doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(nextPage);
string csrf_token = doc.DocumentNode.SelectSingleNode("//input[#type='hidden' and #name='csrf_token']").Attributes["value"].Value;
string cicuid = "51624738";
string timestamp = Convert.ToInt32(DateTime.UtcNow.Subtract(new DateTime(1970, 1, 1)).TotalSeconds).ToString() + "100";
action = "http://eu.finalfantasyxiv.com/lodestone/api/account/select_character/";
sParams = "csrf_token=" + csrf_token + "&cicuid=" + cicuid + "×tamp=" + timestamp;
bData = Encoding.Default.GetBytes(sParams);
request = (HttpWebRequest)WebRequest.Create(action);
request.ContentType = "application/x-www-form-urlencoded";
request.Method = "POST";
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36";
request.CookieContainer = cookies;
request.AllowAutoRedirect = true;
try
{
using (Stream stream = request.GetRequestStream())
{
stream.Write(bData, 0, bData.Length);
}
}
catch (Exception ee)
{
Console.WriteLine(ee.ToString());
}
nextPage = "";
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (var vPage = new StreamReader(response.GetResponseStream()))
{
nextPage = vPage.ReadToEnd();
}
}
return cookies;
}
}
Go below to see answer
I am getting the Facebook access token fine, but where I am having troubles is when I am trying to automate this process.
If I visit this URL is the browser I get the access token just fine.
Example:
I paste this into the browser and hit return.
https://www.facebook.com/dialog/oauth?client_id=324234343434&scope=['ads_read', 'ads_management']&redirect_uri=http://www.kb-demos.com/login_success.html&response_type=token
Then I get sent to this page:
http://www.kb-demos.com/login_success.html?#access_token=34543534534534KJ534LKJLKJLKHLH4534534J5KH345KJ3L4H53KJ5H3K4LJH34KH54K&expires_in=5180653
I changed the access token piece so its not a real token
Viola access token!
What I am trying to do is replicate that same behavior with code. I am getting close but not quite there.
I keep getting the user_denied error.
%3Ferror%3Daccess_denied%26error_code%3D200%26error_description%3DPermissions%2Berror%26error_reason%3Duser_denied%23_%3D_&display=page&locale=en_US&logger_id=786a3153-1d81-415c-8dca-f8fa8b0cd630
I am outputting all headers to the console. It is the location header I am concerned with**
I think it has to do with the 302 redirect?
ApplicationId = request.ClientId;
string permissions = "['ads_management', 'ads_read']";
var destinationURL = String.Format(
#"https://www.facebook.com/dialog/oauth?client_id={0}&scope={1}&redirect_uri=http://www.kb-demos.com/login_success.html&response_type=token",
ApplicationId,
permissions);
// Create a new 'HttpWebRequest' Object to the mentioned URL.
HttpWebRequest myHttpWebRequest = (HttpWebRequest)WebRequest.Create(destinationURL);
myHttpWebRequest.UserAgent = "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36";
myHttpWebRequest.AllowAutoRedirect = false;
// Assign the response object of 'HttpWebRequest' to a 'HttpWebResponse' variable.
HttpWebResponse myHttpWebResponse = (HttpWebResponse)myHttpWebRequest.GetResponse();
//Console.WriteLine("\nThe HttpHeaders are \n\n\tName\t\tValue\n{0}", myHttpWebRequest.Headers); // my http headers
// Print the HTML contents of the page to the console.
var headers = myHttpWebResponse.Headers;
// output all the headers
foreach(var header in headers) {
Console.WriteLine(header.ToString() + ": " + headers[header.ToString()] + "\n" );
}
#region
//Stream streamResponse = myHttpWebResponse.GetResponseStream();
//StreamReader streamRead = new StreamReader(streamResponse);
//Char[] readBuff = new Char[256];
//int count = streamRead.Read(readBuff, 0, 256);
//Console.WriteLine("\nThe HTML contents of page the are : \n\n ");
//while (count > 0)
//{
// String outputData = new String(readBuff, 0, count);
// Console.Write(outputData);
// count = streamRead.Read(readBuff, 0, 256);
//}
//// Close the Stream object.
//streamResponse.Close();
//streamRead.Close();
// Release the HttpWebResponse Resource.
#endregion
myHttpWebResponse.Close();
Console.ReadLine();
I am getting a user_denied error here. But in the browser I am getting a perfectly good token. I cannot figure out why.
The headers in the location header it seems to work when using the browser.
Possible scenario if I cannot get the above to work:
I was wondering if there is a browser with an API? Something I can call from the command line - pass in some arguments - and then get the redirect url is a variable to parse?
This code will automate the retrieval of the access_token. You must have credentials to the account you are requesting an access token for.
Updated
First login to the facebook account.
// LOG INTO FACEBOOK ACCT
string email = "youremail#blah.com";
string pw = "yourPassWord";
CookieContainer cookieJar = new CookieContainer();
HttpWebRequest request1 = (HttpWebRequest)WebRequest.Create("https://www.facebook.com");
request1.CookieContainer = cookieJar;
request1.UserAgent = "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36";
//Get the response from the server and save the cookies from the first request..
HttpWebResponse response = (HttpWebResponse)request1.GetResponse();
var cookies = response.Cookies;
cookieJar.Add(cookies);
response.Close();// close the response
string getUrl = "https://www.facebook.com/login.php?login_attempt=1";
string postData = String.Format("email={0}&pass={1}", email, pw);
HttpWebRequest getRequest = (HttpWebRequest)WebRequest.Create(getUrl);
getRequest.CookieContainer = cookieJar;
//Adding Previously Received Cookies
getRequest.CookieContainer.Add(cookies);
getRequest.Method = WebRequestMethods.Http.Post;
getRequest.UserAgent = "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36";
getRequest.AllowWriteStreamBuffering = true;
getRequest.ProtocolVersion = HttpVersion.Version11;
getRequest.AllowAutoRedirect = true;
getRequest.ContentType = "application/x-www-form-urlencoded";
byte[] byteArray = Encoding.ASCII.GetBytes(postData);
getRequest.ContentLength = byteArray.Length;
Stream newStream = getRequest.GetRequestStream(); //open connection
newStream.Write(byteArray, 0, byteArray.Length); // Send the data.
newStream.Close();
HttpWebResponse getResponse = (HttpWebResponse)getRequest.GetResponse();
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream()))
{
string sourceCode = sr.ReadToEnd();
}
Then request the access_token
ApplicationId = request.ClientId; // your application id
string permissions = "['ads_management', 'ads_read']";
var destinationURL = String.Format(
#"https://www.facebook.com/dialog/oauth?client_id={0}&scope={1}&redirect_uri=http://www.kb-demos.com/login_success.html&response_type=token",
ApplicationId,
permissions);
// Create a new 'HttpWebRequest' Object to the mentioned URL.
HttpWebRequest myHttpWebRequest = (HttpWebRequest)WebRequest.Create(destinationURL);
// use the same cookie container and cookies
myHttpWebRequest.CookieContainer = cookieJar;
myHttpWebRequest.CookieContainer.Add(cookies); //recover cookies First request
myHttpWebRequest.UserAgent = "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36";
myHttpWebRequest.AllowAutoRedirect = false;
// Assign the response object of 'HttpWebRequest' to a 'HttpWebResponse' variable.
HttpWebResponse myHttpWebResponse = (HttpWebResponse)myHttpWebRequest.GetResponse();
Console.WriteLine("\nThe Request HttpHeaders are \n\n\tName\t\tValue\n{0}", myHttpWebRequest.Headers); // my http headers
Console.WriteLine("\nThe Request HttpHeaders are \n\n\tName\t\tValue\n{0}", myHttpWebRequest.CookieContainer); // my http headers
//Console.WriteLine("\nThe Request HttpHeaders are \n\n\tName\t\tValue\n{0}", cookies); // my http headers
var headers = myHttpWebResponse.Headers;
// output all the headers
foreach (var header in headers)
{
Console.WriteLine(header.ToString() + ": " + headers[header.ToString()] + "\n");
}
var cow = GetParams(headers["Location"]);
string accessToken = "";
accessToken = cow["#access_token"];
And the helper method
/// <summary>
/// Helper method to get Params from URL using RegEx
/// </summary>
static Dictionary<string, string> GetParams(string uri)
{
var matches = Regex.Matches(uri, #"[\?&](([^&=]+)=([^&=#]*))", RegexOptions.Compiled);
return matches.Cast<Match>().ToDictionary(
m => Uri.UnescapeDataString(m.Groups[2].Value),
m => Uri.UnescapeDataString(m.Groups[3].Value)
);
}
I'm trying to download this json file to read out some data:
http://ddragon.leagueoflegends.com/cdn/5.7.2/data/en_US/item.json
My problem is now that somehow always the values in the tags attribute is missing.
I already used different methods to retrieve them used different encodings but nothing helped. Also the content length is less than the one which I receive in the browser.
Below are some codes I already tried. As you can see I used the WebClient to download this as a string. Also tried to download it as a file, same result.
Used HttpWebRequest with all header attributes.
Used the HEAD method to receive the content length which is less than in the browser.
I also tried using the MemoryStream and the StreamReader to get the correct data. Nothing worked.
String url = "http://ddragon.leagueoflegends.com/cdn/" + version + "/data/en_US/item.json";
//String json = new WebClient().DownloadString(url);
String json = "";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.UserAgent = "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36";
request.ContentType = "text/plain; charset=utf-8 ";
request.Accept = "*/*";
request.Method = "GET";
//Get the headers associated with the request.
WebHeaderCollection myWebHeaderCollection = request.Headers;
myWebHeaderCollection.Add("DNT", "1");
myWebHeaderCollection.Add("Accept-Encoding", "gzip, deflate, sdch");
myWebHeaderCollection.Add("Accept-Language", "de-DE,de;q=0.8,en-US;q=0.6,en;q=0.4");
HttpWebRequest requestHeader = ((HttpWebRequest) WebRequest.Create(url));
requestHeader.Method = "HEAD";
using (WebResponse resp = requestHeader.GetResponse())
{
Console.WriteLine(resp.ContentLength);
}
Stream responseStream;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
using (responseStream = response.GetResponseStream())
{
if (responseStream != null)
{
Console.WriteLine(response.ContentLength);
Console.WriteLine(response.ContentType);
StreamReader readStream = null;
if (string.IsNullOrEmpty(response.CharacterSet))
{
readStream = new StreamReader(responseStream);
}
else
{
readStream = new StreamReader(responseStream, Encoding.GetEncoding(response.CharacterSet));
}
json = readStream.ReadToEnd();
readStream.Close();
//using (var memoryStream = new MemoryStream())
//{
// responseStream.CopyTo(memoryStream);
// json = Encoding.UTF8.GetString(memoryStream.ToArray());
//}
}
}
Edit:
Sample Outputs:
Browser: http://pastebin.com/nQBj64L0
C# Client: http://pastebin.com/7a0Gxvcg