Develop Reference

c# Huge memory build-up with HttpWebRequest

Hello I am having problems with httpwebrequest, huge memory buildups when running a rather large request based program. The only real place the build up can be happening is reading the stream back after sending the request, how would I combat this or better yet set a limit to this sort of thing?
I also have various (10-15) if statements looking for various things in the body of the response but about to change them to a switch statement, although I doubt they're causing such high memory buildups I thought it'd be best to mention them. My request looks like so:
HttpWebResponse response = null;
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Headers.Set(HttpRequestHeader.CacheControl, "max-age=0");
request.Headers.Add("Upgrade-Insecure-Requests", #"1");
request.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.132 Safari/537.36 OPR/63.0.3368.88";
response = (HttpWebResponse)request.GetResponse();
string _responseData = new StreamReader(response.GetResponseStream()).ReadToEnd();
catch (WebException e)
{
}
catch (UriFormatException p)
{
}
As you can see my stream is read back via _responseData which I then use .contains to see if what I'm looking for is there.

Set HttpWebRequest.AllowWriteStreamBuffering to false.
Dispose HttpWebResponse, Stream and StreamReader.
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.AllowWriteStreamBuffering = false;
request.Headers.Set(HttpRequestHeader.CacheControl, "max-age=0");
request.Headers.Add("Upgrade-Insecure-Requests", #"1");
request.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.132 Safari/537.36 OPR/63.0.3368.88";
string _responseData;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (Stream stream = response.GetResponseStream())
{
using (StreamReader sr = new StreamReader(stream, Encoding.UTF8))
{
_responseData = sr.ReadToEnd();
}
}
}
}
catch (WebException e)
{
}
catch (UriFormatException p)
{
}

Trying to login to website C#

The code I have works and successfully logs me in the first time, but when I try it for a second time the form freezes and I have to restart the program. Here is my code:
private void button1_Click(object sender, EventArgs e)
{
Auth(textBox1.Text, textBox2.Text);
}
private void Auth(string user, string pass)
{
try
{
//GET CSRF TOKEN
var request = (HttpWebRequest)WebRequest.Create("https://accounts.spotify.com/en-US/login?continue=https:%2F%2Fwww.spotify.com%2Fus%2Faccount%2Foverview%2F");
request.Method = "GET";
request.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36";
request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8";
var response1 = (HttpWebResponse)request.GetResponse();
var headers = response1.Headers;
var csrf = headers["Set-Cookie"].Split('=', ';')[1];
//POST LOGIN
var request2 = (HttpWebRequest)WebRequest.Create("https://accounts.spotify.com/api/login");
var postdata = Encoding.ASCII.GetBytes(string.Format("remember=true&username={0}&password={1}&captcha_token=&csrf_token={2}", user, pass, csrf));
request2.Method = "POST";
request2.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36";
request2.ContentType = "application/x-www-form-urlencoded";
request2.ContentLength = postdata.Length;
request2.Headers.Add(string.Format("Cookie: sp_t=4c114efa38d7c3d8510fa41bb7fd76af; spot=%7B%22t%22%3A1523485160%2C%22m%22%3A%22us%22%2C%22p%22%3Anull%7D; _ga=GA1.2.981430147.1523485160; sp_usid=919af976-2997-44dc-872b-1704356316bb; sp_last_utm=%7B%22utm_source%22%3A%22us-en_brand_contextual-desktop_text%22%2C%22utm_medium%22%3A%22paidsearch%22%2C%22utm_campaign%22%3A%22alwayson_ucanz_us_performancemarketing_core_brand+contextual-desktop+text+bmm+us-en+google%22%7D; _gid=GA1.2.411374915.1523901844; _gcl_aw=GCL.1523901845.CjwKCAjwk9HWBRApEiwA6mKWabkt4w1SE6lOm5TjrId-HkZNuakJeNIhOAkuVou7Vmueqb0bbw7sUBoCcvYQAvD_BwE; _gcl_dc=GCL.1523901845.CjwKCAjwk9HWBRApEiwA6mKWabkt4w1SE6lOm5TjrId-HkZNuakJeNIhOAkuVou7Vmueqb0bbw7sUBoCcvYQAvD_BwE; _gac_UA-5784146-31=1.1523902416.CjwKCAjwk9HWBRApEiwA6mKWabkt4w1SE6lOm5TjrId-HkZNuakJeNIhOAkuVou7Vmueqb0bbw7sUBoCcvYQAvD_BwE; _gat_UA-5784146-31=1; _gali=header-login-link; __bon=MHwwfC0zMTA4ODYyMDR8LTEzMDU3MjIwNTY4fDF8MXwxfDE=; fb_continue=https%3A%2F%2Fwww.spotify.com%2Fus%2Faccount%2Foverview%2F; _gat=1; csrf_token={0}; remember=ghyuit78%40gmail.com", csrf));
using (Stream w = request2.GetRequestStream()) //Freezes around here the second time around
{
w.Write(postdata, 0, postdata.Length);
}
var response2 = new StreamReader(((HttpWebResponse)request2.GetResponse()).GetResponseStream()).ReadToEnd();
richTextBox1.Text = "Request1:\n----------------------\n" + response2;
if (response2.Contains("displayName"))
{
label3.ForeColor = Color.Green;
label3.Text = "Successfully logged in!";
}
else
{
label3.ForeColor = Color.Red;
label3.Text = "Invalid username/password. Try again!";
}
}
catch
{
label3.ForeColor = Color.Red;
label3.Text = "ERROR! Try again!";
}
}
I manage to get a successful login the first time, but when I press the button again, it freezes and does not give me an error or anything. I have to open my task manager and end the process.

Download image from google drive

I try to allow users to select images from the users google drive.
I follow this doc: https://developers.google.com/picker/docs/.
When i recive the image url from google ('/drive/v2/files/' + FileId) then i try to download tile image in my .NET code:
var WebRequest = (HttpWebRequest)System.Net.WebRequest.Create(url);
WebRequest.AllowWriteStreamBuffering = true;
WebRequest.Timeout = 30000;
WebRequest.KeepAlive = false;
WebRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36";
WebRequest.AllowAutoRedirect = true;
WebRequest.Headers.Add("Authorization", "Bearer " + token);
byte[] Arr;
using (var WebResponse = WebRequest.GetResponse())
{
using (var Stream = WebResponse.GetResponseStream())
{
Arr = Stream.StreamToByteArray();
}
WebResponse.Close();
}
return Arr;
But all i get is a empty (white) webpage. With the url i get from google, i can se the image in my browser, but i cant download it in my code.
How can i get the image downloaded?

Use instead WebClient:
client = new WebClient();
client.Headers.Add("User-Agent", "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36");
client.Headers.Add("Authorization", "Bearer " + token);
client.AllowWriteStreamBuffering = true;
return client.DownloadData(url); /// DownloadData returns a byte array

Error 500 with authorization while consuming OAuth2 RESTful service through C#

My current job is to consume a RESTful API with OAuth2. Currently I worked out how to get the access token and it is working ok while I use the chrome extension Rest Console, but when I try to do it from my application I always get the error that I am sending an invalid OAuth request. Below you can see three of the ways I tried to consume the API, but to no success. The page always returns error 500. Any help will be appreciated, if I had missed something crucial.
var auth = "Bearer " + item.access_token;
/* First Attempt */
var client = new RestClient("http://<link>");
var request = new RestRequest("sample", Method.GET);
request.AddHeader("Authorization", auth);
request.AddHeader("Content-Type", "application/json;charset=UTF-8");
request.AddHeader("Pragma", "no-cache");
request.AddHeader("User-Agent", "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36");
request.AddHeader("Accept", "application/json");
request.RequestFormat = DataFormat.Json;
var response = client.Execute(request);
var content = response.Content;
/* Second Attempt */
string sURL = "http://<link>/sample";
string result = "";
using (WebClient client = new WebClient())
{
client.Headers["Authorization"] = auth;
client.Headers["Content-Type"] = "application/json;charset=UTF-8";
client.Headers["Pragma"] = "no-cache";
client.Headers["User-Agent"] = "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36";
client.Headers["Accept"] = "application/json";
byte[] byteArray = Encoding.UTF8.GetBytes(parameters);
var result1 = client.DownloadString(sURL);
}
/* Third Attempt */
var request = (HttpWebRequest)WebRequest.Create(sURL);
request.Method = "GET";
request.ContentType = "application/json;charset=UTF-8";
request.Accept = "application/json";
request.Headers["Authorization"] = auth;
request.UserAgent = "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36";
string content;
HttpStatusCode statusCode;
using (var response = request.GetResponse())
using (var stream = response.GetResponseStream())
{
var contentType = response.ContentType;
Encoding encoding = null;
if (contentType != null)
{
var match = Regex.Match(contentType, #"(?<=charset\=).*");
if (match.Success)
encoding = Encoding.GetEncoding(match.ToString());
}
encoding = encoding ?? Encoding.UTF8;
statusCode = ((HttpWebResponse)response).StatusCode;
using (var reader = new StreamReader(stream, encoding))
content = reader.ReadToEnd();
}
--------EDIT--------
For the first attempt I also tried to add the authentication to the client variable client.Authenticator = Authenticate; where OAuth2AuthorizationRequestHeaderAuthenticator Authenticate = new OAuth2AuthorizationRequestHeaderAuthenticator(item.access_token, item.token_type);

The code seems right. The fail attempts you did suggest that the issue is with the token and not the code. Bearer tokens have expiration time. So semms like your token expired between the first time you got it using chrome REST Console extension and when you wrote your code. But the strange situation here is the 500 error code you got. 401 is standard response when you token expired or not exist. 500 error code always mean server error.

Website Login And Scraping HTML

I'm a little bit stuck here. I am busy making an Windows application that reads data from a website. However de website requires a login first and i don't seem to be able to get passes that. I'm fairly new to programming, so i hope someone know a solution.
This is the code i use to login:
private void btnLogin2_Click(object sender, EventArgs e)
{
HttpWebRequest request = WebRequest.Create(LoginPageURL) as HttpWebRequest;
request.KeepAlive = true;
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.UserAgent = "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.2 Safari/537.36";
string postData = "j_username=" + number + "&j_password=" + password;
byte[] dataBytes = UTF8Encoding.UTF8.GetBytes(postData);
request.ContentLength = dataBytes.Length;
using (Stream postStream = request.GetRequestStream())
{
postStream.Write(dataBytes, 0, dataBytes.Length);
}
HttpWebResponse httpResponse = request.GetResponse() as HttpWebResponse;
request = WebRequest.Create(Page2URL) as HttpWebRequest;
request.UserAgent = "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.2 Safari/537.36";
request.CookieContainer = new CookieContainer();
request.CookieContainer.Add(httpResponse.Cookies);
request.Method = "GET";
HttpWebResponse httpResponse2 = request.GetResponse() as HttpWebResponse;
StreamReader stream = new StreamReader(httpResponse2.GetResponseStream(), System.Text.Encoding.UTF8);
string result = stream.ReadToEnd();
stream.Close();
tbOutput2.Text = result;
}
The point is that i get the HTML of the page after the login (Page2URL). But i keep getting the HTML from the Login Page.

You are adding cookies to the request from first response:
request.CookieContainer.Add(httpResponse.Cookies);
Probably the cookies in response are null! To cope up with this issue, read cookie values from response header and add them to the next request like this:
string response_header_cookies = response.Headers.Get("Set-Cookie")
req.Headers.Add("Cookie",response_header_cookies);
In most of the cases this is the more efficient way. Hope this helps!
Source: msdn