this is quiet different than the answers given on the other questions on the stack.
i m confused why i'm getting the exception "Too many requests" when sending a GET request to the server .
the request works fine on BURP-SUIT/PostMan even without headers . and i tried to setup 10 continuous requests on Postman and i received OK status code on all of them with interval of 700 ms . even tho in c# Code i'm still getting this exception .
any help is really appreciated .
EDIT :
var req = WebRequest.Create("example.com") as HttpWebRequest;
req.Method = "GET";
req.UserAgent = "Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0";
req.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
req.GetResponse();
Add this to your request header:
Retry-After: 120
Like this on code :
request.Headers.Add("Retry-After", "120");
Related
I made a program in VB.net a few years back which counted headers from web requests and if it matched 10 it would do something else it would continue checking.
I'm now trying to transfer the program to C# and extend the program, does C# have a similar method? I found this but I am struggling to implement it.
For reference, this is my visual basic code:
Sub CheckLink(ByVal link As String)
Try
If link.Length >= 3 Then
Dim web As New WebClient
web.Headers.Add("user-agent", " Mozilla/5.0 (Windows NT 6.1; WOW64; rv:25.0) Gecko/20100101 Firefox/25.0")
Dim t As Byte() = WC.DownloadData("https://www.google.com")
Dim headers As WebHeaderCollection = WC.ResponseHeaders
WC.Dispose()
If headers.Count = 10 Then
Msgbox("Correct")
Else
// Do nothing
End If
End Try
End Sub
Is there a method for C#? thank you!
EDIT:
I've tried implementing it into my C# but I don't think the user agent is correct as it's not outputting. I have tested the header count on a basic plaintext website without the user agent and it outputted the correct header count but I think that websites with javascript need the user agent provided otherwise it doesn't return the HttpWebResponse. Where have I gone wrong?
static void Main(string[] args)
{
Console.WriteLine("Header count: ");
HttpWebRequest myHttpWebRequest = (HttpWebRequest)WebRequest.Create("http://passport.twitch.tv/usernames/sdjkf3jk");
myHttpWebRequest.UserAgent = " Mozilla/5.0 (Windows NT 6.1; WOW64; rv:25.0) Gecko/20100101 Firefox/25.0";
HttpWebResponse myHttpWebResponse = (HttpWebResponse)myHttpWebRequest.GetResponse();
WebHeaderCollection myWebHeaderCollection = myHttpWebResponse.Headers;
Console.WriteLine(myWebHeaderCollection.Count); // Test write to see if outputting
}
I am posting the following JSON to my Flask server:
{"comment": "astute observation", "ua": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36", "cookie": "ua=f8256d37159e3faf28ae61a6406601c3; platform=pc; bs=bxa7ziiq07dufk31prvoo0mbfm44sayh; ss=994139249854417186; il=v1qyca8PU7X6jSwgiqceXsySwWB60HnCjSJ1HjNmsSxRUxNjQzNjA3OTEycUVTTFoxWUpSZDFTZ3kwclIzenBHa19FbnlmMTlIN0hZeHNDQ1FOLQ..; expiredEnterModalShown=1", "parent": "860245961"}
When sent from Postman, the request works flawlessly. However, when sent from my C# app, like so, the server returns a 400 Bad Request error code.
var data = $"{{\"comment\": \"{text}\", \"ua\": \"{userAgent}\", \"cookie\": \"{cookie}\", \"parent\": \"{parent}\"}}";
var url = "http://127.0.0.1:5000/";
var request = HttpWebRequest.CreateHttp(url);
request.Method = "POST";
request.ContentType = "application/json";
await using var sw = new StreamWriter(request.GetRequestStream());
await sw.WriteAsync(data);
var response = await request.GetResponseAsync();
To make sure that the JSON is properly formatted, I set a breakpoint and inspected the "data" variable. I copied/pasted that value into Postman, set the Content-Type to application/json, and the request succeeds from there, but fails from my C# application.
Server code:
from flask import Flask, request
app = Flask(__name__)
#app.route('/', methods=['GET', 'POST'])
def process():
print(request.json)
return 'Hello'
app.run(debug=True)
Edit: After posting this, I realized I wasn't sending a user agent from my C# app. I tried adding one and the server still responds with a 400 Bad Request error.
Edit 2: I tried simplifying the data variable for testing purposes. var data = "{\"msg\": \"david\"}"; and setting request.ContentLength = data.Length, still having the same problem. Really strange.
Very odd, but this solved the problem. Using the StreamWriter class was causing the issue. Doing this instead results in a 200 OK response finally.
using var stream = request.GetRequestStream();
await stream.WriteAsync(Encoding.UTF8.GetBytes(data));
Consider the following URL: "http://www.bestbuy.com". This resource is quickly and correctly loaded in all browsers and from all locations.
However, basic C# code currently stucks (ends by timeout for any timeout) for this URL:
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
var req = WebRequest.CreateHttp("http://www.bestbuy.com");
req.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
req.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36";
req.Timeout = 30000; // you can set any timeout
using (var resp = (HttpWebResponse)req.GetResponse()) // stucks and finally ends by timeout here
{
Console.WriteLine(resp.StatusCode);
Console.WriteLine(resp.ResponseUri.AbsoluteUri);
}
Fiddle: https://dotnetfiddle.net/M7NZgG
The same code works fine for most other URLs.
Tried different things, but all of them did not help:
direct loading of HTTPS version ("https://www.bestbuy.com")
remove calls for UserAgent, AutomaticDecompression and SecurityProtocol setters
HttpClient class also stucks and ends by timeout for that resource.
In Fiddler the response is quickly returned, but it looks strange - it is completely empty:
I'm trying to get ASP.NET_SessionId from a HttpWebResponse but it seems that no such data comes on the response.
Basically I'm trying to simulate some steps over some pages, where authentication is required. The problem is not in the authentication, my problem is to get ASP.NET_SessionId after the authentication so I can use it in my future requests/steps.
From Chrome on developer tools > network, I can see the ASP.NET_SessionId on the headers, but it does not come in my HttpWebResponse. Any ideia why this is happening?
There is my code:
var httpWebRequest = (HttpWebRequest) WebRequest.Create(url);
httpWebRequest.UserAgent = "Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.93 Safari/537.36"; httpWebRequest.Method = "POST";
httpWebRequest.ContentType = "application/x-www-form-urlencoded";
httpWebRequest.ContentLength = 0;
var httpWebResponse = (HttpWebResponse) httpWebRequest.GetResponse();
After my request I should see a ASP.NET_SessionId header Set-Cookie, but no luck. Any ideia?
I've seen some people say that
httpWebResponse.Headers["ASP.NET_SessionId"]
or
httpWebResponse.Headers["SESSION_ID"]
should work but no, no session id header is set nor any Cookie.
After many research, the answer was here
Basically we have to keep the same CookieContainer object reference across all requests. I was extracting some Set-Cookie from the responses and adding them into my requests, but now I don't need to do anything, CookieContainer manages all of it transparently.
Set-Cookie from responses are set on CookieContainer of your request. It's the way they found to resolve possible security issues, so don't lose more time and just keep a reference to your CookieContainer because you will not be able to access session id (and you don't need it).
There's the example of my code now.
var cookieContainer = new CookieContainer();
var httpWebRequest1 = (HttpWebRequest) WebRequest.Create(url);
httpWebRequest1.CookieContainer = cookieContainer;
// do the request and some logic
var httpWebRequest2 = (HttpWebRequest) WebRequest.Create(anotherUrl);
httpWebRequest2.CookieContainer = cookieContainer; // same cookieContainer reference
Everything is working great now, hope it helps someone.
Trying To Loading Html Content Of "http://links.casemakerlegal.com/states/CA/books/Case_Law/results?search[Cite]=214 Cal.App.3d 533" but HttpWebResponse object Giving This Error "(500) Internal Server Error"
And Code Is------
request = WebRequest.Create(urlCheck); request.Timeout = 100000; response = request.GetResponse(); strmRead = new StreamReader(response.GetResponseStream(),System.Text.Encoding.UTF8); result = strmRead.ReadToEnd();
You need to use a tool like Wireshark or Ethereal, or the developer tools in your browser to investigate this further. It is likely the browser is sending some values in the HTTP Header that your code is not, and the server is returning a 500 due to these missing values. Try replicating all of the headers that the browser is using in your code to see if this resolves the problem.
It is usually browser agent. try adding a valid browser agent to your request headers along with Accept and Accept-Encoding headers,
*Edit: For example:
request.UserAgent = "Mozilla/5.0 (Windows NT 5.2; WOW64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1";
request.Headers.Add("Accept-Encoding: gzip, deflate");
request.Headers.Add("Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
request.Headers.Add("Accept-Language: en;q=0.8");