I 'd like to get captcha images.
I have captured request to get images using fiddler.
this is raw text of request.
GET http://api.minteye.com/slider/image.ashx?CaptchaId=4162&PublicKey=76dd4b3f-92d2-4d29-9cb8-740ccfca4fe4&w=300&h=250&dumm=635086078045177660&reqid=e112b222-0f38-4309-9ab4-791aa11590d8-2eb9&img=1 HTTP/1.1
Host: api.minteye.com
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36
Accept: /
Referer: http://abc-site.com/?param=value
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US;q=0.8,en-US;q=0.6,en;q=0.4
Cookie: __utma=219856005.121997545.1372326418.1372834638.1372992381.5; __utmz=219856005.1372992381.5.2.utmcsr=api.minteye.com|utmccn=(referral)|utmcmd=referral|utmcct=/; ASP.NET_SessionId=oh543ydsa2z033ybv2cg0wwi; ExchangeAds=10959011
After that, i have made request header and send it to getresponse.
but i can't get any image.(can't see anything even in fiddler)
this is my code.
CookieContainer cookie = new CookieContainer();
request = (HttpWebRequest)HttpWebRequest.Create("http://api.minteye.com/slider/image.ashx?CaptchaId=4162&PublicKey=76dd4b3f-92d2-4d29-9cb8-740ccfca4fe4&w=300&h=250&dumm=4211089392&reqid=8075f74e-7878-4a9a-a5a0-3c21b5fbfb79-5f5f&img=5");
request.KeepAlive = true;
request.Accept = "*/*";
request.UserAgent = "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36";
request.Referer = "http://abc-site.com/?param=value";
request.Headers.Add("Accept-Language: en-US;q=0.8,en-US;q=0.6,en;q=0.4");
request.CookieContainer = cookie;
response = (HttpWebResponse)request.GetResponse();
While running this code, fiddler captures requests and its response, but doesn't get any image.
What's wrong?
If you give me your sources, i can look how you became the requestID,
with this requestID can you make the URL for this images.
With this URL, you can download it with a simple WebClient ;)
Related
I try to get a specific value from a specific site...
the site periodically updating the value using an Ajax call to
https://www.plus500.co.il/api/LiveData/FeedUpdate?instrumentId=19
(you can Navigate to the address and see you get the XML response.)
using Postman:
sending
GET /api/LiveData/FeedUpdate?instrumentId=19 HTTP/1.1
Host: www.plus500.co.il
Cache-Control: no-cache
Postman-Token: f823c87d-3edc-68ce-e1e7-02a8fc68be7a
I get a valid Json Response...
Though, when i try it from C#:
var webRequest = WebRequest.CreateHttp(#"https://www.plus500.co.il/api/LiveData/FeedUpdate?instrumentId=19");
webRequest.Method = "GET";
using (var response = webRequest.GetResponse())
{...}
The request Fails with Error-Code 403 (Forbidden)
when adding:
webRequest.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.84 Safari/537.36";
The request Fails with Error-Code 500 (Internal Server Error)
Addition (Edit)
I also initiate with
ServicePointManager.ServerCertificateValidationCallback = delegate { return true; };
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12 |
SecurityProtocolType.Tls11 |
SecurityProtocolType.Tls |
SecurityProtocolType.Ssl3;
Also, I Tried Setting a CookieContainer, but the result is the same 500.
Why is Postman/Chrome Successfuly querying this API while C# Webrequest do not?
What is the difference?
So, the reason that this is failing is because of the headers being included in the client request from postman by default, though not from the C# request.
Using a program like Fiddler (https://www.telerik.com/fiddler) you can watch the request to see that the headers from the postman request are:
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.84 Safari/537.36
Yet from C# are just
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36
Filling in the extra client request headers like this allows it to go through fine:
webRequest.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8";
webRequest.Headers.Add("Accept-Encoding", "gzip deflate,br");
webRequest.Headers.Add("Accept-Language", "en-US,en;q=0.9");
in asp.net pageload method I am trying to get request data using
Request.Form["keyname"]
Request.Query
but all are empty but when I run fiddler in inspector>>raw I am getting this
POST http://localhost:50844/Success.aspx HTTP/1.1
Host: localhost:50844
Connection: keep-alive
Content-Length: 15583
Cache-Control: max-age=0
Upgrade-Insecure-Requests: 1
Origin: null
Content-Type: application/x-www-form-urlencoded
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Cookie: ASP.NET_SessionId=ax0e4j1ucaghsarowzmvgoep
valuestr=abc&randomnum=99389
I need to get values of valuestr and randomnum,how can I do this
You can use the below code as per the values in your URL.
if your URl is = http://localhost:3039/test.aspx?valuestr=abc&randomnum=99389
then you can use the below code
string valuestr = Request.QueryString["valuestr"];
string randomnum= Request.QueryString["valuestr"];
Response.Write(valuestr );
Response.Write(randomnum);
do let me know in case you require any help
I am trying to make a web service call and include and authorization cookie in the header.
I see another site do this, using fiddler and the call looks like -
GET url/services/events HTTP/1.1
Host: url
Connection: keep-alive
Accept: application/json
X-Auth-Token: 33d131bb-37b4-4d87-b646-6edccab4f294
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36
Content-Type: application/json
Referer: url/api-docs/index.html
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8
Cookie: knownUser=true
How do I code in "X-Auth-Token: 33d131bb-37b4-4d87-b646-6edccab4f294".
Here is my C# code.
HttpClient client = new HttpClient();
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
client.DefaultRequestHeaders.Add("X-Auth-Token", "33d131bb-37b4-4d87-b646-6edccab4f294");
HttpResponseMessage response = client.PostAsync(new Uri("url?keywords=Taylor%20Swift&eventType=CONCERT&city=Mexico%20City"), new StringContent(JsonConvert.SerializeObject(sp), Encoding.UTF8, "application/json")).Result;
But I keep getting a 500 error. (StatusCode: 500, ReasonPhrase: 'Internal Server Error')
What is the ocrrect syntax to get that token passed intot he web service call?
I have a webforms application that has a download handler in an ashx file that's redirected to from an urlMapping in the web.config. This works alright, however I'd like to also support continuing downloads- which uses the Range request header. However, though the Range header is sent, it is not visible on the server.
My test request looks like this:
GET http://localhost:81/Downloads/MyFile.zip HTTP/1.1
Host: localhost:81
Connection: keep-alive
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36
Accept-Language: en-US,en;q=0.8,nl;q=0.6
Range: bytes=3-
However, when on the server I try read the Range header, I get nothing. The following code:
var request = HttpContext.Current.Request;
Debug.WriteLine("/" + request.HttpMethod + " " + request.RawUrl);
foreach (var key in headers.AllKeys)
{
Debug.WriteLine(key + ": " + context.Request.Headers[key]);
}
The output that gives me is:
/GET /Downloads/MyFile.zip
Cache-Control: max-age=0
Connection: keep-alive
Content-Length: 0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.8,nl;q=0.6
Host: localhost:81
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36
Somehow, the Range header is gone and a "Content-Length: 0" header has been added.
My question is: Why is the range header being removed, how can I get it back?
I've recently run into an issue which I believe is related with the CookieContainer.
It is a basic HttpWebRequest to a url that was previously working fine (2 days ago) but is now having issues.
Below is the code I am using.
string url = "http://www.footywire.com/afl/footy/ft_match_statistics?mid=5827";
var cookieJar = new CookieContainer();
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.CookieContainer = cookieJar;
request.AllowAutoRedirect = false;
request.Method = "GET";
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:29.0) Gecko/20100101 Firefox/29.0";
request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
WebHeaderCollection myWebHeaderCollection = request.Headers;
myWebHeaderCollection.Add("Accept-Encoding", "gzip, deflate");
myWebHeaderCollection.Add("Accept-Language", "en-US,en;q=0.5");
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
int cookieCount = cookieJar.Count;
response.Close();
I am getting a (500) Internal Server Error during the GetResponse() call.
The URL works fine in the browser, but fails if I use an incognito/private window which makes me think it has to be cookie related.
I've monitored the request/responses using fiddler and can see the request is not handling cookies at all.
Here is a working request through the browser with cookies enabled.
GET http://www.footywire.com/afl/footy/ft_match_statistics?mid=5827 HTTP/1.1
Host: www.footywire.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:29.0) Gecko/20100101 Firefox/29.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Cookie: __utma=123692316.616048912.1368166198.1400452319.1400457786.25; __gads=ID=c95b71c78d8a7516:T=1368166203:S=ALNI_MbHrLnKztzLrfUEkZNoQfcimetGNw; _em_vt=3938ffa289939f5bd80fd60798185181c3303dddd6-9262208553794a3e; __utmz=123692316.1395356585.4.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utmc=123692316; JSESSIONID=62B598BD99DC2D228C4FC1E1549ED6FB; __utmb=123692316.1.10.1400457786; _em_v=5cc514d9ac92155cd445ac583ffd53794a3e7088d0-4657961653794a3e
Connection: keep-alive
This is the request the code is sending. The cookie is clearly missing.
GET http://www.footywire.com/afl/footy/ft_match_statistics?mid=5827 HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:29.0) Gecko/20100101 Firefox/29.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.5
Host: www.footywire.com
Connection: Keep-Alive
I'm at a bit of a loss as to why its not working now, but it was 2 days ago. I'm also not sure of what to do next.
Any help would be appreciated. Thanks.