Develop Reference

Simulate Upload in Chrome Postman WebApi

I am trying to simulate a FileUpload with Chrome Postman to my ASP.NET WebApi method . To do this I perform the following steps in Postman.
1.Open postman and choose POST method from address bar.
2.Enter valid URL.
3.Go to Body section below address bar.
4.From Body section choose Binary option.
5.Hit the "Chose Files" button to select a file to upload.
6.Choose a file
7.Ensure the "Post" option is selected
8.Hit "Send"
I then test for MultiPartContent, however when I step through my method the Request Object doesn't appear to have the info that I need . The exception below will get hit.
public List<Premium.Model.BureauModUpdate> Post() {
HttpRequestMessage request = this.Request;
if (!request.Content.IsMimeMultipartContent()) {
throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
}
}
It doesn't appear as if any info concerning the file upload is in the request. A review of the request content yields this:
{Method: POST, RequestUri: 'http://localhost/WCAPI/Lookup/BureauModUpdate/CreateNyModUpdates', Version: 1.1, Content: System.Web.Http.WebHost.HttpControllerHandler+LazyStreamContent, Headers:
{
Cache-Control: no-cache
Connection: keep-alive
Accept: */*
Accept-Encoding: gzip
Accept-Encoding: deflate
Accept-Encoding: br
Accept-Language: en-US
Accept-Language: en; q=0.8
Host: localhost
User-Agent: Mozilla/5.0
User-Agent: (Windows NT 6.1; WOW64)
User-Agent: AppleWebKit/537.36
User-Agent: (KHTML, like Gecko)
User-Agent: Chrome/55.0.2883.87
User-Agent: Safari/537.36
Postman-Token: 23f64855-3b60-3f87-3aa1-ba7d0a841273
Origin: chrome-extension://fhbjgbiflinjbdggehcddcbncdddomop
Content-Length: 200284
Content-Type: application/x-www-form-urlencoded
}}
Is there something else I need to be doing ? If anybody has any suggestions or can point me in the right direction on this it would be greatly appreciated.

I believe your content type is incorrect. It should be multipart/form-data when uploading files.
application/x-www-form-urlencoded is used only when posting a form with no binary data.

Issue getting text from amazon.com

I am getting shipping text from amazon.com as in the picture below:
But the issue is i cannot get this text even though its shown on the site,not sure whats wrong in it example is this link.
SAMPLE LINK 1
Not sure whats wrong in it, if you see the page source it wont have in it and still its showing on the page which is confusing me, for other this works fine and there are many more pages like this example.Any ideas?

The above post is right, its loaded after the page, so the link you truely need is:
GET http://www.amazon.com/gp/twister/ajax/prefetch/184-8017667-8054405?json=1&sid=184-8017667-8054405&rid=0EY8R4VSQH90CGG2B8WB&parentAsin=B00AZD9YO0&qid=&sr=&asinList=B00AZD9YCC,B00AZD9YO0&productGroupID=health_and_beauty_display_on_website&merchantID=&smid=null&PowerBar=0&pfWrapFeatures=0&rps=0&_=1358853427426 HTTP/1.1
Host: www.amazon.com
Connection: keep-alive
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.97 Safari/537.11
Accept: application/json, text/javascript, */*
Referer: http://www.amazon.com/dp/B00AZD9YCC/?tag=stackoverfl08-20
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-GB,en-US;q=0.8,en;q=0.6
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: x-wl-uid=12Rt0TTSfvFFeeEo/b9iO7gAvvtsw+sHdWjU1SmtKe3jzgy3Bnf5YM/gC+Zm2ZMG5bJiGZ2xkab0=; session-token=QMx3+akd/XUQe2V8GVUQ3fioq2STsxGQWCTw5AUILjVp5itgNLqGYps7hrRzzq2v0LoD8e0VjoC0EH+c8aZE0ybkSB4VNhfnGPLWyC/5s0yDboyQr9bsxEcuJzpSH5lS0xj9bvd6Rz6u195/PCt7G8W/9bPscFcGCOOczEh46FoQ0wM8/I+IZL/tsTtlCoS7WtomqeKvqgUxrqYqFrB0+cNbH0L2vg3Z3zLX45qER4i9LByiil/UeOeNtqtKnN0A; ubid-main=181-8869220-9587219; session-id-time=2082787201l; session-id=184-8017667-8054405; UserPref=ZQYGYZ6aqj9V1WCTNQfIBVHwrxWUiRBbtjYgpE662zHEbrcpva4uwDzapqUVUnEAv7+ZRaU6VjPKllub9eBfBEVyz+RPxbaEmIS3D9nkpoNciDxQuq8F4WFNYy8WKy4AzYftKT9qYMugJLXI2w7L9Jt6780u982A54mduTieZ7XvtM991yXd8WzHgADBGM6vnfjtkwaFeRcJ6jkFysNFob24qESWn1sN2mZT3YVgjfWBqb1X8Z3FZbNmZt4qp8ZEtjl0wXNhZRHlP6OZ3+HbabTWQKGoG1kRzJJ9YQAv4QMKP2zyQnI2/MJp/VohCMF+1kfimtGK8+azjZQClkezqjBdRTYQ6PFuOPB6sVJS1EIY0GGRZ4ugjr/ynWLQrxvO3LUOYDByWeF2GtH+WlscSpdZUaRrxGipc/Ze1nwhz+KyVBv4vnf4kJx93WjpbJbMu6PdOC1fiXg=; csm-hit=3949.77|
You will have to build the Get request using the ASIN number I believe and parse the response to get your data.

It is present, but it's probably being loaded asynchronously after the page itself loads, hence it won't necessarily be visible via whatever method you're viewing the source with.
<span class="plusShippingText">
+ $4.02 shipping
</span>

C# - Connection: keep-alive Header is Not Being Sent During HttpWebRequest

I'm trying to send to send the following header with my HttpWebRequest:
Connection: keep-alive
However, the header is never sent. Fiddler2 is showing that whenever I request the page in Google Chrome, the header is sent. However, my application refuses to send this header for some reason.
I have set the KeepAlive property to true (it's true by default anyway), yet the header still does not get sent.
I am trying to send this header with multiple HttpWebRequests, but they all basically look like this:
HttpWebRequest logIn6 = (HttpWebRequest)WebRequest.Create(new Uri(responseFromLogIn5));
logIn6.CookieContainer = cookies;
logIn6.KeepAlive = true;
logIn6.Referer = "https://login.yahoo.com/config/login?.src=spt&.intl=us&.lang=en-US&.done=http://football.fantasysports.yahoo.com/";
logIn6.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.220 Safari/535.1";
logIn6.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
logIn6.Headers.Add("Accept-Encoding:gzip,deflate,sdch");
logIn6.Headers.Add("Accept-Language:en-US,en;q=0.8");
logIn6.Headers.Add("Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.3");
logIn6.AllowAutoRedirect = false;
HttpWebResponse logIn6Response = (HttpWebResponse)logIn6.GetResponse();
string responseFromLogIn6 = logIn6Response.GetResponseHeader("Location");
cookies.Add(logIn6Response.Cookies);
logIn6Response.Close();
Does anyone know what I have to do to make sure this header is sent?
Fiddler2 Raw From Chrome:
GET xxx HTTP/1.1
Host: accounts.google.com
Connection: keep-alive
Referer: https://login.yahoo.com/config/login?.src=spt&.intl=us&.lang=en-US&.done=http://football.fantasysports.yahoo.com/
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.220 Safari/535.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: xxx
HTTP/1.1 302 Moved Temporarily
Set-Cookie: xxx
Set-Cookie: xxx
Location: xxx
Content-Type: text/html; charset=UTF-8
P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info."
Date: Sat, 17 Sep 2011 22:27:09 GMT
Expires: Sat, 17 Sep 2011 22:27:09 GMT
Cache-Control: private, max-age=0
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Content-Length: 2176
Server: GSE
Fiddler2 Raw From My Application:
GET xxx HTTP/1.1
Referer: https://login.yahoo.com/config/login?.src=spt&.intl=us&.lang=en-US&.done=http://football.fantasysports.yahoo.com/
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.220 Safari/535.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Host: accounts.google.com
HTTP/1.1 302 Moved Temporarily
Location: xxx
Content-Type: text/html; charset=UTF-8
Date: Sun, 18 Sep 2011 00:05:40 GMT
Expires: Sun, 18 Sep 2011 00:05:40 GMT
Cache-Control: private, max-age=0
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Content-Length: 573
Server: GSE
I'm trying to get the second Fiddler2 raw information to look like the first Fiddler2 raw information.

I've had the same issue: The Connection: Keep-Alive header is not sent except the first request, and the server I accessed won't give me the correct response if it is missing. So, here are my workarounds to this issue:
First is set the ProtocolVersion property of HttpWebRequest instance to HttpVersion.Version10. Except the http command will become GET xxx HTTP/1.0, it works and uses only the public API.
The second way uses the reflection to modify the internal property ServicePoint.HttpBehaviour of HttpWebRequest instance, like this:
var req = (HttpWebRequest)WebRequest.Create(someUrl);
var sp = req.ServicePoint;
var prop = sp.GetType().GetProperty("HttpBehaviour",
BindingFlags.Instance | BindingFlags.NonPublic);
prop.SetValue(sp, (byte)0, null);
req.GetResponse().Close();
Hope this helps.

I struggled with this problem for half a day! And dear old Fiddler (my guardian angel) was inadvertantly part of the problem:
Whenever I tested my HTTP POSTs using Fiddler monitoring ON - the problem DIDN'T appear
Whenever I tested my HTTP POSTs with Fiddler monitoring OFF - the problem DID appear
My POSTS were sent with protocol 1.1 and the Keep-Alive was ignored/redundant/why after the initial connection. i.e. I could see it in the header of the first POST (via Fiddler!), but not in subsequent POSTs despite using the same code. Hey ho ...
But the remote server would only respond if Keep-Alive was sent. Now I can't prove this, but I suspect that Fiddler monitoring the connection caused the remote server to think or believe that the connection was still active (despite no Keep-Alives sent after my first POST) and responded correctly. As I said, the second I turned Fiddler off, the absence of Keep-Alives caused the remote server to timeout on me..
I implemented the 1.0 solution described above and my POSTS now work, with or without Fiddler on or off. Hope this helps somebody else stuck somewhere ...

You doing it right. The code should result in following header added:
Connection: Keep-Alive
Post the code that you use for sending request and Raw output from Fiddler if you don't see this header. You may also ignore this because HTTP 1.1 connection is keep-alive by default.
Update: it looks like .NET only sets Keep-Alive explicitly for the first (!) request. Further requests to the same host/url will not have this header presumably because underlying tcp connection is already being reused.

After downloading HttpWebRequest source code, noticed that every property checks some known headers for HeaderCollection. To get rid of that doing some reflection stuff on that collection make it work
var webRequest = (HttpWebRequest) WebRequest.Create(url);
webRequest.Headers.GetType().InvokeMember("ChangeInternal",
BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.InvokeMethod,
Type.DefaultBinder, webRequest.Headers, new object[] {name, value}
);

I know the answer for this as I had the same problem and managed to solve it by inheriting the webclient and overriding it's Get Web Request method.
See the code below:
public class CookieAwareWebClient : WebClient
{
public CookieContainer CookieContainer { get; set; }
public CookieAwareWebClient()
: this(new CookieContainer())
{ }
public CookieAwareWebClient(CookieContainer c)
{
this.CookieContainer = c;
}
protected override WebRequest GetWebRequest(Uri address)
{
WebRequest request = base.GetWebRequest(address);
var castRequest = request as HttpWebRequest;
if (castRequest != null)
{
castRequest.KeepAlive = true; //<-- this what you want! The rest you don't need.
castRequest.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8";
castRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.104 Safari/537.36";
castRequest.Referer = "https://www.jobserve.com/gb/en/Candidate/Login.aspx?url=48BB4C724EA6A1F2CADF4243A0D73C13225717A29AE8DAD6913D";
castRequest.Headers.Add("Accept-Encoding", "gzip,deflate,sdch");
castRequest.Headers.Add("Accept-Language", "en-GB,en-US;q=0.8,en;q=0.6");
castRequest.CookieContainer = this.CookieContainer;
}
return request;
}
}
As you can see I am not only enabling keep-alive but I am utilizing cookies and other headers also!
I hope that helps!
Kiran

How to set Accept and Accept-Language header fields?

I can set Request.Content-Type = ... , Request.Content-Length = ...
How to set Accept and Accept-Language?
I want to upload a file (RFC 1867) and need to create a request like this:
POST /test-upload.php.xml HTTP/1.1
Host: example.com
User-Agent: Mozilla/5.0 (Windows NT 5.2; WOW64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: tr-tr,tr;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-9,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Content-Type: multipart/form-data; boundary=---------------------------21724139663430
Content-Length: 56048

Take a look at Accept property:
HttpWebRequest myHttpWebRequest=(HttpWebRequest)WebRequest.Create(myUri);
myHttpWebRequest.Accept="image/*";
HttpWebResponse myHttpWebResponse=
(HttpWebResponse)myHttpWebRequest.GetResponse();
This MSDN article shows how to add custom headers to your request:
//Get the headers associated with the request.
WebHeaderCollection myWebHeaderCollection = myHttpWebRequest.Headers;
//Add the Accept-Language header (for Danish) in the request.
myWebHeaderCollection.Add("Accept-Language:da");
//Include English in the Accept-Langauge header.
myWebHeaderCollection.Add("Accept-Language","en;q=0.8");

When you want to set the Accept type and content type, just cast the webrequest to HttpwebRequest
var webreq= (HttpWebRequest)WebRequest.Create(requestUri);
webreq.Method = "POST";
webreq.Accept = "application/json";
webreq.ContentType = "application/json";

You need to be sure that you type cast the request to (HttpWebRequest), where the accept header property is available.
In the old WebRequest class, the Accept header is not accessible.

I have to confirm after several annoying attempts to use the headers that the
myWebHeaderCollection.Add("foo","bar"); solution works perfectly.
if you want to set the Language.
myWebHeaderCollection.Add("AcceptCharset", "ISO-8859-1,utf-8;q=0.7,*;q=0.7");
myWebHeaderCollection.Add("TransferEncoding", "gzip,deflate");
Does not set the values however. Which may seem like a logical conclusion given the first one works.

If you are using HttpRequestMessage, set the header using Headers.Add method. In your case :
request.Headers.Add("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");

In a case where I have the pleasure of maintaining 15 year old vb.NET 3.5 code, this workaround was successful for me:
webReq = WebRequest.Create(apiHost)
CType(webReq, HttpWebRequest).Accept = "application/json"

Get content of response with StatusCode 401

I'm trying to establish a connection to a server that sends 401 Authentication Error for all my requests along with the normal html response. e.g.
However, I also want to read the HTML response that is sent alongwith so that I can parse that. An example header exchange captured using LiveHTTPHeaders:
Clearly, content-length is non-zero. Firefox shows it to be javascript.
https://172.31.1.251:1003/fgtauth?73e285357b2dc5cc
Request:
GET /fgtauth?73e285357b2dc5cc HTTP/1.1
Host: 172.31.1.251:1003
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1b4) Gecko/20090423 Firefox/3.5b4 (.NET CLR 3.5.30729)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Response:
HTTP/1.x 401 Unauthorized
WWW-Authenticate: Fortigate
Content-Length: 1091
Connection: Keep-Alive
Cache-Control: no-cache
Content-Type: text/html
At this point, a form opens in firefox that asks me to enter my username and password.
https://172.31.1.251:1003/
Request:
POST / HTTP/1.1
Host: 172.31.1.251:1003
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1b4) Gecko/20090423 Firefox/3.5b4 (.NET CLR 3.5.30729)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: https://172.31.1.251:1003/
Content-Type: application/x-www-form-urlencoded
Content-Length: 93
magic=73e285357b2dc5cc&username=uuxx&password=xxuu&4Tredir=http%3A%2F%2Fwww.google.com%2F
Response:
HTTP/1.x 401 Unauthorized
WWW-Authenticate: Fortigate
Content-Length: 924
Connection: Keep-Alive
Cache-Control: no-cache
Content-Type: text/html
At this point I am redirected to another URL. However, the problem that I have is how to get the content of lenght 924 that is sent along with the 401 Unauthorized, because that content will help me in doing what I want to do further. But the very line:
WebResponse loginResponse = loginRequest.GetResponse();
throws an exception.
I will be grateful for any suggestions to help me get to the actual content.
Thanks.

You need to read the WebException's Response property, like this:
WebResponse loginResponse;
try {
loginResponse = loginRequest.GetResponse();
} catch(WebException ex) {
if (ex.Status == WebExceptionStatus.ProtocolError) {
loginResponse = ex.Response;
} else
throw;
}
//Do something with the response, and remember to dispose it.