I am trying to simulate a real web browser request and turns out when I use this code:
WebClient client = new WebClient();
client.DownloadFile(address, localFilename);
I get only the GET to the address(of course) and the behavior in a browser is many GET requests to images, blogger, etc...
Is there a shortcut to get/simulate the same behavior or the only alternative is to parse the file/string and make all these requests by myself manually?
Yes, a browser processes the specific type of file (typically HTML) and it parses it. Depending on what the file contains (links to other files like images, etc.) the browser will then start up many other connections to get all those other files to display within the browser.
That doesn't come for free--you have to do that yourself. DownloadFile just downloads a file, that may or may not be an HTML file and thus it doesn't handled all possible file types and process all linked files.
Related
I have created a Simple Aspx Page to Log the Output of a Process running on a Remote Machine.
My url is:.com/Log.ashx?Data=SomeString
Should I use WebRequest or a WebClient?
What is more efficient and less resource intensive?
I would need to do this about 20 times in a minute. The ashx file does not show any output.
Neither one is more efficient than the other. WebRequest just offers you more functonality than WebClient. if WebClient exposes enough functionality for what you need than go with that. If it doesn't then use WebRequest.
I want to run my personal web sites via an httphandler (I have a web server and static ip at home.)
Eventually, I will incorporate a data access layer and domain router into the handler, but for now, I am just trying to use it to return static web content.
I have the handler mapped to all verbs and paths with no access restrictions in IIS 7 on Windows 7.
I have added a little file logging at the beginning of process request. As it is the first thing in the handler, I use the logging to tell me when the handler is hit.
At the moment, the handler just returns a single web page that I have already written.
The handler itself is mostly just this:
using (FileStream fs = new FileStream(Request.PhysicalApplicationPath + "index.htm",
FileMode.Open))
{
fs.CopyTo(Response.OutputStream);
}
I understand that this won't work for anything but the one file.
So my issue is this: the HTML file has links to some images in it. I would expect that the browser would come back to the server to get those images as new requests. I would expect those requests to fail (because they'd be mapped to index.htm). But I would expect to see the logging hit at least twice (and potentially hit recursively). However, I only see a single request. The web page comes up and the images are 'X's.
When I refresh the browser, I see another request come through, but only for the root page again. The page is basic HTML, I do not have an asp.net application (nor do I want one, I like HTML/CSS/JS).
What do I have to do to get more than just the first request sent from the browser? I assume I'm just totally off the mark because I wrote an HTTP Module first, but strangely got the same exact behavior. I'm thinking I need to specify some response headers, but don't see that in any example.
I am using WebClient.UploadFileAsync to upload local files to a web server and I would also like to pass some parameters with the post as well. I would like to send a few fields that will tell the PHP code on the server specific locations on where to save the uploaded files.
I tried to put the fields directly into the url, for example:
WebClient client = new WebClient();
Uri uri = new Uri("http://example.com/upload.php?field1=test");
client.UploadFileAsync(uri, "POST", "c:\test.jpg");
The PHP code returns false for isset($_REQUEST['field1']).
Thank you for any suggestions.
NOTE: this question was also asked in very similar format for vb.net a while back, but it did not get any answers,
WebClient's UploadFile is designed to send only a file (as byte[]) as part of request. From my understanding, UploadFile method closes the request stream after writing the binaries.
In your scenario, you actually request is has two parts 1. file as byte[] 2. the file name as string.
To do this, you have to use HttpWebRequest or any other high level class capable of creating request.
Refer to the post http://www.codeproject.com/KB/cs/uploadfileex.aspx?display=Print
which does a similar job
This article goes into detail about what is needed to accomplish posting of fields while uploading files using WebClient.
Unfortunately, most file upload scenarios are HTML form based and may
contain form fields in addition to the file data. This is where
WebClient falls flat. After review of the source code for WebClient,
it is obvious that there is no possibility of reusing it to perform a
file upload including additional form fields.
So, the only option is to create a custom implementation that conforms
to rfc1867, rfc2388 and the W3C multipart/form-data specification that
will enable file upload with additional form fields and exposes
control of cookies and headers.
I would look into using the QueryString property of the WebClient to set the value of field1 (as well as any other QueryString parameters to the request).
NameValueCollection query = new NameValueCollection();
query.Add("field1", "test");
client.QueryString = query;
Reference: http://msdn.microsoft.com/en-us/library/system.net.webclient.querystring(v=VS.100).aspx
So there are many examples of getting the stream from a file locally, but suppose i want to stream an MP3 from another site.
I have a page that lists urls of MP3s, for example, one might be...
www.domain.com/getResource.aspx?ResourceId=123
This page can obviously fairly simply load Resource 123, and send to the browser.
Suppose resource 123 is not local, it is a URL to another site which serves up the mp3
so my question is what is the best way for this Resource Page to get the data from the remote source and send back to the browser. In effect, there should be no differential as far as the client is concerned where the file has come from. It always goes through this aspx (or ashx) page
Your site would need to make its own HTTP request to the remote server. Take the content of the response, and write it to the response stream of the request that came to your site.
Is it possible to get file properties of a web file. ie: PDF file and get the date and time the PDF file was modified. Thanks.
You're probably looking for the HttpWebResponse.LastModified property.
No, you can't. It's not a file, it's a resource.
When you request it from the server you get a response back. In this case the server uses a file as source for the resource, but it could just as well be something that is just created on the fly. You only get the information that the server chooses to put in the HTML header.
You might however be able to get some more information if the server returns directory information. Then you could request the folder and get a listing back, from which you could parse some information. Note however that it's not certain that the server actually uses the files when you request something from the directory, it could still return something completely different. Also, the directory listing is also a resource, so there is no guarantee that it's relevant either.