I'm downloading in image from web to save it locally. It works great with any other image formats but it this method below fails with an argument exception when I try to read a WebP image.
private static Image GetImage(string url)
{
try
{
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
return Image.FromStream(response.GetResponseStream());
}
catch
{
return null;
}
}
How do you read .webp images in C#?
I found this other question that allows for converting between types but I do not want to do that WebP library for C#
Reason I'm not wanting to do it is because I think it might lose some quality. Besides, I want to know why is this not working.
The base class libraries won't help you to deal with WebP images. However, if you only want to save the received file to the disk, you don't have to even know that you are dealing with a WebP images. You can simply treat the received data as a binary blob and dump it to a file, for example using Stream.CopyTo and a FileStream.
The Content-Type HTTP header will give you the mime type of the file you're downloading, and the Content-Disposition header can provide you with a filename and extension (though you might have to do some parsing). You can access those using HttpWebResponse.ContentType and HttpWebResponse.Headers["Content-Disposition"].
#Trillian nailed it. Here is a code snippet for what I did based on his suggestion. Wanted to add code so not posting this as a comment.
To get just the image file extension, you can do this
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
string fileExt = response.ContentType.Replace("image/", string.Empty);
To get the file name with extension, you can do the following and the do parsing like I did above. It just has some more data in it.
response.Headers["Content-Disposition"];
Once you have you file name you want to save as, create a file stream and copy the response stream into it.
FileStream fs = new FileStream(targetPath + fileName, FileMode.Create);
response.GetResponseStream().CopyTo(fs);
Assuming you app has access to the destination, image should get saved. Make sure to add try catch and handle exceptions properly. Also note that FileMode.Create will overwrite if the file already exists!
Related
My problem is that I dont Know how i can download a File withknowing the file name or the file extension in the url, like this http://findicons.com/icon/download/235456/internet_download/128/png?id=235724
I hope you can help me
You could inspect the Content-Disposition response header using an HTTP request to get the filename. This would be a more general solution, so even if the filename is not contained in the URL, it would work:
var url = "http://findicons.com/icon/download/235456/internet_download/128/png?id=235724";
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
var fn = response.Headers["Content-Disposition"].Split(new string[] { "=" }, StringSplitOptions.None)[1];
string basePath = #"X:\Folder\SubFolder"; // Change accordingly...
var responseStream = response.GetResponseStream();
using (var fileStream = File.Create(Path.Combine(basePath, fn)))
{
responseStream.CopyTo(fileStream);
}
}
The above code uses certain methods and functions, you can find more information here:
HttpWebRequest - Usage example here
Saving a stream to a file - see this answer. Just note that when saving an HTTP response stream, you don't need to seek to the beginning, as it already is at the beginning and doing so will throw an exception. So, to be on the safe side, use it like I have in the code above.
Hope this answer helps you :)
It's possible to get the filename since the server is sending the Content-Disposition header. Here's a code example on how to get the filename using the HttpClient class:
var url = "http://findicons.com/icon/download/235456/internet_download/128/png?id=235724";
using (var client = new HttpClient())
using (var response = await client.GetAsync(url))
{
// make sure our request was successful
response.EnsureSuccessStatusCode();
// read the filename from the Content-Disposition header
var filename = response.Content.Headers.ContentDisposition.FileName;
// read the downloaded file data
var stream = await response.Content.ReadAsStreamAsync();
// Where you want the file to be saved
var destinationFile = Path.Combine("C:\\local\\directory", filename);
// write the steam content into a file
using (var fileStream = File.Create(destinationFile))
{
stream.CopyTo(fileStream);
}
}
I've had a hard time solving this issue myself and have come to a solution solving some issues in relation of obtaining the file name automatically.
Some headers does not include the Content Disposition as suggested in
https://stackoverflow.com/a/37228939/8805908, but the ones that do is still used.
I was wondering how Chrome, firefox etc. could obtain the name of the file though this information is not available through any header entry. I found that the links left without the information could be derived via its URL, from which i am using the code of:
http://codesnippets.fesslersoft.de/how-to-get-the-filename-of-url-in-c-and-vb-net/
My conclusion so far is; check the header for Content Disposition, if this does not contain any information check the URL for any file matches. So far I have not found a download link that I have not been able to retrieve the file name of.
I hope this may solve some issues.
--- Edit 12-06-2018
The solution using these methods are satisfying the following links:
5 test cases
I'm writing a web scraping program in C#. So far, I have been able to log in to website, save cookie, and return source code of another page. From this source code, I get a link that takes me to a pdf, but the page doesn't end with .pdf extension. In the browser, this page shows the pdf image and there are controls in the browser including a save button.
I believe the pdf page was created with ColdFusion as it has .cfm, CFID and CFTOKEN in the URL.
How do I save this pdf file programmatically?
Two answers have suggested I save the binary stream to pdf. How do I get the binary data in the first place? I have tried the following:
byte[] result;
byte[] buffer = new byte[4096];
WebRequest wr = WebRequest.Create(billURL);
using (WebResponse response = wr.GetResponse())
{
using (Stream responseStream = response.GetResponseStream())
{
using (MemoryStream memoryStream = new MemoryStream())
{
int count = 0;
do
{
count = responseStream.Read(buffer, 0, buffer.Length);
memoryStream.Write(buffer, 0, count);
} while (count != 0);
result = memoryStream.ToArray();
}
}
}
Do I then want to save result as a pdf, or am I doing something wrong there?
The common method in CF for streaming a PDF to the browser is using this method:
<cfheader name="Content-Disposition" value="attachment;filename=#PDFFileName#">
<cfcontent type="application/pdf" reset="true" variable="#toBinary(PDFinMemory)#">
Use a C# WebRequest to get the URL of the PDf. Then check the response header for a 'Content-Type of 'application/pdf'. If so, save the binary stream to a PDF file on disk.
Assuming the CFID and CFTOKEN are not really needed, (you can test the URL without CFID and CFTOKEN and see if you can still fetches the PDF successfully)
Use WebRequest to make a GET request to that URL (see: http://support.microsoft.com/kb/307023)
Save the binary stream as a PDF File.
I get a link that takes me to a pdf, but the page doesn't end with
.pdf extension ..
How do I get the binary data in the first place?
In addition to the other suggestions, one small clarification. The file extension does not really matter. What is important is the content. A .cfm script can return any content-type, not just text/html. So it can mimic a pdf, image, etcetera. As long as your link returns type application/pdf you should get back a binary stream (ie the pdf) you can save to a file. The original file name can be obtained from the WebResponse headers.
Okay so I want to download a file from a website, but the file is lacking an extension.
(it's an image file, I know this much, but the link does not provide the actual extension)
When I use webrequest, or webclient to download the file I get a "404 file not found" exception.
WebClient wc = new WebClient();
Stream strm = wc.DownloadFile("http://some_site.some_domain/some_image.","C:/some_directory/save_name.some_extention");
Notice the lack of extention at the end of the URL.
The site in question displays the image fine in a webbrowser, but when viewing just the image there is no extension and thus it's treated an unknown file (not showing an image).
So simply put: how do I download a file if there is no extention specified?
Thanks in advance!
So you're trying to determine what extension to give the file after downloading? If the URL doesn't have one you would have to inspect the actual data of the file.
You might be able to inspect the beginning of the file and see if it matches known valid file types. For instance, PNGs seem to have 'PNG' as bytes 2-4 (at least in the ones I've inspected). By looking at that data you should be able to determine the format with a fairly high accuracy.
This would be my best suggestion, if this doesn't work I don't know how to solve you problem...
List<string> fileExtensions = new List<string>(){"png","gif","bmp","jpg"}// other known image file extensions here...
WebClient wc = new WebClient();
foreach(var extension in fileExtensions)
{
try
{ wc.DownloadFile("http://some_site.some_domain/some_image."+extension,"C:/some_directory/save_name."+extension);
break;
}
catch {}
}
This would just be a work around, I guess... Not a real solution...
I'm trying to download remote file using C# (ASP.NET). The problem is when I browse to the file download URL - it downloads perfectly. When I try the WebClient.DownloadData(url) I get "no data to show" response.
If I browse using the built-in VS2010 browser I still get this "error" message.
The file link is: http://www.tase.co.il/TASE/Pages/Export.aspx?sn=he-IL_ds&enumTblType=AllSecurities&Columns=he-IL_Columns&Titles=he-IL_Titles&TblId=0&ExportType=3
(CSV file)
How can the file be downloaded? any ideas?
Many Thanks
This isn't a download problem on your side, that page is referencing something, probably from session...so you would need to login to the site, get a session, basically go through whatever steps you normally would in route to getting the file so it's generated correctly/available.
If I had to take a guess, I'm thinking they do the export to the file system when you ask for it, so it's a temp file somewhere...not something you can just grab, so unless you went through the step that created it (doesn't seem this Export.aspx page does it), it won't be there, and their error message for this is "No data to show".
There's also the possibility they're explicitly blocking you to prevent any kind of bots/leeching from happening...basically preventing exactly what you're trying to do.
Many sites will block downloads based on the Referer or User-Agent header.
Watch a "working" download with
Fiddler
Watch your code's
download with Fiddler
Compare the
two requests
Update your request
to match the "working" case.
I solved the problem by obtaining the remote file as a stream. From that point I was able to read/write the stream to the response.
//The absolute path to your file
string remoteFile = "http://my.cloudfront.net/videos/vehicle_english_v3.mp4";
//The name of the file you want the user to see when they download the file
int pos = remoteFile.LastIndexOf("/") + 1;
string fileName = remoteFile.Substring(pos, remoteFile.Length - pos);
//Obtain your file as a stream
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(remoteFile);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream fileStream = response.GetResponseStream();
//Write the stream to the response
if (fileStream != null && fileStream.CanRead) {
context.Response.AddHeader("Content-Disposition", "attachment; filename=\"" + fileName + "\"");
context.Response.ContentType = "application/octet-stream";
context.Response.ClearContent();
fileStream.CopyTo(context.Response.OutputStream);
}
I have to interface with a slightly archaic system that doesn't use webservices. In order to send data to this system, I need to post an XML document into a form on the other system's website. This XML document can get very large so I would like to compress it.
The other system sits on IIS and I use C# my end. I could of course implement something that compresses the data before posting it, but that requires the other system to change so it can decompress the data. I would like to avoid changing the other system as I don't own it.
I have heard vague things about enabling compression / http 1.1 in IIS and the browser but I have no idea how to translate that to my program. Basically, is there some property I can set in my program that will make my program automatically compress the data that it is sending to IIS and for IIS to seamlessly decompress it so the receiving app doesn't even know the difference?
Here is some sample code to show roughly what I am doing;
private static void demo()
{
Stream myRequestStream = null;
Stream myResponseStream = null;
HttpWebRequest myWebRequest = (HttpWebRequest)System.Net
.WebRequest.Create("http://example.com");
byte[] bytMessage = null;
bytMessage = Encoding.ASCII.GetBytes("data=xyz");
myWebRequest.ContentLength = bytMessage.Length;
myWebRequest.Method = "POST";
// Set the content type as form so that the data
// will be posted as form
myWebRequest.ContentType = "application/x-www-form-urlencoded";
//Get Stream object
myRequestStream = myWebRequest.GetRequestStream();
//Writes a sequence of bytes to the current stream
myRequestStream.Write(bytMessage, 0, bytMessage.Length);
//Close stream
myRequestStream.Close();
WebResponse myWebResponse = myWebRequest.GetResponse();
myResponseStream = myWebResponse.GetResponseStream();
}
"data=xyz" will actually be "data=[a several MB XML document]".
I am aware that this question may ultimately fall under the non-programming banner if this is achievable through non-programmatic means so apologies in advance.
I see no way to compress the data on one side and receiving them uncompressed on the other side without actively uncompressing the data..
No idea if this will work since all of the examples I could find were for download, but you could try using gzip to compress the data, then set the Content-Encoding header on the outgoing message to gzip. I believe that the Length should be the length of the zipped message, although you may want to play with making it the length of the unencoded message if that doesn't work.
Good luck.
EDIT I think the issue is whether the ISAPI filter that supports compression is ever/always/configurably invoked on upload. I couldn't find an answer to that so I suspect that the answer is never, but you won't know until you try (or find the answer that eluded me).