Develop Reference

HttpWebRequest not downloading file correctly

I can't use WebClient, before anyone suggests that because it makes my legit application seem like a virus to McAfee. So please don't suggest that.
I have a binary.txt file stored on my server. It is approximately 1,240kb. However, HttpWebRequest downloads random amounts from 1,300kb to 1,700kb.
HttpWebRequest httpRequest = (HttpWebRequest)
WebRequest.Create("http://deviantsmc.com/binary.txt");
httpRequest.Method = WebRequestMethods.Http.Get;
HttpWebResponse httpResponse = (HttpWebResponse)httpRequest.GetResponse();
Stream httpResponseStream = httpResponse.GetResponseStream();
byte[] buffer = new byte[1240];
int bytesRead = 0;
StringBuilder sb = new StringBuilder();
//FileStream fileStream = File.Create(#"tcontent.txt");
while ((bytesRead = httpResponseStream.Read(buffer, 0, 1240)) != 0)
{
sb.Append(Encoding.ASCII.GetString(buffer));
//fileStream.Write(buffer, 0, bytesRead);
}
File.WriteAllText(#"tcontent1.txt", sb.ToString());
(The contents of the binary.txt file on the server are in ASCII, therefore I get the Encoding string to ASCII as well).
This is how I encoded that text file (on that server)
My file is basically this:
byte[] bytes = File.ReadAllBytes("binary.txt");
String encBytes = Encoding.ASCII.GetString(bytes);
File.WriteAllText(file("binary.txt"), encBytes);
I contacted the AV company about the WebDownloader being seen as some malicious import in C#, but they didn't get back to me, so I'm forced to use HttpWebRequest.

If your only goal is to fetch that binary and write it to disk you can simply copy the stream to a file with the CopyTo method that exists on a Stream object.
Your file looks like a broken zip file btw, given the first characters that are PK, and is used in the zip specificaton.
From wikipedia:
Viewed as an ASCII string this reads "PK", the initials of the inventor Phil Katz. Thus, when a .ZIP file is viewed in a text editor the first two bytes of the file are usually "PK".
I used 7-zip to open your file as the Windows default didn't accept it as a valid file. It contains a manifest.mf file but the content itself seems missing. the file itself has a size of 1.269.519 bytes.
HttpWebRequest httpRequest = (HttpWebRequest)
WebRequest.Create("http://deviantsmc.com/binary.txt");
httpRequest.Method = WebRequestMethods.Http.Get;
HttpWebResponse httpResponse = (HttpWebResponse)httpRequest.GetResponse();
Stream httpResponseStream = httpResponse.GetResponseStream();
// create and open a FileStream, using calls dispose when done
using(var fs= File.Create(#"c:\temp\bin.7z"))
{
// Copy all bytes from the responsestream to the filestream
httpResponseStream.CopyTo(fs);
}

I think you should write the entire binary data to the local file, instead of encoding it segmentally, you can try this:
HttpWebRequest httpRequest = (HttpWebRequest)
WebRequest.Create("http://deviantsmc.com/binary.txt");
httpRequest.Method = WebRequestMethods.Http.Get;
HttpWebResponse httpResponse = (HttpWebResponse)httpRequest.GetResponse();
Stream httpResponseStream = httpResponse.GetResponseStream();
using (BinaryReader responseReader = new BinaryReader(httpResponseStream.GetResponseStream()))
{
byte[] bytes = responseReader.ReadBytes((int)response.ContentLength);
using (BinaryWriter sw = new BinaryWriter(File.OpenWrite("tcontent1.txt")))
{
sw.Write(bytes);
sw.Flush();
sw.Close();
}
}
Hope this can help

URL Formats are not supported

I am reading File with File.OpenRead method, I am giving this path
http://localhost:10001/MyFiles/folder/abc.png
I have tried this as well but no luck
http://localhost:10001//MyFiles//abc.png
but its giving
URL Formats are not supported
When I give physical path of my Drive like this,It works fine
d:\MyFolder\MyProject\MyFiles\folder\abc.png
How can I give file path to an Http path?
this is my code
public FileStream GetFile(string filename)
{
FileStream file = File.OpenRead(filename);
return file;
}

Have a look at WebClient (MSDN docs), it has many utility methods for downloading data from the web.
If you want the resource as a Stream, try:
using(WebClient webClient = new WebClient())
{
using(Stream stream = webClient.OpenRead(uriString))
{
using( StreamReader sr = new StreamReader(stream) )
{
Console.WriteLine(sr.ReadToEnd());
}
}
}

You could either use a WebClient as suggested in other answers or fetch the relative path like this:
var url = "http://localhost:10001/MyFiles/folder/abc.png";
var uri = new Uri(url);
var path = Path.GetFileName(uri.AbsolutePath);
var file = GetFile(path);
// ...
In general you should get rid of the absolute URLs.

The best way to download the HTML is by using the WebClient class. You do this like:
private string GetWebsiteHtml(string url)
{
WebRequest request = WebRequest.Create(url);
WebResponse response = request.GetResponse();
Stream stream = response.GetResponseStream();
StreamReader reader = new StreamReader(stream);
string result = reader.ReadToEnd();
stream.Dispose();
reader.Dispose();
return result;
}
Then, If you want to further process the HTML to ex. extract images or links, you will want to use technique known as HTML scrapping.
It's currently best achieved by using the HTML Agility Pack.
Also, documentation on WebClient class: MSDN

Here I found this snippet. Might do exactly what you need:
using(WebClient client = new WebClient()) {
string s = client.DownloadFile(new Uri("http://.../abc.png"), filename);
}
It uses the WebClient class.

To convert a file:// URL to a UNC file name, you should use the Uri.LocalPath property, as documented.
In other words, you can do this:
public FileStream GetFile(string url)
{
var filename = new Uri(url).LocalPath;
FileStream file = File.OpenRead(filename);
return file;
}

Copy a PDF Stream to File

I'm calling a routine in PHP (TCPDF) from C# via WebRequest using StreamReader. The PDF file is returned as a stream and stored in a string (obv). I know the data being returned to the string is actually a PDF file, as I've tested it in PHP. I'm having a hard time writing the string to a file and actually getting a valid PDF in C#. I know it has something to do with the way I'm trying to encode the file, but the several things I've tried have resulted in 'Not today, Padre' (i.e. they didn't work)
Here's the class I'm using to perform the request (thanks to user 'Paramiliar' for the example I'm using/borrowed/stole):
public class httpPostData
{
WebRequest request;
WebResponse response;
public string senddata(string url, string postdata)
{
// create the request to the url passed in the paramaters
request = (WebRequest)WebRequest.Create(url);
// set the method to POST
request.Method = "POST";
// set the content type and the content length
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = postdata.Length;
// convert the post data into a byte array
byte[] byteData = Encoding.UTF8.GetBytes(postdata);
// get the request stream and write the data to it
Stream dataStream = request.GetRequestStream();
dataStream.Write(byteData, 0, byteData.Length);
dataStream.Close();
// get the response
response = request.GetResponse();
dataStream = response.GetResponseStream();
StreamReader reader = new StreamReader(dataStream);
// read the response
string serverresponse = reader.ReadToEnd();
//Console.WriteLine(serverresponse);
reader.Close();
dataStream.Close();
response.Close();
return serverresponse;
}
} // end class httpPostData
...and my call to it
httpPostData myPost = new httpPostData();
// postData defined (not shown)
string response = myPost.senddata("http://www.example.com/pdf.php", postData);
In case it isn't clear, I'm stuck writing string response to a valid .pdf file. I've tried this (Thanks to user Adrian):
static public void SaveStreamToFile(string fileFullPath, Stream stream)
{
if (stream.Length == 0) return;
// Create a FileStream object to write a stream to a file
using (FileStream fileStream = System.IO.File.Create(fileFullPath, (int)stream.Length))
{
// Fill the bytes[] array with the stream data
byte[] bytesInStream = new byte[stream.Length];
stream.Read(bytesInStream, 0, (int)bytesInStream.Length);
// Use FileStream object to write to the specified file
fileStream.Write(bytesInStream, 0, bytesInStream.Length);
}
}
..and the call to it:
string location = "C:\\myLocation\\";
SaveStreamToFile(location, response); // <<-- this throws an error b/c 'response' is a string, not a stream. New to C# and having some basic issues with things like this
I think I'm close...a nudge in the right direction would be greatly appreciated.

You can use WebClient. Use the method DownloadFile, or the async ones.
Have fun!
Fernando.-
Sorry, I haven't read your comments till now.
I guess you have already done this...
But this may help you (just replace urls and paths) (from: http://msdn.microsoft.com/en-us/library/ez801hhe.aspx )
string remoteUri = "http://www.contoso.com/library/homepage/images/";
string fileName = "ms-banner.gif", myStringWebResource = null;
// Create a new WebClient instance.
WebClient myWebClient = new WebClient();
// Concatenate the domain with the Web resource filename.
myStringWebResource = remoteUri + fileName;
Console.WriteLine("Downloading File \"{0}\" from \"{1}\" .......\n\n", fileName, myStringWebResource);
// Download the Web resource and save it into the current filesystem folder.
myWebClient.DownloadFile(myStringWebResource,fileName);
Console.WriteLine("Successfully Downloaded File \"{0}\" from \"{1}\"", fileName, myStringWebResource);
Console.WriteLine("\nDownloaded file saved in the following file system folder:\n\t" + Application.StartupPath);

IMGUR photo upload POST Request

what im trying to do is to make photo upload to imgur site using their API here http://api.imgur.com/endpoints/image#image-upload. As i read in documentation, image data has to be in "image" parameter. So what im doing is setting POST request data as "image=base64codedfile&title=blabla&type=base64" and it does upload it but file is corrupted. If my request will contain only "base64codedfile" without image, title, type name parameters its working like a charm. Am i doing something wrong?
If i set it as:
string postData = "image="+Convert.ToBase64String(image)+"&type=base64&title=test;
image is corrupted
If it's only raw data:
string postData = Convert.ToBase64String(image);
It's working
Whole code is something like this:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://api.imgur.com/3/upload.xml");
request.Headers.Add("Authorization", "Client-ID >>myclientid<<");
request.Method = "POST";
string filePath = "d:\\test.jpg";
FileStream file = new FileStream(filePath, FileMode.Open);
byte[] image = new byte[file.Length];
file.Read(image, 0, (int)file.Length);
ASCIIEncoding enc = new ASCIIEncoding();
string postData = Convert.ToBase64String(image);
byte[] bytes = enc.GetBytes(postData);
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = bytes.Length;
Stream writer = request.GetRequestStream();
writer.Write(bytes, 0, bytes.Length);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();

I am also trying to do the same and also facing the problem...
I got one solution...
After uploading your image get the id of the image using response.getresponsestream()....
then using this id update that image..and dont forget to use imgur api to update image..to know more about updating imgur image visit this link.

My solution :
using (var reader = new StreamReader(response.GetResponseStream()))
{
var objText = reader.ReadToEnd();
}
in object text you will found the json response

C# Escape Plus Sign (+) in POST using HttpWebRequest

I'm having issues to send POST data that includes characteres like "+" in the password field,
string postData = String.Format("username={0}&password={1}", "anyname", "+13Gt2");
I'm using HttpWebRequest and a webbrowser to see the results, and when I try to log in from my C# WinForms using HttpWebRequest to POST data to the website, it tells me that password is incorrect. (in the source code[richTexbox1] and the webBrowser1). Trying it with another account of mine, that does not contain '+' character, it lets me log in correctly (using array of bytes and writing it to the stream)
byte[] byteArray = Encoding.ASCII.GetBytes(postData); //get the data
request.Method = "POST";
request.Accept = "text/html";
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = byteArray.Length;
Stream newStream = request.GetRequestStream(); //open connection
newStream.Write(byteArray, 0, byteArray.Length); // Send the data.
newStream.Close(); //this works well if user does not includes symbols
From this Question I found that HttpUtility.UrlEncode() is the solution to escape illegal characters, but I can't find out how to use it correctly, my question is, after url-encoding my POST data with urlEncode() how do I send the data to my request correctly?
This is how I've been trying for HOURS to make it work, but no luck,
First method
string urlEncoded = HttpUtility.UrlEncode(postData, ASCIIEncoding.ASCII);
//request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = urlEncoded.Length;
StreamWriter wr = new StreamWriter(request.GetRequestStream(),ASCIIEncoding.ASCII);
wr.Write(urlEncoded); //server returns for wrong password.
wr.Close();
Second method
byte[] urlEncodedArray = HttpUtility.UrlEncodeToBytes(postData,ASCIIEncoding.ASCII);
Stream newStream = request.GetRequestStream(); //open connection
newStream.Write(urlEncodedArray, 0, urlEncodedArray.Length); // Send the data.
newStream.Close(); //The server tells me the same thing..
I think I'm doing wrong on how the url-encoded must be sent to the request, I really ask for some help please, I searched through google and couldn't find more info about how to send encoded url to an HttpWebRequest.. I appreciate your time and attention, hope you can help me. Thank you.

I found my answer using Uri.EscapeDataString it exactly solved my problem, but somehow I couldn't do it with HttpUtility.UrlEncode. Stackoverflowing around, I found this question that is about urlEncode method, in msdn it documentation tells that:
Encodes a URL string.
But I know now that is wrong if used to encode POST data (#Marvin, #Polity, thanks for the correction). After discarding it, I tried the following:
string postData = String.Format("username={0}&password={1}", "anyname", Uri.EscapeDataString("+13Gt2"));
The POST data is converted into:
// **Output
string username = anyname;
string password = %2B13Gt2;
Uri.EscapeDataString in msdn says the following:
Converts a string to its escaped representation.
I think this what I was looking for, when I tried the above, I could POST correctly whenever there's data including the '+' characters in the formdata, but somehow there's much to learn about them.
This link is really helpful.
http://blogs.msdn.com/b/yangxind/archive/2006/11/09/don-t-use-net-system-uri-unescapedatastring-in-url-decoding.aspx
Thanks a lot for the answers and your time, I appreciate it very much. Regards mates.

I'd recommend you a WebClient. Will shorten your code and take care of encoding and stuff:
using (var client = new WebClient())
{
var values = new NameValueCollection
{
{ "username", "anyname" },
{ "password", "+13Gt2" },
};
var url = "http://foo.com";
var result = client.UploadValues(url, values);
}

the postdata you are sending should NOT be URL encoded! it's formdata, not the URL
string url = #"http://localhost/WebApp/AddService.asmx/Add";
string postData = "x=6&y=8";
WebRequest req = WebRequest.Create(url);
HttpWebRequest httpReq = (HttpWebRequest)req;
httpReq.Method = WebRequestMethods.Http.Post;
httpReq.ContentType = "application/x-www-form-urlencoded";
Stream s = httpReq.GetRequestStream();
StreamWriter sw = new StreamWriter(s,Encoding.ASCII);
sw.Write(postData);
sw.Close();
HttpWebResponse httpResp =
(HttpWebResponse)httpReq.GetResponse();
s = httpResp.GetResponseStream();
StreamReader sr = new StreamReader(s, Encoding.ASCII);
Console.WriteLine(sr.ReadToEnd());
This uses the System.Text.Encoding.ASCII to encode the postdata.
Hope this helps,