I can't use WebClient, before anyone suggests that because it makes my legit application seem like a virus to McAfee. So please don't suggest that.
I have a binary.txt file stored on my server. It is approximately 1,240kb. However, HttpWebRequest downloads random amounts from 1,300kb to 1,700kb.
HttpWebRequest httpRequest = (HttpWebRequest)
WebRequest.Create("http://deviantsmc.com/binary.txt");
httpRequest.Method = WebRequestMethods.Http.Get;
HttpWebResponse httpResponse = (HttpWebResponse)httpRequest.GetResponse();
Stream httpResponseStream = httpResponse.GetResponseStream();
byte[] buffer = new byte[1240];
int bytesRead = 0;
StringBuilder sb = new StringBuilder();
//FileStream fileStream = File.Create(#"tcontent.txt");
while ((bytesRead = httpResponseStream.Read(buffer, 0, 1240)) != 0)
{
sb.Append(Encoding.ASCII.GetString(buffer));
//fileStream.Write(buffer, 0, bytesRead);
}
File.WriteAllText(#"tcontent1.txt", sb.ToString());
(The contents of the binary.txt file on the server are in ASCII, therefore I get the Encoding string to ASCII as well).
This is how I encoded that text file (on that server)
My file is basically this:
byte[] bytes = File.ReadAllBytes("binary.txt");
String encBytes = Encoding.ASCII.GetString(bytes);
File.WriteAllText(file("binary.txt"), encBytes);
I contacted the AV company about the WebDownloader being seen as some malicious import in C#, but they didn't get back to me, so I'm forced to use HttpWebRequest.
If your only goal is to fetch that binary and write it to disk you can simply copy the stream to a file with the CopyTo method that exists on a Stream object.
Your file looks like a broken zip file btw, given the first characters that are PK, and is used in the zip specificaton.
From wikipedia:
Viewed as an ASCII string this reads "PK", the initials of the inventor Phil Katz. Thus, when a .ZIP file is viewed in a text editor the first two bytes of the file are usually "PK".
I used 7-zip to open your file as the Windows default didn't accept it as a valid file. It contains a manifest.mf file but the content itself seems missing. the file itself has a size of 1.269.519 bytes.
HttpWebRequest httpRequest = (HttpWebRequest)
WebRequest.Create("http://deviantsmc.com/binary.txt");
httpRequest.Method = WebRequestMethods.Http.Get;
HttpWebResponse httpResponse = (HttpWebResponse)httpRequest.GetResponse();
Stream httpResponseStream = httpResponse.GetResponseStream();
// create and open a FileStream, using calls dispose when done
using(var fs= File.Create(#"c:\temp\bin.7z"))
{
// Copy all bytes from the responsestream to the filestream
httpResponseStream.CopyTo(fs);
}
I think you should write the entire binary data to the local file, instead of encoding it segmentally, you can try this:
HttpWebRequest httpRequest = (HttpWebRequest)
WebRequest.Create("http://deviantsmc.com/binary.txt");
httpRequest.Method = WebRequestMethods.Http.Get;
HttpWebResponse httpResponse = (HttpWebResponse)httpRequest.GetResponse();
Stream httpResponseStream = httpResponse.GetResponseStream();
using (BinaryReader responseReader = new BinaryReader(httpResponseStream.GetResponseStream()))
{
byte[] bytes = responseReader.ReadBytes((int)response.ContentLength);
using (BinaryWriter sw = new BinaryWriter(File.OpenWrite("tcontent1.txt")))
{
sw.Write(bytes);
sw.Flush();
sw.Close();
}
}
Hope this can help
Related
I'm trying to download an Excel file from a Tableau view, using a snippet based on this article: Downloading Files with the WebRequest and WebResponse Classes.
WebResponse response = null;
Stream webStream = null;
Stream localStream = null;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(endPoint);
request.Method = "GET";
request.Headers.Add("X-Tableau-Auth", token);
response = request.GetResponse();
webStream = response.GetResponseStream();
localStream = File.Create("testing.xlsx");
byte[] buffer = new Byte[1024];
int bytesRead;
int bytesProcessed = 0;
do
{
bytesRead = webStream.Read(buffer, 0, buffer.Length);
localStream.Write(buffer, 0, bytesRead);
bytesProcessed += bytesRead;
} while (bytesRead > 0);
response.Close();
webStream.Close();
localStream.Close();
But when I try to open the Excel file it says "Nope, maybe it is corrupt?". The response is a complete Excel file "ready to be saved", which means that it's encoded using UTF-8. Indeed, if I use Postman to do the call, and then save the response, it's saved and opens without any problem.
Before finding the page I said above, I thought the problem was because the response is UTF-8 and the class String is UTF-16. So, I made some testing getting the data from the file generated by Postman and then writing to a new file. The result of the testings: indeed, if the data passes through a String, it's not well saved. Then I tried with that code, and got the same result: fail. I'm pretty sure this code is not using any UTF-16 encoding variable, but maybe I'm wrong.
Anyway, do anyone knows what is my problem with that code, or point me to the right way to accomplish my task? That is, to download a Tableau report to Excel, using the Tableau API.
Thanks in advance.
Unless you're stuck on a particularly old version of .NET, that referenced link is from 2004. The following code will work in .NET 5 / C# 9, and in earlier versions with just some minor tweaking of the using statements. It's showing for Tableau Online, but should work fine for recent versions of Server, if I had to guess. This is sample-grade code, so I would recommend following best practices for the HttpClient if you intend to make this call frequently.
//using System.IO;
//using System.Net.Http;
var token = "your-session-token";
var uri = "https://your-pod.online.tableau.com/api/.../sites/.../views/.../crosstab/excel";
var yourFile = "D:/file/test.xlsx";
using var client = new HttpClient();
var request = new HttpRequestMessage(HttpMethod.Get, uri);
request.Headers.Add("X-Tableau-Auth", token);
using var response = await client.SendAsync(request);
using FileStream outputFileStream = new FileStream(yourFile, FileMode.Create);
await response.Content.CopyToAsync(outputFileStream);
UPDATE: If you're constrained to WebRequest and non-async methods, you can try the following:
var token = "your-session-token";
var uri = "https://your-pod.online.tableau.com/api/.../sites/.../views/.../crosstab/excel";
var yourFile = "D:/file/test.xlsx";
WebRequest request = WebRequest.Create(uri);
request.Method = "GET";
request.Headers.Add("X-Tableau-Auth", token);
var response = request.GetResponse();
if(((HttpWebResponse)response).StatusCode == HttpStatusCode.OK)
{
using (Stream dataStream = response.GetResponseStream())
using (FileStream fileStream = new FileStream(yourFile, FileMode.CreateNew))
{
dataStream.CopyTo(fileStream);
}
}
response.Close();
I'm calling a routine in PHP (TCPDF) from C# via WebRequest using StreamReader. The PDF file is returned as a stream and stored in a string (obv). I know the data being returned to the string is actually a PDF file, as I've tested it in PHP. I'm having a hard time writing the string to a file and actually getting a valid PDF in C#. I know it has something to do with the way I'm trying to encode the file, but the several things I've tried have resulted in 'Not today, Padre' (i.e. they didn't work)
Here's the class I'm using to perform the request (thanks to user 'Paramiliar' for the example I'm using/borrowed/stole):
public class httpPostData
{
WebRequest request;
WebResponse response;
public string senddata(string url, string postdata)
{
// create the request to the url passed in the paramaters
request = (WebRequest)WebRequest.Create(url);
// set the method to POST
request.Method = "POST";
// set the content type and the content length
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = postdata.Length;
// convert the post data into a byte array
byte[] byteData = Encoding.UTF8.GetBytes(postdata);
// get the request stream and write the data to it
Stream dataStream = request.GetRequestStream();
dataStream.Write(byteData, 0, byteData.Length);
dataStream.Close();
// get the response
response = request.GetResponse();
dataStream = response.GetResponseStream();
StreamReader reader = new StreamReader(dataStream);
// read the response
string serverresponse = reader.ReadToEnd();
//Console.WriteLine(serverresponse);
reader.Close();
dataStream.Close();
response.Close();
return serverresponse;
}
} // end class httpPostData
...and my call to it
httpPostData myPost = new httpPostData();
// postData defined (not shown)
string response = myPost.senddata("http://www.example.com/pdf.php", postData);
In case it isn't clear, I'm stuck writing string response to a valid .pdf file. I've tried this (Thanks to user Adrian):
static public void SaveStreamToFile(string fileFullPath, Stream stream)
{
if (stream.Length == 0) return;
// Create a FileStream object to write a stream to a file
using (FileStream fileStream = System.IO.File.Create(fileFullPath, (int)stream.Length))
{
// Fill the bytes[] array with the stream data
byte[] bytesInStream = new byte[stream.Length];
stream.Read(bytesInStream, 0, (int)bytesInStream.Length);
// Use FileStream object to write to the specified file
fileStream.Write(bytesInStream, 0, bytesInStream.Length);
}
}
..and the call to it:
string location = "C:\\myLocation\\";
SaveStreamToFile(location, response); // <<-- this throws an error b/c 'response' is a string, not a stream. New to C# and having some basic issues with things like this
I think I'm close...a nudge in the right direction would be greatly appreciated.
You can use WebClient. Use the method DownloadFile, or the async ones.
Have fun!
Fernando.-
Sorry, I haven't read your comments till now.
I guess you have already done this...
But this may help you (just replace urls and paths) (from: http://msdn.microsoft.com/en-us/library/ez801hhe.aspx )
string remoteUri = "http://www.contoso.com/library/homepage/images/";
string fileName = "ms-banner.gif", myStringWebResource = null;
// Create a new WebClient instance.
WebClient myWebClient = new WebClient();
// Concatenate the domain with the Web resource filename.
myStringWebResource = remoteUri + fileName;
Console.WriteLine("Downloading File \"{0}\" from \"{1}\" .......\n\n", fileName, myStringWebResource);
// Download the Web resource and save it into the current filesystem folder.
myWebClient.DownloadFile(myStringWebResource,fileName);
Console.WriteLine("Successfully Downloaded File \"{0}\" from \"{1}\"", fileName, myStringWebResource);
Console.WriteLine("\nDownloaded file saved in the following file system folder:\n\t" + Application.StartupPath);
I want to save a GIF driven from an internet address.
When I request the file from the browser, I receive these characters:
GIF87a½€ÿÿÿ,½þ„©Ëí£œ´Ú‹³Þ¼û†âH–æ‰Ádk
ÇòòµuÏú>æùôãEX¯–Š¨²µ‚‰"<}O)Ò¨¬:kÜ'Õª]þ¢`ìc
Ú”ìvöhe;Ï]¸;oßÂyþ|Þ‡µe§'hÄÔdhW÷§H#&gö¨ð…G™Gy(‰ x÷j†øyÙˆI8uJZZiª
™ªÚº¹i {ç‰Zj™ˆwšµ7
©;é›9Æy¤VÄ$ú;œË›ÙÛI¬,œ=}|,l¼Û˧¸ìæ%]h~n»8»ÌíIv—Fe6¿‹¸®0YÉ
J7k8”•pÕá C¬4Èráâ;zü2¤È‘$Kš
Can I retrieve GIF format with these characters?
A GIF is a binary file. You shouldn't be reading it as a string. Read the GIF into a byte[], and then save those bytes to a file.
Just do this:
string url = "http://address.com/getFile?s=1234";
string filePath = #"c:\MyFolder\file.gif";
using (WebClient client = new WebClient())
{
client.DownloadFile(url, filePath);
}
and do not forget to add this at the beginning:
using System.Net;
thanks for your answers.
I've tried a way and I found the Issue
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
request.Accept = "text/html, text/plain";
request.AllowAutoRedirect = true;
WebResponse response = request.GetResponse();
Stream stream = response.GetResponseStream();
StreamReader reader = new StreamReader(stream, Encoding.UTF8);
Bitmap bmp = new Bitmap(Image.FromStream(stream, true, false));
bmp.Save("image.gif");
Judging from what I know about file formats and the web, the characters are representations ASCII or another encoding of binary. You may still be able to save it using the BinaryWriter.Write() Method.
Despite trying lots of things (see below), I can't get rid of the "Bytes to be written to the stream exceed the Content-Length bytes size specified." error that's thrown in
writer.Close();
This is the code that tries to post data from an ASP.NET to a php site. The script works fine as long as there are no special characters in the code - note the German Umlaut in 'Wörld'.
Uri uri = new Uri("http://mydomain/test.php");
string data = #"data=Hello Wörld";
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(uri);
request.Method = WebRequestMethods.Http.Post;
request.ContentLength = data.Length;
request.ContentType = "application/x-www-form-urlencoded";
StreamWriter writer = new StreamWriter(request.GetRequestStream());
writer.Write(data);
writer.Close();
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
StreamReader reader = new StreamReader(response.GetResponseStream());
string tmp = reader.ReadToEnd();
response.Close();
Response.Write(tmp);
I have tried different variations using UTF-8 encodings, like:
request.ContentLength = System.Text.Encoding.UTF8.GetByteCount(data);
and/or
StreamWriter writer = new StreamWriter(request.GetRequestStream(), Encoding.UTF8);
I have also tried to convert the data to UTF-8 before sending it (somewhat ugly):
data = System.Text.Encoding.UTF8.GetString(System.Text.Encoding.Convert(System.Text.Encoding.UTF8, System.Text.Encoding.UTF8, System.Text.Encoding.UTF8.GetBytes(data)));
Yet the error remains. My feeling is that I just don't get the UTF-8 handling right. Any help is greatly appreciated, also any hint where I can find a perfectly working script that posts to php from ASP.NET (server side).
use
byte[] bdata = Encoding.UTF8.GetBytes(data);
and
request.ContentLength = bdata.Length;
and
Stream writer = request.GetRequestStream();
writer.Write(bdata, 0, bdata.Length);
writer.Close();
I want to create a file with a .aspx extension(or any other extension) completely in memory. Can this be done?
Right now I have a memory stream that has all the stuff I want to write to this file but I don't actually want to create a physical file on the server since then I probably have to enable write permissions for my server. What I want to do is create the file in memory and upload via ftpWebRequest.
Edit.
I must be doing something wrong because I get weird stuff in my file so weird I can't even paste it into my posts.
Basically it is a bunch of squares between everything. Like it almost seems like it fills it in for the spaces. Like If I look closely I will see the tag still but it will have a square between each letter.
Here is a portion of my code. Maybe I am using the wrong encoding?
using (MemoryStream memory = new MemoryStream())
{
UnicodeEncoding uniEncoding = new UnicodeEncoding();
// readByline is the first bunch of data I want for my new file.
memory.Write(uniEncoding.GetBytes(readByLine), 0, readByLine.Length);
// second bunch of data I want for my new file.
memory.Write(uniEncoding.GetBytes(html), 0, html.Length);
// the follow code just figure out the end of the file that I am
// trying to extract some information out of.
string readToEnd = reader.ReadToEnd();
int endIndex = readToEnd.IndexOf(END_FLAG);
endIndex += END_FLAG.Length;
string restOfFile = readToEnd.Substring(endIndex);
// once found I write it the memory stream.
memory.Write(uniEncoding.GetBytes(restOfFile),0,restOfFile.Length);
// now I want to upload my file. I have the same file name already
// existing on the server? Do I have to tell it override it?
FtpWebRequest request2 = (FtpWebRequest)WebRequest.Create(path);
request2.Method = WebRequestMethods.Ftp.UploadFile;
request2.Credentials = new NetworkCredential(ftpUsername, ftpPassword);
// now I am trying your code.
byte[] fileContents = memory.ToArray();
using (Stream writer = request2.GetRequestStream())
{
writer.Write(fileContents, 0, fileContents.Length);
}
FtpWebResponse test = (FtpWebResponse)request2.GetResponse();
return Content("test");
}
You can convert the MemoryStream to a byte[] and then use WebClient.UploadData to upload the file with FTP to some server without writing it on the client to disk first:
webClient.UploadData(
"ftp://remoteserver/remotepath/file.aspx"
memoryStream.ToArray());
FtpWebRequest works as well, of course, but needs a few more lines of code:
FtpWebRequest ftpRequest;
FtpWebResponse ftpResponse;
ftpRequest = (FtpWebRequest)FtpWebRequest.Create(new Uri("ftp://..."));
ftpRequest.Method = WebRequestMethods.Ftp.UploadFile;
ftpRequest.Proxy = null;
ftpRequest.UseBinary = true;
ftpRequest.Credentials = new NetworkCredential("UserName", "Password");
using (Stream stream = ftpRequest.GetRequestStream())
using (StreamWriter writer = new StreamWriter(stream, Encoding.UTF8))
{
writer.WriteLine("<html><head><title>Hello World</title></head>...");
}
ftpResponse = (FtpWebResponse)ftpRequest.GetResponse();
Well, such a file is really just plain text, with certain formatting - HTML 4.0, XHTML or such.
So yes, you can create that in memory - inside e.g. a StringBuilder, and then save it back out to disk using a StreamWriter or some other means.
If you can't or don't want to write it out to disk, you can of course also put it into a MemoryStream, and interfaces that can read from any arbitrary stream can read from the memory stream, too.
Check out the MSDN docs on FtpWebRequest and on GetRequestStream() method. It has a sample on how to upload from a stream to FTP directly:
// Get the object used to communicate with the server.
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(serverUri);
request.Method = WebRequestMethods.Ftp.UploadFile;
// this could be your MemoryStream, of course, that you're reading from
StreamReader sourceStream = new StreamReader(fileName);
byte[] fileContents = Encoding.UTF8.GetBytes(sourceStream.ReadToEnd());
sourceStream.Close();
request.ContentLength = fileContents.Length;
request.Credentials = new NetworkCredential(ftpUsername, ftpPassword);
Stream requestStream = request.GetRequestStream();
requestStream.Write(fileContents, 0, fileContents.Length);
requestStream.Close();
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
// check the response, do whatever you need to do with it
response.Close();