My issue is this:
I am generating and uploading a SQL file using ASP.NET, but after the file is saved to the FTP server, characters like ü are changed to &uul;, ø to ø and so on... How can I prevent this from happening? I don't want the file to be formatted with ASCII code, but with UTF-8.
The code that generates and uploads the file looks like this:
//request = the object to be made an request out of.
Stream requestStream = request.GetReguestStream();
var encoding = new UTF8Encoding();
//fileContent is the string to be saved in the file
byte[] buffer = encoding.GetBytes(fileContent);
requestStream.Write(buffer, 0, buffer.Length);
requestStream.Close();
As you can see I've tried to use the System.Text.UTF8Encoding, but it doesn't work.
Remember, with streams you can almost always wrap the streams as necessary. If you want to write UTF-8 encoded content you wrap the request stream in a StreamWriter with the correct encoding:
using (Stream requestStream = request.GetRequestStream())
using (StreamWriter writer = new StreamWriter(requestStream, Encoding.UTF8)) {
writer.Write(fileContent);
}
Since you say you're uploading to a web service be sure to set your content encoding as well. Since you haven't posted where the request object comes from, I'll assume it's a normal HttpWebRequest.
With a HttpWebRequest you would tell the server what the content encoding is by using the ContentType property.
request.ContentType = "text/plain;charset=utf-8";
As others have mentioned, though, the FTP transfer itself may be breaking it too. If you can, make sure it's transferred in binary mode, not ASCII mode.
Put it in debug and look at what gets put in 'buffer' after encoding.GetBytes() is called. This will verify if it's the rx side causing it.
Related
I have a program that gets a response from a url in binary format and I do not know how to convert this to a text file.
byte[] postBytes = System.Text.Encoding.UTF8.GetBytes(postString);
request.ContentLength = postBytes.Length;
Stream stream = request.GetRequestStream();
stream.Write(postBytes, 0, postBytes.Length);
stream.Close();
response = (HttpWebResponse)request.GetResponse();
Stream ReceiveStream = response.GetResponseStream();
string filename = "C:\\responseGot.txt";
byte[] buffer = new byte[1024];
FileStream outFile = new FileStream(filename, FileMode.Create);
int bytesRead;
while ((bytesRead = ReceiveStream.Read(buffer, 0, buffer.Length)) != 0)
outFile.Write(buffer, 0, bytesRead);
When I open responseGot.txt it is a binary file how do I get text file.
In what format is the response you get? There is no such thing as a text file. There are only binary files. HTTP is also 100% binary.
Text is the interpretation of bytes, and it only exists as part of running application. You can never, ever write text to a file. You can only convert the text to bytes (using various ways) and write the bytes.
Therefore, ask yourself why the bytes you received cannot be interpreted by notepad.exe as text. Maybe the response is not directly text but a ZIP file or something.
You can guess the format with a hex editor
You can ask the website owner
You don't show in your code sample saving the file anywhere.
But to convert the response to string you can use:
using (HttpWebResponse response = req.GetResponse() as HttpWebResponse)
{
StreamReader reader = new StreamReader(response.GetResponseStream());
string ResponseTXT = reader.ReadToEnd();
}
Then you can save it with usual techniques
http://msdn.microsoft.com/en-us/library/6ka1wd3w%28v=vs.110%29.aspx
Did you mean that?
Every data represented in digital computing these days is based on 2 bits ie. binary (electrical/magnetic signals: on/off or north/south).
Every file written to disk is also a binary file ie. a sequence of (8 bit) bytes.
ASCII/ANSI defines character map for each byte sequence and only about 95 of the 256 bytes are referred to as printable (text) characters.
Your downloaded file seems to have more than just the printable characters (usually referred to as a plain text file).
To view the file as it is (in your current encoding settings):
type <file.ext>
To view in a different code page:
chcp <codepage>
type <file.ext>
To view a (plain)text representation of your file, you'd encode it first (ie. translate it to a text file) eg. hex coded string via some hex editor.
The first few characters of the hex sequence should give a magic number, indicating the type of file being read. You'd then open the file with the associated program (that is capable of opening those types of files).
If it is a text file you were expecting and instead got a file which has more than just printable (plain text) characters, then it's more likely there has been some sort of compression/encryption applied to it. Once again, the magic number should hint how the file should be treated eg. decompressed before attempting to read the data/file. (Encrypted files should come with a decryption hint/key, unless exchanged/agreed earlier)
Use the ReadFully method in this topic Creating a byte array from a stream
Get the string representation to an actual string:
string text = System.Text.Encoding.Default.GetString(byteArray);
And finally create the text file and write the content:
using(StreamWriter sw = new StreamWriter("C:\\responseGot.txt"))
{
sw.WriteLine(text);
}
I am trying to save a string containing Json syntax to a .txt file on an FTP server.
I tried using this example http://msdn.microsoft.com/en-us/library/ms229715.aspx which worked great.
But this example takes an existing .txt local file and uploads it to the ftp server.
I would like to directly create / update a txt file on the ftp server from a string variable. Without having first to create the txt file locally in my pc.
Your example link is exactly what you need, but you need to get your information from a MemoryStream instead of an existing file.
You can turn a string directly into a Stream with this:
MemoryStream memStr = MemoryStream(UTF8Encoding.Default.GetBytes("asdf"));
However, you can shortcut this more by directly turning your string into a byte array, avoiding the need to make a Stream altogether:
System.Text.ASCIIEncoding encoding = new System.Text.ASCIIEncoding();
Byte[] bytes = encoding.GetBytes(yourString);
//and now plug that into your example
Stream requestStream = request.GetRequestStream();
requestStream.Write(bytes, 0, bytes.Length);
requestStream.Close();
I am working on developing an HTTP Server/Client and I can currently send small files over it such as .txt files and other easy to read files that do not require much memory. However when I want to send a larger file say a .exe or large .pdf I get memory errors. This are occurring from the fact that before I try to send or receive a file I have to specify the size of my byte[] buffer. Is there a way to get the size of the buffer while reading it from stream?
I want to do something like this:
//Create the stream.
private Stream dataStream = response.GetResponseStream();
//read bytes from stream into buffer.
byte[] byteArray = new byte[Convert.ToInt32(dataStream.Length)];
dataStream.read(byteArray,0,byteArray.Length);
However when calling "dataStream.Length" it throws the error:
ExceptionError: This stream does not support seek operations.
Can someone offer some advice as to how I can get the length of my byte[] from the stream?
Thanks,
You can use CopyTo method of the stream.
MemoryStream m = new MemoryStream();
dataStream.CopyTo(m);
byte[] byteArray = m.ToArray();
You can also write directly to file
var fs = File.Create("....");
dataStream.CopyTo(fs);
The network layer has no way of knowing how long the response stream is.
However, the server is supposed to tell you how long it is; look in the Content-Length response header.
If that header is missing or incorrect, you're out of luck; you'll need to keep reading until you run out of data.
EDIT: Filezilla caused the problem, when i download files back from server it added new lines. I'm sorry for confusion.
This method upload files to ftp server and it's work fine, but in text files uploaded to server blank lines appear after every line("cr lf" appear), for example:
File:
First line
Second line
Third line
Uploaded file:
First line
Second line
Third line
Origin and uploaded files accordingly have different sizes, non-text files are the same.
Code:
private void sendFile(string In, string Out)
{
FtpWebRequest request = (FtpWebRequest) WebRequest.Create("ftp://domain//" + Out);
request.Method = WebRequestMethods.Ftp.UploadFile;
request.Credentials = new NetworkCredential("username", "password");
FileStream sourceStream = new FileStream(In, FileMode.Open, FileAccess.Read, FileShare.Read);
byte[] fileContents = new byte[sourceStream.Length];
sourceStream.Read(fileContents, 0, (int) sourceStream.Length);
sorceStream.Close();
request.ContentLength = fileContents.Length;
Stream requestStream = request.GetRequestStream();
requestStream.Write(fileContents, 0, fileContents.Length);
requestStream.Close();
}
How can i fix this?
EDIT: As the answer below doesn't seem to have helped (but I'm leaving it there for posterity as it shows better code) here are the next diagnostics steps I'd check:
How are you viewing the files? If at all possible, get onto the server directly rather than fetching the files again via a web browser or whatever.
What's the type of FTP server you're connecting to? Maybe there's a known issue.
Have you tried looking at what's actually being sent via Wireshark?
Have you tried sending the same files via a normal FTP client?
You should set FtpWebRequest.UseBinary to true in order to preserve the exact file contents. Otherwise the two systems will try to figure out line endings themselves, changing line terminators as they see fit. I very rarely think that's a good idea. (EDIT: UseBinary is actually true by default, but this sounds like the kind of problem introduced by using text mode... it certainly does no harm to make this explicit.)
Additionally:
You should be disposing of your FileStream via a using statement
You should be disposing of the request stream via a using statement
You should be taking note of the result of Stream.Read - it needn't always read the whole of the requested data in one go
You can either use File.ReadAllBytes to simply read the complete file data in one go, or use Stream.CopyTo (if you're using .NET 4) to copy the FileStream to the request stream (which won't set the content length, of course; I don't know whether this is a problem)
You're never calling GetResponse; it's unclear exactly what happens if you never fetch the response of an FtpWebRequest
Your parameter names don't match .NET naming conventions, and aren't very descriptive
So I would probably use:
private void SendFile(string inputFile, string outputPath)
{
FtpWebRequest request = (FtpWebRequest) WebRequest.Create
("ftp://domain//" + outputPath);
request.Method = WebRequestMethods.Ftp.UploadFile;
request.UseBinary = true;
request.Credentials = new NetworkCredential("username", "password");
byte[] fileContents = File.ReadAllBytes(inputFile);
request.ContentLength = fileContents.Length;
using (Stream requestStream = request.GetRequestStream())
{
requestStream.Write(fileContents, 0, fileContents.Length);
}
// This *may* be necessary in order to validate that everything has happened
using (WebResponse response = request.GetResponse())
{
}
}
Its strange. I face the same problem and I was unable to fix it until I did not provide an extension in file. For Example if my file name was
abcfile
then I make it abcfile.dat and after that it shows me the uploaded file as actual file. I again upload file with abcfile.txt but this time again empty line problem appear in my uploaded file.
I suggest that you must provide extension to your file any except .txt.
The system that you're sending to uses different line endings to what your system uses. I can assume, because you get an extra line, that you're on Windows, and it uses CRLF endings. The system you're sending to recognises CR and LF as separate endings, so you get the extra lines.
For text, truncate the LF or the CR, see what happens. I have no clue about the differing file sizes.
In the top menu of FileZilla, set:
Transfer menu > Transfer type > binary
In the top menu of FileZilla, set:
Transfer menu > Transfer type > binary
It's working for me.
I need to post raw xml to a site and read the response. With the following code I keep getting an "Unknown File Format" error and I'm not sure why.
XmlDocument sampleRequest = new XmlDocument();
sampleRequest.Load(#"C:\SampleRequest.xml");
byte[] bytes = Encoding.UTF8.GetBytes(sampleRequest.ToString());
string uri = "https://www.sample-gateway.com/gw.aspx";
req = WebRequest.Create(uri);
req.Method = "POST";
req.ContentLength = bytes.Length;
req.ContentType = "text/xml";
using (var requestStream = req.GetRequestStream())
{
requestStream.Write(bytes, 0, bytes.Length);
}
// Send the data to the webserver
rsp = req.GetResponse();
XmlDocument responseXML = new XmlDocument();
using (var responseStream = rsp.GetResponseStream())
{
responseXML.Load(responseStream);
}
I am fairly certain my issue is what/how I am writing to the requestStream so..
How can I modify that code so that I may write an xml located on the hard drive to the request stream?
ok instead of doing sampleRequest.ToString(), you should use sampleRequest.OuterXml, and that would do the magic, you were sending "System.Xml.XmlDocument" instead of the Xml
XmlDocument sampleRequest = new XmlDocument();
sampleRequest.Load(#"C:\SampleRequest.xml");
//byte[] bytes = Encoding.UTF8.GetBytes(sampleRequest.ToString());
byte[] bytes = Encoding.UTF8.GetBytes(sampleRequest.OuterXml);
Two things:
First, whenever you're trying to diagnose a problem with an HTML response, you should always examine what the response stream actually contains. If you had in this case, you would have seen that it contains System.Xml.XmlDocument, which would have told you what was wrong pretty much immediately.
Second, in an application with any kind of transaction volume, you're not going to want to load a static XML file into an XmlDocument before putting it in the response stream; your program's spending time and memory building something that you don't need. (It's even worse than that in your case; your approach not only parses the XML into a DOM object, it then makes an in-memory copy of its OuterXml property when you encode it as UTF-8. Also, do you really need to be doing that?) Instead, you should create a FileStream object and use one of the techniques in this answer to copy it to the response stream.