Develop Reference

Stream was not readable

I have code below that read ftp response stream and write data to two different files (test1.html & test2.html). The 2nd StreamReader throw a stream was not readable error. The response stream should be readable because it's not out of scope yet and the dispose shouldn't be called. Can someone explain why?
static void Main(string[] args)
{
// Make sure it is ftp
if (Properties.Settings.Default.FtpEndpoint.Split(':')[0] != Uri.UriSchemeFtp) return;
// Intitalize object to used to communicuate to the ftp server
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(Properties.Settings.Default.FtpEndpoint + "/test.html");
// Credentials
request.Credentials = new NetworkCredential(Properties.Settings.Default.FtpUser, Properties.Settings.Default.FtpPassword);
// Set command method to download
request.Method = WebRequestMethods.Ftp.DownloadFile;
// Get response
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
using (Stream output = File.OpenWrite(#"C:\Sandbox\vs_projects\FTP\FTP_Download\test1.html"))
using (Stream responseStream = response.GetResponseStream())
{
responseStream.CopyTo(output);
Console.WriteLine("Successfully wrote stream to test.html");
try
{
using (StreamReader reader = new StreamReader(responseStream))
{
string file = reader.ReadToEnd();
File.WriteAllText(#"C:\Sandbox\vs_projects\FTP\FTP_Download\test2.html", file);
Console.WriteLine("Successfully wrote stream to test2.html");
}
}
catch (Exception ex)
{
Console.WriteLine($"Exception: {ex}");
}
}
}

You can't read from the stream twice. After this call:
responseStream.CopyTo(output);
... you've already read all the data in the stream. There's nothing left to read, and you can't "rewind" the stream (e.g. seeking to the beginning) because it's a network stream. Admittedly I'd expect it to just be empty rather than throwing an error, but the details don't really matter much as it's not a useful thing to try to do.
If you want to make two copies of the same data, the best option is to copy it to disk as you're already doing, then read the file that you've just written.
(Alternatively, you could just read it into memory by copying to a MemoryStream, then you can rewind that stream and read from it repeatedly. But if you're already going to save it to disk, you might as well do that first.)

convert pdf to stream from url

I have a PDF which is hosted in say http://test.com/mypdf.pdf.
How can I convert the PDF to Stream and then using this Stream convert it back to PDF.
I tried the following but got an exception(see image):
private static Stream ConvertToStream(string fileUrl)
{
HttpWebResponse aResponse = null;
try
{
HttpWebRequest aRequest = (HttpWebRequest)WebRequest.Create(fileUrl);
aResponse = (HttpWebResponse)aRequest.GetResponse();
}
catch (Exception ex)
{
}
return aResponse.GetResponseStream();
}

This will work:
private static Stream ConvertToStream(string fileUrl)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(fileUrl);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
try {
MemoryStream mem = new MemoryStream();
Stream stream = response.GetResponseStream();
stream.CopyTo(mem,4096);
return mem;
} finally {
response.Close();
}
}
However you are entirely responsible for the lifetime of the returned memory stream.
A better approach is:
private static void ConvertToStream(string fileUrl, Stream stream)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(fileUrl);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
try {
Stream response_stream = response.GetResponseStream();
response_stream.CopyTo(stream,4096);
} finally {
response.Close();
}
}
You can then do something like:
using (MemoryStream mem = new MemoryStream()) {
ConvertToStream('http://www.example.com/',mem);
mem.Seek(0,SeekOrigin.Begin);
... Do something else ...
}
You may also be able to return the response stream directly but you'd have to check on the lifetime of that, releasing the response may release the stream, hence the mem copy.

You may want to take a look at WebClient.DownloadFile.
You give it a URL and local file name and it saves the file straight to disk. Might save you a step or two.
You could also try WebClient.DownloadData which saves the file to an in-memory byte[].
EDIT
You did not specify the protocol of the web-service you are posting the file to. The simplest form (RESTful) would be just to POST the file to data to another URL. Here is how you would do that.
using (WebClient wc = new WebClient())
{
// copy data to byte[]
byte[] data = wc.DownloadData("http://somesite.com/your.pdf");
// POST data to another URL
wc.Headers.Add("Content-Type","application/pdf");
wc.UploadData("http://anothersite.com/your.pdf", data);
}
If you are using SOAP, you would have to convert the file to a Base64 string, but hopefully you are using a generated client which takes care of that for you. If you could elaborate on the type of web-service you are sending the file to, I could probably provide some more information..

Convert .db to binary

I'm trying to convert a .db file to binary so I can stream it across a web server. I'm pretty new to C#. I've gotten as far as looking at code snippets online but I'm not really sure if the code below puts me on the right track. How I can write the data once I read it? Does BinaryReader automatically open up and read the entire file so I can then just write it out in binary format?
class Program
{
static void Main(string[] args)
{
using (FileStream fs = new FileStream("output.bin", FileMode.Create))
{
using (BinaryWriter bw = new BinaryWriter(fs))
{
long totalBytes = new System.IO.FileInfo("input.db").Length;
byte[] buffer = null;
BinaryReader binReader = new BinaryReader(File.Open("input.db", FileMode.Open));
}
}
}
}
Edit: Code to stream the database:
[WebGet(UriTemplate = "GetDatabase/{databaseName}")]
public Stream GetDatabase(string databaseName)
{
string fileName = "\\\\computer\\" + databaseName + ".db";
if (File.Exists(fileName))
{
FileStream stream = File.OpenRead(fileName);
if (WebOperationContext.Current != null)
{
WebOperationContext.Current.OutgoingResponse.ContentType = "binary/.bin";
}
return stream;
}
return null;
}
When I call my server, I get nothing back. When I use this same type of method for a content-type of image/.png, it works fine.

All the code you posted will actually do is copy the file input.db to the file output.bin. You could accomplish the same using File.Copy.
BinaryReader will just read in all of the bytes of the file. It is a suitable start to streaming the bytes to an output stream that expects binary data.
Once you have the bytes corresponding to your file, you can write them to the web server's response like this:
using (BinaryReader binReader = new BinaryReader(File.Open("input.db",
FileMode.Open)))
{
byte[] bytes = binReader.ReadBytes(int.MaxValue); // See note below
Response.BinaryWrite(bytes);
Response.Flush();
Response.Close();
Response.End();
}
Note: The code binReader.ReadBytes(int.MaxValue) is for demonstrating the concept only. Don't use it in production code as loading a large file can quickly lead to an OutOfMemoryException. Instead, you should read in the file in chunks, writing to the response stream in chunks.
See this answer for guidance on how to do that
https://stackoverflow.com/a/8613300/141172

C# HttpListener Response + GZipStream

I use HttpListener for my own http server (I do not use IIS). I want to compress my OutputStream by GZip compression:
byte[] refBuffer = Encoding.UTF8.GetBytes(...some data source...);
var varByteStream = new MemoryStream(refBuffer);
System.IO.Compression.GZipStream refGZipStream = new GZipStream(varByteStream, CompressionMode.Compress, false);
refGZipStream.BaseStream.CopyTo(refHttpListenerContext.Response.OutputStream);
refHttpListenerContext.Response.AddHeader("Content-Encoding", "gzip");
But I getting error in Chrome:
ERR_CONTENT_DECODING_FAILED
If I remove AddHeader, then it works, but the size of response is not seems being compressed. What am I doing wrong?

The problem is that your transfer is going in the wrong direction. What you want to do is attach the GZipStream to the Response.OutputStream and then call CopyTo on the MemoryStream, passing in the GZipStream, like so:
refHttpListenerContext.Response.AddHeader("Content-Encoding", "gzip");
byte[] refBuffer = Encoding.UTF8.GetBytes(...some data source...);
var varByteStream = new MemoryStream(refBuffer);
System.IO.Compression.GZipStream refGZipStream = new GZipStream(refHttpListenerContext.Response.OutputStream, CompressionMode.Compress, false);
varByteStream.CopyTo(refGZipStream);
refGZipStream.Flush();

The first problem (as mentioned by Brent M Spell) is the wrong position of the header. The second is that you don't use properly the GZipStream. This stream requires a "top" stream to write to, meaning an empty stream (you fill it with your buffer). Having an empty "top" stream then all you have to do is to write on GZipStream your buffer. As a result the memory stream will be filled by the compressed content. So you need something like:
byte[] buffer = ....;
using (var ms = new MemoryStream())
{
using (var zip = new GZipStream(ms, CompressionMode.Compress, true))
zip.Write(buffer, 0, buffer.Length);
buffer = ms.ToArray();
}
response.AddHeader("Content-Encoding", "gzip");
response.ContentLength64 = buffer.Length;
response.OutputStream.Write(buffer, 0, buffer.Length);

Hopeful this might help, they discuss how to get GZIP working.
Sockets in C#: How to get the response stream?

Out of Memory Exception When Using HttpWebRequest to Stream Large File

I get an Out of Memory Exception when using Http.Put of a large file. I am using an asynchronous model as shown in the code. I am trying to send 8K blocks of data to a Windows 2008 R2 server. The exception consistently occurs when I attempt to write a block of data that exceeds 536,868,864 bytes. The exception occurs on the requestStream.Write method in the code snippet below.
Looking for reasons why?
Note: Smaller files are PUT OK. Logic also works if I write to a local FileStream. Running VS 2010, .Net 4.0 on Win 7 Ultimate client computer.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("Http://website/FileServer/filename");
request.Method = WebRequestMethods.Http.Put;
request.SendChunked = true;
request.AllowWriteStreamBuffering = true;
...
request.BeginGetRequestStream( new AsyncCallback(EndGetStreamCallback), state);
...
int chunk = 8192; // other values give same result
....
private static void EndGetStreamCallback(IAsyncResult ar) {
long limit = 0;
long fileLength;
HttpState state = (HttpState)ar.AsyncState;
Stream requestStream = null;
// End the asynchronous call to get the request stream.
try {
requestStream = state.Request.EndGetRequestStream(ar);
// Copy the file contents to the request stream.
FileStream stream = new FileStream(state.FileName, FileMode.Open, FileAccess.Read, FileShare.None, chunk, FileOptions.SequentialScan);
BinaryReader binReader = new BinaryReader(stream);
fileLength = stream.Length;
// Set Position to the beginning of the stream.
binReader.BaseStream.Position = 0;
byte[] fileContents = new byte[chunk];
// Read File from Buffer
while (limit < fileLength)
{
fileContents = binReader.ReadBytes(chunk);
// the next 2 lines attempt to write to network and server
requestStream.Write(fileContents, 0, chunk); // causes Out of memory after 536,868,864 bytes
requestStream.Flush(); // I get same result with or without Flush
limit += chunk;
}
// IMPORTANT: Close the request stream before sending the request.
stream.Close();
requestStream.Close();
}
}

You apparently have this documented problem. When AllowWriteStreamBuffering is true, it buffers all the data written to the request! So, the "solution" is to set that property to false:
To work around this issue, set the HttpWebRequest.AllowWriteStreamBuffering property to false.