I use HttpListener for my own http server (I do not use IIS). I want to compress my OutputStream by GZip compression:
byte[] refBuffer = Encoding.UTF8.GetBytes(...some data source...);
var varByteStream = new MemoryStream(refBuffer);
System.IO.Compression.GZipStream refGZipStream = new GZipStream(varByteStream, CompressionMode.Compress, false);
refGZipStream.BaseStream.CopyTo(refHttpListenerContext.Response.OutputStream);
refHttpListenerContext.Response.AddHeader("Content-Encoding", "gzip");
But I getting error in Chrome:
ERR_CONTENT_DECODING_FAILED
If I remove AddHeader, then it works, but the size of response is not seems being compressed. What am I doing wrong?
The problem is that your transfer is going in the wrong direction. What you want to do is attach the GZipStream to the Response.OutputStream and then call CopyTo on the MemoryStream, passing in the GZipStream, like so:
refHttpListenerContext.Response.AddHeader("Content-Encoding", "gzip");
byte[] refBuffer = Encoding.UTF8.GetBytes(...some data source...);
var varByteStream = new MemoryStream(refBuffer);
System.IO.Compression.GZipStream refGZipStream = new GZipStream(refHttpListenerContext.Response.OutputStream, CompressionMode.Compress, false);
varByteStream.CopyTo(refGZipStream);
refGZipStream.Flush();
The first problem (as mentioned by Brent M Spell) is the wrong position of the header. The second is that you don't use properly the GZipStream. This stream requires a "top" stream to write to, meaning an empty stream (you fill it with your buffer). Having an empty "top" stream then all you have to do is to write on GZipStream your buffer. As a result the memory stream will be filled by the compressed content. So you need something like:
byte[] buffer = ....;
using (var ms = new MemoryStream())
{
using (var zip = new GZipStream(ms, CompressionMode.Compress, true))
zip.Write(buffer, 0, buffer.Length);
buffer = ms.ToArray();
}
response.AddHeader("Content-Encoding", "gzip");
response.ContentLength64 = buffer.Length;
response.OutputStream.Write(buffer, 0, buffer.Length);
Hopeful this might help, they discuss how to get GZIP working.
Sockets in C#: How to get the response stream?
Related
How can I upload an image properly from Windows Phone where the image source is
a PhotoResult from a PhotoChooserTask? I'm using RestSharp for this, but it only
uploads a big bunch of zeroes to my server.
Here's the code part I have trobule with:
using (MemoryStream memo = new MemoryStream())
{
appGlobal.appData.selectedPhoto.ChosenPhoto.CopyTo(memo);
byte[] imgArray = new byte[appGlobal.appData.selectedPhoto.ChosenPhoto.Length];
memo.Read(imgArray, 0, imgArray.Length);
request.AddFile("image", imgArray, "image", "image/jpeg");
}
I can't seem to figure out how am I supposed to convert PhotoResult.ChosenPhoto (which is a PhotoStream) to a byte array.
Any thoughts?
Ok I found out what the problem was. It seems that when you get the PhotoResult back from
a chooser task let it be PhotoChooserTask or CameraCaptureTask, the stream's position isn't set to 0. So you'll have to set it manually before reading the bytes out from it. Here's the fixed code for my question:
byte[] imgArray = new byte[(int)appGlobal.appData.selectedPhoto.ChosenPhoto.Length];
appGlobal.appData.selectedPhoto.ChosenPhoto.Position = 0;
appGlobal.appData.selectedPhoto.ChosenPhoto.Read(imgArray, 0, (int)appGlobal.appData.selectedPhoto.ChosenPhoto.Length);
appGlobal.appData.selectedPhoto.ChosenPhoto.Seek(0, SeekOrigin.Begin);
request.AddFile("image", imgArray, "image");
Also thanks for the help KooKiz. :)
You're mixing two approaches in your code.
Either directly read the contents of the stream in a byte array:
byte[] imgArray = new byte[appGlobal.appData.selectedPhoto.ChosenPhoto.Length];
appGlobal.appData.selectedPhoto.ChosenPhoto.Read(imgArray, 0, imgArray.Length);
request.AddFile("image", imgArray, "image", "image/jpeg");
Or copy the stream to a MemoryStream then use the ToArray method:
using (MemoryStream memo = new MemoryStream())
{
appGlobal.appData.selectedPhoto.ChosenPhoto.CopyTo(memo);
byte[] imgArray = memo.ToArray();
request.AddFile("image", imgArray, "image", "image/jpeg");
}
I have been trying get a stream from a byte array in metro style app using the following code.
InMemoryRandomAccessStream memoryStream = new InMemoryRandomAccessStream();
memoryStream.AsStreamForWrite().Write(byteArray, 0, byteArray.Length);
memoryStream.Seek(0);
It executes with no errors but stream size is zero (0). Can anybody tell me why is its size is zero?
You can use the DataWriter and DataReader classes. For example ...
// put bytes into the stream
var ms = new InMemoryRandomAccessStream();
var dw = new Windows.Storage.Streams.DataWriter(ms);
dw.WriteBytes(new byte[] { 0x00, 0x01, 0x02 });
await dw.StoreAsync();
// read them out
ms.Seek(0);
byte[] ob = new byte[ms.Size];
var dr = new Windows.Storage.Streams.DataReader(ms);
await dr.LoadAsync((uint)ms.Size);
dr.ReadBytes(ob);
You can also use the BinaryWriter/BinaryReader to read and write from and to byte[] and Streams.
private Stream ConvertToStream(byte[] raw)
{
Stream streamOutput = new MemoryStream();
using (BinaryWriter writer = new BinaryWriter(streamOutput))
{
writer.Write(raw);
}
return streamOutput;
}
Another option is to use built in extension methods as Marc Gravell already mentioned:
private Stream ConvertToStream(byte[] raw)
{
return raw.AsBuffer().AsStream();
}
The extension methods are commented with [Security Critical] which may indicate a later change. However, after looking around a bit I couldn't find any additional information on the security code comment.
I know this is a very old question, but I was running into this issue myself today and figured it out so I'll leave this here for others.
I realized that the stream wasn't being written if it was too small. To fix this, I explicitly set the length of the stream like this:
ms.AsStreamForWrite(imageBytes.Length).Write(imageBytes, 0, imageBytes.Length);
That should be all you need.
I am trying to send large objects (>30MB) to a MSMQ queue. Due to the large amount of data we are are tring to send the idea was to GZip the objects prior to sending them, then unzipping them on the receiving end.
However, writing the compressed stream to the message.BodyStream property seems to work, but not reading it out from there.
I don't know what's wrong.
Message l_QueueMessage = new Message();
l_QueueMessage.Priority = priority;
using (MessageQueue l_Queue = CreateQueue())
{
GZipStream stream = new GZipStream(l_QueueMessage.BodyStream, CompressionMode.Compress);
Formatter.Serialize(stream, message);
l_Queue.Send(l_QueueMessage);
}
The Formatter is a global property of type BinaryFormatter. This is used to serialize/deserialize to the type of object we want to send/receive, e.g. "ProductItem".
The receving end looks like this:
GZipStream stream = new GZipStream(l_Message.BodyStream, CompressionMode.Decompress);
object decompressedObject = Formatter.Deserialize(stream);
ProductItem l_Item = decompressedObject as ProductItem;
m_ProductReceived(sender, new MessageReceivedEventArgs<ProductItem>(l_Item));
l_ProductQueue.BeginReceive();
I get an EndOfStreamException "{"Unable to read beyond the end of the stream."} trying to deserialize
at System.IO.BinaryReader.ReadByte()
Using the messageBodyStream property I actually circumvent the message.Formatter, which I don't initialize to anything, becasue I'm using my own ser/deser mechanism with the GZipStream. However, I am not sure if that's the correct way of doing this.
What am I missing?
Thanks!
In your original code, the problem is that you need to close the GZipStream in order for a GZip footer to be written correctly, and only then you can send it. If you dont, you end up sending bytes that can not be deserialized. That's also why your new code where sending is done later works.
OK, I made this work. The key was to convert the decompressed stream on the receiver to a byte[] array. Then the deserialization started working.
The sender code (notice the stream is closed before sending the message):
using (MessageQueue l_Queue = CreateQueue())
{
using (GZipStream stream = new GZipStream(l_QueueMessage.BodyStream, CompressionMode.Compress, true))
{
Formatter.Serialize(stream, message);
}
l_Queue.Send(l_QueueMessage);
}
The receiving end (notice how I convert the stream to a byte[] then deserialize):
using (GZipStream stream = new GZipStream(l_QueueMessage.BodyStream, CompressionMode.Decompress))
{
byte[] bytes = ReadFully(stream);
using (MemoryStream ms = new MemoryStream(bytes))
{
decompressedObject = Formatter.Deserialize(ms);
}
}
Still, don't know why this works using the ReadFully() function and not the Stream.CopyTo().
Does anyone?
Btw, ReadFully() is a function that creates a byte[] out of a Stream. I have to credit Jon Skeet for this at http://www.yoda.arachsys.com/csharp/readbinary.html. Thanks!
Try to separate compressing and sending:
byte[] binaryBuffer = null;
using (MemoryStream compressedBody = new MemoryStream())
{
using(GZipStream stream = new GZipStream(compressedBody, CompressionMode.Compress))
{
Formatter.Serialize(compressedBody, message);
binaryBuffer = compressedBody.GetBuffer();
}
}
using (MessageQueue l_Queue = CreateQueue())
{
l_QueueMessage.BodyStream.Write(binaryBuffer, 0, binaryBuffer.Length);
l_QueueMessage.BodyStream.Seek(0, SeekOrigin.Begin);
l_Queue.Send(l_QueueMessage);
}
I'm trying to get an image from an url using a byte stream. But i get this error message:
This stream does not support seek operations.
This is my code:
byte[] b;
HttpWebRequest myReq = (HttpWebRequest)WebRequest.Create(url);
WebResponse myResp = myReq.GetResponse();
Stream stream = myResp.GetResponseStream();
int i;
using (BinaryReader br = new BinaryReader(stream))
{
i = (int)(stream.Length);
b = br.ReadBytes(i); // (500000);
}
myResp.Close();
return b;
What am i doing wrong guys?
You probably want something like this. Either checking the length fails, or the BinaryReader is doing seeks behind the scenes.
HttpWebRequest myReq = (HttpWebRequest)WebRequest.Create(url);
WebResponse myResp = myReq.GetResponse();
byte[] b = null;
using( Stream stream = myResp.GetResponseStream() )
using( MemoryStream ms = new MemoryStream() )
{
int count = 0;
do
{
byte[] buf = new byte[1024];
count = stream.Read(buf, 0, 1024);
ms.Write(buf, 0, count);
} while(stream.CanRead && count > 0);
b = ms.ToArray();
}
edit:
I checked using reflector, and it is the call to stream.Length that fails. GetResponseStream returns a ConnectStream, and the Length property on that class throws the exception that you saw. As other posters mentioned, you cannot reliably get the length of a HTTP response, so that makes sense.
Use a StreamReader instead:
HttpWebRequest myReq = (HttpWebRequest)WebRequest.Create(url);
WebResponse myResp = myReq.GetResponse();
StreamReader reader = new StreamReader(myResp.GetResponseStream());
return reader.ReadToEnd();
(Note - the above returns a String instead of a byte array)
You can't reliably ask an HTTP connection for its length. It's possible to get the server to send you the length in advance, but (a) that header is often missing and (b) it's not guaranteed to be correct.
Instead you should:
Create a fixed-length byte[] that you pass to the Stream.Read method
Create a List<byte>
After each read, call List.AddRange to append the contents of your fixed-length buffer onto your byte list
Note that the last call to Read will return fewer than the full number of bytes you asked for. Make sure you only append that number of bytes onto your List<byte> and not the whole byte[], or you'll get garbage at the end of your list.
If the server doesn't send a length specification in the HTTP header, the stream size is unknown, so you get the error when trying to use the Length property.
Read the stream in smaller chunks, until you reach the end of the stream.
With images, you don't need to read the number of bytes at all. Just do this:
Image img = null;
string path = "http://www.example.com/image.jpg";
WebRequest request = WebRequest.Create(path);
req.Credentials = CredentialCache.DefaultCredentials; // in case your URL has Windows auth
WebResponse resp = req.GetResponse();
using( Stream stream = resp.GetResponseStream() )
{
img = Image.FromStream(stream);
// then use the image
}
Perhaps you should use the System.Net.WebClient API. If already using client.OpenRead(url) use client.DownloadData(url)
var client = new System.Net.WebClient();
byte[] buffer = client.DownloadData(url);
using (var stream = new MemoryStream(buffer))
{
... your code using the stream ...
}
Obviously this downloads everything before the Stream is created, so it may defeat the purpose of using a Stream. webClient.DownloadData("https://your.url") gets a byte array which you can then turn into a MemoryStream.
The length of a stream can not be read from the stream since the receiver does not know how many bytes the sender will send. Try to put a protocol on top of http and send i.e. the length as first item in the stream.
HI
I have an xml file with 500KB size which i need to send it to webservice, so i want to compress this data and send it to the webservice
i have heard of some base24Encoding something...
Can anyone throw more light on this
Suppose if i use GZipStream how can i send the file to the webservice
Thanks in Advance
Something like below (the first part just writes some random xml for us to work with). Your web-service would ideally take a byte[] argument, and would have (if using WSE3 or MCF over basic-http) MTOM enabled, which reduces the base-64 overhead. You just post it the byte[] and then reverse the compression at the other end.
if (File.Exists("my.xml")) File.Delete("my.xml");
using (XmlWriter xmlFile = XmlWriter.Create("my.xml")) {
Random rand = new Random();
xmlFile.WriteStartElement("xml");
for (int i = 0; i < 1000; i++) {
xmlFile.WriteElementString("add", rand.Next().ToString());
}
xmlFile.WriteEndElement();
xmlFile.Close();
}
// now we have some xml!
using (MemoryStream ms = new MemoryStream()) {
int origBytes = 0;
using (GZipStream zip = new GZipStream(ms, CompressionMode.Compress, true))
using (FileStream file = File.OpenRead("my.xml")) {
byte[] buffer = new byte[2048];
int bytes;
while ((bytes = file.Read(buffer, 0, buffer.Length)) > 0) {
zip.Write(buffer, 0, bytes);
origBytes += bytes;
}
}
byte[] blob = ms.ToArray();
string asBase64 = Convert.ToBase64String(blob);
Console.WriteLine("Original: " + origBytes);
Console.WriteLine("Raw: " + blob.Length);
Console.WriteLine("Base64: " + asBase64.Length);
}
Alternatively, consider a different serialization format; there are dense binary protocols that are much smaller (and as a consequence don't benefit from gzip etc). For example, serializing via protobuf-net would give you a very efficient size. But this only applies to an object-model, not to arbitrary xml data.
There are some good articles about that here:
http://www.ibm.com/developerworks/xml/library/x-tipcomp.html
http://www.xml.com/pub/a/2000/03/22/deviant/
The best way to handle this scenario would be to have your web service accept a byte[] parameter which will represent the compressed XML. The Base64 encoding will be done automatically. To improve compression ratio you could use MTOM encoding. This will avoid the Base64 step which consists of converting your byte array to a string in order to send it over the wire and where you loose might in compression ratio.
You have following to choose from:
BinaryFormatter
ArrayList itemsToSerialize = new ArrayList();
itemsToSerialize.Add ( "john" );
itemsToSerialize.Add ( "smith" );
Stream stream = new FileStream( #"MyApplicationData.dat", System.IO.FileMode.Create );
IFormatter formatter = new BinaryFormatter();
formatter.Serialize( stream, itemsToSerialize );
stream.Close();
You could use WCF netTcpBinding
You could use *HttpBinding for WCF service hosted on IIS and follow this blog which walks you through setting up WCF gzip compression through IIS
You could zip response and request and then decompress it
new StreamReader(new GZipStream(webResponse.GetResponseStream(), CompressionMode.Decompress));