How to get the length of the compressed data from DeflateStream? - c#

The following is a simple compression method I wrote using DeflateStream:
public static int Compress(
byte[] inputData,
int inputStartIndex,
int inputLength,
byte[] outputData,
int outputStartIndex,
int outputLength)
{
if (inputData == null)
throw new ArgumentNullException("inputData must be non-null");
MemoryStream memStream = new MemoryStream(outputData, outputStartIndex, outputLength);
using (DeflateStream dstream = new DeflateStream(memStream, CompressionLevel.Optimal))
{
dstream.Write(inputData, inputStartIndex, inputLength);
return (int)(memStream.Position - outputStartIndex);
}
}
What is special in this method is that I didn't use the parameter-less constructor of MemoryStream. This is because it is a high-throughput server. Array outputData is rented from ArrayPool, to be used to hold the compressed bytes, so that after I make use of it I can return it to ArrayPool.
The compression happened properly, and the compressed data is properly placed into outputData, but memStream.Position was zero, so I can't find out how many bytes have been written into the MemoryStream.
Only part of outputData is occupied by the compressed data. How do I find out the length of the compressed data?

MemoryStream.Position is 0 because data was not actually written there yet at the point you read Position. Instead, tell DeflateStream to leave underlying stream (MemoryStream) open, then dispose DeflateStream. At this point you can be sure it's done writing whatever it needs. Now you can read MemoryStream.Position to check how many bytes were written:
public static int Compress(
byte[] inputData,
int inputStartIndex,
int inputLength,
byte[] outputData,
int outputStartIndex,
int outputLength)
{
if (inputData == null)
throw new ArgumentNullException("inputData must be non-null");
using (var memStream = new MemoryStream(outputData, outputStartIndex, outputLength)) {
// leave open
using (DeflateStream dstream = new DeflateStream(memStream, CompressionLevel.Optimal, leaveOpen: true)) {
dstream.Write(inputData, inputStartIndex, inputLength);
}
return (int) memStream.Position; // now it's not 0
}
}
You also don't need to substract outputStartIndex, because Position is already relative to that index you passed to constructor.

Related

Why do I get ArgumentException when I try to convert stream to Image for second time? C#

I have a small problem, I am doing a video transmission between 2 PCs by TCP connection, what I do is take a screenshot of the screen, serialize it and send it to the other PC, the first time it works correctly, but the second time I get "ArgumentException " and the program no longer works, try to do it with BinaryFormatter it works but it is very slow.
both methods are called with an infinite while
public static void SerializeScreen(Stream stream, Bitmap Image)
{
BinaryWriter binaryWriter = new BinaryWriter(stream);
MemoryStream memory = new MemoryStream();
Image.Save(memory, System.Drawing.Imaging.ImageFormat.Jpeg);
binaryWriter.Write(memory.ToArray());
}
public static Image DeserializeScreen(Stream stream)
{
byte[] Buffer = new byte[500000];
BinaryReader binaryReader = new BinaryReader(stream);
int Bytes;
int LastPost = 0;
do
{
Bytes = binaryReader.Read(Buffer, LastPost, Buffer.Length - LastPost);
LastPost += Bytes;
} while (Bytes > 0);
MemoryStream memory = new MemoryStream(Buffer);
//Here is where I get the exception
return Image.FromStream(memory);
}
You are not delimiting your serialized images so that the deserializer knows where one image ends and the next begins. Especially since JPG is a variable-length format, depending on its compression level. Your deserializer is just blindly reading from the input stream until some arbitrary max byte count is reached, or the input stream runs out of bytes (ie, when the peer closes the TCP connection). That kind of logic will never work out.
I suggest you write out the serialized Image's byte count to the destination stream before writing out the actual serialized bytes. That way, the deserializer can then read the count first, allocate that much memory, and read the bytes int it.
Try something more like this:
public static void SerializeScreen(Stream stream, Bitmap Image)
{
MemoryStream memory = new MemoryStream();
Image.Save(memory, System.Drawing.Imaging.ImageFormat.Jpeg);
int numBytes = (int) memory.Length;
BinaryWriter binaryWriter = new BinaryWriter(stream);
binaryWriter.Write(numBytes);
binaryWriter.Write(memory.GetBuffer(), 0, numBytes);
}
public static Image DeserializeScreen(Stream stream)
{
BinaryReader binaryReader = new BinaryReader(stream);
int numBytes = binaryReader.ReadInt32();
byte[] buffer = binaryReader.ReadBytes(numBytes);
MemoryStream memory = new MemoryStream(buffer);
return Image.FromStream(memory);
}

Return buffer while processing Stream

So I have a file upload form which (after uploading) encrypts the file and uploads it to an S3 bucket. However, I'm doing an extra step which I want to avoid. First, I'll show you some code what I am doing now:
using (MemoryStream memoryStream = new MemoryStream())
{
Security.EncryptFile(FileUpload.UploadedFile.OpenReadStream(), someByteArray, memoryStream);
memoryStream.Position = 0; // reset it's position
await S3Helper.Upload(objectName, memoryStream);
}
My Security.EncryptFile method:
public static void EncryptFile(Stream inputStream, byte[] key, Stream outputStream)
{
CryptoStream cryptoStream;
using (SymmetricAlgorithm cipher = Aes.Create())
using (inputStream)
{
cipher.Key = key;
// aes.IV will be automatically populated with a secure random value
byte[] iv = cipher.IV;
// Write a marker header so we can identify how to read this file in the future
outputStream.WriteByte(69);
outputStream.WriteByte(74);
outputStream.WriteByte(66);
outputStream.WriteByte(65);
outputStream.WriteByte(69);
outputStream.WriteByte(83);
outputStream.Write(iv, 0, iv.Length);
using (cryptoStream =
new CryptoStream(inputStream, cipher.CreateEncryptor(), CryptoStreamMode.Read))
{
cryptoStream.CopyTo(outputStream);
}
}
}
The S3Helper.Upload method:
public async static Task Upload(string objectName, Stream inputStream)
{
try
{
// Upload a file to bucket.
using (inputStream)
{
await minio.PutObjectAsync(S3BucketName, objectName, inputStream, inputStream.Length);
}
Console.Out.WriteLine("[Bucket] Successfully uploaded " + objectName);
}
catch (MinioException e)
{
Console.WriteLine("[Bucket] Upload exception: {0}", e.Message);
}
}
So, what happens above is I'm creating a MemoryStream, running the EncryptFile() method (which outputs it back to the stream), I reset the stream position and finally reuse it again to upload it to the S3 bucket (Upload()).
The question
What I'd like to do is the following (if possible): directly upload the uploaded file to the S3 bucket, without storing the full file in memory first (kinda like the code below, even though it's not working):
await S3Helper.Upload(objectName, Security.EncryptFile(FileUpload.UploadedFile.OpenReadStream(), someByteArray));
So I assume it has to return a buffer to the Upload method, which will upload it, and waits for the EncryptFile() method to return a buffer again until the file has been fully read. Any pointers to the right direction will be greatly appreciated.
What you could do is make your own EncryptionStream that overloads the Stream class. When you read from this stream, it will take a block from the inputstream, encrypt it and then output the encrypted data.
As an example, something like this:
public class EncrypStream : Stream {
private Stream _cryptoStream;
private SymmetricAlgorithm _cipher;
private Stream InputStream { get; }
private byte[] Key { get; }
public EncrypStream(Stream inputStream, byte[] key) {
this.InputStream = inputStream;
this.Key = key;
}
public override int Read(byte[] buffer, int offset, int count) {
if (this._cipher == null) {
_cipher = Aes.Create();
_cipher.Key = Key;
// aes.IV will be automatically populated with a secure random value
byte[] iv = _cipher.IV;
// Write a marker header so we can identify how to read this file in the future
// #TODO Make sure the BUFFER is big enough...
var idx = offset;
buffer[idx++] = 69;
buffer[idx++] = 74;
buffer[idx++] = 66;
buffer[idx++] = 65;
buffer[idx++] = 69;
buffer[idx++] = 83;
Array.Copy(iv, 0, buffer, idx, iv.Length);
offset = idx + iv.Length;
// Startup stream
this._cryptoStream = new CryptoStream(InputStream, _cipher.CreateEncryptor(), CryptoStreamMode.Read);
}
// Write block
return this._cryptoStream.Read(buffer, offset, count);
}
protected override void Dispose(bool disposing) {
base.Dispose(disposing);
// Make SURE you properly dispose the underlying streams!
this.InputStream?.Dispose();
this._cipher?.Dispose();
this._cryptoStream?.Dispose();
}
// Omitted other methods from stream for readability...
}
Which allows you to call the stream as:
using (var stream = new EncrypStream(FileUpload.UploadedFile.OpenReadStream(), someByteArray)) {
await S3Helper.Upload(objectName, stream);
}
As I notice your upload method requires the total bytelength of the encrypted data, you can look into this post here to get an idea how you would be able to calculate this.
(I'm guessing that the CryptoStream does not return the expected length of the encrypted data, but please correct me if I'm wrong on this)

Sending and receiving compressed data over a TCP socket

Need help with sending and receiving compressed data over TCP socket.
The code works perfectly fine if I don't use compression, but something very strange happens when I do use compression.. Basically, the problem is that the stream.Read() operation gets skipped and I don't know why..
My code:
using (var client = new TcpClient())
{
client.Connect("xxx.xxx.xx.xx", 6100);
using (var stream = client.GetStream())
{
// SEND REQUEST
byte[] bytesSent = Encoding.UTF8.GetBytes(xml);
// send compressed bytes (if this is used, then stream.Read() below doesn't work.
//var compressedBytes = bytesSent.ToStream().GZipCompress();
//stream.Write(compressedBytes, 0, compressedBytes.Length);
// send normal bytes (uncompressed)
stream.Write(bytesSent, 0, bytesSent.Length);
// GET RESPONSE
byte[] bytesReceived = new byte[client.ReceiveBufferSize];
// PROBLEM HERE: when using compression, this line just gets skipped over very quickly
stream.Read(bytesReceived, 0, client.ReceiveBufferSize);
//var decompressedBytes = bytesReceived.ToStream().GZipDecompress();
//string response = Encoding.UTF8.GetString(decompressedBytes);
string response = Encoding.UTF8.GetString(bytesReceived);
Console.WriteLine(response);
}
}
You will notice some extension methods above. Here is the code in case you are wondering if something is wrong there.
public static MemoryStream ToStream(this byte[] bytes)
{
return new MemoryStream(bytes);
}
public static byte[] GZipCompress(this Stream stream)
{
using (var memoryStream = new MemoryStream())
{
using (var gZipStream = new GZipStream(memoryStream, CompressionMode.Compress))
{
stream.CopyTo(gZipStream);
}
return memoryStream.ToArray();
}
}
public static byte[] GZipDecompress(this Stream stream)
{
using (var memoryStream = new MemoryStream())
{
using (var gZipStream = new GZipStream(stream, CompressionMode.Decompress))
{
gZipStream.CopyTo(memoryStream);
}
return memoryStream.ToArray();
}
}
The extensions work quite well in the following, so I'm sure they're not the problem:
string original = "the quick brown fox jumped over the lazy dog";
byte[] compressedBytes = Encoding.UTF8.GetBytes(original).ToStream().GZipCompress();
byte[] decompressedBytes = compressedBytes.ToStream().GZipDecompress();
string result = Encoding.UTF8.GetString(decompressedBytes);
Console.WriteLine(result);
Does anyone have any idea why the Read() operation is being skipped when the bytes being sent are compressed?
EDIT
I received a message from the API provider after showing them the above sample code. They had this to say:
at a first glance I guess the header is missing. The input must start
with a 'c' followed by the length of the input
(sprintf(cLength,"c%09d",hres) in our example). We need this because
we can't read until we find a binary 0 to recognize the end.
They previously provided some sample code in C, which I don't fully understand 100%, as follows:
example in C:
#include <zlib.h>
uLongf hres;
char cLength[COMPRESS_HEADER_LEN + 1] = {'\0'};
n = read(socket,buffer,10);
// check if input is compressed
if(msg[0]=='c') {
compressed = 1;
}
n = atoi(msg+1);
read.....
hres = 64000;
res = uncompress((Bytef *)msg, &hres, (const Bytef*)
buffer/*compressed*/, n);
if(res == Z_OK && hres > 0 ){
msg[hres]=0; //original
}
else // errorhandling
hres = 64000;
if (compressed){
res = compress((Bytef *)buffer, &hres, (const Bytef *)msg, strlen(msg));
if(res == Z_OK && hres > 0 ) {
sprintf(cLength,"c%09d",hres);
write(socket,cLength,10);
write(socket, buffer, hres);
}
else // errorhandling
makefile: add "-lz" to the libs
They're using zlib. I don't suspect that to make any difference, but I did try using zlib.net and I still get no response anyway.
Can someone give me an example of how exactly I'm supposed to send this input length in C#?
EDIT 2
In response to #quantdev, here is what I am trying now for the length prefix:
using (var client = new TcpClient())
{
client.Connect("xxx.xxx.xx.xx", 6100);
using (var stream = client.GetStream())
{
// SEND REQUEST
byte[] bytes = Encoding.UTF8.GetBytes(xml);
byte[] compressedBytes = ZLibCompressor.Compress(bytes);
byte[] prefix = Encoding.UTF8.GetBytes("c" + compressedBytes.Length);
byte[] bytesToSend = new byte[prefix.Length + compressedBytes.Length];
Array.Copy(prefix, bytesToSend, prefix.Length);
Array.Copy(compressedBytes, 0, bytesToSend, prefix.Length, compressedBytes.Length);
stream.Write(bytesToSend, 0, bytesToSend.Length);
// WAIT
while (client.Available == 0)
{
Thread.Sleep(1000);
}
// GET RESPONSE
byte[] bytesReceived = new byte[client.ReceiveBufferSize];
stream.Read(bytesReceived, 0, client.ReceiveBufferSize);
byte[] decompressedBytes = ZLibCompressor.DeCompress(bytesReceived);
string response = Encoding.UTF8.GetString(decompressedBytes);
Console.WriteLine(response);
}
}
You need to check the return value of the Read() calls you are making on the TCP stream: it is the number of bytes effectively read.
MSDN says :
Return Value
The total number of bytes read into the buffer. This can be less than the number of bytes requested if that many bytes are not
currently available, or zero (0) if the end of the stream has been
reached.
If the socket is closed, the call will return immediately 0 (which is what might be happening here).
If is not 0, then you must check how many bytes you did actually received, if it is less than client.ReceiveBufferSize, you will need additional calls to Read to retrieve the remaining bytes.
Prior to you call to read, check that some data is actually available on the socket :
while(client.Available == 0)
// wait ...
http://msdn.microsoft.com/en-us/library/system.net.sockets.tcpclient.available%28v=vs.110%29.aspx
I think you may have the end of file or so. Can you try setting the stream position before reading the stream
stream.position = 0;
http://msdn.microsoft.com/en-us/library/vstudio/system.io.stream.read
Encoding.UTF8.GetString shouldn't be used on arbitrary byte array.
e.g.: The compressed bytes may contain NULL character, which is not allowed in UTF-8 encoded text except for being used as terminator.
If you want to print the received bytes for debugging, maybe you should just print them as integers.

c# Stream Reading and Deserialization

I have this code:
public static List<ReplicableObject> ParseStreamForObjects(Stream stream)
{
List<ReplicableObject> result = new List<ReplicableObject>();
while (true)
{
// HERE I want to check that there's at least four bytes left in the stream
BinaryReader br = new BinaryReader(stream);
int length = br.ReadInt32();
// HERE I want to check that there's enough bytes left in the stream
byte[] bytes = br.ReadBytes(length);
MemoryStream ms = new MemoryStream(bytes);
ms.Position = 0;
result.Add((ReplicableObject) Formatter.Deserialize(ms));
ms.Close();
br.Close();
}
return result;
}
Unfortunately, the stream object is always going to be a TCP stream, which means no seek operations. So how can I check to make sure that I'm not over-running the stream where I've put the // HERE comments?
I don't think there's any way to query a NetworkStream to find the data you're looking for. What you'll probably need to do is buffer whatever data the stream makes available into another data structure, then parse objects out of that structure once you know it's got enough bytes in it.
The NetworkStream class provides a DataAvailable property that tells you if any data is available to be read, and the Read() method returns a value indicating how many bytes it actually retrieved. You should be able to use those values to do the buffering you need.
See Mr. Skeets page
Sometimes, you don't know the length of the stream in advance (for instance a network stream) and just want to read the whole lot into a buffer. Here's a method to do just that:
/// <summary>
/// Reads data from a stream until the end is reached. The
/// data is returned as a byte array. An IOException is
/// thrown if any of the underlying IO calls fail.
/// </summary>
/// <param name="stream">The stream to read data from</param>
public static byte[] ReadFully (Stream stream)
{
byte[] buffer = new byte[32768];
using (MemoryStream ms = new MemoryStream())
{
while (true)
{
int read = stream.Read (buffer, 0, buffer.Length);
if (read <= 0)
return ms.ToArray();
ms.Write (buffer, 0, read);
}
}
}
This should give you some ideas. Once you have the byte array, checking the Length will be easy to do.
In your example, it would look something like this:
int bytes_to_read = 4;
byte[] length_bytes = new byte[bytes_to_read];
int bytes_read = stream.Read(length_bytes, 0, length_bytes.Length);
// Check that there's at least four bytes left in the stream
if(bytes_read != bytes_to_read) break;
int bytes_in_msg = BitConverter.ToInt32(length_bytes);
byte[] msg_bytes = new byte[bytes_in_msg];
bytes_read = stream.Read(msg_bytes, 0, msg_bytes.Length);
// Check that there's enough bytes left in the stream
if(bytes_read != bytes_in_msg ) break;
...

Creating a byte array from a stream

What is the prefered method for creating a byte array from an input stream?
Here is my current solution with .NET 3.5.
Stream s;
byte[] b;
using (BinaryReader br = new BinaryReader(s))
{
b = br.ReadBytes((int)s.Length);
}
Is it still a better idea to read and write chunks of the stream?
It really depends on whether or not you can trust s.Length. For many streams, you just don't know how much data there will be. In such cases - and before .NET 4 - I'd use code like this:
public static byte[] ReadFully(Stream input)
{
byte[] buffer = new byte[16*1024];
using (MemoryStream ms = new MemoryStream())
{
int read;
while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
{
ms.Write(buffer, 0, read);
}
return ms.ToArray();
}
}
With .NET 4 and above, I'd use Stream.CopyTo, which is basically equivalent to the loop in my code - create the MemoryStream, call stream.CopyTo(ms) and then return ms.ToArray(). Job done.
I should perhaps explain why my answer is longer than the others. Stream.Read doesn't guarantee that it will read everything it's asked for. If you're reading from a network stream, for example, it may read one packet's worth and then return, even if there will be more data soon. BinaryReader.Read will keep going until the end of the stream or your specified size, but you still have to know the size to start with.
The above method will keep reading (and copying into a MemoryStream) until it runs out of data. It then asks the MemoryStream to return a copy of the data in an array. If you know the size to start with - or think you know the size, without being sure - you can construct the MemoryStream to be that size to start with. Likewise you can put a check at the end, and if the length of the stream is the same size as the buffer (returned by MemoryStream.GetBuffer) then you can just return the buffer. So the above code isn't quite optimised, but will at least be correct. It doesn't assume any responsibility for closing the stream - the caller should do that.
See this article for more info (and an alternative implementation).
While Jon's answer is correct, he is rewriting code that already exists in CopyTo. So for .Net 4 use Sandip's solution, but for previous version of .Net use Jon's answer. Sandip's code would be improved by use of "using" as exceptions in CopyTo are, in many situations, quite likely and would leave the MemoryStream not disposed.
public static byte[] ReadFully(Stream input)
{
using (MemoryStream ms = new MemoryStream())
{
input.CopyTo(ms);
return ms.ToArray();
}
}
Just want to point out that in case you have a MemoryStream you already have memorystream.ToArray() for that.
Also, if you are dealing with streams of unknown or different subtypes and you can receive a MemoryStream, you can relay on said method for those cases and still use the accepted answer for the others, like this:
public static byte[] StreamToByteArray(Stream stream)
{
if (stream is MemoryStream)
{
return ((MemoryStream)stream).ToArray();
}
else
{
// Jon Skeet's accepted answer
return ReadFully(stream);
}
}
MemoryStream ms = new MemoryStream();
file.PostedFile.InputStream.CopyTo(ms);
var byts = ms.ToArray();
ms.Dispose();
just my couple cents... the practice that I often use is to organize the methods like this as a custom helper
public static class StreamHelpers
{
public static byte[] ReadFully(this Stream input)
{
using (MemoryStream ms = new MemoryStream())
{
input.CopyTo(ms);
return ms.ToArray();
}
}
}
add namespace to the config file and use it anywhere you wish
You can simply use ToArray() method of MemoryStream class, for ex-
MemoryStream ms = (MemoryStream)dataInStream;
byte[] imageBytes = ms.ToArray();
You can even make it fancier with extensions:
namespace Foo
{
public static class Extensions
{
public static byte[] ToByteArray(this Stream stream)
{
using (stream)
{
using (MemoryStream memStream = new MemoryStream())
{
stream.CopyTo(memStream);
return memStream.ToArray();
}
}
}
}
}
And then call it as a regular method:
byte[] arr = someStream.ToByteArray()
I get a compile time error with Bob's (i.e. the questioner's) code. Stream.Length is a long whereas BinaryReader.ReadBytes takes an integer parameter. In my case, I do not expect to be dealing with Streams large enough to require long precision, so I use the following:
Stream s;
byte[] b;
if (s.Length > int.MaxValue) {
throw new Exception("This stream is larger than the conversion algorithm can currently handle.");
}
using (var br = new BinaryReader(s)) {
b = br.ReadBytes((int)s.Length);
}
In case anyone likes it, here is a .NET 4+ only solution formed as an extension method without the needless Dispose call on the MemoryStream. This is a hopelessly trivial optimization, but it is worth noting that failing to Dispose a MemoryStream is not a real failure.
public static class StreamHelpers
{
public static byte[] ReadFully(this Stream input)
{
var ms = new MemoryStream();
input.CopyTo(ms);
return ms.ToArray();
}
}
The one above is ok...but you will encounter data corruption when you send stuff over SMTP (if you need to). I've altered to something else that will help to correctly send byte for byte:
'
using System;
using System.IO;
private static byte[] ReadFully(string input)
{
FileStream sourceFile = new FileStream(input, FileMode.Open); //Open streamer
BinaryReader binReader = new BinaryReader(sourceFile);
byte[] output = new byte[sourceFile.Length]; //create byte array of size file
for (long i = 0; i < sourceFile.Length; i++)
output[i] = binReader.ReadByte(); //read until done
sourceFile.Close(); //dispose streamer
binReader.Close(); //dispose reader
return output;
}'
Combinig two of the most up-voted answers into an extension method:
public static byte[] ToByteArray(this Stream stream)
{
if (stream is MemoryStream)
return ((MemoryStream)stream).ToArray();
else
{
using MemoryStream ms = new();
stream.CopyTo(ms);
return ms.ToArray();
}
}
Create a helper class and reference it anywhere you wish to use it.
public static class StreamHelpers
{
public static byte[] ReadFully(this Stream input)
{
using (MemoryStream ms = new MemoryStream())
{
input.CopyTo(ms);
return ms.ToArray();
}
}
}
In namespace RestSharp.Extensions there is method ReadAsBytes. Inside this method is used MemoryStream and there is the same code like in some examples on this page but when you are using RestSharp this is easiest way.
using RestSharp.Extensions;
var byteArray = inputStream.ReadAsBytes();
If a stream supports the Length property, a byte array can be directly created. The advantage is that MemoryStream.ToArray creates the array twice. Plus, probably some unused extra bytes in the buffer. This solution allocates the exact array needed. If the stream does not support the Length property, it will throw NotSupportedException exception.
It is also worth noting that arrays cannot be bigger than int.MaxValue.
public static async Task<byte[]> ToArrayAsync(this Stream stream)
{
var array = new byte[stream.Length];
await stream.ReadAsync(array, 0, (int)stream.Length);
return array;
}
Complete code which switches between both versions based on whether the stream supports seeking or not. It includes checks for Position and unreliable Length. That might slightly reduce speed. In my tests ToArrayAsyncDirect is about 3 times faster compared to ToArrayAsyncGeneral.
public static class StreamExtensions
{
public static readonly byte[] TempArray = new byte[4];
/// <summary>
/// Converts stream to byte array.
/// </summary>
/// <param name="stream">Stream</param>
/// <param name="cancellationToken">Cancellation token</param>
/// <returns>Stream data as array</returns>
/// <returns>Binary data from stream in an array</returns>
public static async Task<byte[]> ToArrayAsync(this Stream stream, CancellationToken cancellationToken)
{
if (!stream.CanRead)
{
throw new AccessViolationException("Stream cannot be read");
}
if (stream.CanSeek)
{
return await ToArrayAsyncDirect(stream, cancellationToken);
}
else
{
return await ToArrayAsyncGeneral(stream, cancellationToken);
}
}
/// <summary>
/// Converts stream to byte array through MemoryStream. This doubles allocations compared to ToArrayAsyncDirect.
/// </summary>
/// <param name="stream">Stream</param>
/// <param name="cancellationToken">Cancellation token</param>
/// <returns></returns>
private static async Task<byte[]> ToArrayAsyncGeneral(Stream stream, CancellationToken cancellationToken)
{
using MemoryStream memoryStream = new MemoryStream();
await stream.CopyToAsync(memoryStream, cancellationToken);
return memoryStream.ToArray();
}
/// <summary>
/// Converts stream to byte array without unnecessary allocations.
/// </summary>
/// <param name="stream">Stream</param>
/// <param name="cancellationToken">Cancellation token</param>
/// <returns>Stream data as array</returns>
/// <exception cref="ArgumentException">Thrown if stream is not providing correct Length</exception>
private static async Task<byte[]> ToArrayAsyncDirect(Stream stream, CancellationToken cancellationToken)
{
if (stream.Position > 0)
{
throw new ArgumentException("Stream is not at the start!");
}
var array = new byte[stream.Length];
int bytesRead = await stream.ReadAsync(array, 0, (int)stream.Length, cancellationToken);
if (bytesRead != array.Length ||
await stream.ReadAsync(TempArray, 0, TempArray.Length, cancellationToken) > 0)
{
throw new ArgumentException("Stream does not have reliable Length!");
}
return array;
}
}
This is the function which I am using, tested and worked well.
please bear in mind that 'input' should not be null and 'input.position' should reset to '0' before reading otherwise it will break the read loop and nothing will read to convert to array.
public static byte[] StreamToByteArray(Stream input)
{
if (input == null)
return null;
byte[] buffer = new byte[16 * 1024];
input.Position = 0;
using (MemoryStream ms = new MemoryStream())
{
int read;
while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
{
ms.Write(buffer, 0, read);
}
byte[] temp = ms.ToArray();
return temp;
}
}
Since there's no modern (i.e. async) version of this answer, this is the extension method I use for this purpose:
public static async Task<byte[]> ReadAsByteArrayAsync(this Stream source)
{
// Optimization
if (source is MemoryStream memorySource)
return memorySource.ToArray();
using var memoryStream = new MemoryStream();
await source.CopyToAsync(memoryStream);
return memoryStream.ToArray();
}
The optimization is based on the fact the source code for ToArray calls some internal methods.
You can use this extension method.
public static class StreamExtensions
{
public static byte[] ToByteArray(this Stream stream)
{
var bytes = new List<byte>();
int b;
// -1 is a special value that mark the end of the stream
while ((b = stream.ReadByte()) != -1)
bytes.Add((byte)b);
return bytes.ToArray();
}
}
i was able to make it work on a single line:
byte [] byteArr= ((MemoryStream)localStream).ToArray();
as clarified by johnnyRose, Above code will only work for MemoryStream

Categories

Resources