I am serializing an object to a stream to store as file and then retrieving and trying to deserialize the object, but get an error parsing. Below is code:
var content = JsonConvert.SerializeObject(data);
var output = new MemoryStream();
var writer = new StreamWriter(output, Encoding.UTF8);
writer.Write(content);
writer.Flush();
//write to some file...
//when reading the file
Stream filestream;
//filestream opens some file stream
byte[] buffer = new byte[4096]
using(MemoryStream ms = new MemoryStream()){
int read;
while((read = filestream.Read(buffer, 0, buffer.Length)) > 0){
ms.Write(buffer, 0, read);
}
var data = Encoding.UTF8.GetString(ms.ToArray());
//encounters error here. I can see that first few chars of the string are question marks.
JsonConvert.DeserializeObject<T>(data);
A couple ideas. Are you getting an exception?
Try encapsulating your write & reads into functions and wrap the memoryStream and streamWriter in using statements as well
using var writer = new StreamWriter(...
Additionally you don't need to set the length of the buffer to 4096, that's probably messing up your encoding. Read your memorystream to an array like so
var buffer = ms.ToArray();
How can I write MemoryStream to byte[]
It looks like you're having an issue copying the memory between the 2 memory streams though. Ideally you would write and read to a file or some other source rather than between 2 memory stream objects but you can copy the contents directly using a copyto
https://learn.microsoft.com/en-us/dotnet/api/system.io.stream.copyto?view=net-6.0
Edit: Adding pseudocode
Assuming your content has been read into
Stream filestream;
using var MemoryStream ms = new MemoryStream();
filestream.CopyTo(ms);
var data = Encoding.UTF8.GetString(ms.ToArray());
return JsonConvert.DeserializeObject<T>(data);
Related
i'm trying to read .gz files using binary reader by first unzipping with gzipstream, and then creating a new binary reader with the gzipstream. however, when i try to use the BaseStream.Position and BaseStream.Length of BinaryReader (to know when i'm at the end of my file), i get a NotSupportedException, checking the doc for these fields in GZipStream Class shows:
Length
This property is not supported and always throws a NotSupportedException.(Overrides Stream.Length.)
Position
This property is not supported and always throws a NotSupportedException.(Overrides Stream.Position.)
so my question is how can i know when i'm at the end of my file when reading a decompressed GZipStream using BinaryReader? thanks
here is my code:
Stream stream = new MemoryStream(textAsset.bytes);
GZipStream zippedStream = new GZipStream(stream, CompressionMode.Decompress);
using (BinaryReader reader = new BinaryReader(zippedStream))
while(reader.BaseStream.Position != reader.BaseStream.Length)
{
//do stuff with BinaryReader
}
the above throws:
NotSupportedException: Operation is not supported. System.IO.Compression.DeflateStream.get_Position()
due to the BaseStream.Position call in the while()
You can copy your zippedStream to MemoryStream instance, that can be read fully using ToArray function. That is the easiest solution I can think of.
Stream stream = new MemoryStream(textAsset.bytes);
byte[] result;
using (GZipStream zippedStream = new GZipStream(stream, CompressionMode.Decompress))
{
using (MemoryStream reader = new MemoryStream())
{
zippedStream.CopyTo(reader);
result = reader.ToArray();
}
}
Alternatively if you want to read stream in chunks
using (GZipStream zippedStream = new GZipStream(stream, CompressionMode.Decompress))
{
byte[] buffer = new byte[16 * 1024];
int read;
while ((read = zippedStream.Read(buffer, 0, buffer.Length)) > 0)
{
// do work
}
}
Depending on what you are decoding you can read the first type into a byte array using the BinaryReader and then use BitConverter to convert these bytes into the type you want. You can then use BinaryReader as normal until the start of the next record.
byte[] headermarker = new byte[4];
int count;
// if bytes available in underlying stream.
while ((count = br.Read(headermarker, 0, 4) > 0 )
{
Int32 marker = BitConverter.ToInt32(headermarker, 0);
//
// now use Binary Reader for the rest of the record until we loop
//
}
Does anyone know why I'm getting the "Unexpected end of data" error message when un-gzipping the gzip file?
To verify the bytes data is not corrupted, I use the FooTest4.csv to write to file and was able to opened the file successfully.
Both 'FooTest3.csv.gzand 'FooTest2.csv.gz ran into "Unexpected end of data" when un-gzipping.
public static List<byte> CompressFile(List<byte> parmRawBytes)
{
//Initialize variables...
List<byte> returnModifiedBytes = null;
File.WriteAllBytes(#"X:\FooTest4.csv", parmRawBytes.ToArray());
using (var memoryStream = new MemoryStream())
{
using (var gzipStream = new GZipStream(memoryStream, CompressionMode.Compress, false))
{
gzipStream.Write(parmRawBytes.ToArray(), 0, parmRawBytes.ToArray().Length);
gzipStream.Flush();
File.WriteAllBytes(#"X:\FooTest3.csv.gz", memoryStream.ToArray());
returnModifiedBytes = memoryStream.ToArray().ToList();
}
}
File.WriteAllBytes(#"X:\FooTest2.csv.gz", returnModifiedBytes.ToArray());
return returnModifiedBytes;
}
GZipStream needs to be closed so it can write some terminating data to the end of the buffer to complete the gzip encoding.
byte[] inputBytes = ...;
using (var compressedStream = new MemoryStream())
{
using (var compressor = new GZipStream(compressedStream, CompressionMode.Compress))
{
compressor.Write(inputBytes, 0, inputBytes.Length);
}
// get bytes after the gzip stream is closed
File.WriteAllBytes(pathToFile, compressedStream.ToArray());
}
Instead of loading the bytes, compressing and saving them you could do do compression and writing at once. Also I don't know why you're using List<Byte> instead of byte[], maybe this could be it.
void CompressFile(string inputPath, string outputPath)
{
Stream readStream = new FileStream(inputPath, Filemode.Open);
Stream writeStream = new FileStream(outputPath, FileMode.Create);
Stream compressionStream = new GZipStream(writeStream. CompressionMode.Compress);
byte[] data = new byte[readStream.Length];
readStream.Read(data, 0, data.Length);
compressionStream.Write(data, 0, data.Length);
readStream.Close();
writeStream.Close();
}
byte[] CompressFile(string inputPath)
{
byte[] data = File.ReadAllBytes(inputPath);
MemoryStream memStream = new MemoryStream(data);
var gzipStream = new GZipStream(memStream, CompressionMode.Compress);
gzipStream.Write(data, 0, data.Length);
gzipStream.Close();
return gzipStream.ToArray();
}
PS: I wrote the code in the text editor, so there might be errors. Also you say the error is on the "unzippiing", why no show us the unzip code?
I am testing some code. I am stuck with the following. What ever I write as text, the length of the zipped stream is always 10? What am I doing wrong?
var inStream = new MemoryStream();
var inWriter = new StreamWriter(inStream);
str text = "HelloWorldsasdfghj123455667880fgsjfhdfasdferrbvbyjun hbwecwcxqsz edcrgvebrjnuj5juerqwetsrgfggshurhtnbvzkfjhguhgrgal;kjhao;rhl;zkfhg;aorihghg;oahrgarhguhh';aaeaeiaijeihjrhfidfhfidfidhh953453453";
inWriter.WriteLine(text);
inWriter.Flush();
inStream.Position = 0;
var outStream = new MemoryStream();
var compressStream = new GZipStream(outStream, CompressionMode.Compress);
inStream.CopyTo(compressStream);
compressStream.Flush();
outStream.Flush();
compressStream.Flush();
outStream.Position = 0;
Console.WriteLine(outStream.Position);
Console.WriteLine(outStream.Length);
Until you Close it the compression stream doesn't know you've finished writing to it - so cannot complete its compression algorithm. Flushing flushes those parts it can flush, but until its been told you have completed adding new bytes it cannot flush its last package of compressed data.
so, I am editing a word document, using OpenXML. And for some reasons, I convert it all into a string:
//conversion du byte en memorystream
using (var file = new MemoryStream(text))
using (var reader = new StreamReader(file))
{
WordprocessingDocument wordDoc = WordprocessingDocument.Open(file, true);
using (StreamReader sr = new StreamReader(wordDoc.MainDocumentPart.GetStream()))
{
docText = sr.ReadToEnd();
}
}
And then, I convert it as a byte.
But, a simple convert will not work:
byte[] back2Byte = System.Text.Encoding.ASCII.GetBytes(docText );
Because the string is a open xml string.
Tried this, but always got a corrupted file when I tried to open it with Word:
var repo = new System.IO.MemoryStream(System.Text.Encoding.UTF8.GetBytes(docText));
byte[] buffer = new byte[16 * 1024];
MemoryStream ms = new MemoryStream();
int read;
while ((read = repo.Read(buffer, 0, buffer.Length)) > 0)
{
ms.Write(buffer, 0, read);
}
byte[] back2Byte = ms.ToArray();
So, this doesn't work either:
byte[] back2Byte = new byte[docText.Length * sizeof(char)];
System.Buffer.BlockCopy(docText.ToCharArray(), 0, back2Byte, 0, back2Byte.Length);
edit : After some checkings, it seems it is write as a openxml document into the database, and so, word cannot read it. There is no error when i open it with notepad
How can I correct this?
So, the real issue is, how can I convert a OpenXML string to a byte that can be open in word?
You cannot do this sort of thing. You are getting the bytes for only one part of an OpenXML document. By definition, all Microsoft Office documents are multi-part OpenXML documents. You could theoretically capture the bytes for all the parts using a technique like you're currently using, but you would also have to capture all the part/relationship information necessary to reconstruct the multi-part document. You'd be better off just reading all the bytes of the file and storing them as-is:
// to read the file as bytes
var fileName = #"C:\path\to\the\file.xlsx";
var fileBytes = File.ReadAllBytes(fileName);
// to recreate the file from the bytes
File.WriteAllBytes(fileName, fileBytes)
If you need a string form of those bytes, try this:
// to convert bytes to a (non-readable) text form
var fileContent = Convert.ToBase64String(fileBytes);
// to convert base-64 back to bytes
var fileBytes = Convert.FromBase64String(fileContent);
Either way, there is absolutely no need to use the OpenXML SDK for your use case.
I am trying to read an Http response stream twice via the following:
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
stream = response.GetResponseStream();
RssReader reader = new RssReader(stream);
do
{
element = reader.Read();
if (element is RssChannel)
{
feed.Channels.Add((RssChannel)element);
}
} while (element != null);
StreamReader sr = new StreamReader(stream);
feed._FeedRawData = sr.ReadToEnd();
However when the StreamReader code executes there is no data returned because the stream has now reached the end. I tried to reset the stream via stream.Position = 0 but this throws an exception (I think because the stream can't have its position changed manually).
Basically, I would like to parse the stream for XML and have access to the raw data (in string format).
Any ideas?
Copy it into a new MemoryStream first. Then you can re-read the MemoryStream as many times as you like:
Stream responseStream = CopyAndClose(resp.GetResponseStream());
// Do something with the stream
responseStream.Position = 0;
// Do something with the stream again
private static Stream CopyAndClose(Stream inputStream)
{
const int readSize = 256;
byte[] buffer = new byte[readSize];
MemoryStream ms = new MemoryStream();
int count = inputStream.Read(buffer, 0, readSize);
while (count > 0)
{
ms.Write(buffer, 0, count);
count = inputStream.Read(buffer, 0, readSize);
}
ms.Position = 0;
inputStream.Close();
return ms;
}
Copying the stream to a MemoryStream as suggested by Iain is the right approach. But since
.NET Framework 4 (released 2010) we have Stream.CopyTo. Example from the docs:
// Create the streams.
MemoryStream destination = new MemoryStream();
using (FileStream source = File.Open(#"c:\temp\data.dat",
FileMode.Open))
{
Console.WriteLine("Source length: {0}", source.Length.ToString());
// Copy source to destination.
source.CopyTo(destination);
}
Console.WriteLine("Destination length: {0}", destination.Length.ToString());
Afterwards you can read destination as many times as you like:
// re-set to beginning and convert stream to string
destination.Position = 0;
StreamReader streamReader = new StreamReader(destination);
string text = streamReader.ReadToEnd();
// re-set to beginning and read again
destination.Position = 0;
RssReader cssReader = new RssReader(destination);
(I have seen Endy's comment but since it is an appropriate, current answer, it should have its own answer entry.)
have you tried resetting the stream position?
if this does not work you can copy the stream to a MemoryStream and there you can reset the position (i.e. to 0) as often as you want.