I've this class
class CacheHelper() {
private Dictionary<string, MemoryStream> cacher;
// ... other porps, f's...etc
public MemoryStream GetImageStream(string fileName)
{
if (!cacher.ContainsKey(fileName))
return null;
MemoryStream memStream = null;
cacher.TryGetValue(fileName, out memStream); // TODO
return memStream;
}
}
and I'm using it like this:
Stream fileStream = _cacheHelper.GetImageStream(filePath);
and When I'm done I'm closing fileStream like this:
if(fileStream!=null)
fileStream.Dispose();
I'm not sure what's going on underneath Stream implementation in C#, so I'm afraid that I'm closing the original MemoryStream (the one inside the internal cacher Dictionary) if I'm closing fileStream, i.e. implemented on top of pointers, or something.
A MemoryStream is a class. All classes are reference types which means that the variable you have is indeed a kind of pointer to the actual instance. What happens is that you pass a reference of your memory stream somewhere. If you don't want to close that stream, you should not do so.
A better implementation might be to either cache byte arrays or handle everything using streams inside the cache itself. Passing a stateful object from your cache to somewhere it's used and expecting it to keep it's original state is not such a good design. It's very easy to make mistakes that way.
Related
I need to return a MemoryStream, and currently I do it like this:
String ICallbackEventHandler.GetCallbackResult()
{
using (var stream = new MemoryStream())
{
this._synth.Rate = this.Rate;
this._synth.Volume = this.Volume;
this._synth.SetOutputToWaveStream(stream);
string sText = this.Context.Items["data"].ToString();
this._synth.Speak(sText);
//rewind to beginning of stream
stream.Seek(0, SeekOrigin.Begin);
using (var nMp3Stream = new MemoryStream())
using (var rdr = new WaveFileReader(stream))
using (var wtr = new LameMP3FileWriter(nMp3Stream, rdr.WaveFormat, LAMEPreset.STANDARD))
{
rdr.CopyTo(wtr);
nMp3Stream.Position = 0;
return (String.Concat("data:audio/mpeg;base64,", Convert.ToBase64String(nMp3Stream.ToArray())));
}
}
}
But I don't think that's the proper way. I guess I shouldn't return a using variable, right?
If not, how could I do this? I need to dispose the MemoryStream, I think. When should I do this, or should I let the GC do this?
I guess I shouldn't return a using variable, right?
You don't return "using var" at all.
You convert content of it into the string , and return that string instead.
return (String.Concat("data:audio/mpeg;base64,",
Convert.ToBase64String(nMp3Stream.ToArray())));
Here you create a new instance of a string, fill it with content of MemoryStream and return that instance.
I need to dispose the MemoryStream, I think.
Yes, you have to think about lifetime of your instances.
When should I do this, or should I let the GC do this?
In most scenarios it's you who have take care of it and you're doing it proper way, at least judging from the code sample provided.
But, a lot depends on your concrete application design and execution dynamics.
For example:
consider performance implication of converting memory stream into the string
consider that at the moment of conversion you allocate almost 2x memory: one for MemoryStream another for string
You are not actually returning the using variable in this code. The ToArray() call created a new array object with memory separate from the nMp3Stream object. Additionally, Convert.ToBase64String() created a new string object separate from the array, and String.Concat() created yet another string separate from the first string.
So while I question the efficiency of this code, especially as it relates to address exhaustion via the Garbage Collector Large Object Heap, it's certainly not going to run into any problems with using.
If you're interested in fixing the performance issues, that's a much bigger question which is likely to involve changing how this method is used in the first place.
For the situations where you really do want to return the subject of a using block from a method, the pattern is typically to remove the using block, and instead move that block to the caller. In the case where the using subject needs a longer life span, perhaps as a class member, then the class should be written to implement IDisposable, so class instances can themselves be the subject of a using block.
Given a stream by a user such that we expect them to manage the disposal of it through typical using
using(var stream = new MemoryStream())
{
MyMethod(stream);
}
Is there any risk to copying back to the stream after working on it. Specifically we have a method that populates the data, but we have a conditional need to sort the data. So MyMethod is something like this:
void MyMethod(Stream stream, bool sort = false)
{
//Stream is populated
stream.Position = 0;
if(sort)
{
Sort(stream);
}
}
void Sort(Stream stream)
{
using(var sortedStream = new MemoryStream)
{
//Sort per requirements into the new sorted local stream
sortedStream.Position = 0;
//Is this safe? Any risk of losing data or memory leak?
sortedStream.CopyTo(stream);
}
}
The thing to notice is we are populating the stream provided by the user and then sorting it into a local stream. Since the local stream is owned by the local method it is cleaned up but in converse we can NOT clean up the provided stream, but want to populate it with the local results.
To reiterate my question, is there anything wrong with this? Is there a risk of garbage data being in the stream or some other issue I am not thinking of?
Stream is an abstract class, and has a lot of different implementations. Not all streams can be written to, so in some cases the code may not work as expected, or could crash.
sortedStream.Position = 0;
sortedStream.CopyTo(stream);
You would need to check the CanSeek and CanWrite properties beforehand:
if (sortedStream.CanSeek & stream.CanWrite)
{
sortedStream.Position = 0;
sortedStream.CopyTo(stream);
}
else
{
// not supported
}
Whether a given stream support moving the position around and re-writing data over itself is going to depend on the specific stream. Some support it, and some don't. Not all streams are allowed to change their position, not all are able to write, not all are able to overwrite existing data, and some are able to do all of those things.
A well behaved stream shouldn't leak resources if you do any of those unsupported things; it ought to just throw an exception, but of course technically a custom stream could do whatever it wants, so you most certainly could write your own stream that leaks resources when changing the position. But of course at that point the bug of leaking a resource is in that stream's implementation, not in your code that sorts the data in the stream. The code you've shown here only needs to worry about a stream throwing an exception if an unsupported operation is performed.
I have no idea why you don't sort it before you insert it into the stream or why you use a stream at all when you access seems to be random-access, but technically, it's fine. You can do it. It will work.
I'm working with Ionic.Zlib.DeflateStream (I think aka DotNetZip) in C# code and notice it doesn't have a BaseStream property like System.IO.Compression.DeflateStream does. Is there any simple way to access this? Maybe a partial class or extension (not really familiar with those concepts) or just something I'm overlooking, or an updated version of this library?
Update: I have function deep inside a large project that is given an Ionic.Zlib.DeflateStream as a paramater. I know that the underlying stream is a MemoryStream, and I want to modify the code to seek to Position 0 in the underlying stream, write a few bytes, then return to the previos Position. This is what we call a "kludge", or dirty-hack, as opposed to rewriting a lot of code... but this is the solution we are looking for at this time, as opposed to something else that would require more retesting. The few bytes in this part of the MemoryStream that need to be updated are not compressed, so modifying them outside the DeflateStream in this matter is fine.
I'd still like to know other options for future projects, or if this answer could cause issues, but I think I did find one option...
When I create the object like this:
MemoryStream ms = new MemoryStream();
DeflateStream ds = new DeflateStream(ms,...);
If instead I create a class like:
class MyDeflateStream : DeflateStream
{
public MemoryStream RootStream;
}
I can change the above code to:
MemoryStream ms = new MemoryStream();
MyDeflateStream ds = new MyDeflateStream (ms,...);
ds.RootStream = ms;
Then make the function where I need access to it something like this:
void Whatever(DeflateStream ds)
{
MyDeflateStream mds = (MyDeflateStream)ds;
MemoryStream ms = mds.RootStream;
}
Ideally I'd only have to modify the Whatever() function, because sometimes I might not have access to the code that created the object in the first place, but in this case I do. So still hoping for an answer, even though I found one possible way to handle this.
I am trying to use the EndianBinaryReader and EndianBinaryWriter that Jon Skeet wrote as part of his misc utils lib. It works great for the two uses I have made of it.
The first reading from a Network Stream (TCPClient) where I sit in a loop reading the data as it comes in. I can create a single EndianBinaryReader and then just dispose of it on the shut down of the application. I construct the EndianBinaryReader by passing the TCPClient.GetStream in.
I am now trying to do the same thing when reading from a UdpClient but this does not have a stream as it is connection less. so I get the data like so
byte[] data = udpClientSnapShot.Receive(ref endpoint);
I could put this data into a memory stream
var memoryStream = new MemoryStream(data);
and then create the EndianBinaryReader
var endianbinaryReader = new EndianBinaryReader(
new BigEndianBitConverter(), memoryStream,Encoding.ASCII);
but this means I have to create a new endian reader every time I do a read. Id there a way where I can just create a single stream that I can just keep updateing the inputstream with the data from the udp client?
I can't remember whether EndianBinaryReader buffers - you could overwrite a single MemoryStream? But to be honest there is very little overhead from an extra object here. How big are the packets? (putting it into a MemoryStream will clone the byte[]).
I'd be tempted to use the simplest thing that works and see if there is a real problem. Probably the one change I would make is to introduce using (since they are IDisposable):
using(var memoryStream = new MemoryStream(data))
using(var endianbinaryReader = ..blah..) {
// use it
}
Your best option is probably an override of the .NET Stream class to provide your custom functionality. The class is designed to be overridable with custom behavior.
It may look daunting because of the number of members, but it is easier than it looks. There are a number of boolean properties like "CanWrite", etc. Override them and have them all return "false" except for the functionality that your reader needs (probably CanRead is the only one you need to be true.)
Then, just override all of the methods that start with the phrase "When overridden in a derived class" in the help for Stream and have the unsupported methods return an "UnsupportedException" (instead of the default "NotImplementedException".
Implement the Read method to return data from your buffered UDP packets using perhaps a linked list of buffers, setting used buffers to "null" as you read past them so that the memory footprint doesn't grow unbounded.
As discussed before, when a BinaryReader or BinaryWriter gets closed, its underlying Stream get closed as well (aargh). Consider this situation: a routine R is passed a MemoryStream, say M; I would like to write some stuff to M and then pass it to another routine for more processing (not necessarily writing). For convenience, I'd like to wrap M in a BinaryWriter to do my writing. After writing, I'm done with the BinaryWriter but not with M.
void R(MemoryStream M)
{
using (B = new BinaryWriter(M))
{
// write some stuff using B
}
S(M); // now pass M to another routine for further processing
}
But, I can't dispose of the BinaryStream without closing M.
Q: Is there a way to do any of the following?
extract the underlying byte[] from a MemoryStream,
clone a Stream
reopen a Stream after it's been closed
You should better get the underlying byte[] buffer using
byte[] buffer = ms.GetBuffer();
And then copy the byte data using the Array.Copy() method.
You are free to create a new stream with it.
You can use things like the MiscUtil.IO.NonClosingStreamWrapper in MiscUtil, which wraps a Stream and simply ignores Close/Dispose requests. For just this purpose.
void R(MemoryStream M)
{
using (B = new BinaryWriter(new NonClosingStreamWrapper(M)))
{
// write some stuff using B
}
S(M); // now pass M to another routine for further processing
}
You can:
Call M.ToArray() to get the stream as an array of bytes.
Subclass BinaryWriter and override the Dispose method to prevent closing of the child stream
Thanks to several who suggested ToArray, I was led to right answer, which is `M.GetBuffer'. ToArray is not too bad, but it
makes a copy
gets only part of the buffer
GetBuffer just grabs a reference to the underlying byte[], which is what I'm after.
Just to add it in here, a very simple solution would be not to Dispose() the writer.
void R(MemoryStream M)
{
B = new BinaryWriter(M);
// write some stuff using B
B.Flush();
S(M); // now pass M to another routine for further processing
}
Now you only have to worry about keeping B in scope, which it will be during R().
This may not be the best solution here, but it is worth noting that the Readers and Writers don't need Disposing themselves.
A somewhat naive approach is to use
byte buf[] = MemoryStream.ToArray();
To copy the stream contents to a byte array. You can turn it back into a stream with
MemoryStream ms = new MemoryStream(buf);
Accoring to this M.Clone(); should work. But i may be wrong...