BinaryReader does not have EndOfStream property. Is it safe to use following code to check if end of stream is reached?
reader.BaseStream.Length>reader.BaseStream.Position
The easiest way I found is to check the returned value of the BinaryReader's PeekChar() method. If it returns -1, then you reached the end of the stream.
It depends. There are various stream types that do not implement the Length or Position property, you'd get a NotSupportedException. NetworkStream for example. Of course, if you'd use such a stream then you really do have to know up front how often to call the BinaryReader.Read() method. So, yes, it's fine.
This won't work as a general solution because it assumes that the BaseStream value supports the Length property. Many Stream implementation do not and instead throw a NotSupportedException. In particular any networking base stream such as HttpRequestStream and NetworkStream
I've noticed that comparing Position to Length DOES NOT work on StreamReader even if the underlying BaseStream supports seeking. It seems that StreamReader buffers read-ahead from the BaseStream. This must be why StreamReader supplies an EndOfStream property, which is a good thing, and I wish BinaryReader did the same.
Checking these values (Length and Position) on the underlying base stream counts on BinaryReader to not behave as StreamReader does, i.e. relies on BinaryReader only grabbing the exact number of bytes from BaseStream needed to fulfill a user method call. Presumably if BinaryReader in fact operates this way internally it is why it does not need to supply an EndOfStream, but I sure wish it did supply one so that I knew that end of file was being correctly handled for clients in an implementation independent way.
Of course Readers are not Streams, but with respect to end of file behavior it would be nice if there were a common interface that enabled clients of input/output classes to know if A. end of file is a sensible concept for the underlying source of data, and B. when end of file occurs if A is sensible.
Check the Streams CanSeek property. If this property returns true then you can compare the streams Length to the stream's Position to tell if you are at the end of the stream. If this property returns false then this won't work.
For Network streams you may need to distinguish between the end of the available bytes (the client on the other end still have more to write but hasn't yet) and the stream being closed. The IsConnected property for an underlying Tcp connection isn't reliable for knowing when the stream has closed. It is possible to enumerate the connections that the computer has and see if the stream you are using is among them. This is more reliable, but more complex. It may be better to just handle IOExceptions when you can't read any
Related
I have been using this code to capture the webcam and I have been trying to learn from it and make it better. Rider IDE suggested I should use an async variant of MemoryMappedViewStream.Read but it doesn't work at all. It produces all-black images suggesting the async and sync methods are totally different. I am wondering why that's the case?
// Working:
sourceStream.Read(MemoryMarshal.AsBytes(image.GetPixelMemoryGroup().Single().Span));
// NOT Working:
var bytes = MemoryMarshal.AsBytes(image.GetPixelMemoryGroup().Single().Span).ToArray();
await sourceStream.ReadAsync(bytes, 0, bytes.Length, token);
Repository and line of code
Those two versions are not the same. In "sync" version you obtain a reference to memory location of an image via image.GetPixelMemoryGroup(). Then you read data from sourceStream directly into that location.
In "async" version you again obtain reference to memory location via image.GetPixelMemoryGroup but then you do something different - you call ToArray. This extension method copies bytes from image memory location into new array, the one you hold in bytes variable. You then read data from sourceStream into that bytes array, NOT directly into image memory locaiton. Then you discard bytes array, so you read them to nowhere basically.
Now,MemoryMappedViewStream inherits from UnmanagedMemoryStream and all read\write operations are implemented in UnmanagedMemoryStream. This kind of stream represents data in memory and there is nothing async it can do. The only reason it even has ReadAsync is because base stream class (Stream) has those methods. Even if you manage to make ReadAsync work - in this case it will not be asynchornous anyway. As far as I know - MemoryMappedViewStream does now allow real asynchronous access, even though it could make sense, since it has underlying file.
In short - I'd just continue with sync version, because there is no benefit in this case to use "async" one. Static analyzer of course doesn't know that, it only sees that there is Async-named analog of the method you use.
await sourceStream.ReadAsync(bytes, 0, bytes.Length, token).ConfigureAwait(false);
Check like this
Given a stream by a user such that we expect them to manage the disposal of it through typical using
using(var stream = new MemoryStream())
{
MyMethod(stream);
}
Is there any risk to copying back to the stream after working on it. Specifically we have a method that populates the data, but we have a conditional need to sort the data. So MyMethod is something like this:
void MyMethod(Stream stream, bool sort = false)
{
//Stream is populated
stream.Position = 0;
if(sort)
{
Sort(stream);
}
}
void Sort(Stream stream)
{
using(var sortedStream = new MemoryStream)
{
//Sort per requirements into the new sorted local stream
sortedStream.Position = 0;
//Is this safe? Any risk of losing data or memory leak?
sortedStream.CopyTo(stream);
}
}
The thing to notice is we are populating the stream provided by the user and then sorting it into a local stream. Since the local stream is owned by the local method it is cleaned up but in converse we can NOT clean up the provided stream, but want to populate it with the local results.
To reiterate my question, is there anything wrong with this? Is there a risk of garbage data being in the stream or some other issue I am not thinking of?
Stream is an abstract class, and has a lot of different implementations. Not all streams can be written to, so in some cases the code may not work as expected, or could crash.
sortedStream.Position = 0;
sortedStream.CopyTo(stream);
You would need to check the CanSeek and CanWrite properties beforehand:
if (sortedStream.CanSeek & stream.CanWrite)
{
sortedStream.Position = 0;
sortedStream.CopyTo(stream);
}
else
{
// not supported
}
Whether a given stream support moving the position around and re-writing data over itself is going to depend on the specific stream. Some support it, and some don't. Not all streams are allowed to change their position, not all are able to write, not all are able to overwrite existing data, and some are able to do all of those things.
A well behaved stream shouldn't leak resources if you do any of those unsupported things; it ought to just throw an exception, but of course technically a custom stream could do whatever it wants, so you most certainly could write your own stream that leaks resources when changing the position. But of course at that point the bug of leaking a resource is in that stream's implementation, not in your code that sorts the data in the stream. The code you've shown here only needs to worry about a stream throwing an exception if an unsupported operation is performed.
I have no idea why you don't sort it before you insert it into the stream or why you use a stream at all when you access seems to be random-access, but technically, it's fine. You can do it. It will work.
I am trying to use the EndianBinaryReader and EndianBinaryWriter that Jon Skeet wrote as part of his misc utils lib. It works great for the two uses I have made of it.
The first reading from a Network Stream (TCPClient) where I sit in a loop reading the data as it comes in. I can create a single EndianBinaryReader and then just dispose of it on the shut down of the application. I construct the EndianBinaryReader by passing the TCPClient.GetStream in.
I am now trying to do the same thing when reading from a UdpClient but this does not have a stream as it is connection less. so I get the data like so
byte[] data = udpClientSnapShot.Receive(ref endpoint);
I could put this data into a memory stream
var memoryStream = new MemoryStream(data);
and then create the EndianBinaryReader
var endianbinaryReader = new EndianBinaryReader(
new BigEndianBitConverter(), memoryStream,Encoding.ASCII);
but this means I have to create a new endian reader every time I do a read. Id there a way where I can just create a single stream that I can just keep updateing the inputstream with the data from the udp client?
I can't remember whether EndianBinaryReader buffers - you could overwrite a single MemoryStream? But to be honest there is very little overhead from an extra object here. How big are the packets? (putting it into a MemoryStream will clone the byte[]).
I'd be tempted to use the simplest thing that works and see if there is a real problem. Probably the one change I would make is to introduce using (since they are IDisposable):
using(var memoryStream = new MemoryStream(data))
using(var endianbinaryReader = ..blah..) {
// use it
}
Your best option is probably an override of the .NET Stream class to provide your custom functionality. The class is designed to be overridable with custom behavior.
It may look daunting because of the number of members, but it is easier than it looks. There are a number of boolean properties like "CanWrite", etc. Override them and have them all return "false" except for the functionality that your reader needs (probably CanRead is the only one you need to be true.)
Then, just override all of the methods that start with the phrase "When overridden in a derived class" in the help for Stream and have the unsupported methods return an "UnsupportedException" (instead of the default "NotImplementedException".
Implement the Read method to return data from your buffered UDP packets using perhaps a linked list of buffers, setting used buffers to "null" as you read past them so that the memory footprint doesn't grow unbounded.
I'm just curious about this. It strikes me that the behavior of a StringBuilder is functionally (if not technically) the same as a Stream -- it's a bin of data to which other data can be added.
Again, just curious.
Stream is an input and output of binary data.
StringBuilder is means of building up text data.
Beyond that, there's the issue of state - a StringBuilder just has the current value, with no idea of "position". It allows you to access and mutate data anywhere within it. A stream, on the other hand, is logically a potentially infinite stream of data, with a cursor somewhere in the middle to say where you've got to. You generally just read/write forwards, with Seek/Position to skip to a specific part of the data stream.
Try to imagine implementing the Stream API with StringBuilder... it just doesn't fit. You could sort of do it, but you'd end up with StringReader and StringWriter, basically.
StringBuilder has more than just Append functions. It also has insert functions which is unnatural for a stream. Use the StringWriter class if you want a stream that wraps a StringBuilder.
A stream normally refers to an external input/output source (file, network). StringBuilder has no such characteristic.
Because it's not really a stream. It's more of a buffer that grows.
While both can have data added to them, the functionality as a whole is different.
A Stream is for inputting or outputting data from/to some source, not for building something. StringBuilder does not need functionality that Stream provides, like Buffering, etc to build a resource.
On the other hand you will find the classes StringReader/Writer in System.IO. The StringWriter e.g. implements TextWriter against an underlying StringBuilder.
Personally I've never used it but if you have a text file writing routine you could make it work against a TextWriter. Then in your test, instead of instantiating a StreamWriter you instantiate a StringWriter and you could then check what was written by looking at the underlying StringBuilder.
Now I'm dizzy...
This is untested as i need to write more code. But is this correct and i feel like i am missing something, like this could be better written. Do i need the c.lose at the end? should i flush anything(i'll assume no if i do close())?
Byte[] buffer;
using (var m = new MemoryStream())
{
using (var binWriter = new BinaryWriter(m))
{
foreach (var v in wordIDs)
binWriter.Write(v);
binWriter.Close();
}
buffer = m.GetBuffer();
m.Close();
}
You don't need the .Close() calls (the automatic .Dispose() the using block generates takes care of those).
Also, you'll want to use .ToArray() on the MemoryStream, not .GetBuffer(). GetBuffer() returns the underlying buffer, no matter how much of it is used. ToArray() returns a copy that is the perfect length.
If you're using this to communicate with another program, make sure you and it agree on the order of the bytes (aka endianness). If you're using network byte-order, you'll need to flip the order of the bytes (using something like IPAddress.HostToNetworkOrder()), as network byte-order is big-endian, and BinaryWriter uses little-endian.
What is wordIDs, is it an enumeration or is it an Int32[]? You can use the following if it is just Int32[]:
byte[] bytes = new byte[wordIDs.Length * 4];
Buffer.BlockCopy(wordIDs, 0, bytes, 0, bytes.Length);
Otherwise, if wordIDs is an enumeration that you must step through, all you need to change is remove the m.Close (as mentioned) and use MemoryStream.ToArray (as mentioned).
Close is not needed here. The using statements will ensure the Dispose method on these types are called on exit and this will have the same effect as calling Close. In fact if you look at the code in reflector, you'll find that Close in both cases just proxies off to the Dispose method on both types.
Thus sayeth Skeet:
There's no real need to close either
a MemoryStream or a BinaryWriter, but
I think it's good form to use a using
statement to dispose of both - that
way if you change at a later date to
use something that really does need
disposing, it will fit into the same
code.
So you don't need the Close or the using statement, but using is idiomatic C#.
JaredPar's and Jonathan's answers are correct. If you want an alternative, you use BitConverter.GetBytes(int). So now your code turns into this
wordIDs.SelectMany(i => BitConverter.GetBytes(i));
I disagree with the Skeet here.
Whilst you may not need close by using using you are relying on the implementation of BinaryWriter and MemoryStream to do it for you in the Dispose method. This is true for framework types, but what if someone writes a Writer or Stream which doesn't do it?
Adding close does no harm and protects you against badly written classes.