Develop Reference

Calling OpenReadStream of an IFormFile multiple times

I need to have the stream of the file in 2 different locations. In the code the IFormFile is already passed as parameter in the 2 methods. I thought of either modifying the methods and calling the OpenReadStream in the beginning and pass the stream as param or calling OpenReadStream separately.
I inspected the dissasembled code and OpenReadStream does this:
return new ReferenceReadStream(_baseStream, _baseStreamOffset, Length);
and the ReferenceReadStream class does this in the constructor:
public ReferenceReadStream(Stream inner, long offset, long length)
{
if (inner == null)
{
throw new ArgumentNullException("inner");
}
_inner = inner;
_innerOffset = offset;
_length = length;
_inner.Position = offset;
}
In my understanding the base stream is the same and it doesn't matter calling OpenReadStream multiple times.
What worries me is if I'll run into problems when I start using Seek method.
Does anyone know what's the correct usage of OpenReadStream in this senario?

Apparently it's not safe to call OpenReadStream multiple times.
When Read method is called, it calls this method:
private void VerifyPosition()
{
if (_inner.Position == _innerOffset + _position)
{
return;
}
throw new InvalidOperationException("The inner stream position has changed unexpectedly.");
}
I was able to trigger this exception with the following code:
var s = file.OpenReadStream();
s.Seek(10, SeekOrigin.Begin);
var b = new byte[2];
var c = s.Read(b);
var s2 = file.OpenReadStream();
c = s.Read(b);

How to raise a delegate event using a ref parameter in NSubstitute 3.1?

I'm working on a C# project that uses a 3rd party library. This library defines a rather unusual delegate event using a ref parameter:
event GetDataHandler OnGetData;
public delegate bool GetDataHandler(string name, ref byte[] data);
I'm trying to raise this event in a unit test via NSubstitute (version 3.1) but I can't get it to work. I tried this code (and basically every variation of it that I could think of):
var theKey = "test";
byte[] theData = null;
_theObject.OnGetData += Raise.Event<GetDataHandler>(theKey, ref theData);
But this does not compile. The compiler says: Argument 2 may not be passed with the 'ref' keyword.
I'm aware that the out/ref mechanism has changed with NSubstitute 4.x but my company has not upgraded to the newer version yet.
Is there any way to get this up and running using NSubstitute 3.1? Thanks a lot!
Best regards,
Oliver

That Raise.Event overload takes parameters as a params object[]. We can pass the byref byte array as a standard value in this params array (which means we don't get compile-time safety for the event args we're passing, but the test will pick this up pretty quickly if we get it wrong :) ):
_theObject.OnGetData += Raise.Event<GetDataHandler>("name", theData);
Here's an executable example:
using NSubstitute;
using Xunit;
public delegate bool GetDataHandler(string name, ref byte[] data);
public interface ISomeType {
event GetDataHandler OnGetData;
}
public class SampleFixture {
string lastNameUsed = "";
byte[] lastBytesUsed = new byte[0];
[Fact]
public void SomeTest() {
var sub = Substitute.For<ISomeType>();
var data = new byte[] { 0x42 };
sub.OnGetData += Sub_OnGetData;
sub.OnGetData += Raise.Event<GetDataHandler>("name", data);
Assert.Equal("name", lastNameUsed);
Assert.Equal(data, lastBytesUsed);
}
private bool Sub_OnGetData(string name, ref byte[] data) {
lastNameUsed = name;
lastBytesUsed = data;
return true;
}
}
Edit after more info provided in comment.
I don't think NSubstitute supports inspecting the value that comes back in this case.
Without knowing exactly what you're trying to test, I can suggest a couple of general approaches for testing this sort of thing.
First option is to hand code a test double (in this case an implementation of ISomeType) that you have full control over. Unless the interface is huge, I'd recommend this approach.
Another option is to test the delegate and the wire-up separately. For example, given this class:
public class ClassUnderTest {
public ClassUnderTest(ISomeType dep) {
dep.OnGetData += Dep_OnGetData;
}
public static bool Dep_OnGetData(string name, ref byte[] data) {
data = System.Text.Encoding.UTF8.GetBytes(name);
return true;
}
}
We can test the delegate independently, and then test we've hooked up that delegate:
[Fact]
public void TestDelegate() {
byte[] data = new byte[0];
var result = ClassUnderTest.Dep_OnGetData("hi", ref data);
Assert.True(result);
Assert.Equal(new byte[] { 104, 105 }, data);
}
[Fact]
public void TestWireup() {
var sub = Substitute.For<ISomeType>();
var subject = new ClassUnderTest(sub);
sub.Received().OnGetData += ClassUnderTest.Dep_OnGetData;
}
I think the delegate test in this case is potentially very useful, but the wire-up test is probably not great because it is very specific to the particular implementation, rather than the behaviour/outcome required. But in this case observing the particular effect is difficult, so it is a potential answer.
Thirdly, we may be able to use a more testable wrapper over the library in question. Or this test may be at the wrong level entirely -- be wary of mocking types we don't own. (I've written a bit about this here.)
If you can provide a little more info on what you're trying to test in this scenario I'm happy to try to come up with a more reasonable answer. :)

C# Using span with SocketAsyncEventArgs

I would like to use new Span to send unmanaged data straight to the socket using SocketAsyncEventArgs but it seems that SocketAsyncEventArgs can only accept Memory<byte> which cannot be initialized with byte * or IntPtr.
So please is there a way to do use span with SocketAsyncEventArgs?
Thank you for your help.

As already mentioned in the comments, Span is the wrong tool here - have you looked at using Memory instead? As you stated, the SetBuffer method does accept that as a parameter - is there a reason you can't use it?
See also this article for a good explanation on how stack vs heap allocation applies to Span and Memory. It includes this example, using a readonly Memory<Foo> buffer:
public struct Enumerable : IEnumerable<Foo>
{
readonly Stream stream;
public Enumerable(Stream stream)
{
this.stream = stream;
}
public IEnumerator<Foo> GetEnumerator() => new Enumerator(this);
IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
public struct Enumerator : IEnumerator<Foo>
{
static readonly int ItemSize = Unsafe.SizeOf<Foo>();
readonly Stream stream;
readonly Memory<Foo> buffer;
bool lastBuffer;
long loadedItems;
int currentItem;
public Enumerator(Enumerable enumerable)
{
stream = enumerable.stream;
buffer = new Foo[100]; // alloc items buffer
lastBuffer = false;
loadedItems = 0;
currentItem = -1;
}
public Foo Current => buffer.Span[currentItem];
object IEnumerator.Current => Current;
public bool MoveNext()
{
if (++currentItem != loadedItems) // increment current position and check if reached end of buffer
return true;
if (lastBuffer) // check if it was the last buffer
return false;
// get next buffer
var rawBuffer = MemoryMarshal.Cast<Foo, byte>(buffer);
var bytesRead = stream.Read(rawBuffer);
lastBuffer = bytesRead < rawBuffer.Length;
currentItem = 0;
loadedItems = bytesRead / ItemSize;
return loadedItems != 0;
}
public void Reset() => throw new NotImplementedException();
public void Dispose()
{
// nothing to do
}
}
}

You should copy the data to managed memory first, use Marshal or Buffer class.
If not think about when the C code delete the returned pointer what will happen to the send data?

There's a complete example (and implementation of the class) on the MSDN page for the SocketAsyncEventArgs (just follow the link). It shows the proper use of the class and may give you the guidence you're looking for.
Also, as Shingo said, it should all be in managed code, not in pointers.

IObservable<byte[]> conversion

How can I convert IObservable to byte[]?
I want to convert IObservable to a wav file and save it on the disk.
DisOutput d;
File.WriteAllBytes("outputsend.wav", d);
I have an array IObservable which I want to convert to byte[] so that I can write to a file. How can I convert IObservable to byte[] because WriteAllBytes takes byte[] as input

I assume that you have a framework that generates a WAV file on the fly in the form of an IObservable<Byte>. Presumably, when the generation is complete the IObservable<Byte> will fire OnCompleted to signal this.
IObservable<Byte> observable;
The simplest way to do what you want is to use ToList which will generate an IObservable<IList<Byte>> that fires when the source observable sequence completes:
observable.ToList().Subscribe(list => File.WriteAllBytes("outputsend.wav", list.ToArray());
What happens is that the ToList operator will collect all incoming bytes in a list that grows over time. When the incoming sequence completes the subscribers are notified and in this case the list of bytes is written to a file.
However, there is no need to buffer the bytes in memory. Instead they can be written directly to the file. It is important that the file is closed when the incoming stream of bytes completes and this can be achieved using this somewhat more complicated but also more efficient extension method:
static class ObservableExtensions {
public static IObservable<Unit> WriteToFile(this IObservable<Byte> source, String fileName) {
return Observable.Create<Unit>(
observer => {
var fileStream = new SerialDisposable();
return new CompositeDisposable(
source.Subscribe(
value => {
try {
if (fileStream.Disposable == null)
fileStream.Disposable = File.Create(fileName);
((FileStream) fileStream.Disposable).WriteByte(value);
}
catch (SystemException ex) {
observer.OnError(ex);
}
},
observer.OnError,
() => {
observer.OnNext(Unit.Default);
observer.OnCompleted();
}
),
fileStream
);
}
);
}
}
You can use this extension method like this:
observable.WriteToFile("outputsend.wav").Subscribe(_ => Console.WriteLine("Done"));
To handle errors you can use another overload of Subscribe:
observable.WriteToFile("outputsend.wav").Subscribe(
_ => Console.WriteLine("Done"),
ex => Console.WriteLine(ex)
);
This will write exceptions to the console. A more sophisticated approach would be required in a production quality application.

What is the proper way to create an Observable which reads a stream to the end

I'm struggling here. Normally I'd read a book but there aren't any yet. I've found countless examples of various things to do with reading streams using RX but I'm finding it very hard to get my head around.
I know I can use Observable.FromAsyncPattern to create a wrapper of the Stream's BeginRead/EndRead or BeginReadLine/EndReadLine methods.
But this only reads once -- when the first observer subscribes.
I want an Observable which will keep reading and pumping OnNext until the stream errors or ends.
In addition to this, I'd also like to know how I can then share that observable with multiple subscribers so they all get the items.

You can use Repeat in order to keep reading lines until the end of the stream and Publish or Replay in order to control sharing across multiple readers.
An example of a simple, full Rx solution for reading lines from any stream until the end would be:
public static IObservable<string> ReadLines(Stream stream)
{
return Observable.Using(
() => new StreamReader(stream),
reader => Observable.FromAsync(reader.ReadLineAsync)
.Repeat()
.TakeWhile(line => line != null));
}
This solution also takes advantage of the fact that ReadLine returns null when the end of the stream is reached.

Adding to Lee's answer, using rxx:
using (new FileStream(#"filename.txt", FileMode.Open)
.ReadToEndObservable()
.Subscribe(x => Console.WriteLine(x.Length)))
{
Console.ReadKey();
}
The length of the read buffers will be outputted.

Heh - gonna reuse one of my other answers here (well, part of it, anyways):
Ref: Reading from NetworkStream corrupts the buffer
In that, I've got an extension method like this:
public static class Ext
{
public static IObservable<byte[]> ReadObservable(this Stream stream, int bufferSize)
{
// to hold read data
var buffer = new byte[bufferSize];
// Step 1: async signature => observable factory
var asyncRead = Observable.FromAsyncPattern<byte[], int, int, int>(
stream.BeginRead,
stream.EndRead);
return Observable.While(
// while there is data to be read
() => stream.CanRead,
// iteratively invoke the observable factory, which will
// "recreate" it such that it will start from the current
// stream position - hence "0" for offset
Observable.Defer(() => asyncRead(buffer, 0, bufferSize))
.Select(readBytes => buffer.Take(readBytes).ToArray()));
}
}
You can probably use this as written in a form like so:
// Note: ToEnumerable works here because your filestream
// has a finite length - don't do this with infinite streams!
var blobboData = stream
.ReadObservable(bufferSize)
// take while we're still reading data
.TakeWhile(returnBuffer => returnBuffer.Length > 0)
.ToEnumerable()
// mash them all together
.SelectMany(buffer => buffer)
.ToArray();

The solution is to use Observable.Create
Here is an example which can be adapated for reading any kind of stream
public static IConnectableObservable<Command> GetReadObservable(this CommandReader reader)
{
return Observable.Create<Command>(async (subject, token) =>
{
try
{
while (true)
{
if (token.IsCancellationRequested)
{
subject.OnCompleted();
return;
}
//this part here can be changed to something like this
//int received = await Task.Factory.FromAsync<int>(innerSocket.BeginReceive(data, offset, size, SocketFlags.None, null, null), innerSocket.EndReceive);
Command cmd = await reader.ReadCommandAsync();
subject.OnNext(cmd);
}
}
catch (Exception ex)
{
try
{
subject.OnError(ex);
}
catch (Exception)
{
Debug.WriteLine("An exception was thrown while trying to call OnError on the observable subject -- means you're not catching exceptions everywhere");
throw;
}
}
}).Publish();
}
Don't forget to call Connect() on the returned IConnectableObservable