Develop Reference

NAudio stream Read method latency

I made my own beep code as an exercise. However it is lagging.
2-3 beeps lagging then 3rd or 4th comes quicker.
Can someone please explain why? And how do I rid off the latency?
I used bits from Mark Heath blogs and NAudio github code..
This is main console code:
var waveSine = new BeepStream(waveWhite.WaveFormat);
var sineChannel = new WaveChannel32(waveSine) { PadWithZeroes = false };
List<WaveChannel32> inputs = new List<WaveChannel32>();
inputs.Add(sineChannel);
var mixer = new MixingWaveProvider32(inputs);
var output = new WaveOut();
output.Init(mixer);
output.Play();
output.Volume = 0.5f;
while (true)
{
Thread.Sleep(1000);
waveSine.Beep(1000, 150);
}
This is beep code (it basically outputs 0s, but when Beep is called it pushes cached sine samples):
(note1: not production code so please ignore obvious inconsistencies)
(note1: in reality there are at least 2 channels - one is always playing, this is why I had to make second - the beep channel - also always playing. I would gladly write SamplesProviders instead but couldnt find good example)
public class BeepStream : WaveStream
...
public override int Read(byte[] buffer, int offset, int count)
{
int totalBytesRead = 0;
int beepBytesRead = 0;
while (totalBytesRead < count)
{
if(playingNow == null)
{
// silence
buffer[totalBytesRead] = 0;
totalBytesRead += 1;// bytesRead;
}
else
{
// beep
buffer[totalBytesRead] = playingNow[beepBytesRead++];
totalBytesRead += 1;// bytesRead;
if (beepBytesRead >= playingNow.Length)
playingNow = null;
}
}
return totalBytesRead;
}
An observation - Reads come with count 52920 always. What is this magic number? Can I reduce it? The wave format is float 44100 if that helps..

Found solution by simply trying everything.
Use of exclusive WASAPI output:
var output = new WasapiOut(NAudio.CoreAudioApi.AudioClientShareMode.Exclusive, 3);
output.Init(mixer);
output.Play();

Suspicious behavior of Windows Form after using unsafe code with ZeroMQ

I am using ZeroMQ inside Windows Form application in C#. The ZeroMQ is using Publisher-Subscriber pattern and consists of 3 publishers to publish 3 different types of data. Below is the code snippet-
var ZmqContext = new ZContext();
ZmqContext.SetOption(ZContextOption.IO_THREADS, 3);
var DataType1Publisher = new ZSocket(zmqContext, ZSocketType.PUB);
var DataType2Publisher = new ZSocket(zmqContext, ZSocketType.PUB);
var DataType3Publisher = new ZSocket(zmqContext, ZSocketType.PUB);
DataType1Publisher.SetOption(ZSocketOption.CONFLATE, 1);
DataType1Publisher.Bind("tcp://*:10001");
DataType2Publisher.SetOption(ZSocketOption.CONFLATE, 1);
DataType2Publisher.Bind("tcp://*:10002");
DataType3Publisher.SetOption(ZSocketOption.CONFLATE, 1);
DataType3Publisher.Bind("tcp://*:10003");
DataType1Publisher.Send(new ZFrame(DataType1ByteArray));
DataType2Publisher.Send(new ZFrame(DataType2ByteArray));
DataType3Publisher.Send(new ZFrame(DataType3ByteArray));
The DataType1ByteArray, DataType2ByteArray and DataType3ByteArray are byte array manually composed by converting uint and float data into byte array as follows-
int MAX_LENGTH = 100;
byte[] DataType1ByteArray = new byte[MAX_LENGTH];
int DataType1ByteArrayIndex = 0;
unsafe private void AddUint(uint point)
{
byte* bytePtr = (byte*)&point;
for (int i = 0; i < sizeof(uint); i++)
{
DataType1ByteArray[DataType1ByteArrayIndex++] = bytePtr[i];
}
}
unsafe private void AddFloat(float point)
{
byte* bytePtr = (byte*)&point;
for (int i = 0; i < sizeof(float); i++)
{
DataType1ByteArray[DataType1ByteArrayIndex++] = bytePtr[i];
}
}
Below are my observations-
The code works smoothly for few hours and successfully transmits data at 700 MBPS rate approximately.
It stops transmitting any data after 5-6 hours.
There is no error reported in Visual Studio.
When I see the "Task Manager" in PC (Windows 10), the ethernet shows 0Kbps.
Please note that the code works fine with BitConverter class. However, for faster speed, I used byte* bytePtr = (byte*)&point since it is much faster than BitConverter.
Though I am not sure it seems some kind of memory leakage. How to debug this issue and make it working? Any workaround, please?

I have a log file with RTP packets: now what?

I have a log file with RTP packets coming off of a black box device. I also have a corresponding SDP file (RTSP DESCRIBE) for that. I need to convert this file into some kind of playable video file. Can I pass these two files to FFMpeg or VLC or something else and have it mux that data into something playable?
As an alternate plan, I can loop through the individual packets in code and do something with each packet. However, it seems that there are existing libraries for parsing this data. And it seems to do it by hand would be asking for a large project. Is there some kind of video file format that is a pretty raw mix of SDP and RTP? Thanks for your time.
Is there a way for FFmpeg or VLC to open an SDP file and then get their input packets through STDIN?
I generally use C#, but I could use C if necessary.
Update 1: Here is my unworking code. I'm trying to get some kind of output to play with ffplay, but I haven't had any luck yet. It gives me invalid data errors. It does go over all the data correctly as far as I can tell. My output is nearly as big as my input (at about 4MB).
public class RtpPacket2
{
public byte VersionPXCC;
public byte MPT;
public ushort Sequence; // length?
public uint Timestamp;
public uint Ssrc;
public int Version { get { return VersionPXCC >> 6; } }
public bool Padding { get { return (VersionPXCC & 32) > 0; } }
public bool Extension { get { return (VersionPXCC & 16) > 0; } }
public int CsrcCount { get { return VersionPXCC & 0xf; } } // ItemCount
public bool Marker { get { return (MPT & 0x80) > 0; } }
public int PayloadType { get { return MPT & 0x7f; } } // PacketType
}
static void Main(string[] args)
{
if (args.Length != 2)
{
Console.WriteLine("Usage: <input RTP file> <output 3GP file>");
return;
}
var inputFile = args[0];
var outputFile = args[1];
if(File.Exists(outputFile)) File.Delete(outputFile);
// FROM the SDP : fmtp 96 profile-level-id=4D0014;packetization-mode=0
var sps = Convert.FromBase64String("Z0LAHoiLUFge0IAAA4QAAK/IAQ=="); // BitConverter.ToString(sps) "67-42-C0-1E-88-8B-50-58-1E-D0-80-00-03-84-00-00-AF-C8-01" string
var pps = Convert.FromBase64String("aM44gA=="); // BitConverter.ToString(pps) "68-CE-38-80" string
var sep = new byte[] { 00, 00, 01 };
var packet = new RtpPacket2();
bool firstFrame = true;
using (var input = File.OpenRead(inputFile))
using (var reader = new BinaryReader(input))
using (var output = File.OpenWrite(outputFile))
{
//output.Write(header, 0, header.Length);
output.Write(sep, 0, sep.Length);
output.Write(sps, 0, sps.Length);
output.Write(sep, 0, sep.Length);
output.Write(pps, 0, pps.Length);
output.Write(sep, 0, sep.Length);
while (input.Position < input.Length)
{
var size = reader.ReadInt16();
packet.VersionPXCC = reader.ReadByte();
packet.MPT = reader.ReadByte();
packet.Sequence = reader.ReadUInt16();
packet.Timestamp = reader.ReadUInt32();
packet.Ssrc = reader.ReadUInt32();
if (packet.PayloadType == 96)
{
if (packet.CsrcCount > 0 || packet.Extension) throw new NotImplementedException();
var header0 = reader.ReadByte();
var header1 = reader.ReadByte();
var fragmentType = header0 & 0x1F; // should be 28 for video
if(fragmentType != 28) // 28 for video?
{
input.Position += size - 14;
continue;
}
var nalUnit = header0 & ~0x1F;
var nalType = header1 & 0x1F;
var start = (header1 & 0x80) > 0;
var end = (header1 & 0x40) > 0;
if(firstFrame)
{
output.Write(sep, 0, sep.Length);
output.WriteByte((byte)(nalUnit | fragmentType));
firstFrame = false;
}
for (int i = 0; i < size - 14; i++)
output.WriteByte(reader.ReadByte());
if (packet.Marker)
firstFrame = true;
}
else input.Position += size - 12;
}
}
}

http://www.bogotobogo.com/VideoStreaming/videostreaming_etc.php
Stay away from doing anything at the packet level because you may get bogged down in the details of how encoded streams are packetized.
look thru the above link. SDP / RTP / RTSP streaming are pretty involved protocols that usually do not work when you try to hook them up directly to players expecting to just open a local media file.
If you are handling streams and you want to save a file from the stream , you might want to google 'filesinks' with any of the big media projects ( ffmpeg, vlc, live555, openrtsp ) because those projects already have opensource fileSink implementations (C, C++).
For example, in live555, all of the codecs have fileSink implementations in ./live/liveMedia directory.
If you have a stream from an SDP source, you can handle each of the tracks (audio, video) with a fileSink for the codec used by that track. Then you can mux those tracks to a player or play the tracks independently.

Open the SDP file in vlc and then write a small app to re-play your RTP packets out to the network again so that VLC can receive them.
Make sure that you play them out to 127.0.0.1 and a port number that matches the SDP file.
VLC will wait until it receives some packets, then use VLC to save the media without transcoding into a MP4 or similar file format.

playing sound with SlimDX and DirectSound (C#)

(apologies if this is a duplicate ... i posted but saw no evidence that it actually made it to the forum)
I've been trying to get SlimDX DirectSound working. Here's the code I have. It fills the secondary buffer from a wav file and then, in a thread loop, alternately fills the lower or upper halves of the buffer.
It plays the first load of the buffer fine. The AutoResetEvents fire when they should and the lower half then upper half of the buffer are populated (verified with Debug statements). But playing does not continue after the first load of the buffer. So somehow the repopulation of the buffer doesn't work as it should.
Ideas?
(I'm using DirectSound because it's the only way I've found to set the guid of the audio device that I want to use. Am open to other .NET-friendly approaches.)
private void PlaySound(Guid soundCardGuid, string audioFile) {
DirectSound ds = new DirectSound(soundCardGuid);
ds.SetCooperativeLevel(this.Handle, CooperativeLevel.Priority);
WaveFormat format = new WaveFormat();
format.BitsPerSample = 16;
format.BlockAlignment = 4;
format.Channels = 2;
format.FormatTag = WaveFormatTag.Pcm;
format.SamplesPerSecond = 44100;
format.AverageBytesPerSecond = format.SamplesPerSecond * format.BlockAlignment;
SoundBufferDescription desc = new SoundBufferDescription();
desc.Format = format;
desc.Flags = BufferFlags.GlobalFocus;
desc.SizeInBytes = 8 * format.AverageBytesPerSecond;
PrimarySoundBuffer pBuffer = new PrimarySoundBuffer(ds, desc);
SoundBufferDescription desc2 = new SoundBufferDescription();
desc2.Format = format;
desc2.Flags = BufferFlags.GlobalFocus | BufferFlags.ControlPositionNotify | BufferFlags.GetCurrentPosition2;
desc2.SizeInBytes = 8 * format.AverageBytesPerSecond;
SecondarySoundBuffer sBuffer1 = new SecondarySoundBuffer(ds, desc2);
NotificationPosition[] notifications = new NotificationPosition[2];
notifications[0].Offset = desc2.SizeInBytes / 2 + 1;
notifications[1].Offset = desc2.SizeInBytes - 1; ;
notifications[0].Event = new AutoResetEvent(false);
notifications[1].Event = new AutoResetEvent(false);
sBuffer1.SetNotificationPositions(notifications);
byte[] bytes1 = new byte[desc2.SizeInBytes / 2];
byte[] bytes2 = new byte[desc2.SizeInBytes];
Stream stream = File.Open(audioFile, FileMode.Open);
Thread fillBuffer = new Thread(() => {
int readNumber = 1;
int bytesRead;
bytesRead = stream.Read(bytes2, 0, desc2.SizeInBytes);
sBuffer1.Write<byte>(bytes2, 0, LockFlags.None);
sBuffer1.Play(0, PlayFlags.None);
while (true) {
if (bytesRead == 0) { break; }
notifications[0].Event.WaitOne();
bytesRead = stream.Read(bytes1, 0, bytes1.Length);
sBuffer1.Write<byte>(bytes1, 0, LockFlags.None);
if (bytesRead == 0) { break; }
notifications[1].Event.WaitOne();
bytesRead = stream.Read(bytes1, 0, bytes1.Length);
sBuffer1.Write<byte>(bytes1, desc2.SizeInBytes / 2, LockFlags.None);
}
stream.Close();
stream.Dispose();
});
fillBuffer.Start();
}
}

You haven't set it to loop on the play buffer. Change your code to:
sBuffer1.Play(0, PlayFlags.Looping);

Need help in translating C++ to C#

I don't know C++ very well especially the IO part. Can anyone please help me to translate the following C++ code into C#?
unsigned *PostingOffset, *PostingLength, NumTerms;
void LoadSubIndex(char *subindex) {
FILE *in = fopen(subindex, "rb");
if (in == 0) {
printf("Error opening sub-index file '%s'!\n", subindex);
exit(EXIT_FAILURE);
}
int len=0;
// Array of terms
char **Term;
char *TermList;
fread(&NumTerms, sizeof(unsigned), 1, in);
PostingOffset = (unsigned*)malloc(sizeof(unsigned) * NumTerms);
PostingLength = (unsigned*)malloc(sizeof(unsigned) * NumTerms);
Term = (char**)malloc(sizeof(char*) * NumTerms);
Term = (char**)malloc(sizeof(char*) * NumTerms);
// Offset of each posting
fread(PostingOffset, sizeof(unsigned), NumTerms, in);
// Length of each posting in bytes
fread(PostingLength, sizeof(unsigned), NumTerms, in);
// Number of bytes in the posting terms array
fread(&len, sizeof(unsigned), 1, in);
TermList = (char*)malloc(sizeof(char) * len);
fread(TermList, sizeof(unsigned)*len, 1, in);
unsigned k=1;
Term[0] = &TermList[0];
for (int i=1; i<len; i++) {
if (TermList[i-1] == '\0') {
Term[k] = &TermList[i];
k++;
}
}
fclose(in);
}
Thanks in advance.

I'll give you a headstart.
using(var reader = new BinaryReader(new FileStream(subindex, FileMode.Open)) {
int numTerms = reader.ReadUInt32();
postingOffset = new UInt32[numTerms];
postingLength = new UInt32[numTerms];
var term = new byte[numTerms];
for(int i=0;i<numTerms;i++)
postingOffset[i] = reader.ReadUInt32();
for(int i=0;i<numTerms;i++)
postingLength[i] = reader.ReadUInt32();
var len = reader.ReadInt32();
var termList = new ... // byte[] or uint32[] ??
//etc
}
There's no need to close the file handle here - it will close when the using { } block loses scope.
I didn't finish it because there are some flaws in your code. With TermList you are reading in 4 times as much data as you've allocated. You shouldn't be allocating Term twice either - that will result in leaking memory.
To turn Term back into a string, use Encoding.ASCII.GetString(term).TrimEnd('\0');