Develop Reference

Using the same function yields different results?

Going from Framework 4.5 to Net 6.0 for an old project, there is an function that has broken immensely. When using the function for the 1st time, it returns the value 1651. Running the same function for a 2nd time, it returns a 0 value. This has caused an issue where our program cannot open the PKG file properly (it fails reading the 2nd value count.). I am wondering if there's a quick fix to be done?
The File Check (The code using the broken function): Github ArkPackage.cs
private void readNewFileTable(Stream header, bool readHash = false)
{
uint numFiles = header.ReadUInt32LE();
var files = new OffsetFile[numFiles];
for (var i = 0; i < numFiles; i++)
{
// Version 3 uses 32-bit file offsets
long arkFileOffset = header.ReadInt64LE();
string path = header.ReadLengthPrefixedString(System.Text.Encoding.UTF8);
var flags = header.ReadInt32LE();
uint size = header.ReadUInt32LE();
if (readHash) header.Seek(4, SeekOrigin.Current); // Skips checksum
var finalSlash = path.LastIndexOf('/');
var fileDir = path.Substring(0, finalSlash < 0 ? 0 : finalSlash);
var fileName = path.Substring(finalSlash < 0 ? 0 : (finalSlash + 1));
var parent = makeOrGetDir(fileDir);
var file = new OffsetFile(fileName, parent, contentFileMeta, arkFileOffset, size);
file.ExtendedInfo["id"] = i;
file.ExtendedInfo["flags"] = flags;
files[i] = file;
parent.AddFile(file);
}
var numFiles2 = header.ReadUInt32LE();
if(numFiles != numFiles2)
throw new Exception("Ark header appears invalid (file count mismatch)");
for(var i = 0; i < numFiles2; i++)
{
files[i].ExtendedInfo["flags2"] = header.ReadInt32LE();
}
The Broken Function (L154 is a variation that uses 161-171): Github StreamExtensions.cs
public static uint ReadUInt32LE(this Stream s) => unchecked((uint)s.ReadInt32LE());
/// <summary>
/// Read a signed 32-bit little-endian integer from the stream.
/// </summary>
/// <param name="s"></param>
/// <returns></returns>
public static int ReadInt32LE(this Stream s)
{
int ret;
byte[] tmp = new byte[4];
s.Read(tmp, 0, 4);
ret = tmp[0] & 0x000000FF;
ret |= (tmp[1] << 8) & 0x0000FF00;
ret |= (tmp[2] << 16) & 0x00FF0000;
ret |= (tmp[3] << 24);
return ret;
}
Before (Framework 4.5):
After (Net 6.0):

Starting in .Net 5, I believe it was, FileStream.Read doesn't block until the entire amount of data requested is available, it instead reads at most as many characters as instructed. Check the return value to see how much it read
If you want to read exactly as many characters as you want, either use a loop or use BinaryReader.ReadBytes which implements the loop for you.
Also that bit of code speaks volumes about the general quality of whatever library that is. You shouldn't be allocating arrays like that anymore, this isn't the 90's.

Suspicious behavior of Windows Form after using unsafe code with ZeroMQ

I am using ZeroMQ inside Windows Form application in C#. The ZeroMQ is using Publisher-Subscriber pattern and consists of 3 publishers to publish 3 different types of data. Below is the code snippet-
var ZmqContext = new ZContext();
ZmqContext.SetOption(ZContextOption.IO_THREADS, 3);
var DataType1Publisher = new ZSocket(zmqContext, ZSocketType.PUB);
var DataType2Publisher = new ZSocket(zmqContext, ZSocketType.PUB);
var DataType3Publisher = new ZSocket(zmqContext, ZSocketType.PUB);
DataType1Publisher.SetOption(ZSocketOption.CONFLATE, 1);
DataType1Publisher.Bind("tcp://*:10001");
DataType2Publisher.SetOption(ZSocketOption.CONFLATE, 1);
DataType2Publisher.Bind("tcp://*:10002");
DataType3Publisher.SetOption(ZSocketOption.CONFLATE, 1);
DataType3Publisher.Bind("tcp://*:10003");
DataType1Publisher.Send(new ZFrame(DataType1ByteArray));
DataType2Publisher.Send(new ZFrame(DataType2ByteArray));
DataType3Publisher.Send(new ZFrame(DataType3ByteArray));
The DataType1ByteArray, DataType2ByteArray and DataType3ByteArray are byte array manually composed by converting uint and float data into byte array as follows-
int MAX_LENGTH = 100;
byte[] DataType1ByteArray = new byte[MAX_LENGTH];
int DataType1ByteArrayIndex = 0;
unsafe private void AddUint(uint point)
{
byte* bytePtr = (byte*)&point;
for (int i = 0; i < sizeof(uint); i++)
{
DataType1ByteArray[DataType1ByteArrayIndex++] = bytePtr[i];
}
}
unsafe private void AddFloat(float point)
{
byte* bytePtr = (byte*)&point;
for (int i = 0; i < sizeof(float); i++)
{
DataType1ByteArray[DataType1ByteArrayIndex++] = bytePtr[i];
}
}
Below are my observations-
The code works smoothly for few hours and successfully transmits data at 700 MBPS rate approximately.
It stops transmitting any data after 5-6 hours.
There is no error reported in Visual Studio.
When I see the "Task Manager" in PC (Windows 10), the ethernet shows 0Kbps.
Please note that the code works fine with BitConverter class. However, for faster speed, I used byte* bytePtr = (byte*)&point since it is much faster than BitConverter.
Though I am not sure it seems some kind of memory leakage. How to debug this issue and make it working? Any workaround, please?

Capture audio from WasapiLoopbackCapture, and convert to muLaw

I'm capturing audio with WasapiLoopbackCapture
- format = IeeeFloat
- SampleRate = 48000
- BitsPerSample = 32
I need to convert this to muLaw (8Khz, 8 bit, mono) - eventually it'll be sent to a phone via SIP trunking. I've tried 100s of samples (most of them with NAudio) and solutions but still have no clue how to do this ...

The Mu-Law tools in NAudio are limited so you might have to roll your own.
You'll need to set up a chain of IWaveProvider filters to convert to mono, change bit-rate, and change bit-depth.
waveBuffer = new BufferedWaveProvider(waveIn.WaveFormat);
waveBuffer.DiscardOnBufferOverflow = true;
waveBuffer.ReadFully = false; // leave a buffer?
sampleStream = new WaveToSampleProvider(waveBuffer);
// Stereo to mono
monoStream = new StereoToMonoSampleProvider(sampleStream)
{
LeftVolume = 1f,
RightVolume = 1f
};
// Downsample to 8000
resamplingProvider = new WdlResamplingSampleProvider(monoStream, 8000);
// Convert to 16-bit in order to use ACM or MuLaw tools.
ieeeToPcm = new SampleToWaveProvider16(resamplingProvider);
Then create a custom IWaveProvider for the next step.
// In MuLawConversionProvider
public int Read(byte[] destinationBuffer, int offset, int readingCount)
{
// Source buffer has twice as many items as the output array.
var sizeOfPcmBuffer = readingCount * 2;
_sourceBuffer = BufferHelpers.Ensure(_sourceBuffer, sizeOfPcmBuffer);
var sourceBytesRead = _sourceProvider.Read(_sourceBuffer, offset * 2, sizeOfPcmBuffer);
var samplesRead = sourceBytesRead / 2;
var outIndex = 0;
for (var n = 0; n < sizeOfPcmBuffer; n += 2)
{
destinationBuffer[outIndex++] = MuLawEncoder.LinearToMuLawSample(BitConverter.ToInt16(_sourceBuffer, offset + n));
}
return samplesRead * 2;
}
The new provider can be sent directly to WaveOut
outputStream = new MuLawConversionProvider(ieeeToPcm);
waveOut.Init(outputStream);
waveOut.Play();
These filters remain in place with the BufferedWaveProvider as the "root". Whenever you call BufferedWaveProvider.AddSamples(), the data will go through all these filters.

I have a log file with RTP packets: now what?

I have a log file with RTP packets coming off of a black box device. I also have a corresponding SDP file (RTSP DESCRIBE) for that. I need to convert this file into some kind of playable video file. Can I pass these two files to FFMpeg or VLC or something else and have it mux that data into something playable?
As an alternate plan, I can loop through the individual packets in code and do something with each packet. However, it seems that there are existing libraries for parsing this data. And it seems to do it by hand would be asking for a large project. Is there some kind of video file format that is a pretty raw mix of SDP and RTP? Thanks for your time.
Is there a way for FFmpeg or VLC to open an SDP file and then get their input packets through STDIN?
I generally use C#, but I could use C if necessary.
Update 1: Here is my unworking code. I'm trying to get some kind of output to play with ffplay, but I haven't had any luck yet. It gives me invalid data errors. It does go over all the data correctly as far as I can tell. My output is nearly as big as my input (at about 4MB).
public class RtpPacket2
{
public byte VersionPXCC;
public byte MPT;
public ushort Sequence; // length?
public uint Timestamp;
public uint Ssrc;
public int Version { get { return VersionPXCC >> 6; } }
public bool Padding { get { return (VersionPXCC & 32) > 0; } }
public bool Extension { get { return (VersionPXCC & 16) > 0; } }
public int CsrcCount { get { return VersionPXCC & 0xf; } } // ItemCount
public bool Marker { get { return (MPT & 0x80) > 0; } }
public int PayloadType { get { return MPT & 0x7f; } } // PacketType
}
static void Main(string[] args)
{
if (args.Length != 2)
{
Console.WriteLine("Usage: <input RTP file> <output 3GP file>");
return;
}
var inputFile = args[0];
var outputFile = args[1];
if(File.Exists(outputFile)) File.Delete(outputFile);
// FROM the SDP : fmtp 96 profile-level-id=4D0014;packetization-mode=0
var sps = Convert.FromBase64String("Z0LAHoiLUFge0IAAA4QAAK/IAQ=="); // BitConverter.ToString(sps) "67-42-C0-1E-88-8B-50-58-1E-D0-80-00-03-84-00-00-AF-C8-01" string
var pps = Convert.FromBase64String("aM44gA=="); // BitConverter.ToString(pps) "68-CE-38-80" string
var sep = new byte[] { 00, 00, 01 };
var packet = new RtpPacket2();
bool firstFrame = true;
using (var input = File.OpenRead(inputFile))
using (var reader = new BinaryReader(input))
using (var output = File.OpenWrite(outputFile))
{
//output.Write(header, 0, header.Length);
output.Write(sep, 0, sep.Length);
output.Write(sps, 0, sps.Length);
output.Write(sep, 0, sep.Length);
output.Write(pps, 0, pps.Length);
output.Write(sep, 0, sep.Length);
while (input.Position < input.Length)
{
var size = reader.ReadInt16();
packet.VersionPXCC = reader.ReadByte();
packet.MPT = reader.ReadByte();
packet.Sequence = reader.ReadUInt16();
packet.Timestamp = reader.ReadUInt32();
packet.Ssrc = reader.ReadUInt32();
if (packet.PayloadType == 96)
{
if (packet.CsrcCount > 0 || packet.Extension) throw new NotImplementedException();
var header0 = reader.ReadByte();
var header1 = reader.ReadByte();
var fragmentType = header0 & 0x1F; // should be 28 for video
if(fragmentType != 28) // 28 for video?
{
input.Position += size - 14;
continue;
}
var nalUnit = header0 & ~0x1F;
var nalType = header1 & 0x1F;
var start = (header1 & 0x80) > 0;
var end = (header1 & 0x40) > 0;
if(firstFrame)
{
output.Write(sep, 0, sep.Length);
output.WriteByte((byte)(nalUnit | fragmentType));
firstFrame = false;
}
for (int i = 0; i < size - 14; i++)
output.WriteByte(reader.ReadByte());
if (packet.Marker)
firstFrame = true;
}
else input.Position += size - 12;
}
}
}

http://www.bogotobogo.com/VideoStreaming/videostreaming_etc.php
Stay away from doing anything at the packet level because you may get bogged down in the details of how encoded streams are packetized.
look thru the above link. SDP / RTP / RTSP streaming are pretty involved protocols that usually do not work when you try to hook them up directly to players expecting to just open a local media file.
If you are handling streams and you want to save a file from the stream , you might want to google 'filesinks' with any of the big media projects ( ffmpeg, vlc, live555, openrtsp ) because those projects already have opensource fileSink implementations (C, C++).
For example, in live555, all of the codecs have fileSink implementations in ./live/liveMedia directory.
If you have a stream from an SDP source, you can handle each of the tracks (audio, video) with a fileSink for the codec used by that track. Then you can mux those tracks to a player or play the tracks independently.

Open the SDP file in vlc and then write a small app to re-play your RTP packets out to the network again so that VLC can receive them.
Make sure that you play them out to 127.0.0.1 and a port number that matches the SDP file.
VLC will wait until it receives some packets, then use VLC to save the media without transcoding into a MP4 or similar file format.

Convert Amplitude BACK to .wav file (C#)

What I have done is converted a wave file to amplitude values in a short[] array as found here Mean amplitude of a .wav in C#
I modified the values and now want to convert back to .wav format or a byte[] array which when can be written to a byte file.

void SetShortToBuffer(short val,byte[] outArray,int Offset)
{
outArray[Offset] = (byte)(val & 0x00FF);
Offset++;
outArray[Offset] = (byte)((val >> 8) & 0x00FF);
}
byte[] ConvertShortArray(short[] Data,int Offset,int Count)
{
byte[] helper = new byte[Count * sizeof(short)];
int end = Offset+Count;
int io=0;
for (int i = Offset; i < end; i++)
{
SetShortToBuffer(Data[i], helper, io);
io+=sizeof(short);
}
return helper;
}
In C this would not be an issue, you could simply tell the compiler that your previously declared short array should now be treated as a byte array (simple cast) but after failing to do so in C# outside of unsafe context I came up with this code :)
You can use ConvertShortArray function to get chunks of data in case your wave is large
EDIT:
Quick and dirty wave header creator, not tested
byte[] CreateWaveFileHeader(int SizeOfData, short ChannelCount, uint SamplesPerSecond, short BitsPerSample)
{
short BlockAlign = (short)(ChannelCount * (BitsPerSample / 8));
uint AverageBytesPerSecond = SamplesPerSecond * BlockAlign;
List<byte> pom = new List<byte>();
pom.AddRange(ASCIIEncoding.ASCII.GetBytes("RIFF"));
pom.AddRange(BitConverter.GetBytes(SizeOfData + 36)); //Size + up to data
pom.AddRange(ASCIIEncoding.ASCII.GetBytes("WAVEfmt "));
pom.AddRange(BitConverter.GetBytes(((uint)16))); //16 For PCM
pom.AddRange(BitConverter.GetBytes(((short)1))); //PCM FMT
pom.AddRange(BitConverter.GetBytes(((short)ChannelCount)));
pom.AddRange(BitConverter.GetBytes((uint)SamplesPerSecond));
pom.AddRange(BitConverter.GetBytes((uint)AverageBytesPerSecond));
pom.AddRange(BitConverter.GetBytes((short)BlockAlign));
pom.AddRange(BitConverter.GetBytes((short)BitsPerSample));
pom.AddRange(ASCIIEncoding.ASCII.GetBytes("data"));
pom.AddRange(BitConverter.GetBytes(SizeOfData));
return pom.ToArray();
}