I have a log file with RTP packets coming off of a black box device. I also have a corresponding SDP file (RTSP DESCRIBE) for that. I need to convert this file into some kind of playable video file. Can I pass these two files to FFMpeg or VLC or something else and have it mux that data into something playable?
As an alternate plan, I can loop through the individual packets in code and do something with each packet. However, it seems that there are existing libraries for parsing this data. And it seems to do it by hand would be asking for a large project. Is there some kind of video file format that is a pretty raw mix of SDP and RTP? Thanks for your time.
Is there a way for FFmpeg or VLC to open an SDP file and then get their input packets through STDIN?
I generally use C#, but I could use C if necessary.
Update 1: Here is my unworking code. I'm trying to get some kind of output to play with ffplay, but I haven't had any luck yet. It gives me invalid data errors. It does go over all the data correctly as far as I can tell. My output is nearly as big as my input (at about 4MB).
public class RtpPacket2
{
public byte VersionPXCC;
public byte MPT;
public ushort Sequence; // length?
public uint Timestamp;
public uint Ssrc;
public int Version { get { return VersionPXCC >> 6; } }
public bool Padding { get { return (VersionPXCC & 32) > 0; } }
public bool Extension { get { return (VersionPXCC & 16) > 0; } }
public int CsrcCount { get { return VersionPXCC & 0xf; } } // ItemCount
public bool Marker { get { return (MPT & 0x80) > 0; } }
public int PayloadType { get { return MPT & 0x7f; } } // PacketType
}
static void Main(string[] args)
{
if (args.Length != 2)
{
Console.WriteLine("Usage: <input RTP file> <output 3GP file>");
return;
}
var inputFile = args[0];
var outputFile = args[1];
if(File.Exists(outputFile)) File.Delete(outputFile);
// FROM the SDP : fmtp 96 profile-level-id=4D0014;packetization-mode=0
var sps = Convert.FromBase64String("Z0LAHoiLUFge0IAAA4QAAK/IAQ=="); // BitConverter.ToString(sps) "67-42-C0-1E-88-8B-50-58-1E-D0-80-00-03-84-00-00-AF-C8-01" string
var pps = Convert.FromBase64String("aM44gA=="); // BitConverter.ToString(pps) "68-CE-38-80" string
var sep = new byte[] { 00, 00, 01 };
var packet = new RtpPacket2();
bool firstFrame = true;
using (var input = File.OpenRead(inputFile))
using (var reader = new BinaryReader(input))
using (var output = File.OpenWrite(outputFile))
{
//output.Write(header, 0, header.Length);
output.Write(sep, 0, sep.Length);
output.Write(sps, 0, sps.Length);
output.Write(sep, 0, sep.Length);
output.Write(pps, 0, pps.Length);
output.Write(sep, 0, sep.Length);
while (input.Position < input.Length)
{
var size = reader.ReadInt16();
packet.VersionPXCC = reader.ReadByte();
packet.MPT = reader.ReadByte();
packet.Sequence = reader.ReadUInt16();
packet.Timestamp = reader.ReadUInt32();
packet.Ssrc = reader.ReadUInt32();
if (packet.PayloadType == 96)
{
if (packet.CsrcCount > 0 || packet.Extension) throw new NotImplementedException();
var header0 = reader.ReadByte();
var header1 = reader.ReadByte();
var fragmentType = header0 & 0x1F; // should be 28 for video
if(fragmentType != 28) // 28 for video?
{
input.Position += size - 14;
continue;
}
var nalUnit = header0 & ~0x1F;
var nalType = header1 & 0x1F;
var start = (header1 & 0x80) > 0;
var end = (header1 & 0x40) > 0;
if(firstFrame)
{
output.Write(sep, 0, sep.Length);
output.WriteByte((byte)(nalUnit | fragmentType));
firstFrame = false;
}
for (int i = 0; i < size - 14; i++)
output.WriteByte(reader.ReadByte());
if (packet.Marker)
firstFrame = true;
}
else input.Position += size - 12;
}
}
}
http://www.bogotobogo.com/VideoStreaming/videostreaming_etc.php
Stay away from doing anything at the packet level because you may get bogged down in the details of how encoded streams are packetized.
look thru the above link. SDP / RTP / RTSP streaming are pretty involved protocols that usually do not work when you try to hook them up directly to players expecting to just open a local media file.
If you are handling streams and you want to save a file from the stream , you might want to google 'filesinks' with any of the big media projects ( ffmpeg, vlc, live555, openrtsp ) because those projects already have opensource fileSink implementations (C, C++).
For example, in live555, all of the codecs have fileSink implementations in ./live/liveMedia directory.
If you have a stream from an SDP source, you can handle each of the tracks (audio, video) with a fileSink for the codec used by that track. Then you can mux those tracks to a player or play the tracks independently.
Open the SDP file in vlc and then write a small app to re-play your RTP packets out to the network again so that VLC can receive them.
Make sure that you play them out to 127.0.0.1 and a port number that matches the SDP file.
VLC will wait until it receives some packets, then use VLC to save the media without transcoding into a MP4 or similar file format.
Related
I need to check in realtime if a specific process try to read a wav of flac file.
I hare created a routine with 2 parameters: the process name, and the file that try to open:
public static bool Scan(string ProcessName,string TextToFind)
{
// getting minimum & maximum address
SYSTEM_INFO sys_info = new SYSTEM_INFO();
GetSystemInfo(out sys_info);
int MatchCount=0;
IntPtr proc_min_address = sys_info.minimumApplicationAddress;
IntPtr proc_max_address = sys_info.maximumApplicationAddress;
// saving the values as long ints so I won't have to do a lot of casts later
long proc_min_address_l = (long)proc_min_address;
long proc_max_address_l = (long)proc_max_address;
Process[] Arrprocess = Process.GetProcessesByName(ProcessName);
if (Arrprocess.Length == 0) return false;
// notepad better be runnin'
Process process = Arrprocess[0];
// opening the process with desired access level
IntPtr processHandle = OpenProcess(PROCESS_QUERY_INFORMATION | PROCESS_WM_READ, false, process.Id);
// this will store any information we get from VirtualQueryEx()
MEMORY_BASIC_INFORMATION mem_basic_info = new MEMORY_BASIC_INFORMATION();
int bytesRead = 0; // number of bytes read with ReadProcessMemory
// long milliseconds_start = DateTime.Now.Ticks / TimeSpan.TicksPerMillisecond;
while (proc_min_address_l < proc_max_address_l)
{
// 28 = sizeof(MEMORY_BASIC_INFORMATION)
VirtualQueryEx(processHandle, proc_min_address, out mem_basic_info, 28);
// if this memory chunk is accessible
if (mem_basic_info.Protect == PAGE_READWRITE && mem_basic_info.State == MEM_COMMIT && (mem_basic_info.lType == MEM_MAPPED || mem_basic_info.lType == MEM_PRIVATE))
{
byte[] buffer = new byte[mem_basic_info.RegionSize];
// read everything in the buffer above
ReadProcessMemory((int)processHandle, mem_basic_info.BaseAddress, buffer, mem_basic_info.RegionSize, ref bytesRead);
string result = System.Text.Encoding.ASCII.GetString(buffer);
if (result.Contains(TextToFind))
return true;
}
// move to the next memory chunk
proc_min_address_l += mem_basic_info.RegionSize;
proc_min_address = new IntPtr(proc_min_address_l);
}
return false;
}
this code work only the first time, because when the process close the flac file, the string keep in memory until the process will close.
I need to check every time (with precision of milliseconds) the process try to load the file that I pass as parameter.
I don't known if i'am in the right way ...
Can someone suggest me a code (in c# or c++) that detect in realtime if a process x try to read a file name y ?
Thank you !
Windows provides methods to let you know when files are opened/edited/renamed/etc.
https://learn.microsoft.com/en-us/windows/win32/projfs/file-system-operation-notifications
Is there a way to convert any audio file to ogg with .net core?
I figured out how to convert with NAudio.Vorbis from ogg to wav:
using (var vorbis = new VorbisWaveReader(inputFile))
{
WaveFileWriter.CreateWaveFile(outputFile, vorbis);
}
But I could not find a way to convert any audio file to ogg.
Does NAudio provide the functionallity I require or are there any other libraries better suited?
You could try this library:
Vorbis Encoder
(this is on nuget, the linked GitHub project provides the source and the following example of encoding)
using System;
using System.IO;
namespace OggVorbisEncoder.Example
{
public class Encoder
{
private const int SampleSize = 1024;
[STAThread]
private static void Main()
{
var stdin = new FileStream(#"unencoded.raw", FileMode.Open, FileAccess.Read);
var stdout = new FileStream(#"encoded.ogg", FileMode.Create, FileAccess.Write);
// StripWavHeader(stdin);
// Stores all the static vorbis bitstream settings
var info = VorbisInfo.InitVariableBitRate(2, 44100, 0.1f);
// set up our packet->stream encoder
var serial = new Random().Next();
var oggStream = new OggStream(serial);
// =========================================================
// HEADER
// =========================================================
// Vorbis streams begin with three headers; the initial header (with
// most of the codec setup parameters) which is mandated by the Ogg
// bitstream spec. The second header holds any comment fields. The
// third header holds the bitstream codebook.
var headerBuilder = new HeaderPacketBuilder();
var comments = new Comments();
comments.AddTag("ARTIST", "TEST");
var infoPacket = headerBuilder.BuildInfoPacket(info);
var commentsPacket = headerBuilder.BuildCommentsPacket(comments);
var booksPacket = headerBuilder.BuildBooksPacket(info);
oggStream.PacketIn(infoPacket);
oggStream.PacketIn(commentsPacket);
oggStream.PacketIn(booksPacket);
// Flush to force audio data onto its own page per the spec
OggPage page;
while (oggStream.PageOut(out page, true))
{
stdout.Write(page.Header, 0, page.Header.Length);
stdout.Write(page.Body, 0, page.Body.Length);
}
// =========================================================
// BODY (Audio Data)
// =========================================================
var processingState = ProcessingState.Create(info);
var buffer = new float[info.Channels][];
buffer[0] = new float[SampleSize];
buffer[1] = new float[SampleSize];
var readbuffer = new byte[SampleSize*4];
while (!oggStream.Finished)
{
var bytes = stdin.Read(readbuffer, 0, readbuffer.Length);
if (bytes == 0)
{
processingState.WriteEndOfStream();
}
else
{
var samples = bytes/4;
for (var i = 0; i < samples; i++)
{
// uninterleave samples
buffer[0][i] = (short) ((readbuffer[i*4 + 1] << 8) | (0x00ff & readbuffer[i*4]))/32768f;
buffer[1][i] = (short) ((readbuffer[i*4 + 3] << 8) | (0x00ff & readbuffer[i*4 + 2]))/32768f;
}
processingState.WriteData(buffer, samples);
}
OggPacket packet;
while (!oggStream.Finished
&& processingState.PacketOut(out packet))
{
oggStream.PacketIn(packet);
while (!oggStream.Finished
&& oggStream.PageOut(out page, false))
{
stdout.Write(page.Header, 0, page.Header.Length);
stdout.Write(page.Body, 0, page.Body.Length);
}
}
}
stdin.Close();
stdout.Close();
}
/// <summary>
/// We cheat on the WAV header; we just bypass the header and never
/// verify that it matches 16bit/stereo/44.1kHz.This is just an
/// example, after all.
/// </summary>
private static void StripWavHeader(BinaryReader stdin)
{
var tempBuffer = new byte[6];
for (var i = 0; (i < 30) && (stdin.Read(tempBuffer, 0, 2) > 0); i++)
if ((tempBuffer[0] == 'd') && (tempBuffer[1] == 'a'))
{
stdin.Read(tempBuffer, 0, 6);
break;
}
}
}
}
I'm capturing audio with WasapiLoopbackCapture
- format = IeeeFloat
- SampleRate = 48000
- BitsPerSample = 32
I need to convert this to muLaw (8Khz, 8 bit, mono) - eventually it'll be sent to a phone via SIP trunking. I've tried 100s of samples (most of them with NAudio) and solutions but still have no clue how to do this ...
The Mu-Law tools in NAudio are limited so you might have to roll your own.
You'll need to set up a chain of IWaveProvider filters to convert to mono, change bit-rate, and change bit-depth.
waveBuffer = new BufferedWaveProvider(waveIn.WaveFormat);
waveBuffer.DiscardOnBufferOverflow = true;
waveBuffer.ReadFully = false; // leave a buffer?
sampleStream = new WaveToSampleProvider(waveBuffer);
// Stereo to mono
monoStream = new StereoToMonoSampleProvider(sampleStream)
{
LeftVolume = 1f,
RightVolume = 1f
};
// Downsample to 8000
resamplingProvider = new WdlResamplingSampleProvider(monoStream, 8000);
// Convert to 16-bit in order to use ACM or MuLaw tools.
ieeeToPcm = new SampleToWaveProvider16(resamplingProvider);
Then create a custom IWaveProvider for the next step.
// In MuLawConversionProvider
public int Read(byte[] destinationBuffer, int offset, int readingCount)
{
// Source buffer has twice as many items as the output array.
var sizeOfPcmBuffer = readingCount * 2;
_sourceBuffer = BufferHelpers.Ensure(_sourceBuffer, sizeOfPcmBuffer);
var sourceBytesRead = _sourceProvider.Read(_sourceBuffer, offset * 2, sizeOfPcmBuffer);
var samplesRead = sourceBytesRead / 2;
var outIndex = 0;
for (var n = 0; n < sizeOfPcmBuffer; n += 2)
{
destinationBuffer[outIndex++] = MuLawEncoder.LinearToMuLawSample(BitConverter.ToInt16(_sourceBuffer, offset + n));
}
return samplesRead * 2;
}
The new provider can be sent directly to WaveOut
outputStream = new MuLawConversionProvider(ieeeToPcm);
waveOut.Init(outputStream);
waveOut.Play();
These filters remain in place with the BufferedWaveProvider as the "root". Whenever you call BufferedWaveProvider.AddSamples(), the data will go through all these filters.
I'm currently using the FFMpeg AutoGen project (in C# of course) to decode frames from an audio file and add them to a new stream that is being written to a video. This all works correctly, but I was wondering how one would go about mixing two AVFrame* objects together (after they are decoded).
I've mixed PCM data before but was wondering if FFmpeg had a built in API to do the work more effectively.
This is how I'm currently doing it:
short* baseFrameBuffer1 = (short*)baseFrame->data_0;
short* baseFrameBuffer2 = (short*)baseFrame->data_1;
short* readFrameBuffer1 = (short*)readFrame->data_0;
short* readFrameBuffer2 = (short*)readFrame->data_1;
for (int frameIndex = 0; frameIndex< 1024; frameIndex++)
{
int dataSample1 = GetInRangeSample(baseFrameBuffer1[frameIndex] + readFrameBuffer1[frameIndex]);
int dataSample2 = GetInRangeSample(baseFrameBuffer2[frameIndex] + readFrameBuffer2[frameIndex]);
baseFrame->data_0[frameIndex] = (byte)dataSample1;
baseFrame->data_1[frameIndex] = (byte)dataSample2;
}
private static int GetInRangeSample(int sample)
{
if (sample > short.MaxValue)
{
sample = short.MaxValue;
}
if (sample < short.MinValue)
{
sample = short.MinValue;
}
return sample;
}
You can use the amix filter in libavfilter.
I have a C# client/server network program I've written using
TCPListener and TCPClient classes. The server is reading everything
from the client (small amounts of xml) just fine until I try to send a
large file (100k) back to the client.
I'm using stream functions for
both client and server with non-blocking socket functions. When I do a
socket.SendFile("filename") back to the client, the file is getting
cut off - I've set the receive buffer size on the client to well past
100k but it still gets cut off around 25k and the communication
between client and server is unreliable afterwords.
My basic question
is what happens if data is somehow left in the pipe ? i.e.. will it be
read by the next socket.Read... Does every Send call require exactly
one and only one Read ? Maybe I'm not giving the client enough time to
read the file but their both on the same machine and I've tried
sleeping for a few seconds in various places w/o success.
It is very possible that you cannot read the entire message through one Read call (perhaps all data has not arrived yet). In network programming, you would often place the call to Read in a while loop and simply Read() until you have received the entire expected message.
1 Send call might take more than one Read call to receive, and 1 Read call might read the data send by several Send call.
TCP just provides a stream, so it's up to you to define how the data or messages you send are partitioned.
In this case, you probably just need to loop ,doing Read until the stream is closed.
You probably want something like this:
socket.Blocking = false;
const int ChunkSize = 1492;
const int ReceiveTimeout = 10000;
const int SendTimeout = 10000;
public void Send(Byte[] data)
{
var sizeBytes = BitConverter.GetBytes(data.Length);
SendInternal(sizeBytes);
SendInternal(data);
}
public Byte[] Receive()
{
var sizeBytes = ReceiveInternal(4);
var size = BitConverter.ToInt32(sizeBytes, 0);
var data = ReceiveInternal(size);
return data;
}
private void SendInternal(Byte[] data)
{
var error = SocketError.Success;
var lastUpdate = Environment.TickCount;
var size = data.Length;
var count = 0;
var sent = 0;
while (sent < size)
{
count = Math.Min(ChunkSize, size - sent);
count = socket.Send(data, sent, count, SocketFlags.None, out error);
if (count > 0)
{
sent += count;
lastUpdate = Environment.TickCount;
}
if (error != SocketError.InProgress && error != SocketError.Success && error != SocketError.WouldBlock)
throw new SocketException((Int32)error);
if (Environment.TickCount - lastUpdate > SendTimeout)
throw new TimeoutException("Send operation timed out.");
if (count == 0 && !socket.Poll(100, SelectMode.SelectWrite))
throw new SocketException((Int32)SocketError.Shutdown);
}
}
private Byte[] ReceiveInternal(Int32 size)
{
var error = SocketError.Success;
var lastUpdate = Environment.TickCount;
var buffer = new Byte[ChunkSize];
var count = 0;
var received = 0;
using (var ms = new MemoryStream(size))
{
while (received < size)
{
count = Math.Min(ChunkSize, size - received);
count = socket.Receive(buffer, 0, count, SocketFlags.None, out error);
if (count > 0)
{
ms.Write(buffer, 0, count);
received += count;
lastUpdate = Environment.TickCount;
}
if (error != SocketError.InProgress && error != SocketError.Success && error != SocketError.WouldBlock)
throw new SocketException((Int32)error);
if (Environment.TickCount - lastUpdate > ReceiveTimeout)
throw new TimeoutException("Receive operation timed out.");
if (count == 0 && socket.Poll(100, SelectMode.SelectRead) && socket.Available == 0)
throw new SocketException((Int32)SocketError.Shutdown);
}
return ms.ToArray();
}
}
What I would usually do is create a header structure that is sent
Header Size (int, 4 bytes)
File Name Offset (int, 4 bytes)
File Name Size (int , 4 bytes)
File Data Offset (int, 4 bytes)
File Data Size (int , 4 bytes)
[ message data here]
and then that header is read using either a BinaryReader or copying the bytes to a struct using Marshal. This way you always know what data is arriving and how many times you need to call Read().
The header size field is also helps with versioning the protocol (keep the structure the same but add to it for later clients so you can keep backwards compatibility). If you define the structure in C# be sure to do it like so:
[StructLayout LayoutKind.Sequential]
struct MessageHeader
{
public int HeaderSize;
public int FileNameOffset;
public int FileNameSize;
public int FileDataOffset;
public int FileDataSize;
}
Then Marshal.PtrToStructure will allow you do create an instance of this struct right from the byte[] you read from the socket
Try sending the chunk from the server side in chunks. Just as the other said, posting the code would be of great help to us.