I have a video analytics program that processes assorted frames from a video. (Several hours long)
The video is likely going to be an MP4 but may be other formats going forwards.
At the moment, I have a C# wrapper around an ffmpeg call to extract an individual frame at the requested time. (I'm using the ffmpeg.exe binary. Not the libraries directly)
At the moment, this all works. But it's slow. Very slow.
I've found ways to improve the speed by storing the extracted frames in a ramdisk while they're being processed. Changing the stored image format etc...
I just wanted to check if anyone could think of any way to pull individual frames out. At split-second accuracy.
I know this is probably possible with DShow etc... I went straight to FFMPEG as I've used it before. But if DShow is likely to be faster I'll gladly change!
In Windows you have native APIs to process, and in particular read from, media files:
DirectShow
Media Foundation
Both provide support for MP4 (H.264 video), DirectShow as a framework extended by third party MP4 Demultiplexer and H.264 decoder (of needed, also Windows 7 provides build it), and Media Foundation - natively or extended by third party extensions depending on OS version.
Both can be interfaced from .NET via open source wrappers DirectShow.NET and Media Foundation .NET respectively. This works out way faster then FFmpeg CLI for individual frames. Also note that you would be able to obtain frames incrementally without need to locate specific time and do excessive duplicated work, not even to mention process startup/initialization overhead. Alternatively you could use FFmpeg/Libav binaries through wrapper into C# and get similar performance.
You can change the position of the offset parameters. The order matters for the speed if the video contains valid meta data you can seek through the video faster.
If you put the offset before the input file the offset will be calculated with the bit rate with is not every time exactly (in case of a variable bit rate), but it is much faster. The correct way is to walk through the video (offset parameter is after the input file) but this takes time.
Related
I´m doing a "Whatsapp" like app and I need to send user videos (from camera/gallery).
I need to send video from ios to android and from android to ios (windows phone in the future).
First thing I thought is to use camera params to record the video in low resolution, but that won´t help with recorded videos stored in the phone already.
Second thought was to zip the video file, but I guess this is not enough for very large files.
Third: actually compressing the video file generating a new file, and then zip it before sending it through the network.
So this is what I need before actually sending the video:
Compress the video file, generating a new file that will play nicely in
both platforms (ios and android)
Make the compressing process aysnc(as I don´t want to block the UI
thread for a really long time)
Zip it (this is the easy part, just for the record)
Any ideas or help are appreciated
You would best need to use your platforms framework to also leverage existing hardware support for encoding (mainly h.264 hardware encoding). A PCL solution would eat to much battery as it would need to run on CPU only giving you bad performance and even worst battery live.
This ties in with 1. Just use your platforms native method to execute the frameworks methods async.
Skip this part. It will increase overhead and disallow video streaming There are virtually 0 benefits from using a zip algorithm on top of an already compressed video stream.
Just make sure that you end up with a cross platform compatible video format like H264.
Is there any easy way to get from original video detailed information as duration and frame rate??
I have to divide originalvideo to single frames and to do it I want to use NReco.VideoConverter but I in the API I could not find function which allows me to get video properties. If there is not possibility to do it, can you recommend me other useful .net wrapper to FFMpeg?
I saw of course Using FFmpeg in .net?
and Solid FFmpeg wrapper for C#/.NET but there are recommended wrappers which are quite old or have only basic functions. I look wrapper which allows me to get details from original video, divide it to single frames (i need to improve quality) and concat it to new one avi or other file of course with audio with possibility to change paramters(resolution, frame rate etc..)
Maybe this can help, it shows a way to get all metadata form a file using Windows Media Format SDK 11:
https://msdn.microsoft.com/en-us/library/windows/desktop/dd743789(v=vs.85).aspx
recently i have started workin on project of my own which is capturing a camera output using DirectShow .Net. There are few problems that i don't know how ot solve
1) How can i encode the captured stream into H.264 format.I understand i should some how add filter in the filter graph. But i wasn't able to find where and how . Also i was not able to find if there is standart H.264 filter or should i download it form some where? If i need to download it can it just be a dll to wich i add reference or should it be installer?
2) Is there a way to save the captured output into a memory object, some kind of stream or it can only be written to file?
Best Regards,
Iordan
You're can use commercial software from VisioForge or Viscomsoft.
AForge potencial problem - no audio during capture, only video. Also output formats is a very limited. But - it's free and open source, if you're have any DirectShow experiense you can add audio support.
You will need to use something like FFMpeg or Handbrake. Check out http://vidcoder.codeplex.com/.
AForge also has some video editing abilities and you can also pass filters to it. There are also several FFMpeg C# wrappers you could use such as https://github.com/crazyender/FFMPEG.net
You should use AForge.net. All of the hard work is done for you already. Use VideoFileWriter http://www.aforgenet.com/framework/docs/html/4ee1742c-44d3-b250-d6aa-90cd2d606611.htm. I appears that the AForge framework uses FFMpeg under the hood as well (See AForge.Video.FFMPEG namespace). You just create a writer and pass it the bitmap/frame writer.WriteVideoFrame(bitmap);
You should fully investigate the video abstractions in AForge. You could save yourself considerable amounts of time.
Example: http://www.aforgenet.com/framework/samples/video.html
1) Yes, You should download encoder filter library. Most of decoders are supplied with free. But encoder is not. If you don't want to pay, you have to find open source Encoder.
And To use at the graphedit program, you should do dll register with that dll file. Or the installer program do this instead. And you also need to check container type like mp4, mkv... In other words you need mux filter to save that in a file. I think someone will link available urls. Sorry I don't have urls now.
2)What is that the means of capture? Is that Movie? or Image?
In case of Image, There are many sample projects and SampleGrabbers. You can save it to both of a file and memory.
In case of Movie, I'm sure your pc memory is not enougth to them with out encoding. Make memory stream and pass it to capture manager.
You can also make a filter in order to customize filter to your needs. All of information are included in Windows SDK samples.
Where I can found something suitable (library) for .NET 4.0 (C#) capable for following features:
+ Reading mp3/wav samples with direct access to samplebuffer of loaded samples? (for example I want to load mp3 sample and programatically add reverb, chorus, and more custom effects implemented by me)
+ Playing directly audio buffers (arrays of floats)
+ Saving audio buffers to disk as mp3 or wav
Some time ago i found ASIO for .NET and this only works with .NET 3.5, is there something for .NET 4.0? Thank's a lot for your help.
ASIO for .NET won't help you in the cracking of mp3 or wav files. The most modern API in Windows for doing this is Microsoft Media Foundation. These are all COM APIs though, so you're either going to be doing (perhaps painful) COM interop from .NET or (easier) writing a C++/CLI wrapper. If you go the C++/CLI wrapper, you will need to be aware of performance issues (especially with how critical latency is to audio programming).
I'm skeptical that you will get the audio latency you need when programming in .NET. A good audio driver (like ASIO) will get you down to <3ms of latency. So if you are targetting "live audio", you will need to be generating audio buffers quicker than that (unless you are fine with longer latencies). To put it this way, the "time intervals" that the Windows APIs deal with are in 100 nanosecond intervals :).
You likely don't want to have to crack the files yourself. It becomes tedious as it's not only just mp3 and wav. You also have to be aware of how the wav is formatted as well (to account for different bit rates, number of channels, etc.). Using Media Foundation, it will automatically load the write decoder for you, you just give it the file path. Check out this tutorial that shows opening an existing WAV file and writing a new WAV file. I just recently went down this path for a drum sequencer I'm creating, and it's not very painful at all (if you're familiar with COM programming).
The central component in MF that makes this possible is the MFSourceReader.
If you're wanting to play the audio after you modify it, you can look at the sample "RenderExclusiveEventDriven" in the Windows SDK (under "audio" I believe). That's what I did for the drum sequencer as well. Latency won't be an issue and you're just dealing with byte arrays, so manipulating the raw data becomes very easy. Though at this point, you can probably stick with the ASIO .NET route and just use that to play the raw data you get from MFSourceReader.
I don't think there are .NET wrappers around Media Foundation yet (though if somebody has done that work already, feel free to post here as it would be awesome to know).
I want to make a program that takes an MP3 file and breaks it into many smaller mp3 files based on 1-2 seconds of no sound (silence).
What is the easiest way to do this in c#?
Bass library. Bass has everything you need. It can access, record and edit media streams like mp3s, allowing you to sample the volume at different time points. It has a .net api, so you can use it in c#. Unfortunately it does cost money if you are using it for a commercial application, but they do provide a free non-commercial license.
Sox is a command-line tool which has an option to split an mp3 on n seconds of silence. You could always use the system command to call sox from c#.
Other related links.
Ripping a CD to mp3 in C# - third party component or api out there?
Audio Libraries for MP3 editing
How do I merge/join mp3 files with c#
This code shows a way to make a CD
ripper in C#. There are APIs from some
vendors that allow reading audio CD
tracks but it is also possible to do
it using APIs that allow low level
access to CD drives such as ASPI from
Adaptec or IOCTL control codes. The
latter method is used in this case,
because there is no need to install
any third party software, it is
completely covered by Win32 API
functions.
http://www.codeproject.com/KB/cs/csharpripper.aspx
Splitting the MP3 stream will be difficult to do with any degree of precision. The compressed MP3 data exists as sequential chunks of audio data comprised of many samples. The easiest way to perform this would be to decode the stream either progressively or in its entirety, perform your manipulation, then re-encode it (which as I understand is how most jukebox software does it)
Having a solid knowledge of the file's binary format would be a good place to start. That done, you'll know what silence looks like in the file. You may have to define exactly what silence is. Presuming that, like most audio, it started from an analog source, there's almost certainly some noise buried in the "silence". What will your tolerance for ambient/background noise be?
Once you know what you're looking for, just scan through the file, looking for "it".
Simple ...
A program to do this already exists:
http://mp3splt.sourceforge.net/mp3splt_page/home.php