GStreamer# equivalent for video overlay functions

GStreamer# equivalent for video overlay functions - c#

I have a C#/Mono application for rendering video streams and, until recently, I've been using my own in-house developed InterOp bindings.
I'm now moving from that method to use those of GStreamer#, since that's likely to be much less maintenance effort, at least for me :-)
Since I need to bind independent video streams to a specific DrawableArea widgets, my method captured the GStreamer messages then checked them with the function gst_is_video_overlay_prepare_window_handle_message(msg)(a).
If that returned true, I then responded with a call to gst_video_overlay_set_window_handle(xid), where xid was the handle for the widget obtained earlier with gdk_x11_window_get_xid().
My problem is this: searching through the GStreamer# code, I cannot find an equivalent function for doing the binding so I was wondering how this is meant to be done.
Anyone have any advice or information to offer?
The source code for those two functions are in gst-plugins-base-1.4.4/gst-libs/gst/video/videooverlay.c so, in a pinch, I could dummy up my own functions to do the job (or just stick with our bindings for that one little bit) but it seems to me this would have been included in GStreamer# since rendering to specific widgets seems like a very handy facility.
(a) Those GStreamer wags. They must replace their keyboards quite a bit with all that unnecessary typing :-)

Turns out those functions are available, but in a separate library and sub-namespace to the baseline GStreamer stuff.
The getting of the Xid is still done with a self-made InterOp binding, to wit:
[DllImport("libgdk-3", EntryPoint = "gdk_x11_window_get_xid")]
private extern static IntPtr GdkX11WindowGetXid(IntPtr window);
You also have instance-level variables for the playbin, bus and X11 window ID:
private IntPtr windowXid;
private Gst.Element playBin;
private Gst.Bus bus;
Then, when instantiating the class, you capture the Realized signal and ensure that all messages on the GStreamer bus go to your callback function:
this.Realized += OnRealized;
playBin = Gst.ElementFactory.Make('playbin', 'playbin');
bus = playBin.Bus;
bus.AddSignalWatch();
bus.Message += MessageCallback;
In that realisation function, you save the Xid for later use (in an instance variable):
void OnRealized(object o, EventArgs args) {
windowXid = GdkX11WindowGetXid(this.Window.Handle);
}
and, when asked by GStreamer, provide this handle in response to the request:
private void MessageCallback(object o, MessageArgs args) {
Gst.Message msg = args.Message;
if (! Gst.Video.Global.IsVideoOverlayPrepareWindowHandleMessage(msg))
return;
Gst.Element src = msg.Src as Gst.Element;
if (src == null)
return;
Gst.Element overlay = null;
if (src is Gst.Bin)
overlay = ((Gst.Bin)src).GetByInterface
Gst.Video.VideoOverlayAdapter = new Gst.Video.VideoOverlayAdapter(overlay.Handle);
adapter.WindowHandle = windowXid;
}
Note that I've fully qualified all the GStreamer objects so it's absolutely clear where they can be found. The code would probably be a lot cleaner without them (and with using var for the variables) but I wanted to ensure all information was available.

Related

Console.WriteLine is not a part of my "using System;" (System.Console does not exist) [duplicate]

Is there a way to write to console / command prompt / powershell (like Console.WriteLine()) or anything similar in UWP apps?
If console is unavailable is there a proper alternative that I can use instead to write to the screen large amounts of text?
I, of course, can make a XAML control and output to it, but it doesn't seem to be convenient in comparison to simple Console.WriteLine().
There is also very old discussion on WPF console, but nothing seems to work from there (at least, I couldn't find Project-Properties-Application tab-Output Type-Console Application and Trace.WriteLine("text") is unavailable).

You can use Debug.WriteLine method from System.Diagnostics namespace
MSDN Link
When you start debugging your application those messages will be displayed in the Output Window (Standard VS shortcut is Ctrl+Alt+O, ReSharper shortcut is Ctrl+W, O)

Starting with RS4 (the release coming out mid-2018) you can build command - line apps with UWP or output info to the command line. The pre-release SDK is already available and you can watch a Channel 9 video.

You can use the LoggingChannel class. to create ETW trace events.
The cool thing with LoggingChannel is you can do sophisticated traces (and use advanced tools like PerfView, etc.), but you can also have a simple equivalent of Debug.WriteLine in terms of simplicity with the LoggingChannel.LogMessage Method
public void LogMessage(String eventString)
or
public void LogMessage(String eventString, LoggingLevel level)
This has numerous advantages over Debug.WriteLine:
it's much faster, you can log millions of messages easily, while Debug.WriteLine is dog slow (based on the archaic Windows' OutputDebugString function).
it doesn't block the sender nor the receiver.
each channel is identified by its own guid, while with Debug.WriteLine you get all traces from everywhere, everyone, it's a bit messy to find your own ones.
you can use a trace level (Critical, Error, Information, Verbose, Warning)
you can use PerfView (if you really want to) or Device Portal or any other ETW tool.
So, to send some traces, just add this:
// somewhere in your initialization code, like in `App` constructor
private readonly static LoggingChannel _channel = new LoggingChannel("MyApp",
new LoggingChannelOptions(),
new Guid("01234567-01234-01234-01234-012345678901")); // change this guid, it's yours!
....
// everywhere in your code. add simple string traces like this
_channel.LogMessage("hello from UWP!");
....
Now, if you want a simple way to display those traces on your local machine, beyond using PerfView or other ETW tools, you can use a free open source GUI tool I write called WpfTraceSpy available here: https://github.com/smourier/TraceSpy#wpftracespy or here is a sample .NET Framework Console app that will output all traces and their level to the console:
using System;
using System.Runtime.InteropServices;
using Microsoft.Diagnostics.Tracing; // you need to add the Microsoft.Diagnostics.Tracing.TraceEvent nuget package
using Microsoft.Diagnostics.Tracing.Session;
namespace TraceTest
{
class Program
{
static void Main()
{
// create a real time user mode session
using (var session = new TraceEventSession("MySession"))
{
// use UWP logging channel provider
session.EnableProvider(new Guid("01234567-01234-01234-01234-012345678901")); // use the same guid as for your LoggingChannel
session.Source.AllEvents += Source_AllEvents;
// Set up Ctrl-C to stop the session
Console.CancelKeyPress += (object s, ConsoleCancelEventArgs a) => session.Stop();
session.Source.Process(); // Listen (forever) for events
}
}
private static void Source_AllEvents(TraceEvent obj)
{
// note: this is for the LoggingChannel.LogMessage Method only! you may crash with other providers or methods
var len = (int)(ushort)Marshal.ReadInt16(obj.DataStart);
var stringMessage = Marshal.PtrToStringUni(obj.DataStart + 2, len / 2);
// Output the event text message. You could filter using level.
// TraceEvent also contains a lot of useful informations (timing, process, etc.)
Console.WriteLine(obj.Level + ":" + stringMessage);
}
}
}

I just want to also add that debug.writeline seems to best work on the main thread so if async/await is used use something like Device.BeginInvokeOnMainThread(() => Debug.WriteLine(response)); to print to the console

UI Automation events stop being received after a while monitoring an application and then restart after some time

We are using Microsoft's UIAutomation framework to develop a client that monitors events of a specific application and responds to them in different ways. We've started with the managed version of the framework, but due to delay issues, moved to the native version wrapped in UIACOMWrapper. After more issues with performance inside our (massive) WPF application, we decided to move it to a separate terminal application (transfer the events to our WPF app through UDP) which seemed to fix all the performance issues. The only problem is that it seems that every several minutes, the events for TabSelection, StructureChanged, WindowOpened and WindowClosed stop being captured for a few minutes. Surprisingly PropertyChanged events are still received and handled while this happens. I will post the relevant code of our event monitor, but this is probably irrelevant as we have seen similar behavior when using Microsoft's own AccEvent utility. I can't post the code of the monitored application as it is proprietary and confidential as well, I can say that it is a WinForms application that hosts WPF windows and also quite massive.
Has anyone seen this sort of behavior while working with the UI Automation framework?
Thank you for your time.
Here's the monitor code (I know the event handling is on the UI Automation threads here but moving it to a dedicated thread did not change anything):
public void registerHandlers()
{
//Register on structure changed and window opened events
System.Windows.Automation.Automation.AddStructureChangedEventHandler(
this.getMsAutomationElement(), System.Windows.Automation.TreeScope.Subtree, this.handleStructureChanged);
System.Windows.Automation.Automation.AddAutomationEventHandler(
System.Windows.Automation.WindowPattern.WindowOpenedEvent,
this.getMsAutomationElement(),
System.Windows.Automation.TreeScope.Subtree,
this.handleWindowOpened);
System.Windows.Automation.Automation.AddAutomationEventHandler(
System.Windows.Automation.WindowPattern.WindowClosedEvent,
System.Windows.Automation.AutomationElement.RootElement,
System.Windows.Automation.TreeScope.Subtree,
this.handleWindowClosed);
this.registerValueChanged();
this.registerTextNameChange();
this.registerTabSelected();
this.registerRangeValueChanged();
}
private void registerRangeValueChanged()
{
if (this.getMsAutomationElement() != null)
{
System.Windows.Automation.Automation.AddAutomationPropertyChangedEventHandler(
this.getMsAutomationElement(),
System.Windows.Automation.TreeScope.Subtree, this.handlePropertyChange,
System.Windows.Automation.RangeValuePattern.ValueProperty);
}
}
private void unregisterRangeValueChanged()
{
System.Windows.Automation.Automation.RemoveAutomationPropertyChangedEventHandler(
this.getMsAutomationElement(),
this.handlePropertyChange);
}
private void registerValueChanged()
{
if (this.getMsAutomationElement() != null)
{
System.Windows.Automation.Automation.AddAutomationPropertyChangedEventHandler(
this.getMsAutomationElement(),
System.Windows.Automation.TreeScope.Subtree, this.handlePropertyChange,
System.Windows.Automation.ValuePattern.ValueProperty);
}
}
private void unregisterValueChanged()
{
System.Windows.Automation.Automation.RemoveAutomationPropertyChangedEventHandler(
this.getMsAutomationElement(),
this.handlePropertyChange);
}
private void registerTextNameChange()
{
if (this.getMsAutomationElement() != null)
{
System.Windows.Automation.Automation.AddAutomationPropertyChangedEventHandler(
this.getMsAutomationElement(),
System.Windows.Automation.TreeScope.Subtree, this.handlePropertyChange,
System.Windows.Automation.AutomationElement.NameProperty);
}
}
private void unregisterTextNameChange()
{
System.Windows.Automation.Automation.RemoveAutomationPropertyChangedEventHandler(
this.getMsAutomationElement(),
this.handlePropertyChange);
}
private void handleWindowOpened(object src, System.Windows.Automation.AutomationEventArgs e)
{
Console.ForegroundColor = ConsoleColor.Magenta;
Console.WriteLine(DateTime.Now.ToShortTimeString() + " " + "Window opened:" + " " +
(src as System.Windows.Automation.AutomationElement).Current.Name);
System.Windows.Automation.AutomationElement element = src as System.Windows.Automation.AutomationElement;
//this.sendEventToPluginQueue(src, e, element.GetRuntimeId(), this.getAutomationParent(element).GetRuntimeId());
//Fill out the fields of the control added message
int[] parentId = this.getAutomationParent(element).GetRuntimeId();
this.copyToIcdArray(parentId,
this.protocol.getMessageSet().outgoing.ControlAddedMessage.Data.controlAdded.parentRuntimeId);
this.copyToIcdArray(element.GetRuntimeId(),
this.protocol.getMessageSet().outgoing.ControlAddedMessage.Data.controlAdded.runtimeId);
//Send the message using the protocol
this.protocol.send(this.protocol.getMessageSet().outgoing.ControlAddedMessage);
}
private void copyToIcdArray(int[] runtimeId, ICD.UI_AUTOMATION.RuntimeId icdRuntimeId)
{
icdRuntimeId.runtimeIdNumberOfItems.setVal((byte)runtimeId.Count());
for (int i = 0; i < runtimeId.Count(); i++)
{
icdRuntimeId.runtimeIdArray.getElement(i).setVal(runtimeId[i]);
}
}
private void handleWindowClosed(object src, System.Windows.Automation.AutomationEventArgs e)
{
if (src != null)
{
Console.ForegroundColor = ConsoleColor.Cyan;
Console.WriteLine(DateTime.Now.ToShortTimeString() + " " + "Window closed:" + " " +
(src as System.Windows.Automation.AutomationElement).GetRuntimeId().ToString());
System.Windows.Automation.AutomationElement element = src as System.Windows.Automation.AutomationElement;
this.copyToIcdArray(element.GetRuntimeId(),
this.protocol.getMessageSet().outgoing.ControlRemovedMessage.Data.controlRemoved.runtimeId);
//Send the message using the protocol
this.protocol.send(this.protocol.getMessageSet().outgoing.ControlRemovedMessage);
//this.sendEventToPluginQueue(src, e, element.GetRuntimeId());
}
}
EDIT:
I forgot to mention that I strongly suspect that the issue is that one of the UI-Automation event handler threads gets stuck somehow. The reason I believe this, is that when the problem occurred in my monitor, I started an instance of AccEvent and it received all the missing events that my monitor was not getting. This means that the events are being fired but not passed to my monitor.
EDIT2:
I forgot to mention that this happens running in Windows 8 with the specific target application, I have not seen this phenomenon on my own Windows 7 machine with other applications. Another interesting thing is that it seems to happen periodically more or less, but regardless of when I subscribe to events, i.e. it can happen almost immediately after subscribing but then it takes several minutes to reoccur.

I'm afraid I don't know the cause of the delays that you're seeing, but here are some thoughts on this...
Everything I say below relates to the native UIA API in Windows, not the managed .NET UIA API. All improvements to UIA in recent years have been made to the Windows UIA API. So whenever I write UIA client C# code, I call UIA through a managed wrapper that I generate with the tlbimp.exe SDK tool.
That is, I first generate the wrapper with a command like...
"C:\Program Files (x86)\Microsoft SDKs\Windows\v8.1A\bin\NETFX 4.5.1 Tools\x64\tlbimp.exe" c:\windows\system32\uiautomationcore.dll /out:Interop.UIAutomationCore.dll
Then I include a reference to the Interop.UIAutomationCore.dll in my C# project, add "using Interop.UIAutomationCore;" to my C# file, and then I can do things like...
IUIAutomation uiAutomation = new CUIAutomation8();
IUIAutomationElement rootElement = uiAutomation.GetRootElement();
uiAutomation.AddAutomationEventHandler(
20016, // UIA_Window_WindowOpenedEventId
rootElement,
TreeScope.TreeScope_Descendants,
null,
this);
...
public void HandleAutomationEvent(IUIAutomationElement sender, int eventId)
{
// Got a window opened event...
}
In Windows 7, there were some important constraints around UIA event handlers. It was easy to write event handlers which didn't account for those constraints, and that could lead to long delays when interacting with UIA. For example, it was important to not add or remove a UIA event handler from inside an event handler. So at the time, I intentionally made no UIA calls at all from inside my event handlers. Instead, I'd post myself a message or add some action to a queue, allow my event handler to return, and take whatever action I wanted to in response to the event shortly afterwards on another thread. This required some more work on my part, but I didn't want to risk hitting delays. And any threads I created would be running in an MTA.
An example of the action described above is in my old focus tracking sample up at https://code.msdn.microsoft.com/windowsapps/Windows-7-UI-Automation-6390614a/sourcecode?fileId=21469&pathId=715901329. The file FocusEventHandler.cs creates the MTA thread and queues messages to avoid making UIA calls inside the event hander.
Since Window 7, I know the constraints in UIA relating to threading and delays have been relaxed, and the likelihood of encountering delays has been reduced. More recently, there were some improvements between Windows 8.1 and Windows 10 in this area, so if it'd be practical to run your code on Windows 10, it would be interesting to see if the delays still repro there.
I know this is time consuming, but you might be interested in removing the interaction with UIA inside your event handlers and seeing if the delays go away. If they do, it'd be a case of determining which action seems to trigger the problem, and seeing if there's an alternative way of achieving your goals without performing the UIA interaction in the event handlers.
For example, in your event handler, you call...
this.getAutomationParent(element).GetRuntimeId();
I expect this will lead to two calls back into the provider app which generated the event. The first call is to get the parent of the source element, and the second call is to get the RuntimeId of that parent. So while UIA is waiting for your event handler to return, you've called twice back into UIA. While I don't know that that's a problem, I'd avoid it.
Sometimes you can avoid a cross-proc call back to the provider process by having some data of interest cached with the event itself. For example, say I know I'm going to want the RuntimeId of an element that raised a WindowOpened event. I can ask UIA to cache that data with the events I receive, when I register for the events.
int propertyRuntimeId = 30000; // UIA_RuntimeIdPropertyId
...
IUIAutomationCacheRequest cacheRequestRuntimeId = uiAutomation.CreateCacheRequest();
cacheRequestRuntimeId.AddProperty(propertyRuntimeId);
uiAutomation.AddAutomationEventHandler(
20016, // UIA_Window_WindowOpenedEventId
rootElement,
TreeScope.TreeScope_Descendants,
cacheRequestRuntimeId,
this);
...
public void HandleAutomationEvent(IUIAutomationElement sender, int eventId)
{
// Got a window opened event...
// Get the RuntimeId from the source element. Because that data is cached with the
// event, we don't have to call back through UIA into the provider process here.
int[] runtimeId = sender.GetCachedPropertyValue(propertyRuntimeId);
}
On a side note, when practical, I always cache data when dealing with events or accessing elements through UIA, (by using calls such as FindFirstBuildCache(),) as I want to avoid as many cross-proc calls as possible.
So my advice would be:
Use the native Windows UIA API with a managed wrapper generated by tlbimp.exe.
Cache as much data as possible with the events, to avoid having to call back into the provider process unnecessarily later.
Avoid calls back into UIA from inside a UIA event handler.
Thanks,
Guy

I have seen this behavior in my project. The solution was unsubscribes and resubscribe to the events using a timer.
In addition, I set off any action following the events in a new task (running in an STA thread pool).

Live video encoding using ffmpeg or expression encoder [duplicate]

So I know its a fairly big challenge but I want to write a basic movie player/converter in c# using the FFmpeg library. However, the first obstacle I need to overcome is wrapping the FFmpeg library in c#. I've downloaded ffmpeg but couldn't compile it on Windows, so I downloaded a precompiled version for me. Ok awesome. Then I started looking for C# wrappers.
I have looked around and have found a few wrappers such as SharpFFmpeg (http://sourceforge.net/projects/sharpffmpeg/) and ffmpeg-sharp (http://code.google.com/p/ffmpeg-sharp/). First of all, I wanted to use ffmpeg-sharp as its LGPL and SharpFFmpeg is GPL. However, it had quite a few compile errors. Turns out it was written for the mono compiler, I tried compiling it with mono but couldn't figure out how. I then started to manually fix the compiler errors myself, but came across a few scary ones and thought I'd better leave those alone. So I gave up on ffmpeg-sharp.
Then I looked at SharpFFmpeg and it looks like what I want, all the functions P/Invoked for me. However its GPL? Both the AVCodec.cs and AVFormat.cs files look like ports of avcodec.c and avformat.c which I reckon I could port myself? Then not have to worry about licencing.
But I want to get this right before I go ahead and start coding. Should I:
Write my own C++ library for interacting with ffmpeg, then have my C# program talk to the C++ library in order to play/convert videos etc.
OR
Port avcodec.h and avformat.h (is that all i need?) to c# by using a whole lot of DllImports and write it entirely in C#?
First of all consider that I'm not great at C++ as I rarely use it but I know enough to get around. The reason I'm thinking #1 might be the better option is that most FFmpeg tutorials are in C++ and I'd also have more control over memory management than if I was to do it in c#.
What do you think?
Also would you happen to have any useful links (perhaps a tutorial) for using FFmpeg?

The original question is now more than 5 years old. In the meantime there is now a solution for a WinRT solution from ffmpeg and an integration sample from Microsoft.

a few other managed wrappers for you to check out
FFMpeg.NET
FFMpeg-Sharp
Writing your own interop wrappers can be a time-consuming and difficult process in .NET. There are some advantages to writing a C++ library for the interop - particularly as it allows you to greatly simplify the interface that the C# code. However, if you are only needing a subset of the library, it might make your life easier to just do the interop in C#.

You can use this nuget package:
Install-Package Xabe.FFmpeg
I'm trying to make easy to use, cross-platform FFmpeg wrapper.
You can find more information about this at Xabe.FFmpeg
More info in documentation
Conversion is simple:
var conversion = await FFmpeg.Conversions.FromSnippet.ToMp4(Resources.MkvWithAudio, output);
await conversion.Start();

GPL-compiled ffmpeg can be used from non-GPL program (commercial project) only if it is invoked in the separate process as command line utility; all wrappers that are linked with ffmpeg library (including Microsoft's FFMpegInterop) can use only LGPL build of ffmpeg.
You may try my .NET wrapper for FFMpeg: Video Converter for .NET (I'm an author of this library). It embeds FFMpeg.exe into the DLL for easy deployment and doesn't break GPL rules (FFMpeg is NOT linked and wrapper invokes it in the separate process with System.Diagnostics.Process).

A solution that is viable for both Linux and Windows is to just get used to using console ffmpeg in your code. I stack up threads, write a simple thread controller class, then you can easily make use of what ever functionality of ffmpeg you want to use.
As an example, this contains sections use ffmpeg to create a thumbnail from a time that I specify.
In the thread controller you have something like
List<ThrdFfmpeg> threads = new List<ThrdFfmpeg>();
Which is the list of threads that you are running, I make use of a timer to Pole these threads, you can also set up an event if Pole'ing is not suitable for your application.
In this case thw class Thrdffmpeg contains,
public class ThrdFfmpeg
{
public FfmpegStuff ffm { get; set; }
public Thread thrd { get; set; }
}
FFmpegStuff contains the various ffmpeg functionality, thrd is obviously the thread.
A property in FfmpegStuff is the class FilesToProcess, which is used to pass information to the called process, and receive information once the thread has stopped.
public class FileToProcess
{
public int videoID { get; set; }
public string fname { get; set; }
public int durationSeconds { get; set; }
public List<string> imgFiles { get; set; }
}
VideoID (I use a database) tells the threaded process which video to use taken from the database.
fname is used in other parts of my functions that use FilesToProcess, but not used here.
durationSeconds - is filled in by the threads that just collect video duration.
imgFiles is used to return any thumbnails that were created.
I do not want to get bogged down in my code when the purpose of this is to encourage the use of ffmpeg in easily controlled threads.
Now we have our pieces we can add to our threads list, so in our controller we do something like,
AddThread()
{
ThrdFfmpeg thrd;
FileToProcess ftp;
foreach(FileToProcess ff in `dbhelper.GetFileNames(txtCategory.Text))`
{
//make a thread for each
ftp = new FileToProcess();
ftp = ff;
ftp.imgFiles = new List<string>();
thrd = new ThrdFfmpeg();
thrd.ffm = new FfmpegStuff();
thrd.ffm.filetoprocess = ftp;
thrd.thrd = new `System.Threading.Thread(thrd.ffm.CollectVideoLength);`
threads.Add(thrd);
}
if(timerNotStarted)
StartThreadTimer();
}
Now Pole'ing our threads becomes a simple task,
private void timerThreads_Tick(object sender, EventArgs e)
{
int runningCount = 0;
int finishedThreads = 0;
foreach(ThrdFfmpeg thrd in threads)
{
switch (thrd.thrd.ThreadState)
{
case System.Threading.ThreadState.Running:
++runningCount;
//Note that you can still view data progress here,
//but remember that you must use your safety checks
//here more than anywhere else in your code, make sure the data
//is readable and of the right sort, before you read it.
break;
case System.Threading.ThreadState.StopRequested:
break;
case System.Threading.ThreadState.SuspendRequested:
break;
case System.Threading.ThreadState.Background:
break;
case System.Threading.ThreadState.Unstarted:
//Any threads that have been added but not yet started, start now
thrd.thrd.Start();
++runningCount;
break;
case System.Threading.ThreadState.Stopped:
++finishedThreads;
//You can now safely read the results, in this case the
//data contained in FilesToProcess
//Such as
ThumbnailsReadyEvent( thrd.ffm );
break;
case System.Threading.ThreadState.WaitSleepJoin:
break;
case System.Threading.ThreadState.Suspended:
break;
case System.Threading.ThreadState.AbortRequested:
break;
case System.Threading.ThreadState.Aborted:
break;
default:
break;
}
}
if(flash)
{//just a simple indicator so that I can see
//that at least one thread is still running
lbThreadStatus.BackColor = Color.White;
flash = false;
}
else
{
lbThreadStatus.BackColor = this.BackColor;
flash = true;
}
if(finishedThreads >= threads.Count())
{
StopThreadTimer();
ShowSample();
MakeJoinedThumb();
}
}
Putting your own events onto into the controller class works well, but in video work, when my own code is not actually doing any of the video file processing, poling then invoking an event in the controlling class works just as well.
Using this method I have slowly built up just about every video and stills function I think I will ever use, all contained in the one class, and that class as a text file is useable on the Lunux and Windows version, with just a small number of pre-process directives.

How Can I Hook a Youtube Video (Flash Player?) To Slow Down Playback?

The only good software I know which can decelerate and accelerate the playback of a YouTube video in any browser without first downloading it (because that would be cumbersome), is Enounce MySpeed.
Unfortunately, this software is not free, and my trial version ran out. I was playing around with its registry settings and noticed a few keys:
ProgramsToHook: iexplore.exe;firefox.exe;plugin-container.exe;chrome.exe;safari.exe;opera.exe;maxthon.exe;feeddemon.exe;realplay.exe;flvplayer.exe;flv player.exe;flock.exe;adobe media player.exe
UseFlashAdapter: 1
LLModules: ole32.dll;nspr4.dll;chrome.exe;realplay.exe;objb3201.dll;oleaut32.dll;rpflashplayer.dll
ModulesToIntercept: flash10*;flash9*;npswf32.dll;gcswf32.dll;fldbg10*;flashplayer.3.1.1k.ocx;adobe media player.exe
Based on the names and values of these registry keys, I'm guessing the MySpeed software hooks some function(s) in the listed modules (but modules are or aren't the same as DLLs?..) and does so for each process listed in ProgramsToHook. This is what I don't understand. What is the concept of the MySpeed software. Obviously it's hooking something, but I'm not too familiar with the intricacies of Windows hooks so I came to ask you experts. I'm thinking if I can understand how this hook process works, I can make my own version of the software using EasyHook, which is a fantastic .NET library to perform user-mode and kernel-mode hooks.
I thought that Windows user-mode hooking goes something like this. You choose one function in one DLL, and you intercept that function (a.k.a hook) in one process you want. If you want to hook the DLL in multiple processes, you just have to repeat the procedure for each process.
And then kernel-mode hooking is just choosing one function in one DLL and intercepting that function in every process that calls it (hence kernel-mode). But surely there are tons of ways to hook; I'm not too sure on whats the difference between these two hooks and DLL injection either.
So the point is, I'd like to know how MySpeed works. What is their hooking concept? If I can know this then I can make such a software in .NET!
Thanks in advance.

I can't provide you with an accurate explanation as I don't know the API calls or capabilites, but it goes something like this:
You app looks for iexplore.exe where it intercepts calls to certain modules. The module is mainly flash player. Flash has support for playing the video slower so you modify the call from iexplore.exe (JavaScript play button on webpage) or make an additional call to set playback speed.
What you need to do:
Use this tool to check what is actually happening: http://www.nektra.com/products/deviare-api-hook-windows/
Learn how to ask Flash Player to slow down a video (probably in Flash API docs). One Simple approach could be to see what MySpeed is actually doing using the Deviare API hook tool.
Write a program that replicates this procedure. It involves intercepting messages sent from one handle (iexplore.exe) to another (flash .dll). This can't be done externally, it has to be done internally, so this may be of help: http://www.codeproject.com/KB/threads/winspy.aspx
On hooks: http://msdn.microsoft.com/en-gb/library/ms644960.aspx
I don't think many people has done this in C#, so it could offer a challenge. I would though be interested in the progress (obstacles) if you have a blog or something to share the gory details on. :)
EDIT: The Deviare API Hook software seems not only to spy on calls, but also allow you to intercept them. So its a all-in-one package for your needs. :)
EDIT2: Relevant question: How do I intercept messages being sent to a window?

The key to speeding up or slowing down a video is to convince multimedia players that your computer is slower or faster than it really is. This can be accomplished hooking timeGetTime().
This is an extremely easy C# code to accomplish it:
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Diagnostics;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using Nektra.Deviare2;
namespace DeviareTest
{
public partial class Form1 : Form
{
private int nSpeed;
private uint nTime;
private NktSpyMgr _spyMgr;
public Form1()
{
InitializeComponent();
_spyMgr = new NktSpyMgr();
_spyMgr.Initialize();
_spyMgr.OnFunctionCalled += new DNktSpyMgrEvents_OnFunctionCalledEventHandler(OnFunctionCalled);
}
private void Form1_Load(object sender, EventArgs e)
{
NktHook hook = _spyMgr.CreateHook("WINMM.dll!timeGetTime", (int)(eNktHookFlags.flgOnlyPostCall));
hook.Hook(true);
bool bProcessFound = false;
NktProcessesEnum enumProcess = _spyMgr.Processes();
NktProcess tempProcess = enumProcess.First();
while (tempProcess != null)
{
if (tempProcess.Name.Equals("iexplore.exe", StringComparison.InvariantCultureIgnoreCase) && tempProcess.PlatformBits == 32)
{
hook.Attach(tempProcess, true);
bProcessFound = true;
}
tempProcess = enumProcess.Next();
}
if(!bProcessFound)
{
MessageBox.Show("Please run \"iexplore.exe\" before!", "Error");
Environment.Exit(0);
}
}
private void OnFunctionCalled(NktHook hook, NktProcess process, NktHookCallInfo hookCallInfo)
{
nTime++;
if (nSpeed==-2)
hookCallInfo.Result().LongVal = hookCallInfo.Result().LongVal - (int)(nTime * 0.2);
else if(nSpeed==2)
hookCallInfo.Result().LongVal = hookCallInfo.Result().LongVal + (int)(nTime * 3);
}
private void SlowButton_CheckedChanged(object sender, EventArgs e)
{
nSpeed = -2;
}
private void FastButton_CheckedChanged(object sender, EventArgs e)
{
nSpeed = 2;
}
}
}
I just published an article with a code example showing how to do this with the Deviare hooking engine. The sample code only works with the video part (not audio) and it is available here.

Youtube now has an html5 player with playback speed controls.
All you have to do is enable html5 here http://www.youtube.com/html5
Only some of the videos support the html5 player yet, though.
Hope this helps.

The key to speeding up or slowing down a video is to convince multimedia players that your computer is slower or faster than it really is
manipulating the system time will be a VERY dangerous and idiotic thing to do - not only will you break user-mode threadslices and hence have a serious impact on system-performance but you also will break many logging-functionalities and even user-mode reflectors which control KM-drivers ... this could both crash and physically harm (!) your system because modern hardware is programmable, given the correct (& proprietary, of course) set of func-calls and such. I would highly advise to NOT do reproduce this, even a few AV-apps will intercept this because of its dangerous nature.
But you're somewhat lucky : the kernel uses its own time, synced to hardware so windows itself COULD remain stable for a limited amount of time.
I think you should get back to the drawing-board, manipulating essential structures of your operating-system certainly is not the right way to accomplish your goal.

Using FFmpeg in .net?

The original question is now more than 5 years old. In the meantime there is now a solution for a WinRT solution from ffmpeg and an integration sample from Microsoft.

a few other managed wrappers for you to check out
FFMpeg.NET
FFMpeg-Sharp
Writing your own interop wrappers can be a time-consuming and difficult process in .NET. There are some advantages to writing a C++ library for the interop - particularly as it allows you to greatly simplify the interface that the C# code. However, if you are only needing a subset of the library, it might make your life easier to just do the interop in C#.

You can use this nuget package:
Install-Package Xabe.FFmpeg
I'm trying to make easy to use, cross-platform FFmpeg wrapper.
You can find more information about this at Xabe.FFmpeg
More info in documentation
Conversion is simple:
var conversion = await FFmpeg.Conversions.FromSnippet.ToMp4(Resources.MkvWithAudio, output);
await conversion.Start();

GPL-compiled ffmpeg can be used from non-GPL program (commercial project) only if it is invoked in the separate process as command line utility; all wrappers that are linked with ffmpeg library (including Microsoft's FFMpegInterop) can use only LGPL build of ffmpeg.
You may try my .NET wrapper for FFMpeg: Video Converter for .NET (I'm an author of this library). It embeds FFMpeg.exe into the DLL for easy deployment and doesn't break GPL rules (FFMpeg is NOT linked and wrapper invokes it in the separate process with System.Diagnostics.Process).

A solution that is viable for both Linux and Windows is to just get used to using console ffmpeg in your code. I stack up threads, write a simple thread controller class, then you can easily make use of what ever functionality of ffmpeg you want to use.
As an example, this contains sections use ffmpeg to create a thumbnail from a time that I specify.
In the thread controller you have something like
List<ThrdFfmpeg> threads = new List<ThrdFfmpeg>();
Which is the list of threads that you are running, I make use of a timer to Pole these threads, you can also set up an event if Pole'ing is not suitable for your application.
In this case thw class Thrdffmpeg contains,
public class ThrdFfmpeg
{
public FfmpegStuff ffm { get; set; }
public Thread thrd { get; set; }
}
FFmpegStuff contains the various ffmpeg functionality, thrd is obviously the thread.
A property in FfmpegStuff is the class FilesToProcess, which is used to pass information to the called process, and receive information once the thread has stopped.
public class FileToProcess
{
public int videoID { get; set; }
public string fname { get; set; }
public int durationSeconds { get; set; }
public List<string> imgFiles { get; set; }
}
VideoID (I use a database) tells the threaded process which video to use taken from the database.
fname is used in other parts of my functions that use FilesToProcess, but not used here.
durationSeconds - is filled in by the threads that just collect video duration.
imgFiles is used to return any thumbnails that were created.
I do not want to get bogged down in my code when the purpose of this is to encourage the use of ffmpeg in easily controlled threads.
Now we have our pieces we can add to our threads list, so in our controller we do something like,
AddThread()
{
ThrdFfmpeg thrd;
FileToProcess ftp;
foreach(FileToProcess ff in `dbhelper.GetFileNames(txtCategory.Text))`
{
//make a thread for each
ftp = new FileToProcess();
ftp = ff;
ftp.imgFiles = new List<string>();
thrd = new ThrdFfmpeg();
thrd.ffm = new FfmpegStuff();
thrd.ffm.filetoprocess = ftp;
thrd.thrd = new `System.Threading.Thread(thrd.ffm.CollectVideoLength);`
threads.Add(thrd);
}
if(timerNotStarted)
StartThreadTimer();
}
Now Pole'ing our threads becomes a simple task,
private void timerThreads_Tick(object sender, EventArgs e)
{
int runningCount = 0;
int finishedThreads = 0;
foreach(ThrdFfmpeg thrd in threads)
{
switch (thrd.thrd.ThreadState)
{
case System.Threading.ThreadState.Running:
++runningCount;
//Note that you can still view data progress here,
//but remember that you must use your safety checks
//here more than anywhere else in your code, make sure the data
//is readable and of the right sort, before you read it.
break;
case System.Threading.ThreadState.StopRequested:
break;
case System.Threading.ThreadState.SuspendRequested:
break;
case System.Threading.ThreadState.Background:
break;
case System.Threading.ThreadState.Unstarted:
//Any threads that have been added but not yet started, start now
thrd.thrd.Start();
++runningCount;
break;
case System.Threading.ThreadState.Stopped:
++finishedThreads;
//You can now safely read the results, in this case the
//data contained in FilesToProcess
//Such as
ThumbnailsReadyEvent( thrd.ffm );
break;
case System.Threading.ThreadState.WaitSleepJoin:
break;
case System.Threading.ThreadState.Suspended:
break;
case System.Threading.ThreadState.AbortRequested:
break;
case System.Threading.ThreadState.Aborted:
break;
default:
break;
}
}
if(flash)
{//just a simple indicator so that I can see
//that at least one thread is still running
lbThreadStatus.BackColor = Color.White;
flash = false;
}
else
{
lbThreadStatus.BackColor = this.BackColor;
flash = true;
}
if(finishedThreads >= threads.Count())
{
StopThreadTimer();
ShowSample();
MakeJoinedThumb();
}
}
Putting your own events onto into the controller class works well, but in video work, when my own code is not actually doing any of the video file processing, poling then invoking an event in the controlling class works just as well.
Using this method I have slowly built up just about every video and stills function I think I will ever use, all contained in the one class, and that class as a text file is useable on the Lunux and Windows version, with just a small number of pre-process directives.

Develop Reference

C# (C-Sharp) is a programming language developed by Microsoft that runs on the .NET Framework.