I am working for a company where we are developing video chat support on an existing application. I have looked at various solutions for this like
Using Managed Direct show for video capture and streaming in C#
Some code samples in code project where we take an image and pass it over the network (I would call it rather a crude solution as this would eat up lot of bandwidth.
Code a compression algorithm from scratch from scratch and use it to compress-decompress video.
Now the challenge is that we are looking to achieve very high quality video streaming and the container application is coded in C#.NET
This is what I have proposed so far. The network logic to stream data is written in C# , the video compression to be written in VC++ and call this VC++ dll using pinvoke or either CLI which way possible.
I am looking for some one more experienced that me in this field who can suggest me if Iam going correct or can this be still improved.
The ultimate goal is high quality video streaming.
The codec can be any anything like h.2633, h.264 etc.
I've used several ways to get video streaming/conferencing with .net easily, without need to dig into directshow. (ok, dig some, but not deep :)
1) Use of plain Windows Media Encoder components. It is documented with samples in Windows Media Encoder SDK. Good for any high resolution streaming, but delay is too big for realtime chat (0.5-2 seconds at best). Modern Express Encoder SDK another option.
2) Microsoft Research ConferenceXP http://cct.cs.washington.edu/ Full featured conferencing API including application streaming. They too low level Windows Media coded filters and wrapped them into managed code. Works well. Easily customizable. Looks bit abandoned now.
3) Microsoft RTC Client up to version 1.3 - core of windows messenger.
pros: managed samples from Microsoft, good docs, reliable performance, freely redistributable, microsoft compatible (good) SIP stack included. Major conferencing vendors like Emblaze VCON based their solutions on it in some near past, not sure about this days, but I know that Tandberg licensed Microsft's VC-1.
cons: version up to 1.3 support h261-h263 video only. modern version with support of VC-1(h264) codec does not allow direct serverless ip-ip connections. It does at require Microsoft Live Communications server. Newer version SDK does not cover well video conferencing calls.
http://msdn.microsoft.com/en-us/library/ms775892(VS.85).aspx
Please let us know what platform you have chosen. By the way, I've even used ConferenceXP video rtp part with RTC 1.3 voice/SIP features together to improve video quality, so you have wide choice of managed technologies here. Another thing is Live Meeting at which I had no chance to take good look yet.
Save yourself the trouble and use VLC. There are some decent .NET wrappers for it (http://forum.videolan.org/viewtopic.php?f=32&t=52021&start=30)
We are using C# and VLC for an IPTV network. We take input off DISH network satellites via Osprey-450 video capture devices on a Windows XP server. From there, we have a .NET server component that we wrote in C# that uses VLC behind the scenes (starting separate processes in .NET to control the vlc.exe instances). The VLC processes transcode and stream the signals over a network (.h264 or MPEG-4, we've successfully done both).
On the client side we have a C# WinForm application that uses an embedded VLC Viewer to view multicast signals. This application is mainly for command & control. The real use of the multicast signals happens when our set top boxes attached to our TV's decode and display the streams.
We thought we were going to have to write our own DirectX encoders too, but don't go to all the trouble. VLC works really well and has enough C# support to be very useful. Feel free to e-mail me if you have specific questions about implementation.
You should check out the Ucentrik SDK. This SDK will enable you to integrate rich-media functionality such as video, audio, chat, remote-desktop sharing and control and video recording in your applications. The video codecs supported are VP8 (Google), Theora, and x.264. Additionally, the rich media traffic is encapsulated within an HTTP protocol to enable it to traverse firewalls that enable normal web traffic. This technology is completely free and you can download the SDK and request an API key so that you can evaluate without investing any time on setting up the infrastructure. In the next few months, we are releasing a server component so that you can download and host the infrastructure your self or your customer. The technology supports 1-to-many connections which means that you can create video conferences if you like. The features are highly modular so that you can integrate just the video or audio or desktop share or a combination of the technologies using the same SDK... You should request for an SDK available here: www.ucentrik.com. Additionally, there are some video's here: http://www.youtube.com/user/ucentrik
good luck.
Ucentrik has just released an open-source call-center application that integrates the CTX technology. This call-center application implements the video, audio, desktop/application sharing (with control), text chat functionality available from the CTX API. The application also includes some business logic specifically around providing the ability to route the calls to an agent that is available or have specific skill-set. The project is available at http://vcca.codeplex.com - please note that you will require access to the Ucentrik CTX SDK which is available for request on the Ucentrik website (www.ucentrik.com). Good luck.
Related
I am making a C# app to capture graphic from an avermedia pcie capture card.
But it seems that there are no out of box tools to do so.
So I made a C++ directshow app to do the capture, which is a console app and opens a capture window when running.
How can I redirect the output to a C# app? for example, to a CaptureElement?
So you want to have XAML CaptureElement connected to AverMedia PCIe capture card. This sounds like a well-understood challenge overall, however every other piece of technology you mentioned is eventually a bad choice: DirectShow, multiple apps with piping, redirection and fitting of cutsom code to XAML CaptureElement control.
Microsoft has intentionally been limited ways you can integrate different APIs and so there are not so many ways to get everything together.
Let us go over the supposed integration path. The capture card is supposed to be shipped with a compatible driver:
Video capture devices are supported through the UVC class driver and must be compatible with UVC 1.1
When this is the case, such devices are visible to Media Foundation API handling video capture among tasks. XAML CaptureElement would be able to see a video capture device through this API and this way everything is supposed to work without need to fit anything from your end.
If this is not happening, it suggests you are dealing with an unsupported device coming without suitable or compatible driver.
Previous media API in Windows was DirectShow but its days are gone. It remains perfectly working as a legacy framework, a lot of applications out there are still relying on it. Specifically it will not integrate with new technology like XAML and UWP. More to that, even Media Foundation itself, the current media API, in its public offering is lagging behind when it comes to fitting with most recent technology. Having said that it is a good idea to stay well clear of DirectShow here if this is at all possible.
I see no need for cross-process design with video travelling between process through piping. There is no good reason for such design and even though this can work efficiently (Windows itself proves it can work great in terms of performance by having so called Frame Server service in it), this is not to be built on piping. In your case it is unlikely to be have to be built on multiple processes either. Instead you can develop a native code DLL project that takes care of video acquisition and connects to managed code via suitable glue layer: C++/CLI, COM, C++/WinRT and such.
Then next thing is fitting to XAML CaptureElement. The control is designed to work with Windows.Media.Capture.MediaCapture class that talks to hardware and you don't have suitable hardware as you plan to implement your own acquisition layer. Long story short you are not supposed to forward external data to CaptureElement and you would have hard time doing this. Your best strategy is to upload externally obtained data to Windows.Graphics.Imaging.SoftwareBitmap or alike and take involved performance impact as acceptable. That is, you will be dealing with video frames as images.
An alternative way is to upload acquired video frames into Direct 3D 11 textures and it would open you a more performant way of integration with video related controls, such as Windows.UI.Xaml.Controls.SwapChainPanel however it would also require that you put much more development effort in there.
Since this topic is a bit out dated I would like to re-discuss it here.
After searching the web, I came across the following link:
http://archive.msdn.microsoft.com/nesl which runs only out of browser because Silverlight (in browser) can't access certain COM libraries that are related to windows.
I wish (for obvious performance purposes) to perform the speech recognition through Silverlight (on the client machine) and then send the result (text) to the server via a postback to perform the corresponding action.
I already achieved a way to get the voice from the microphone and store it in Silverlight in a byte array. Is there a way to convert the speech byte array to text?
HTML5 Google service is not an acceptable approach since IE is required.
My final goal is to implement a speech recognition in ASP.NET Web Application.
Any suggestion is appreciated.
You can't do it in Silverlight. You'll need to send the audio somewhere. You can call some third-party service (I'm sure there are plenty--and it shouldn't matter that you're using IE) or your own ASP.NET (which can call System.Speech or any other free or commercial system). But before you do that, you should compress the audio. There aren't a lot of options in Silverlight. I recommend NSpeex, or at least convert it to 16kHz PCM (either linear or a-law).
Here's a list of Speech SDKs (many of which have a cloud service component): http://www.toolsjournal.com/mobile-articles/item/918-top-10-sdks-to-voice-enable-mobile-apps-quickly
To make Trusted In-browser Silverlight application:
http://msdn.microsoft.com/en-us/library/gg192793(v=vs.95).aspx
http://www.pitorque.de/MisterGoodcat/post/Silverlight-5-Tidbits-Trusted-applications.aspx
And for security background:
http://msdn.microsoft.com/en-us/library/ee721083%28v=vs.95%29.aspx
Note that NESL doesn't support DictionaryGrammar. Grammar needs to be pre-defined:
http://archive.msdn.microsoft.com/nesl/Thread/View.aspx?ThreadId=4905
I am experimenting with C#, and I wanted to create a fun/useful network program. I've programmed for most of my years using C++, C# seems a lot cleaner and easier to program in. I mostly programmed data structures and algorithms. I haven't really touched networking much.
I have video files on my computer that I would like to be able to share/stream/send to other computers on my network. I'm going to eventually expand on it and add a lot of features, but I want to conquer the hardest part first.
Is there a library out that helps with the data management for this?
I see accomplishing this three ways, Idk what's easiest and best.
Maybe using Windows File Sharing (Like how other computers on a network can open videos in a shared folder?)
Streaming the video data to the client computer? Then having their native video program open the data stream? (Buffer-like on youtube?)
Silverlight or some other Library. I can use the built in video player, etc to run it
Features:
I want to allow the client to be able to copy the video tutorial file to their own computer eventually if necessary, so idk. Maybe buffering is the best solution.
Want to allow the client to pause/download the video.
Hopefully I can learn a lot in this project.
You can use Microsoft Expression Encoder SDK to push video stream to a local port, or publish it in Windows or IIS Media Services. Windows Media Player, Silverlight or player-based application can be used for playback on another computer. Also, the are some options for playback on Apple devices. For H.264 support, you would need Pro version of the encoder.
For more information see the SDK documentation on MSDN, and articles Getting started with IIS Live Smooth Streaming and Apple HTTP Live Streaming with IIS Media Services.
You should be able to use vlc to transcode the file (or just stream it) then connect to the stream it produces. I know you're experimenting with C#, but it seems odd to re-invent the wheel, especially when it's such a good one!
I'm sure you'd have some fun automating vlc.
How can I create an ASP.NET page, that allows users to communicate with audio voice.
What must I do to accomplish this job.
Thanks.
Using ASP.NET only? I'm not sure you can do this without some sort of browser plugin. I suspect that it might be advantageous to leverage the Flash Player to make the call. You'd still need a communication server to do the plumbing. Check out Red5 (Open Source), FlashMediaServer (Adobe's product), Wowza Media Server (cheaper than FMS).
spender's answer is correct. Current versions of HTML nor the proposed HTML5 provide any support for audio pick up. Additionally, the audio playback features are insufficient to provide the kind of audio streaming controls that you will need. Look at the variety of plug-in technologies available or write your own browser specific, operating specific plug in.
If you are going to target devices such as the iPad and iPhone that won't support plug-ins, you are going to be forced to build native applications for those platforms.
I'm working on a home project that involves comparing images to a database of images (using a quadrant - or so - histogram approach). I wanted to know what my options are in regards to web cams or other image capture devices that:
Are easy to work with with the
Windows SDK (particularly
DirectShow, which I plan to use
with C#)
Have drivers for both
64-bit and 32-bit Windows Vista (and
Server 2008)
I'm asking primarily so I can avoid pitfalls that other people may have experienced with web cams and to see if there are other image capture devices (or C# usable APIs) available that I should look at. I suspect that any old web cam will do but I'd rather be safe than sorry.
If you only want images, many web cams support TWAIN -- you can use with .NET using this code
http://www.codeproject.com/KB/dotnet/twaindotnet.aspx
I'm doing something very similar with webcams and have found that logitech and Microsoft branded webcams work just fine with DirectShow.
I've also found that many NEW webcams don't support twain, and WIA support for doing live captures has been removed in Vista, and doesn't exist in 2000, so Directshow has been the only thing that seems to work reliably accross OS's.