I want to stream real-time video from the PC's screen capture or the webcam connected on PC to the HoloLens 2. I have tried to use obs-studio to capture video on PC and use vlc to receive and play the stream on HoloLens 2. But the delay is more than two seconds. Are there some other more effective methods to achieve this?
Thank you very much!
You can use MR-WebRTC for that. Just follow the tuts here.
Regular low latency video streaing methods result in subsecond latency. Effectively the end-to-end latency can be well under 100 ms, however this requires that you use proper mix of technologies including: screen capture, video codecs, hardware acceleraton, networking, Unity integration. The technologies you mentioned are far from optimal stack and hence the result you are observing.
The question is simple, but I guess the answer might not be.
I want to create a display device, on which the GPU would render a desktop or a video game. However, this device would not be connected to any physical screen on the video card port. The data rendered would be retrieved and streamed somewhere else over network.
A bit similar to what OnLive did, but I would like to stream that video output over LAN. Obviously it must be a full and real display so that existing applications or video games could work properly on it.
Is it even possible in C#?
surely you could link it up as an output stream with your processed frames, piped to a network socket or such?
I need to be able to record video from an external camera in a C# application.
Unfortunately a webcam is pretty much out of the question as the application will record outside and during the evening/night. That is why I was thinking of a camcorder since it also has manual control over exposure and focus, lower noise and better sensor.
So far I would use the AV/S-Video output from the camcorder and send the signal to a USB capture card (the computer is a laptop so no PCI-E cards).
How would I be able to access the video stream from the C# application, now that it comes from the capture card ?
Does my proposed system seem feasible (achievable, good video quality, good fps)? Does anybody have another working solution?
Thanks
This Code Project Article could be of a good starting point.
The Author mentions :
The main goal of the application was to make it flexible and
extensible. The application itself can communicate with any video
source – it may be an IP video camera or a server, it may be a local
camera attached to USB, it may be an MMS stream from a remote server,
or it may be any other video source. And more of it, the application
can work with all these video sources simultaneously, displaying them
all on a single screen.
The solution I used in the end was Microsoft Expression Encoder.
I'm currently working on some software which captures your monitor image and sends it over to clients over the internet.
So far I have it working in my local area network but when I go to test it over the internet, hardly any of the images get through to the client.
I am using Lidgren for my networking. At the moment, I get a Bitmap from the screen, convert it to a JPEG with 30 quality, G-zip it and send it on its way. Each image is about 80KB in size, and I try to send 10 images a second to the client. Now that's like a 7mbit upload connection required, mines is only 2mbit.
So basically, is anyone aware of any compression libraries or techniques which will dramatically decrease the file size of each image. This might be completely impossible but I thought that I would give it a go.
Any help is much appreciated, Thanks!
Do you really need to send the whole frame each time? Could you not just send what has changed between the current and the previous frame and then apply these changes to the client frame to bring it up to date? This should be pretty quick assuming the server isn't watching a video or some-such. This answer suggests this is what both RDP (Microsoft) and VNC use for remote desktop viewing.
See https://stackoverflow.com/a/4098515/171703 and https://stackoverflow.com/a/1876848/171703 for some ideas on how to do this.
I want to make a similar application to Skype, and the main problem is working with video and audio. The first problem is how to get a bytes array of the video (to be specific, I need to get bytes which represent the video, so that I can send them over the internet), and same with audio. The second problem is to play bytes that come from the other computer.
I've been thinking to do that in WPF. I'm new in WPF (I have practiced a little bit, and made couple of programs among which is a basic chat program). I'm doing this for practice, and I want to code by myself as much as I can, server, client, transmision of data, and so on...
I've been searching over the internet, and only one solution seems to me to be good, or better to say feasible, is to use DirectShow.
Just to add, I know that camera and microphone is supported in Silverlight, and I've tried that (actually, I've tried to host an HTML page with silverlight project in WPF project in which were webbrowser control, and I've succeeded to show video from my webcam), but I don't know how to get bytes which represent video.
Is that possible to do with WPF or silverlight?
I'll be very grateful for suggesting any solution, advice, or useful links.
Using DirectShow filter graphs, you'll have a direct access to image and audio buffers from input devices (such as cameras and microphones) as bytes array, sample by sample. You'll be able to directly manipulate the data, to chose a coding or compression format (using specific filters), and to control the data rate and synchronization.
However :
if you've entirely new to this environment, it will be hard. Also, I know it works nicely with C++, but I've never coded any Directshow application in C#. (You may want to look this way : CodeProject Tutorials, MSDN DirectShow topics, and tests using graphedit)
streaming media accross a network and receiving it with Directshow is not trivial and can be quite a pain. Network renderers and network source filters are available all around, but are always difficult to use in my opinion. And depending on your video format (H264, MPEG, MJPEG...) and network protocol (RTSP, plain old simple UDP...) choices, you might end up having to write your own stream/source filters, which is hard and time consuming.
Nevertheless, it IS feasible, and if your main objective is practice with coding, then why not !
(Never used WPF, maybe it's actually way simpler !)
I can't speak to WPF or Silverlight, but I've done this in DirectShow, and it's a pain in the ass.
If you want to use .NET, there's an open source wrapper called DirectShow.NET, that helps alot, and it's still a pain in the ass.
Microsoft did a good job with DirectShow and the whole Filter-Graph thing, but then they sort of dropped the ball a while ago and haven't updated it in years.
I'd recommend looking for a different technology(although it probably sits on DirectShow), and I'd be interested to hear what you find.
To all who are interested in this subject,
After spending hours and hours searching the internet, i managed to find a solution that should work. With Silverlight i take captures, resizing them to 160x120 (or less), and than convert them with imagetools. One thread that is responsible for taking pictures, starts capture, and when it is finished (capturing is asynchronous, so you need semaphores to use) it sleeps for 200ms; thats almost equivalent to 5 frames per second. I'm doing all of this because i have slow upload bandwidth, about 16 kilobytes per second, so i have to compress one frame as much as i can. Result is low detailed picture, but if u use 100x100 rectangle for viewing it, it isn't too bad. I haven't tried it with the internet yet, but, as i have said, it should work. I've also tried using compress methods, to compress picture a little more, if it is possible, but i don't know how to use that class (something is not working well), so left that for another time. Now i just want to make it work, and latter i'll try to make better performance.
Oh, one more thing, I also have to solve problem with audio transmission, and that needs a lot of work.
So, hear latter.