How to save all depth frames in OpenNI kinect - c#

I've wrote a program to save all the depth frames of the Kinect depth images in OpenNI. I've used the simple viewer sample. The problem that not all the frames are saved!! I run my program for 10 sec and around 20 images only are saved although the application is set with 30fps!!
Could anyone please advise?

My colleague uses a 2-phase extraction. First write the images in binary, in order to avoid losing time during encoding or conversions. (You can use System.IO.FileStream and BinaryWriter for that). And then in another program, read the binary files to get raw depth or color images. You might make use of Matlab, OpenCV or another utility for this second part.
But keep in mind that, even this approach might cause some skipped/dropped frames. Personally, I've never achieved to obtain a constant 30 fps for a long period.

Related

Play memory streams avoiding gap between them in WP8.1 app

I have 2 memory streams which are representing wav files in my Windows Phone 8.1 app. I want to play them but avoiding gaps between them, is there any way to do that without using Sleep methods or something like that?
I've tried already the Thread.Sleep() but I think it is making lot of gaps as my 2 files are 20ms duration each.
Assuming the audio is the same format and the wave headers have been stripped you could just concatenate the memory streams.
streamTwo.CopyTo(streamOne);
If the wave headers are still embedded then you'd need to skip over the one in the second stream - generally 44 bytes. If the formats are different then you'll need to find another technique.

Show webcam stream with delay

I've been trying to find a way to show the stream of my webcam with a 30 second delay.
I found out that VLC is able to do this, but the problem is that the framerate is much too low and it's not smooth or viewable at all. I have to run this on a fairly old laptop with a better webcam that I own now, so I guess it's not an option.
I am able somewhat familiar with c#/c++ and python so I thought that I might make a solution of my own as the task seems fairly easy. Though, the problem is, I don't know where to start and any nudges in the right direction would be much appreciated.
My initial idea was to record first 30 seconds of the stream to the disk, then use VLC to view partial file (AFAIK it's able to do that). Is it an idea worth working on or should I scratch it and use some sort of a buffer for the video frames in the last 30 seconds?
Again, any nudges in the right direction will be much appreciated, thanks!
Take a look at OpenCV.
It can retrieve and show images from a webcam.
A lot a of tutorials are also available; e.g. http://opencv.willowgarage.com/wiki/CameraCapture
So simply create an array (or whatever) to hold the amount of pictures to be expected in 30 sec (FPS*30).
Start to retrieve images and as soon as the array is filled start to play from position 0.
Newly retrieved images can than be stored at the position from the "just" shown image.
Miguel Grinberg has written an excellent video streaming tutorial in Python which sends JPEG frames successively. Check his blog post here:
http://blog.miguelgrinberg.com/post/video-streaming-with-flask1
Each of these JPEG can be quickly reviewed and then broadcasted. [To take the required Delay in consideration]
As far as getting the input video feed is concerned, you can interface a webcam using OPENCV. OpenCV uses VideoCapture which returns raw images as bytes. These bytes needs to be encoded to JPEG and interfaced with Miguel's code.
import cv2
class VideoCamera(object):
def __init__(self):
# Using OpenCV to capture from device
self.video = cv2.VideoCapture(0)
def __del__(self):
self.video.release()
def get_frame(self):
success, image = self.video.read()
# We are using Motion JPEG, but OpenCV defaults to capture raw images,
# so we must encode it into JPEG in order to correctly display the
# video stream.
ret, jpeg = cv2.imencode('.jpg', image)
return jpeg.tobytes()
This approach will help you cater all the required features:
No internet required
Adjustable delay - Easily control the delay and the processing you want to perform on each frame
Good Quality
Record on Demand - Store the captured frames as per need
Have a record back feature, by just saving the previous 24*x frames (24fps stream)

Faster frame extraction than ffmpeg

I have a video analytics program that processes assorted frames from a video. (Several hours long)
The video is likely going to be an MP4 but may be other formats going forwards.
At the moment, I have a C# wrapper around an ffmpeg call to extract an individual frame at the requested time. (I'm using the ffmpeg.exe binary. Not the libraries directly)
At the moment, this all works. But it's slow. Very slow.
I've found ways to improve the speed by storing the extracted frames in a ramdisk while they're being processed. Changing the stored image format etc...
I just wanted to check if anyone could think of any way to pull individual frames out. At split-second accuracy.
I know this is probably possible with DShow etc... I went straight to FFMPEG as I've used it before. But if DShow is likely to be faster I'll gladly change!
In Windows you have native APIs to process, and in particular read from, media files:
DirectShow
Media Foundation
Both provide support for MP4 (H.264 video), DirectShow as a framework extended by third party MP4 Demultiplexer and H.264 decoder (of needed, also Windows 7 provides build it), and Media Foundation - natively or extended by third party extensions depending on OS version.
Both can be interfaced from .NET via open source wrappers DirectShow.NET and Media Foundation .NET respectively. This works out way faster then FFmpeg CLI for individual frames. Also note that you would be able to obtain frames incrementally without need to locate specific time and do excessive duplicated work, not even to mention process startup/initialization overhead. Alternatively you could use FFmpeg/Libav binaries through wrapper into C# and get similar performance.
You can change the position of the offset parameters. The order matters for the speed if the video contains valid meta data you can seek through the video faster.
If you put the offset before the input file the offset will be calculated with the bit rate with is not every time exactly (in case of a variable bit rate), but it is much faster. The correct way is to walk through the video (offset parameter is after the input file) but this takes time.

Capturing Desktop activities as recorded movie in C#

My requirement is to create an application that records desktop activities, along with audio, as a movie. After searching, I found that Microsoft Expression Encoder can be used to record desktop activities as a movie but the output file size is very large. For 10 seconds of video, it occupied around 30 to 40 MB. Also, it uses xesc format.
Is there is any other free API available to do this job?
Before you give up on Expression Encoder try adjusting:
ScreenCaptureJob.ScreenCaptureVideoProfile.Quality
Reducing the quality can greatly reduce the file size. Try it and see if the results are acceptable for you.
Reducing the framerate is actually unhelpful; I guess it forces a fixed framerate, whereas the default is to use a variable framerate based on activity.
If you don't like .xesc files you can transcode the video after you've captured it.
But 30 to 40MB for ten seconds is still way too much. I recorded ten seconds of (admittedly not very large, 1366x768) full-screen video at the default quality. With not much going on it took 300K; with lots of activity (constantly switching between full-screen apps) it took at most 1.5MB.
Reducing the quality reduced file sizes by about 50%.
Unless you're playing a full-screen video and trying to record that, you shouldn't see anything like 30 to 40MB. Perhaps you should look at your audio settings.
ScreenRecorderLib from nuget is good.
SharpAVI is taking too much of my disc space.
You will need to be careful for ScreenRecorderLib, it would require some time saving mp4 file in the end.
Make sure your program won't end before that happens.
I use FileInfo.Length to check if the file size is not growing anymore. This will determine if the saving is finished or not.

high compress Image

I'm trying to do a "remote desktop viewer".
For this I need to send the user's desktop - and it's alot of infomation for sockets...(especially if the resolution is high and the information can approach to 5.3MB (1680X1050))
So I started to compress with GZIP stream and the 5.3MB became 500KB, then I added my own compress algorithm (I think it called RLE) - taking near pixels and write it in format that 1) have 256 >> 3 = 32 colors(for red,blue,green each) and write how many pixels in a row have the same color. + GZIP.
That brought the compression to be in average 60~65KB - up to 200KB and it can be also under 5000 if the screen is totally white.
Now - I thought (and haven't implemented yet) about passing the difference between each frame - for each line I write where the difference(between the pixels) is starting and how long is the difference.
well, it can help - maybe I could get 30KB for each frame in average. but for sockets it's alot.
has anyone ever succeed to fit with this problem? (and how of course...)
There are standard algorithms for compressing images: e.g. JPEG.
A further optimization is to know something about the image: for example on a desktop, items like the Windows Start button, and various application icons, and widgets on the title bar, are standard: so instead of sending their pixels values, you can send their logical identifiers.
Yes people have succeeded with this problem: people who write remote desktop software, including the open source VNC.
You may wish to review the source code of a VNC.
Most VNC servers implement several different forms of compression.

Categories

Resources