I want to create a Windows Store app in C# to process to camera RAW images. It looks like I can load the RAW images using Microsoft's Windows Imaging Component (WIC), but when then I want to manipulate the images in a Windows Store app, I run into a problem.
To manipulate images in Windows Store apps, the primary option I found is using a WriteableBitmap object, but WriteableBitmap objects have only 8 bits per channel. Most DSLRs, including mine, have 14 bits per channel. I don't want to lose that much color depth. I suppose I could go the DirectX approach, but I'd rather not have to deal with C++ for this project. There's SharpDX, which lacks a bit in documentation, but I'd tackle it anyway if it's the best choice. Are there other ideas?
So what are the options, and which solution would you favor in my position? How can I load RAW images into a Windows Store application and make changes to those images without losing color depth?
Related
Situation
I have a video stream coming from a native library in iOS. I'm trying to display the image in an iPhoneOSGameView using glTexImage2D and glTexSubImage2D for updates. I can update subregions of the image, I receive a structure that tells me which rectangle has to be updated on the gpu.
The issue
Framerate is quite low. After much profiling both in OpenGLES and the application code, I have concluded that the application usually is waiting on the texture upload. The slow function is glClear, but I suspect there's an intrinsic glFlush in there.
My question
I've seen some people talking about glMapBuffer that could allow me to stream the video directly to the texture in user-space. I've looked at pixel buffer objects, but they require OpenGLES 3.0 or an extension in 2.0. Is there an efficient way (for mobile) to stream a texture with minimal memory copying OR a way to transfer the texture from different thread?
Additional information
I'm working in C# Xamarin and I'm testing on different devices such as an iPod Touch Gen3, a iPad Air 2 and a iPad Pro 12".
there an efficient way (for mobile) to stream a texture with minimal memory copying
Most operating systems have some media framework which allows import of images directly using the EGL_image_external extension avoiding the need to upload. I'm not sure how it works on iOS, but I strongly suspect it should be possible. It's OS-specific unfortunately, so no standard way of doing it.
OR a way to transfer the texture from different thread?
Just create two GL Contexts in the same share group.
https://developer.apple.com/library/ios/documentation/3DDrawing/Conceptual/OpenGLES_ProgrammingGuide/ConcurrencyandOpenGLES/ConcurrencyandOpenGLES.html
I have tried to approach this problem several ways.
I have a web application that will load a sprite (which is a large jpeg that 'holds' 100 frames. Using the offset of div properties I am able to 'move' the image along so it looks like a movie clip.
This works well.
I need to produce the same on mobile devices.
I have to support Windows Mobile 6.5.
I was using jQuery/ajax to move the sprite image along to stop any flickering of the image within the browser.
Windows mobile 6.5 does not support this.
So, i decided to do a native app using C# Framework 2.0.
I thought about converting the sprite to a mp4 file and playing within a web browser control. but IE on the device will not support html5.
I have been told there is a CAB file from adobe so that I can play flash videos on the IE. But, apart from using ffmpeg to convert these images to a flv there is not much control as to adding menu like pause,stop,play etc when playing the flv file. I have not started any of this by the way i assume that would be the case.
So.. I thought if I download the sprite to the device i could use the graphics object of a picture box and render the rectangle of interest from large sprite to picture box.
I could not download the Sprite and save it to an image though. I get 'out of memory exception'.
I used the OpenNetCF.Drawing as it is supposed to support large files. I still get the error message. I have reduced the size of the image and it will work on about 20 frames.
I would then have to 'stack' these images containing 20 frames and load them up when necessary.
I am beginning to wonder now whether i am doing the right thing?
Is there a different way of trying to achieve what i am trying to achieve that is quicker and more efficient as I have run out of ideas.
If I am OK doing it this way then what is the best way to render images in a picturebox like a video. Memory efficiency says load the image from the file on the device when needed and dispose of it but that would cause 'flickering'. What IS the best way to handle multitude of images on a device?
i have plenty of code I can upload but I was worried the question would look 'messy' as I have tried so many things. I will of course edit/upload the code as required.
Thanks.
We have a c# application that performs processing on video streams. This is a low-level application that receives each frame in Bitmap format, so basically we need 25 images each second. This application is already working for some of our media sources, but we now need to add a webcam as an input device.
So we basically need to capture bitmap images from a webcam continuously so that we can pass all these frames as a "stream" to our application.
What is the best and simplest way to access the webcam and read the actual frames directly from the webcam as individual images? I am still in the starting blocks.
There are a multitude of libraries out there that allows one to access the webcam, preview the content of the webcam on a windows panel and then use screen capturing to capture this image again. This, unfortunately, will not give us the necessary performance when capturing 25 frames per second. IVMRWindowlessControl9::GetCurrentImage has been mentioned as another alternative, but this again seems to be aimed at an infrequent snapshot rather than a constant stream of images. Directshow.Net is mentioned by many as a good candidate, but it is unclear how to simply grab the images from the webcam. Also, many sources state a concern about Microsoft no longer supporting Directshow. Also, implementations I've seen of this requires ImageGrabber which is apparently also no longer supported. The newer alternative from MS seems to be Media Foundation, but my research hasn't turned up any working examples of how this can be implemented (and I'm not sure if this will run on older versions of windows such as XP). DirectX.Capture is an awesome library (see a nice implementation) but seems to lack the filters and methods to get the video images directly. I have also started looking at Filters and Filter Graphs but this seems awfully complex and does feel a bit like "reinventing the wheel".
Overall, all the solutions briefly mentioned above seem to rather old. Can someone please point me in the direction of a step-by-step guide for getting a webcam working in C# and grabbing several images per second from it? (We will also have to do audio at some point, so a solution that does not exclude video would be most helpful).
I use AForge.Video (find it here: code.google.com/p/aforge/) because it's a very fast c# implementation. i am very pleased with the performance and it effortlessly captures from two HD webcams at 30fps on an 8 year old PC. the data is supplied as a native IntPtr so it's ideal for further processing using native code or opencv.
opencv wrappers emgu and opencvsharp both implement a rudimentary video capture functionality which might be sufficient for your purposes. clearly if you are going perform image processing / computer vision you might want to use those anyway.
As dr.mo suggests, Aforge was the answer.
I used the tutorial from here: http://en.code-bude.net/2013/01/02/how-to-easily-record-from-a-webcam-in-c/
In the tutorial, they have an event handler fire each time a frame is received from the webcam. In the original tutorial, this bitmap is used to write the image to a PictureBox. I have simply modified it to save the bitmap image to a file rather than to a picturebox. So I have replaced the following code:
pictureBoxVideo.BackgroundImage = (Bitmap)eventArgs.Frame.Clone();
with the following code:
Bitmap myImage = (Bitmap)eventArgs.Frame.Clone();
string strGrabFileName = String.Format("C:\\My_folder\\Snapshot_{0:yyyyMMdd_hhmmss.fff}.bmp", DateTime.Now);
myImage.Save(strGrabFileName, System.Drawing.Imaging.ImageFormat.Bmp);
and it works like a charm!
I need a web based application where the user needs to fill in 3 fields and taking a high resolution image (2mpix) with a webcam. Once these are ready I need to post them to the server so they are saved in a database.
1) What is the best way/tools available?
2) I saw thisjquery webcam plugin - is it possible to capture photos through your webcam up to 1920×1080?
Thanks
The quality of the pictures you take with ANY camera depends on the sophistication of the camera.
Cameras have maximum resolutions and settings to configure them to a lower one as needed.
If the webcam on the machine has a resolution as high as the one you need, you'll probably be able to do this.
And its worth mentioning that the jquery webcam plugin requires flash to be available on the machine.
Since you've tagged this question with the ".net" and "c#" tags i guess i should mention that you can access the webcam via a silverlight app as well.
I want to create a simple video renderer to play around, and do stuff like creating what would be a mobile OS just for fun. My father told me that in the very first computers, you would edit a specific memory address and the screen would update. I would like to simulate this inside a window in Windows. Is there any way I can do this with C#?
This used to be done because you could get direct access to the video buffer. This is typically not available with today's systems, as the video memory is managed by the video driver and OS. Further, there really isn't a 1:1 mapping of video memory buffer and what is displayed anymore. With so much memory available, it became possible to have multiple buffers and switch between them. The currently displayed buffer is called the "front buffer" and other, non-displayed buffers are called "back buffers" (for more, see https://en.wikipedia.org/wiki/Multiple_buffering). We typically write to back buffers and then have the video system update the front buffer for us. This provides smooth updates, as the video driver synchronizes the update with the scan rate of the monitor.
To write to back buffers using C#, my favorite technique is to use the WPF WritableBitmap. I've also used the System.Drawing.Bitmap to update the screen by writing pixels to it via LockBits.
It's a full featured topic that's outside the scope (it won't fit, not that i won't ramble about it for hours :-) of this answer..but this should get you started with drawing in C#
http://www.geekpedia.com/tutorial50_Drawing-with-Csharp.html
Things have come a bit from the old days of direct memory manipulation..although everything is still tied to pixels.
Edit: Oh, and if you run into flickering problems and get stuck, drop me a line and i'll send you a DoubleBuffered panel to paint with.