Develop Reference

Cannot convert YUV420 frame to Bitmap

I am trying to convert YUV420 frames to Bitmap or Image. I am reading these frames from an MP4 video in C# using the AVBlocks library. So, after creating an input and output socket using AVBlocks classes, I then pull each frame from the video with a YUV420 color format and UncompressedVideo stream type. I basically do this by calling Transcoder.Pull(int outputIndex, MediaSample outputData) and then the MediaBuffer that's part of the outputData has the data in an array of bytes. So I am trying to convert these bytes to a Bitmap or Image so that I can eventually show each frame into a PictureBox in the Winforms application.
What I've tried:
I have tried using a MemoryStream, as shown below, but I get an unhandled ArgumentException saying that the parameter is not valid. I tried using ImageConverter() as well to convert to an Image, but I get the same exception. Then, I converted the byte array from YUV to RGB format and gave the updated array as a parameter to the MemoryStream, but again no luck. I also tried changing the color format of the output socket from YUV420 to a BGR format, but it resulted in the same issue as above. The code that tries to convert to a bitmap using MemoryStream:
while (transcoder.Pull(out inputIndex, yuvFrame))
{
buffer = (MediaBuffer) yuvFrame.Buffer.Clone();
Bitmap b;
byte[] temp = new byte[buffer.DataSize];
Array.Copy(buffer.Start, buffer.DataOffset, temp, 0, buffer.DataSize);
var ms = new MemoryStream(temp);
b = new Bitmap(ms);
}
The aforementioned exception is thrown in the last line of the code. I'm not sure if it's the color format or the stream type, or something else that's causing the problem. If someone wants to see more of the code (setting up input & output sockets etc), let me know. For reference, the link to the example I've been following from AVBlocks is this and the link to MediaBuffer class is this.

The Bitmap(MemoryStream ms) constructor expects the bytes from an actual file, like a png, jpeg, bmp or gif. If I'm reading this correctly, you don't have that; you only have pure RGB triplets data. That isn't enough, because it lacks all information about the image's width, height, colour depth etc.
You will need to actually construct an image object from the RGB data. This isn't really trivial; it means you need to make a new image object with the correct dimensions and colour format, then access its backing bytes array, and write your data into it. The actual code for creating an image out of a byte array can be found in this answer.
Note that you'll have to take into account the actual stride in the resulting data you get; the amount of bytes on each line of the image. Images are saved per line, and those lines are usually padded to a multiple of 4 bytes. This obviously messes up a lot if you don't take it into account.
If your data is completely compact, then the stride to give to the BuildImage function I linked to will just be your image width multiplied by the amount of bytes per pixel (should be 3 for 24bpp RGB), but if not, you'll have to pad it to the next multiple of 4.

C# byte array - writing in the middle

I have the task of reading data from a source in chunks, and storing the entire result in a byte array. Specifically, I need to make subsequent calls to "Socket.Receive". I would like to allocate the byte array with the final size in advance, and each time give the position within the array to copy data into. This, to avoid an extra copy.
In C++, you simply give the offset of the array. Could not figure out how to give the Receive method a location in the middle of the byte array...
Can this be done in C#?

There are overloads to Receive that accept the offset and count to read. You can use them: https://msdn.microsoft.com/en-us/library/system.net.sockets.socket.receive(v=vs.110).aspx - for a specific example: https://msdn.microsoft.com/en-us/library/w3xtz6a5(v=vs.110).aspx

Importing an Image from an XML File

I'm trying to produce a system which takes an image in OpenCV, writes it to XML, reads it from XML, and displays it in an Image control in a WPF application.
The problem I'm having is that I'm unclear as to how .NET classes (e.g. BitmapImage) process images from a byte array. I've tried a number of methods and gotten various errors - too numerous to post unless required.
I was wondering if someone could point me in the right direction regarding this? What I want to know is how does the data need to be presented to a .NET object in order to display it in an Image control?
Thanks in advance for your help - it is much appreciated.
EDIT: I expect what I need to do is convert the XML into a byte array and use that, along with a definition of the format, to instantiate a BitmapImage object or equivalent to use as source for the Image control. What I'm not sure of is how to do this.

Use int cvSaveImage(const char* filename, const CvArr* image)
to save the file as an Image.The image format is chosen based on the filename extension
No need to store it in XML if you can directly store it as Image file.
The .NET classes requires an Image.That's it.
Note:
If you want to store red,green,blue values as comma separated strings, it would be very inefficient.Save it in Base64 instead.You can then convert it into bytes and feed it to any .NET image class.Also the .NET ImageConverter Class may be very helpful.

I found that the issue was (presumably) that I wasn't Base64 encoding the XML data before writing it. Thus the solution is to Base64 encode the data before writing to XML, Base64 decoding it on the other side, reading the data as a byte array and using that to instantiate a MemoryStream and then a BitmapImage object.
Thanks for your help everyone.

You can use:
public static BitmapSource Create(
int pixelWidth,
int pixelHeight,
double dpiX,
double dpiY,
PixelFormat pixelFormat,
BitmapPalette palette,
Array pixels,
int stride
)

Processing on large bitmaps (up to 3GB)

I'm working on some university project and got stuck with memory issue.
I load a bitmap which takes about 1,5GB on HDD with code below:
Bitmap bmp = new Bitmap(pathToFile);
The issue is that the newly created Bitmap object uses about 3,5GB of RAM which is something I can't understand (that's really BIG wrapper :E). I need to get to the pixel array, and the use of Bitmap class is really helpful (I use LockBits() method later, and process the array byte per byte) but in this case it's total blocker. So here is my question:
Is there any easy way to extract the pixel array without lending additional 2gb?
I'm using c# just to extract the needed array, which is later processed in c++ - maybe I can extract all needed data in c++ (but conversion issue appears here - I'm concentrating on 24bgr format)?
PS: I need to keep the whole bitmap in memory so splitting it into parts is no solution.
PS2: Just to clarify some issues: I know the difference between file extension and file format. The loaded file is uncompressed bitmap 3 bytes per pixel of size ~1.42GB (16k x 32k pixels), so why Bitmap object is more than two times bigger? Any decompressing issues and converting into other format aren't taking place.

Consider using Memory Mapped Files to access your HUGE data :).
An example focused on what you need can be found here: http://visualstudiomagazine.com/articles/2010/06/23/memory-mapped-files.aspx
It's in managed code but you might as well use it from equivalent native code.
Let me know if you need more details.

You can use this solution , Work with bitmaps faster in C#
http://www.codeproject.com/Tips/240428/Work-with-bitmap-faster-with-Csharp
Or you can use memory mapped files
http://visualstudiomagazine.com/articles/2010/06/23/memory-mapped-files.aspx

You can stop memory caching.
Instead of
Bitmap bmp = new Bitmap(pathToFile);
Use
var bmp = (Bitmap)Image.FromStream(sourceFileStream, false, false);
see https://stackoverflow.com/a/47424918/887092

Efficient byte[] to BitmapImage and vice versa conversion in C#

I need to get the BitmapImage from the byte array. Currently I'm working with MemoryStream
(byte[] -> MemoryStream -> BitmapImage) but it seems to be not efficient.
The same situation in the other way round - I need to get the byte array representing the BitmapFrame. Again, I'm doing that with MemoryStream.
What is the most efficient way to accomplish that task? Is it possible to speedup the conversion?
Thank you in advance for the replies and hints!
Cheers

Im using BitmapSource.CopyPixels to extract the bytes from the BitmapSource and then BitmapSource.Create to create a new image from the byte array.
I'm not sure how effective it is though, but it is sufficient for my purposes (I can extract the pixels, recolor them and then paste them back on a 2000*2000 image with barely noticeable delay).