Cannot convert YUV420 frame to Bitmap - c#

I am trying to convert YUV420 frames to Bitmap or Image. I am reading these frames from an MP4 video in C# using the AVBlocks library. So, after creating an input and output socket using AVBlocks classes, I then pull each frame from the video with a YUV420 color format and UncompressedVideo stream type. I basically do this by calling Transcoder.Pull(int outputIndex, MediaSample outputData) and then the MediaBuffer that's part of the outputData has the data in an array of bytes. So I am trying to convert these bytes to a Bitmap or Image so that I can eventually show each frame into a PictureBox in the Winforms application.
What I've tried:
I have tried using a MemoryStream, as shown below, but I get an unhandled ArgumentException saying that the parameter is not valid. I tried using ImageConverter() as well to convert to an Image, but I get the same exception. Then, I converted the byte array from YUV to RGB format and gave the updated array as a parameter to the MemoryStream, but again no luck. I also tried changing the color format of the output socket from YUV420 to a BGR format, but it resulted in the same issue as above. The code that tries to convert to a bitmap using MemoryStream:
while (transcoder.Pull(out inputIndex, yuvFrame))
{
buffer = (MediaBuffer) yuvFrame.Buffer.Clone();
Bitmap b;
byte[] temp = new byte[buffer.DataSize];
Array.Copy(buffer.Start, buffer.DataOffset, temp, 0, buffer.DataSize);
var ms = new MemoryStream(temp);
b = new Bitmap(ms);
}
The aforementioned exception is thrown in the last line of the code. I'm not sure if it's the color format or the stream type, or something else that's causing the problem. If someone wants to see more of the code (setting up input & output sockets etc), let me know. For reference, the link to the example I've been following from AVBlocks is this and the link to MediaBuffer class is this.


The Bitmap(MemoryStream ms) constructor expects the bytes from an actual file, like a png, jpeg, bmp or gif. If I'm reading this correctly, you don't have that; you only have pure RGB triplets data. That isn't enough, because it lacks all information about the image's width, height, colour depth etc.
You will need to actually construct an image object from the RGB data. This isn't really trivial; it means you need to make a new image object with the correct dimensions and colour format, then access its backing bytes array, and write your data into it. The actual code for creating an image out of a byte array can be found in this answer.
Note that you'll have to take into account the actual stride in the resulting data you get; the amount of bytes on each line of the image. Images are saved per line, and those lines are usually padded to a multiple of 4 bytes. This obviously messes up a lot if you don't take it into account.
If your data is completely compact, then the stride to give to the BuildImage function I linked to will just be your image width multiplied by the amount of bytes per pixel (should be 3 for 24bpp RGB), but if not, you'll have to pad it to the next multiple of 4.

Related

Loading an image from a NES CHR file (i.e., a binary file) and showing on a Windows Form

I'm working on a C# form that reads graphics from a binary file and allows me to create maps with it. I'm having trouble trying to display an image on the form based on the bytes in the binary file. This binary file stores a bunch of 8x8 pixel images and each row of this image is stored in two different bytes, which I have to do some processing on. After reading all the bytes, I need to get palette data based on these bytes and then I can finally get my image colored properly.
I won't get into details of how the image processing works, since I think I can do that by myself (unless someone is curious, then I can explain in more detail - I'm basically reading a CHR file that contains patterns for a NES game and displaying a tile using the NES palette). By the end of all this processing, I have the RGB values for each pixel of the image stored as a byte array. My problem is that I don't know how to convert this byte array into a 8x8 image.
So for example, suppose I have a 8x8 tile that is just a red tile. By the end of the processing, this tile would be represented by an array like this (each pixel uses 3 bytes: byte 1 = R, byte 2 = G, byte 3 = B):
byte[] tilecolors = {255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0,255,0,0, }
So, binary file has a bunch of these images stored in it as bytes and I want to do is transform those bytes into 8x8 images that will be displayed in an app (probably inside a picturebox, since I want the app user to use these tiles to create maps and stuff).
I tried using a function I saw on another website that reads loads an image from bytes using MemoryStream, but I get a "Parameter is not valid" error. This is the function I tried:
public Image byteArrayToImage(byte[] byteArrayIn)
{
MemoryStream ms = new MemoryStream(byteArrayIn);
Image returnImage = Image.FromStream(ms);
return returnImage;
}
EDIT:
I got it to work by following #ckuri 's suggestion. After I calculate the RGB colors from reading the binary file, I can use the Bitmap class's SetPixel method to draw the image.

Using ImageSharp what's the best way to convert incoming buffer containing Bgra32 to an ImageSharp Image<Rgba24> image?

I get PixelFormats.Bgra32 images from UAP and WPF and I would like to know the fastest way to convert them to and from SixLabors.ImageSharp Image<Rgba32> images. Is there some magic "mutate" that can flip the pixel bytes after I've copied the buffer using TryGetSinglePixelSpan?

There's no shortcut for mutating an existing image but you can create a new one with the pixel format you need from the first.
Image<Rgba32> rgba;
// Load decodes
// LoadPixelData copies
// WrapMemory wraps
using (Image<Bgra32> bgra = Image.LoadPixelData<Bgra32>(...))
{
rgba = bgra.CloneAs<Rgba32>(bgra.GetConfiguration());
}
You can then do whatever you need with the Rgba32 image.

Converting Byte Array To Image Throws "Parameter is not valid."

I'm trying to convert several byte arrays (stored in SQL Server 2008 R2 as varbinary(8000)) into images so I can attach them to a PDF file (an example of the data is below). I keep getting "Parameter is not valid." when creating the Image from the MemoryStream. Any help on this?
MemoryStream memoryStream = new MemoryStream(byteArray);
Image image = Image.FromStream(memoryStream);
Bitmap bitmap = new Bitmap(image);
0x0A020101000000005E006500FA00FA0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010C00010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000C1C0C7000FC2FFC1FE80C70007C2FFC1FE1FC7FFC1F03FC1FFC1FE3FC7FFC1F807C1FFC1FE3FC8FFC1F3C1FFC1FE3FC8FFC1FEC1FFC1FE3FC8FFC1FE7FC1FE3FC9FFBFC1FE3FC9FFC1DFC1FE3FC9FFC1CFC1FE3FC9FFC1F3C1FE3FC9FFC1FBC1FE3FC9FFC1F9C1FE3FC9FFC1F9C1FE3FC9FFC1FCC1FE3FC9FFC1FE7E3FC9FFC1FE7E3FCAFF7E3FC2FFC1FCC20007C4FF3E3FC2FFC1F8C300C4FF3E3FC2FFC1F1C2FFC1FC7FC3FF3E3FC2FFC1F1C2FFC1FC3FC3FF3E3FC2FFC1F1C3FF9FC3FF3E3FC2FFC1F1C3FFC1CFC3FF3E3FC2FFC1F1C3FFC1E7C3FF3E3FC2FFC1F1C3FFC1F3C3FF3E3FC2FFC1F1C3FFC1F3C3FF3E3FC2FFC1F1C3FFC1F3C3FF3E3FC2FFC1F1C3FFC1F3C3FF3E3FC2FFC1F1C3FFC1F3C3FF3E3FC2FFC1F1C3FFC1F3C3FF3E3FC2FFC1F1C3FFC1F3C2FFC1FE7E3FC2FFC1F1C3FFC1E7C2FFC1FE7E3FC2FFC1F1C3FFC1C7C2FFC1FCC1FE3FC2FFC1F1C3FFC1C7C2FFC1F9C1FE3FC2FFC1F1C3FFC1DFC2FFC1F9C1FE3FC2FFC1F1C3FFBFC2FFC1F9C1FE3FC2FFC1F1C2FFC1FC3FC2FFC1F3C1FE3FC2FFC1F0C2FFC1F8C3FFC1E7C1FE3FC2FFC1FEC2000FC3FFC1CFC1FE3FC3FFC2000FC3FF9FC1FE3FC9FF3FC1FE3FC8FFC1FE7FC1FE3FC8FFC1F0C1FFC1FE3FC8FFC1E7C1FFC1FE3FC8FFC1CFC1FFC1FE3FC8FF8FC1FFC1FE3FC8FFC1E0C1FFC1FE3FC8FFC1F8C1FFC1FE3FC8FFC1FC1FC1FE3FC9FF8FC1FE3FC9FFC1E7C1FE3FC9FFC1F0C1FE3FC9FFC1FE7E3FCAFF7E3FC2FFC1FCC300C4FF3E3FC2FFC1F8C3001FC3FF3E3FC2FFC1F1C3FF8FC3FF9E3FC2FFC1F1C3FF87C3FFC1CE3FC2FFC1F1C3FFC1F3C3FFC1CE3FC2FFC1F1C3FFC1F9C3FFC1EE3FC2FFC1F1C3FFC1FCC3FFC1E43FC2FFC1F1C3FFC1FE7FC2FFC1E43FC2FFC1F1C4FF3FC2FFC1E43FC2FFC1F1C4FF1FC2FFC1F03FC2FFC1F1C4FF1FC2FFC1F83FC2FFC1F1C4FF1FC2FFC1F83FC2FFC1F1C4FF1FC2FFC1F83FC2FFC1F1C4FF1FC2FFC1F83FC2FFC1F1C4FF1FC2FFC1F83FC2FFC1F1C4FF1FC2FFC1F83FC2FFC1F1C4FF1FC2FFC1F83FC2FFC1F1C3FFC1FE7FC2FFC1F83FC2FFC1F1C3FFC1FEC3FFC1F83FC2FFC1F1C3FFC1F9C3FFC1F83FC2FFC1F1C3FFC1F3C3FFC1F83FC2FFC1F1C3FFC1F7C3FFC1F83FC2FFC1F1C2FFC1FE0FC3FFC1F83FC2FFC1F0C3003FC3FFC1F83FC2FFC1F8C20001C4FFC1F83FC2FFC1FCC20003C4FFC1FC3FCAFFC1E43FCAFFC1E43FCAFFC1E43FCAFFC1EE3FCAFFC1EE3FCAFFC1CE3FCAFF3E3FCAFF7E3FC9FFC1FE7E3FC9FFC1FE7E3FC9FFC1FCC1FE3FC9FFC1F1C1FE3FC9FFC1C3C1FE3FC9FFC1DFC1FE3FC9FF9FC1FE3FC8FFC1E07FC1FE3FC8FFC1C1C1FFC1FE1FC7FF001FC1FFC1FEC9003FC1FFC1FE80C700C3FFC1FEC1C0C700C3FFC1FE

It's a
% file output
output: PCX ver. 2.8 image data, with palette
under linux I just transformed your hex stream from plain text to a binary file called output
echo
"0A020101000000005E006500FA00FA0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010C00010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000C1C0C7000FC2FFC1FE80C70007C2FFC1FE1FC7FFC1F03FC1FFC1FE3FC7FFC1F807C1FFC1FE3FC8FFC1F3C1FFC1FE3FC8FFC1FEC1FFC1FE3FC8FFC1FE7FC1FE3FC9FFBFC1FE3FC9FFC1DFC1FE3FC9FFC1CFC1FE3FC9FFC1F3C1FE3FC9FFC1FBC1FE3FC9FFC1F9C1FE3FC9FFC1F9C1FE3FC9FFC1FCC1FE3FC9FFC1FE7E3FC9FFC1FE7E3FCAFF7E3FC2FFC1FCC20007C4FF3E3FC2FFC1F8C300C4FF3E3FC2FFC1F1C2FFC1FC7FC3FF3E3FC2FFC1F1C2FFC1FC3FC3FF3E3FC2FFC1F1C3FF9FC3FF3E3FC2FFC1F1C3FFC1CFC3FF3E3FC2FFC1F1C3FFC1E7C3FF3E3FC2FFC1F1C3FFC1F3C3FF3E3FC2FFC1F1C3FFC1F3C3FF3E3FC2FFC1F1C3FFC1F3C3FF3E3FC2FFC1F1C3FFC1F3C3FF3E3FC2FFC1F1C3FFC1F3C3FF3E3FC2FFC1F1C3FFC1F3C3FF3E3FC2FFC1F1C3FFC1F3C2FFC1FE7E3FC2FFC1F1C3FFC1E7C2FFC1FE7E3FC2FFC1F1C3FFC1C7C2FFC1FCC1FE3FC2FFC1F1C3FFC1C7C2FFC1F9C1FE3FC2FFC1F1C3FFC1DFC2FFC1F9C1FE3FC2FFC1F1C3FFBFC2FFC1F9C1FE3FC2FFC1F1C2FFC1FC3FC2FFC1F3C1FE3FC2FFC1F0C2FFC1F8C3FFC1E7C1FE3FC2FFC1FEC2000FC3FFC1CFC1FE3FC3FFC2000FC3FF9FC1FE3FC9FF3FC1FE3FC8FFC1FE7FC1FE3FC8FFC1F0C1FFC1FE3FC8FFC1E7C1FFC1FE3FC8FFC1CFC1FFC1FE3FC8FF8FC1FFC1FE3FC8FFC1E0C1FFC1FE3FC8FFC1F8C1FFC1FE3FC8FFC1FC1FC1FE3FC9FF8FC1FE3FC9FFC1E7C1FE3FC9FFC1F0C1FE3FC9FFC1FE7E3FCAFF7E3FC2FFC1FCC300C4FF3E3FC2FFC1F8C3001FC3FF3E3FC2FFC1F1C3FF8FC3FF9E3FC2FFC1F1C3FF87C3FFC1CE3FC2FFC1F1C3FFC1F3C3FFC1CE3FC2FFC1F1C3FFC1F9C3FFC1EE3FC2FFC1F1C3FFC1FCC3FFC1E43FC2FFC1F1C3FFC1FE7FC2FFC1E43FC2FFC1F1C4FF3FC2FFC1E43FC2FFC1F1C4FF1FC2FFC1F03FC2FFC1F1C4FF1FC2FFC1F83FC2FFC1F1C4FF1FC2FFC1F83FC2FFC1F1C4FF1FC2FFC1F83FC2FFC1F1C4FF1FC2FFC1F83FC2FFC1F1C4FF1FC2FFC1F83FC2FFC1F1C4FF1FC2FFC1F83FC2FFC1F1C4FF1FC2FFC1F83FC2FFC1F1C3FFC1FE7FC2FFC1F83FC2FFC1F1C3FFC1FEC3FFC1F83FC2FFC1F1C3FFC1F9C3FFC1F83FC2FFC1F1C3FFC1F3C3FFC1F83FC2FFC1F1C3FFC1F7C3FFC1F83FC2FFC1F1C2FFC1FE0FC3FFC1F83FC2FFC1F0C3003FC3FFC1F83FC2FFC1F8C20001C4FFC1F83FC2FFC1FCC20003C4FFC1FC3FCAFFC1E43FCAFFC1E43FCAFFC1E43FCAFFC1EE3FCAFFC1EE3FCAFFC1CE3FCAFF3E3FCAFF7E3FC9FFC1FE7E3FC9FFC1FE7E3FC9FFC1FCC1FE3FC9FFC1F1C1FE3FC9FFC1C3C1FE3FC9FFC1DFC1FE3FC9FF9FC1FE3FC8FFC1E07FC1FE3FC8FFC1C1C1FFC1FE1FC7FF001FC1FFC1FEC9003FC1FFC1FE80C700C3FFC1FEC1C0C700C3FFC1FE"
| xxd -p -r >output
and then the easy part with file output to get the name of the file format from the utility file.

First, make sure you know the encoding, as the Image.FromStream method knows only a limited number of formats.
Make sure that the Position of your MemoryStream is set properly to 0, and not to the end - otherwise, any reads will begin at the end appear to be empty, instead of starting at the beginning.
Also, note that
This constructor does not expose the underlying stream. GetBuffer throws UnauthorizedAccessException.1
Thus, this could be caused by some of the way Bitmap is handling the underlying stream. IF need be, try making an empty memorystream, copying the byte into it, and then resetting the position
MemoryStream memoryStream = new MemoryStream();
foreach(var b in byteArray) memoryStream.WriteByte(b);
memoryStream.Position = 0;
Image image = Image.FromStream(memoryStream);

The byte array you gave is 1117 bytes long. That's not evenly divisible by 4 or 3, so I'm pretty certain that the image is not raw bytes and is actually encoded in an image format of some sort.
The Image.FromStream() method can't decode an encoded image without knowing the format, and the byte array you gave doesn't specify what that format is. Therefore the parameter you gave is invalid. If you know the format of the image you could attempt to insert a header into the byte array before the rest of the image, and see if that helps. It's curious you have the image but not its header, although it doesn't appear to be a base-64 string either.
Without knowing the image format it could be difficult to convert these byte arrays into images. Is there any way you could find out what they are? How is the image data retrieved to be stored?

Creating a high quality ico file programmatically

I'm trying to create a high quality icon (means: suitable for Win Vista/7/8) from a PNG file programmatically in C# for use as shortcut icons. Since the Bitmap.GetHIcon() function doesn't support these kind of icons, and I want to avoid external dependencies or libraries, I'm currently using a slightly modified ICO writer I found here on SO.
I have working code but I'm experiencing some glitches in the way Windows displays these icons.
The relevant code is:
// ImageFile contains the path to PNG file
public static String IcoFromImageFile(String ImageFile) {
//...
Image iconfile = Image.FromFile(ImageFile);
//Returns a correctly resized Bitmap
Bitmap bm = ResizeImage(256,256,iconfile);
SaveAsIcon(bm, NewIconFile);
return NewIconFile;
}
// From: https://stackoverflow.com/a/11448060/368354
public static void SaveAsIcon(Bitmap SourceBitmap, string FilePath) {
FileStream FS = new FileStream(FilePath, FileMode.Create);
// ICO header
FS.WriteByte(0); FS.WriteByte(0);
FS.WriteByte(1); FS.WriteByte(0);
FS.WriteByte(1); FS.WriteByte(0);
// Image size
// Set to 0 for 256 px width/height
FS.WriteByte(0);
FS.WriteByte(0);
// Palette
FS.WriteByte(0);
// Reserved
FS.WriteByte(0);
// Number of color planes
FS.WriteByte(1); FS.WriteByte(0);
// Bits per pixel
FS.WriteByte(32); FS.WriteByte(0);
// Data size, will be written after the data
FS.WriteByte(0);
FS.WriteByte(0);
FS.WriteByte(0);
FS.WriteByte(0);
// Offset to image data, fixed at 22
FS.WriteByte(22);
FS.WriteByte(0);
FS.WriteByte(0);
FS.WriteByte(0);
// Writing actual data
SourceBitmap.Save(FS, System.Drawing.Imaging.ImageFormat.Png);
// Getting data length (file length minus header)
long Len = FS.Length - 22;
// Write it in the correct place
FS.Seek(14, SeekOrigin.Begin);
FS.WriteByte((byte)Len);
FS.WriteByte((byte)(Len >> 8));
FS.Close();
}
This compiles and works, but with one problem. Windows displays the icon on the shortcut incorrectly. I do this also programatically, but it occurs even if I do it manually (via File Properties, Change Icon). The problem is that the icon is cut off (the image itself displays correctly). It depends on the image, but usually only around 20% of the actual icon is shown. If I open the file in an image viewer like XNView it displays completely and correct, but MS Paint doesn't.
I made this screenshot, along with a correctly displayed icon for comparison
I suspect the error lies in the ICO saving method, but even after comparing them to normally displayed ICOs in a Hex editor, the header gets written correctly but the PNG image part itself seems different. Has anyone an idea? I also welcome better, less hacky solutions.

Your ico file is set to save the length of the embedded bitmap with only 16-bit precision, but the PNG file is too large (larger than 65535 bytes) so the length record overflows.
I.e. the following lines are incomplete:
// Write it in the correct place
FS.Seek(14, SeekOrigin.Begin);
FS.WriteByte((byte)Len);
FS.WriteByte((byte)(Len >> 8));
You could add these lines:
FS.WriteByte((byte)(Len >> 16));
FS.WriteByte((byte)(Len >> 24));
As a matter of cleanliness and performance, I'd generally avoid all those separate writes and just use the write overload with the byte array parameter. Also, instead of the somewhat tricky Save-To-File then seek, you might consider a Save-To-MemoryStream then a single Write for the header (which can now use the PNG's length in bytes) and a single write to copy the PNG data from the memory stream to the file.
Another point you really should address is disposing IDisposable resources. Even if you don't need to yet since you haven't encountered any problems, it will bite you someday and if you have even a fairly small codebase with all kind of undisposed disposables you'll have a very hard time finding the source of your leak and/or deadlock. In general: Never call Close unless you really can't avoid it - instead wrap your FileStream in a using block. Similarly, Image and Bitmap are disposable and allocate native resources, though at least you can't get any locking issues with those (AFAIK - but better to be safe than sorry).

Reading one bit from a pbm (portable bitmap image)

I want to read a pbm (portable bitmap format) image having depth one bit per pixel and store that data in a file, so I can perform some operation on bits.
I think in C# the smallest data that can be read is 1 byte so how can I read a bit?

You can't read a bit directly. But you can read a byte and then get the bits from that byte. Or, better yet, read a bunch of bytes into an array and then create a BitArray.
Or ... is this "portable bitmap format" supported directly by the Bitmap class? Bitmap does support a 1 bit per pixel format. It might be as simple as calling the Bitmap constructor to load from a file or a stream.
More information on your problem will get you a better answer.

Categories

Resources