I am developing a C# .NET application which will display the image captured from a camera (smart camera). I am using the function below to convert the raw data received from the camera (which is a byte array) to a bitmap image. The problem I face with the below code is that, the image displayed in the C# .NET application's picture box (which should be a live image ideally) looks like a movie reel and it keeps scrolling to the right. Any idea how to solve this issue?
private Image ByteArrayToImage(byte[] myByteArray)
{
if (myByteArray != null)
{
MemoryStream ms = new MemoryStream(myByteArray);
int Height = 504; /*The exact height of image from Smart Camera*/
int Width = 664; /*The exact width of image from Smart Camera*/
BitmapData bmpData =
bmp.LockBits(new Rectangle(0, 0, bmp.Width, bmp.Height),
ImageLockMode.WriteOnly, bmp.PixelFormat);
Marshal.Copy(myByteArray, 0, bmpData.Scan0, myByteArray.Length);
Bitmap bmp = new Bitmap(Width, Height, PixelFormat.Format24bppRgb);
BitmapData bmpData =
bmp.LockBits(new Rectangle(0, 0, bmp.Width, bmp.Height),
ImageLockMode.ReadWrite, bmp.PixelFormat);
int offset = 0;
long ptr = bmpData.Scan0.ToInt64();
for (int i = 0; i < Height; i++)
{
Marshal.Copy(myByteArray, offset, new IntPtr(ptr), Width);
offset += (Width);
ptr += bmpData.Stride;
}
bmp.UnlockBits(bmpData);
return bmp;
}
return null;
}
#Mark, find below the code segment i use for receiving data from camera ...
Logic : I receive the sizeof image first and then open socket connection n loop till i get the sizeof bytes from the camera.
public static byte[] receivedata(Socket clientSock)
{
if (clientSock != null)
{
byte[] size = new byte[10000000];
byte[] buffer = null;
int ch, received, offset;
try
{
clientSock.Receive(size);
//MessageBox.Show("size array is " + size[0]
// + " " + size[1] + " " + size[2] + " " );
ch = BitConverter.ToInt32(size, 0);
//ch = 334656;
//MessageBox.Show("Sizeofimg = " + ch + "");
buffer = new byte[ch];
//MessageBox.Show("Start receiving image from camera");
received = 0;
offset = 0;
do
{
received += clientSock.Receive(buffer,
offset + received,
ch - received,
SocketFlags.None);
} while (received < ch);
//MessageBox.Show("Received " + received + " values");
System.Threading.Thread.Sleep(50);
}
catch (Exception e)
{
MessageBox.Show("Error receiving ...");
MessageBox.Show(e.StackTrace);
}
return buffer;
}
else
{
return null;
}
}
}
Kindly point out the problem ...
Thanks in advance.
"Scrolling to the right" tells me that each image is decoding consistently, but successive images are off in relation to one another. Correct? That means the problem is not in the code you've shown us, but in the code that reads from the camera.
Edit: it seems likely that your width and height are off by a little bit. The only documentation I've found for a 664x504 camera states that the "active area" (the part that actually contains a picture) is really 648x488. If this is the case, your picture would also seem to slant a little as each line would be offset from the next, and would shift up as each frame you read would get part of the next frame's data.
Related
I am receiving the following exception:
Exception thrown:
'System.AccessViolationException' in System.Drawing.dll
When calling the Save function of a Bitmap. The procedure works fine the first time around, but subsequent calls throw this exception.
My application takes a single long image and vertically tiles it out into several separate images. I do this by first breaking out the whole image into bytes, then in a Parallel.For loop I generate the bitmap from a subset byte array.
// Generate Bitmap from width, height and bytes
private Bitmap GenerateBitmap(int width, int height, byte[] bytes)
{
Bitmap bmp = new Bitmap(width, height, Stride(width),
PixelFormat.Format8bppIndexed,
Marshal.UnsafeAddrOfPinnedArrayElement(bytes, 0));
bmp.SetPalette();
return bmp;
}
That is the bitmap generation routine.
Here is the loop body that calls it.
Parallel.For(0, tileCount, i =>
{
byte[] bytes = new byte[imageWidth * tileHeight];
for (int j = 0; j < bytes.Length; j++)
{
bytes[j] = imageBytes[j + (imageWidth * (tileHeight * i))];
}
arr[i] = GenerateBitmap(imageWidth, tileHeight, bytes);
});
And here is the code elsewhere that the exception is thrown.
foreach(Bitmap tile in pattern.Tiles)
{
Console.WriteLine("Creating Tile " + count);
using (Bitmap bmp = new Bitmap(tile))
{
bmp.Save(Globals.patternOutputPath + "tile_" + count + ".png");
}
count += 1;
}
Where Tiles is a property of the pattern that calls the for loop function (which returns a list of Bitmaps).
I am assuming I'm missing some clean up somewhere in here.
Additional info: all images (input and output) are 256 (index) color format.
Edit: The comments below address the problem at hand, and I THINK I've solved the problem. I changed the GenerateBitmap routine to the following and am no longer getting this exception, but I have some more testing to do.
private Bitmap GenerateBitmap(int width, int height, byte[] bytes)
{
Bitmap bmp = new Bitmap(width, height, PixelFormat.Format8bppIndexed);
BitmapData bmpData = bmp.LockBits(new Rectangle(0, 0, bmp.Width, bmp.Height), ImageLockMode.WriteOnly, bmp.PixelFormat);
Marshal.Copy(bytes, 0, bmpData.Scan0, bytes.Length);
bmp.UnlockBits(bmpData);
return bmp;
/*Bitmap bmp = new Bitmap(width, height, Stride(width),
PixelFormat.Format8bppIndexed,
Marshal.UnsafeAddrOfPinnedArrayElement(bytes, 0));
bmp.SetPalette();
return bmp;*/
}
I'm making a program which can capture a small area on screen and will run something if there is any color on image that match the target colors. My program run as the following Sequence:
Get image from a specific area from screen
Save to a folder
using CountPixel to detect any target_color
However, after I click the button5 twice times (not double click), it through an exception at below line :
b.Save(#"C:\Applications\CaptureImage000.jpg", ImageFormat.Jpeg);
Exception :
An unhandled exception of type
'System.Runtime.InteropServices.ExternalException' occurred in
System.Drawing.dll
Additional information: A generic error occurred in GDI+
My questions are :
How can i fix this exception ?
I want to use another method instead of CountPixel() to improve performance, because I just need to detect only one target color to rise event.
Step 2 is troublesome. I wonder if i can skip it and use the other way to call: (#"C:\Applications\CaptureImage000.jpg", ImageFormat.Jpeg) , because using this long string isn't comfortable and result error when im trying to use with GetPixel,... or add it into some "value example" code on internet for improvement purpose.
private int CountPixels(Bitmap bm, Color target_color)
{
// Loop through the pixels.
int matches = 0;
for (int y = 0; y < bm.Height; y++)
{
for (int x = 0; x < bm.Width; x++)
{
if (bm.GetPixel(x, y) == target_color) matches++;
}
}
return matches;
}
private Bitmap CapturedImage(int x, int y)
{
Bitmap b = new Bitmap(XX, YY);
Graphics g = Graphics.FromImage(b);
g.CopyFromScreen(x, y, 0, 0, new Size(XX, YY));
b.Save(#"C:\Applications\CaptureImage000.jpg", ImageFormat.Jpeg);
/* Run 3 line below will lead to question 1 - through exception
Bitmap bm = new Bitmap(#"C:\Applications\CaptureImage000.jpg");
int black_pixels = CountPixels(b, Color.FromArgb(255, 0, 0, 0));
textBox3.Text = black_pixels + " black pixels";
*/
return b;
}
private void button5_Click(object sender, EventArgs e)// Do screen cap
{
Bitmap bmp = null;
bmp = CapturedImage(X0, Y0);
}
[EDIT] Worked on this tonight with OP, made some improvements
This now accounts for endianness of the machine and correctly compares colors by converting them to integers with the Color.ToArgb() function
the below code will work, I have added comments for clarity and given you some options. I wrote the code without an IDE but I am confident it is fine.
In both cases below, just keep the handle to the bitmap, don't need to save and reopen regardless of if you need a copy.
Exception issue and improvements to CapturedImage function
option A (recommended)
Don't save the bitmap, you already have a handle, the graphics object just modified the BMP. Just leave the below code as is for this function and it will work fine without un-commenting one of the other options.
Code and other options:
private Bitmap CapturedImage(Bitmap bm, int x, int y)
{
Bitmap b = new Bitmap(XX, YY);
Graphics g = Graphics.FromImage(b);
g.CopyFromScreen(x, y, 0, 0, new Size(XX, YY));
//option B - If you DO need to keep a copy of the image use PNG and delete the old image
/*
try
{
if(System.IO.File.Exists(#"C:\Applications\CaptureImage.png"))
{
System.IO.File.Delete(#"C:\Applications\CaptureImage.png");
}
b.Save(#"C:\Applications\CaptureImage.png", ImageFormat.Png);
}
catch (System.Exception ex)
{
MessageBox.Show("There was a problem trying to save the image, is the file in open in another program?\r\nError:\r\n\r\n" + ex.Message);
}
*/
//option C - If you DO need to keep a copy of the image AND keep all copies of all images when you click the button use PNG and generate unique filename
/*
int id = 0;
while(System.IO.File.Exists(#"C:\Applications\CaptureImage" + id.ToString().PadLeft('0',4) + ".png"))
{
//increment the id until a unique file name is found
id++;
}
b.Save(#"C:\Applications\CaptureImage" + id.ToString().PadLeft('0',4) + ".png", ImageFormat.Png);
*/
int black_pixels = CountPixels(b, Color.FromArgb(255, 0, 0, 0));
textBox3.Text = black_pixels + " black pixels";
return b;
}
Now for the CountPixels function, you have 3 options but really, you have one really solid option, so I am omitting the others.
This locks the bits in the BMP, uses marshalling to copy the data into an array and scans the array for data, very, very fast, and you will likely not even need to remove the count. If you do STILL want to remove the count, just add "return 1;" right underneath where it increments the matches variable.
Speed issue and improvements to CountPixels function
private int CountPixels(Bitmap bm, Color target_color)
{
int matches = 0;
Bitmap bmp = (Bitmap)bm.Clone();
BitmapData bmpDat = bmp.LockBits(new Rectangle(0, 0, bmp.Width, bmp.Height), ImageLockMode.ReadWrite, bmp.PixelFormat);
int size = bmpDat.Stride * bmpDat.Height;
byte[] subPx = new byte[size];
System.Runtime.InteropServices.Marshal.Copy(bmpDat.Scan0, subPx, 0, size);
//change the 4 (ARGB) to a 3 (RGB) if you don't have an alpha channel, this is for 32bpp images
//ternary operator to check endianess of machine and organise pixel colors as A,R,G,B or B,G,R,A (little endian is reversed);
Color temp = BitConverter.IsLittleEndian ? Color.FromArgb(subPx[i + 2], subPx[i + 1], subPx[i]) : Color.FromArgb(subPx[i + 1], subPx[i + 2], subPx[i + 3]);
for (int i = 0; i < size; i += 4 ) //4 bytes per pixel A, R, G, B
{
if(temp.ToArgb() == target_color.ToArgb())
{
matches++;
}
}
System.Runtime.InteropServices.Marshal.Copy(subPx, 0, bmpDat.Scan0, subPx.Length);
bmp.UnlockBits(bmpDat);
return matches;
}
Finally the same function but allowing for a tolerance percent
private int CountPixels(Bitmap bm, Color target_color, float tolerancePercent)
{
int matches = 0;
Bitmap bmp = (Bitmap)bm.Clone();
BitmapData bmpDat = bmp.LockBits(new Rectangle(0, 0, bmp.Width, bmp.Height), ImageLockMode.ReadWrite, bmp.PixelFormat);
int size = bmpDat.Stride * bmpDat.Height;
byte[] subPx = new byte[size];
System.Runtime.InteropServices.Marshal.Copy(bmpDat.Scan0, subPx, 0, size);
for (int i = 0; i < size; i += 4 )
{
byte r = BitConverter.IsLittleEndian ? subPx[i+2] : subPx[i+3];
byte g = BitConverter.IsLittleEndian ? subPx[i+1] : subPx[i+2];
byte b = BitConverter.IsLittleEndian ? subPx[i] : subPx[i+1];
float distancePercent = (float)Math.Sqrt(
Math.Abs(target_color.R-r) +
Math.Abs(target_color.G-g) +
Math.Abs(target_color.B-b)
) / 7.65f;
if(distancePercent < tolerancePercent)
{
matches++;
}
}
System.Runtime.InteropServices.Marshal.Copy(subPx, 0, bmpDat.Scan0, subPx.Length);
bmp.UnlockBits(bmpDat);
return matches;
}
I wanted to create animated gif image from several images using c#, so i have used below github solution to do so.
https://github.com/DataDink/Bumpkit
I am using below code to do it
using (var gif = File.OpenWrite(#"C:\IMG_TEST.gif"))
using (var encoder = new GifEncoder(gif))
for (int i = 0, count = imageFilePaths.Length; i < count; i++)
{
Image image = Image.FromFile(imageFilePaths[i]);
encoder.AddFrame(image,0,0);
}
it is working like a charm, but it is creating gif of size 45 MB. If i check my actual image size , then it is only 11MB with total 47 images. but somehow gif is generating with big size.
Now i want to compress the size of gif image which is being created using c#.
So is there any way i can compress the size of gif image?
I know this is an old question, but figured I'd share this solution.
I ran into the same issue and found that each frame should contain only the differences in pixels from the previous frame. I thought that the encoder would do the image diff for me, but apparently it doesn't.
So before adding each frame, I compare it to the previous frame using this method I wrote. I then add the resulting image that contains only the changed pixels.
Here's where I'm using it: https://github.com/Jay-Rad/CleanShot/blob/master/CleanShot/Classes/GIFRecorder.cs
public class ImageDiff
{
public static Bitmap GetDifference(Bitmap bitmap1, Bitmap bitmap2)
{
if (bitmap1.Height != bitmap2.Height || bitmap1.Width != bitmap2.Width)
{
throw new Exception("Bitmaps are not of equal dimensions.");
}
if (!Bitmap.IsAlphaPixelFormat(bitmap1.PixelFormat) || !Bitmap.IsAlphaPixelFormat(bitmap2.PixelFormat) ||
!Bitmap.IsCanonicalPixelFormat(bitmap1.PixelFormat) || !Bitmap.IsCanonicalPixelFormat(bitmap2.PixelFormat))
{
throw new Exception("Bitmaps must be 32 bits per pixel and contain alpha channel.");
}
var newImage = new Bitmap(bitmap1.Width, bitmap1.Height);
var bd1 = bitmap1.LockBits(new System.Drawing.Rectangle(0, 0, bitmap1.Width, bitmap1.Height), ImageLockMode.ReadOnly, bitmap1.PixelFormat);
var bd2 = bitmap2.LockBits(new System.Drawing.Rectangle(0, 0, bitmap2.Width, bitmap2.Height), ImageLockMode.ReadOnly, bitmap2.PixelFormat);
// Get the address of the first line.
IntPtr ptr1 = bd1.Scan0;
IntPtr ptr2 = bd2.Scan0;
// Declare an array to hold the bytes of the bitmap.
int bytes = Math.Abs(bd1.Stride) * bitmap1.Height;
byte[] rgbValues1 = new byte[bytes];
byte[] rgbValues2 = new byte[bytes];
// Copy the RGBA values into the array.
Marshal.Copy(ptr1, rgbValues1, 0, bytes);
Marshal.Copy(ptr2, rgbValues2, 0, bytes);
// Check RGBA value for each pixel.
for (int counter = 0; counter < rgbValues1.Length - 4; counter += 4)
{
if (rgbValues1[counter] != rgbValues2[counter] ||
rgbValues1[counter + 1] != rgbValues2[counter + 1] ||
rgbValues1[counter + 2] != rgbValues2[counter + 2] ||
rgbValues1[counter + 3] != rgbValues2[counter + 3])
{
// Change was found.
var pixel = counter / 4;
var row = (int)Math.Floor((double)pixel / bd1.Width);
var column = pixel % bd1.Width;
newImage.SetPixel(column, row, Color.FromArgb(rgbValues1[counter + 3], rgbValues1[counter + 2], rgbValues1[counter + 1], rgbValues1[counter]));
}
}
bitmap1.UnlockBits(bd1);
bitmap2.UnlockBits(bd2);
return newImage;
}
}
I know how to do it in WPF but I have problem for capturing depth in winforms application.
I found some code as below:
private void Kinect_DepthFrameReady(object sender, DepthImageFrameReadyEventArgs e)
{
using (DepthImageFrame depthFrame = e.OpenDepthImageFrame())
{
if (depthFrame != null)
{
Bitmap DepthBitmap = new Bitmap(depthFrame.Width, depthFrame.Height, PixelFormat.Format32bppRgb);
if (_depthPixels.Length != depthFrame.PixelDataLength)
{
_depthPixels = new DepthImagePixel[depthFrame.PixelDataLength];
_mappedDepthLocations = new ColorImagePoint[depthFrame.PixelDataLength];
}
//Copy the depth frame data onto the bitmap
var _pixelData = new short[depthFrame.PixelDataLength];
depthFrame.CopyPixelDataTo(_pixelData);
BitmapData bmapdata = DepthBitmap.LockBits(new Rectangle(0, 0, depthFrame.Width,
depthFrame.Height), ImageLockMode.WriteOnly, DepthBitmap.PixelFormat);
IntPtr ptr = bmapdata.Scan0;
Marshal.Copy(_pixelData, 0, ptr, depthFrame.Width * depthFrame.Height);
DepthBitmap.UnlockBits(bmapdata);
pictureBox2.Image = DepthBitmap;
}
}
}
but this is not giving me the greyScale depth and it's purple. Any improvement or help?
I found the solution myself, by a function to convert the depth frame:
void Kinect_DepthFrameReady(object sender, DepthImageFrameReadyEventArgs e)
{
using (DepthImageFrame depthFrame = e.OpenDepthImageFrame())
{
if (depthFrame != null)
{
this.depthFrame32 = new byte[depthFrame.Width * depthFrame.Height * 4];
//Update the image to the new format
this.depthPixelData = new short[depthFrame.PixelDataLength];
depthFrame.CopyPixelDataTo(this.depthPixelData);
byte[] convertedDepthBits = this.ConvertDepthFrame(this.depthPixelData, ((KinectSensor)sender).DepthStream);
Bitmap bmap = new Bitmap(depthFrame.Width, depthFrame.Height, PixelFormat.Format32bppRgb);
BitmapData bmapdata = bmap.LockBits(new Rectangle(0, 0, depthFrame.Width, depthFrame.Height), ImageLockMode.WriteOnly, bmap.PixelFormat);
IntPtr ptr = bmapdata.Scan0;
Marshal.Copy(convertedDepthBits, 0, ptr, 4 * depthFrame.PixelDataLength);
bmap.UnlockBits(bmapdata);
pictureBox2.Image = bmap;
}
}
}
private byte[] ConvertDepthFrame(short[] depthFrame, DepthImageStream depthStream)
{
//Run through the depth frame making the correlation between the two arrays
for (int i16 = 0, i32 = 0; i16 < depthFrame.Length && i32 < this.depthFrame32.Length; i16++, i32 += 4)
{
// Console.WriteLine(i16 + "," + i32);
//We don’t care about player’s information here, so we are just going to rule it out by shifting the value.
int realDepth = depthFrame[i16] >> DepthImageFrame.PlayerIndexBitmaskWidth;
//We are left with 13 bits of depth information that we need to convert into an 8 bit number for each pixel.
//There are hundreds of ways to do this. This is just the simplest one.
//Lets create a byte variable called Distance.
//We will assign this variable a number that will come from the conversion of those 13 bits.
byte Distance = 0;
//XBox Kinects (default) are limited between 800mm and 4096mm.
int MinimumDistance = 800;
int MaximumDistance = 4096;
//XBox Kinects (default) are not reliable closer to 800mm, so let’s take those useless measurements out.
//If the distance on this pixel is bigger than 800mm, we will paint it in its equivalent gray
if (realDepth > MinimumDistance)
{
//Convert the realDepth into the 0 to 255 range for our actual distance.
//Use only one of the following Distance assignments
//White = Far
//Black = Close
//Distance = (byte)(((realDepth – MinimumDistance) * 255 / (MaximumDistance-MinimumDistance)));
//White = Close
//Black = Far
Distance = (byte)(255 - ((realDepth - MinimumDistance) * 255 / (MaximumDistance - MinimumDistance)));
//Use the distance to paint each layer (R G & of the current pixel.
//Painting R, G and B with the same color will make it go from black to gray
this.depthFrame32[i32 + RedIndex] = (byte)(Distance);
this.depthFrame32[i32 + GreenIndex] = (byte)(Distance);
this.depthFrame32[i32 + BlueIndex] = (byte)(Distance);
}
//If we are closer than 800mm, the just paint it red so we know this pixel is not giving a good value
else
{
this.depthFrame32[i32 + RedIndex] = 0;
this.depthFrame32[i32 + GreenIndex] = 0;
this.depthFrame32[i32 + BlueIndex] = 0;
}
}
so i presume that rgb frame is working out for you in that case:
first to enable depth camera you need to call:
sensor->NuiInitialize(NUI_INITIALIZE_FLAG_USES_DEPTH|all stuff you use also);
second to start streaming you need to call:
if (int(streams&_Kinect_zed)) ret=sensor->NuiImageStreamOpen(
NUI_IMAGE_TYPE_DEPTH, // Depth camera or rgb camera?
NUI_IMAGE_RESOLUTION_640x480, // Image resolution
NUI_IMAGE_STREAM_FLAG_DISTINCT_OVERFLOW_DEPTH_VALUES, // Image stream flags // NUI_IMAGE_STREAM_FLAG_ENABLE_NEAR_MODE nefunguje !!!
2, // Number of frames to buffer
NULL, // Event handle
&stream_hzed); else stream_hzed=NULL;
beware not all resolution/flags combinations work on all models of kinect !!!
this one above is safe even for the older models like mine
this is how i capture frame (called repeatedly from timer or thread loop)
ret=sensor->NuiImageStreamGetNextFrame(stream_hzed,0,&imageFrame); if (ret>=0)
{
// copy data from frame
imageFrame.pFrameTexture->LockRect(0, &LockedRect, NULL, 0);
if (LockedRect.Pitch!=0)
{
const BYTE* curr = (const BYTE*) LockedRect.pBits;
union _col { BYTE u8[2]; WORD u16; } col;
col.u16=0;
pnt3d p;
long ax,ay;
float mxs=float(xs)/(62.0*deg),mys=float(ys)/(48.6*deg);
for(int x=0,y=0;;)
{
col.u8[0]=*curr; curr++;
col.u8[1]=*curr; curr++;
p.raw=col.u16;
p.rgb=&rgb_default;
if (p.raw==0x0000) p.z=0.0; // p.z je kolma vzdialenost od senzora (kinect to correctuje sam)
else if (p.raw>=0x8000) p.z=4.0;
else p.z=0.8+(float(p.raw-6576)*0.00012115165336374002280501710376283);
// depth FOV correction
p.x=zx[x]*p.z;
p.y=zy[y]*p.z;
// color FOV correction zed 58.5° x 45.6° | rgb 62.0° x 48.6° | 25mm distance
if (p.z>0.0)
{
ax=(((x+10-xs2)*241)>>8)+xs2; // cameras x-offset and different FOV
ay=(((y+30-ys2)*240)>>8)+ys2; // cameras y-offset??? and different FOV
if ((ax>=0)&&(ax<xs))
if ((ay>=0)&&(ay<ys)) p.rgb=&rgb[ay][ax];
}
xyz[y][x]=p;
x++; if (x>=xs) { x=0; y++; if (y>=ys) break; }
}
}
// release frame
imageFrame.pFrameTexture->UnlockRect(0);
ret=sensor->NuiImageStreamReleaseFrame(stream_hzed, &imageFrame);
stream_changed|=_Kinect_zed;
}
Sorry for incomplete source code ...
- all is copy pasted from my kinect class (BDS2006 Turbo C++)
- so you need to check your code if you do not forget something
- and if yes then transform my code to C# (i am not C# user)
- most likely you forget to NUIinitialize with depth flag
- or set invalid resolution/flags/ precision or framerate for your HW
if nothing work at all then you need to initialize the sensor in the first place
int sensors;
INuiSensor *sensor;
if ((NUIGetSensorCount(&sensors)<0)||(sensors<1)) return false;
if (NUICreateSensorByIndex(0,&sensor)<0) return false;
if you link to dll on your own then link only these functions:
typedef HRESULT(__stdcall *_NuiGetSensorCount )(int * pCount); _NuiGetSensorCount NUIGetSensorCount =NULL;
typedef HRESULT(__stdcall *_NuiCreateSensorByIndex)(int index,INuiSensor **ppNuiSensor); _NuiCreateSensorByIndex NUICreateSensorByIndex=NULL;
Every other function (must) is obtained via COM inside SDK headers !!!
if you link and use them on your own then you will not be connected to your physical Kinect !!!
Basically kinect sdk is developed for WPf application. In windows form you have convert the short array of the depth data to the BItmap to display it on picturebox. And based on my expriment WPF is better for programming with kinect.
Below is the function that I used to convert depth frame to Bitmap for showing in picture box.
private Bitmap ImageToBitmap(DepthImageFrame Image)
{
short[] pixeldata = new short[Image.PixelDataLength];
int stride = Image.Width * 2;
Image.CopyPixelDataTo(pixeldata);
Bitmap bmap = new Bitmap(Image.Width, Image.Height, PixelFormat.Format16bppRgb555);
BitmapData bmapdata = bmap.LockBits(new Rectangle(0, 0, Image.Width, Image.Height), ImageLockMode.WriteOnly, bmap.PixelFormat);
IntPtr ptr = bmapdata.Scan0;
Marshal.Copy(pixeldata, 0, ptr, Image.PixelDataLength);
bmap.UnlockBits(bmapdata);
return bmap;
}
You may call it like this:
DepthImageFrame VFrame = e.OpenDepthImageFrame();
if (VFrame == null) return;
short[] pixelS = new short[VFrame.PixelDataLength];
Bitmap bmap = ImageToBitmap(VFrame);
how do I extract Images, which are FlateDecoded (such like PNG) out of a PDF-Document with PDFSharp?
I found that comment in a Sample of PDFSharp:
// TODO: You can put the code here that converts vom PDF internal image format to a
// Windows bitmap
// and use GDI+ to save it in PNG format.
// [...]
// Take a look at the file
// PdfSharp.Pdf.Advanced/PdfImage.cs to see how we create the PDF image formats.
Does anyone have a solution for this problem?
Thanks for your replies.
EDIT: Because I'm not able to answer on my own Question within 8 hours, I do it on that way:
Thanks for your very fast reply.
I added some Code to the Method "ExportAsPngImage", but I didn't get the wanted results. It is just extracting a few more Images (png) and they don't have the right colors and are distorted.
Here's my actual Code:
PdfSharp.Pdf.Filters.FlateDecode flate = new PdfSharp.Pdf.Filters.FlateDecode();
byte[] decodedBytes = flate.Decode(bytes);
System.Drawing.Imaging.PixelFormat pixelFormat;
switch (bitsPerComponent)
{
case 1:
pixelFormat = PixelFormat.Format1bppIndexed;
break;
case 8:
pixelFormat = PixelFormat.Format8bppIndexed;
break;
case 24:
pixelFormat = PixelFormat.Format24bppRgb;
break;
default:
throw new Exception("Unknown pixel format " + bitsPerComponent);
}
Bitmap bmp = new Bitmap(width, height, pixelFormat);
var bmpData = bmp.LockBits(new Rectangle(0, 0, width, height), ImageLockMode.WriteOnly, pixelFormat);
int length = (int)Math.Ceiling(width * bitsPerComponent / 8.0);
for (int i = 0; i < height; i++)
{
int offset = i * length;
int scanOffset = i * bmpData.Stride;
Marshal.Copy(decodedBytes, offset, new IntPtr(bmpData.Scan0.ToInt32() + scanOffset), length);
}
bmp.UnlockBits(bmpData);
using (FileStream fs = new FileStream(#"C:\Export\PdfSharp\" + String.Format("Image{0}.png", count), FileMode.Create, FileAccess.Write))
{
bmp.Save(fs, System.Drawing.Imaging.ImageFormat.Png);
}
Is that the right way? Or should I choose another way? Thanks a lot!
I know this answer might be a few years to late, but maybe it will help others.
The disortion occurs in my case because image.Elements.GetInteger(PdfImage.Keys.BitsPerComponent) seems to not return the correct value. As Vive la déraison pointed out under your question, you get the BGR Format for using Marshal.Copy. So reversing the Bytes and rotating the Bitmap after executing Marshal.Copy will do the job.
The resulting code looks like this:
private static void ExportAsPngImage(PdfDictionary image, ref int count)
{
int width = image.Elements.GetInteger(PdfImage.Keys.Width);
int height = image.Elements.GetInteger(PdfImage.Keys.Height);
var canUnfilter = image.Stream.TryUnfilter();
byte[] decodedBytes;
if (canUnfilter)
{
decodedBytes = image.Stream.Value;
}
else
{
PdfSharp.Pdf.Filters.FlateDecode flate = new PdfSharp.Pdf.Filters.FlateDecode();
decodedBytes = flate.Decode(image.Stream.Value);
}
int bitsPerComponent = 0;
while (decodedBytes.Length - ((width * height) * bitsPerComponent / 8) != 0)
{
bitsPerComponent++;
}
System.Drawing.Imaging.PixelFormat pixelFormat;
switch (bitsPerComponent)
{
case 1:
pixelFormat = System.Drawing.Imaging.PixelFormat.Format1bppIndexed;
break;
case 8:
pixelFormat = System.Drawing.Imaging.PixelFormat.Format8bppIndexed;
break;
case 16:
pixelFormat = System.Drawing.Imaging.PixelFormat.Format16bppArgb1555;
break;
case 24:
pixelFormat = System.Drawing.Imaging.PixelFormat.Format24bppRgb;
break;
case 32:
pixelFormat = System.Drawing.Imaging.PixelFormat.Format32bppArgb;
break;
case 64:
pixelFormat = System.Drawing.Imaging.PixelFormat.Format64bppArgb;
break;
default:
throw new Exception("Unknown pixel format " + bitsPerComponent);
}
decodedBytes = decodedBytes.Reverse().ToArray();
Bitmap bmp = new Bitmap(width, height, pixelFormat);
BitmapData bmpData = bmp.LockBits(new Rectangle(0, 0, bmp.Width, bmp.Height), ImageLockMode.WriteOnly, bmp.PixelFormat);
int length = (int)Math.Ceiling(width * (bitsPerComponent / 8.0));
for (int i = 0; i < height; i++)
{
int offset = i * length;
int scanOffset = i * bmpData.Stride;
Marshal.Copy(decodedBytes, offset, new IntPtr(bmpData.Scan0.ToInt32() + scanOffset), length);
}
bmp.UnlockBits(bmpData);
bmp.RotateFlip(RotateFlipType.Rotate180FlipNone);
bmp.Save(String.Format("exported_Images\\Image{0}.png", count++), System.Drawing.Imaging.ImageFormat.Png);
}
The code might need some optimisation, but it did export FlateDecoded Images correctly in my case.
To get a Windows BMP, you just have to create a Bitmap header and then copy the image data into the bitmap. PDF images are byte aligned (every new line starts on a byte boundary) while Windows BMPs are DWORD aligned (every new line starts on a DWORD boundary (a DWORD is 4 bytes for historical reasons)).
All information you need for the Bitmap header can be found in the filter parameters or can be calculated.
The color palette is another FlateEncoded object in the PDF. You also copy that into the BMP.
This must be done for several formats (1 bit per pixel, 8 bpp, 24 bpp, 32 bpp).
Here's my full code for doing this.
I'm extracting a UPS shipping label from a PDF so I know the format in advance. If your extracted image is of an unknown type then you'll need to check the bitsPerComponent and handle it accordingly. I also only handle the first image here on the first page.
Note: I'm using TryUnfilter to 'deflate' which uses whatever filter is applied and decodes the data in-place for me. No need to call 'Deflate' explicitly.
var file = #"c:\temp\PackageLabels.pdf";
var doc = PdfReader.Open(file);
var page = doc.Pages[0];
{
// Get resources dictionary
PdfDictionary resources = page.Elements.GetDictionary("/Resources");
if (resources != null)
{
// Get external objects dictionary
PdfDictionary xObjects = resources.Elements.GetDictionary("/XObject");
if (xObjects != null)
{
ICollection<PdfItem> items = xObjects.Elements.Values;
// Iterate references to external objects
foreach (PdfItem item in items)
{
PdfReference reference = item as PdfReference;
if (reference != null)
{
PdfDictionary xObject = reference.Value as PdfDictionary;
// Is external object an image?
if (xObject != null && xObject.Elements.GetString("/Subtype") == "/Image")
{
// do something with your image here
// only the first image is handled here
var bitmap = ExportImage(xObject);
bmp.Save(#"c:\temp\exported.png", System.Drawing.Imaging.ImageFormat.Bmp);
}
}
}
}
}
}
Using these helper functions
private static Bitmap ExportImage(PdfDictionary image)
{
string filter = image.Elements.GetName("/Filter");
switch (filter)
{
case "/FlateDecode":
return ExportAsPngImage(image);
default:
throw new ApplicationException(filter + " filter not implemented");
}
}
private static Bitmap ExportAsPngImage(PdfDictionary image)
{
int width = image.Elements.GetInteger(PdfImage.Keys.Width);
int height = image.Elements.GetInteger(PdfImage.Keys.Height);
int bitsPerComponent = image.Elements.GetInteger(PdfImage.Keys.BitsPerComponent);
var canUnfilter = image.Stream.TryUnfilter();
var decoded = image.Stream.Value;
Bitmap bmp = new Bitmap(width, height, System.Drawing.Imaging.PixelFormat.Format8bppIndexed);
BitmapData bmpData = bmp.LockBits(new Rectangle(0, 0, bmp.Width, bmp.Height), ImageLockMode.WriteOnly, bmp.PixelFormat);
Marshal.Copy(decoded, 0, bmpData.Scan0, decoded.Length);
bmp.UnlockBits(bmpData);
return bmp;
}
So far... my code... it works with many png files, but not the one that comes from adobe photoshop with colorspace indexed:
private bool ExportAsPngImage(PdfDictionary image, string SaveAsName)
{
int width = image.Elements.GetInteger(PdfSharp.Pdf.Advanced.PdfImage.Keys.Width);
int height = image.Elements.GetInteger(PdfSharp.Pdf.Advanced.PdfImage.Keys.Height);
int bitsPerComponent = image.Elements.GetInteger(PdfSharp.Pdf.Advanced.PdfImage.Keys.BitsPerComponent);
var ColorSpace = image.Elements.GetArray(PdfImage.Keys.ColorSpace);
System.Drawing.Imaging.PixelFormat pixelFormat= System.Drawing.Imaging.PixelFormat.Format24bppRgb; //24 just for initalize
if (ColorSpace is null) //no colorspace.. bufferedimage?? is in BGR order instead of RGB so change the byte order. Right now it works
{
byte[] origineel_byte_boundary = image.Stream.UnfilteredValue;
bitsPerComponent = (origineel_byte_boundary.Length) / (width * height);
switch (bitsPerComponent)
{
case 4:
pixelFormat = System.Drawing.Imaging.PixelFormat.Format32bppPArgb;
break;
case 3:
pixelFormat = System.Drawing.Imaging.PixelFormat.Format24bppRgb;
break;
default:
{
MessageBox.Show("Unknown pixel format " + bitsPerComponent, "Error", MessageBoxButtons.OK, MessageBoxIcon.Warning);
return false;
}
break;
}
Bitmap bmp = new Bitmap(width, height, pixelFormat); //copy raw bytes to "master" bitmap so we are out of pdf format to work with
System.Drawing.Imaging.BitmapData bmd = bmp.LockBits(new Rectangle(0, 0, width, height), System.Drawing.Imaging.ImageLockMode.WriteOnly, pixelFormat);
System.Runtime.InteropServices.Marshal.Copy(origineel_byte_boundary, 0, bmd.Scan0, origineel_byte_boundary.Length);
bmp.UnlockBits(bmd);
Bitmap bmp2 = new Bitmap(width, height, pixelFormat);
for (int indicex = 0; indicex < bmp.Width; indicex++)
{
for (int indicey = 0; indicey < bmp.Height; indicey++)
{
Color nuevocolor = bmp.GetPixel(indicex, indicey);
Color colorintercambiado = Color.FromArgb(nuevocolor.A, nuevocolor.B, nuevocolor.G, nuevocolor.R);
bmp2.SetPixel(indicex, indicey, colorintercambiado);
}
}
using (FileStream fs = new FileStream(SaveAsName, FileMode.Create, FileAccess.Write))
{
bmp2.Save(fs, System.Drawing.Imaging.ImageFormat.Png);
}
bmp2.Dispose();
bmp.Dispose();
}
else
{
// this is the case of photoshop... work needs to be done here. I ´m able to get the color palette but no idea how to put it back or create the png file...
switch (bitsPerComponent)
{
case 4:
pixelFormat = System.Drawing.Imaging.PixelFormat.Format32bppArgb;
break;
default:
{
MessageBox.Show("Unknown pixel format " + bitsPerComponent, "Error", MessageBoxButtons.OK, MessageBoxIcon.Warning);
return false;
}
break;
}
if ((ColorSpace.Elements.GetName(0) == "/Indexed") && (ColorSpace.Elements.GetName(1) == "/DeviceRGB"))
{
//we need to create the palette
int paletteColorCount = ColorSpace.Elements.GetInteger(2);
List<System.Drawing.Color> paletteList = new List<Color>();
//Color[] palette = new Color[paletteColorCount+1]; // no idea why but it seams that there´s always 1 color more. ¿transparency?
PdfObject paletteObj = ColorSpace.Elements.GetObject(3);
PdfDictionary paletteReference = (PdfDictionary)paletteObj;
byte[] palettevalues = paletteReference.Stream.Value;
for (int index = 0; index < (paletteColorCount + 1); index++)
{
//palette[index] = Color.FromArgb(1, palettevalues[(index*3)], palettevalues[(index*3)+1], palettevalues[(index*3)+2]); // RGB
paletteList.Add(Color.FromArgb(1, palettevalues[(index * 3)], palettevalues[(index * 3) + 1], palettevalues[(index * 3) + 2])); // RGB
}
}
}
return true;
}
PDF may contain images with masks and with different colorspace options that is why simply decoding an image object may not work properly in some cases.
So the code also needs to check for image masks (/ImageMask) and other properties of image objects (to see if image should also use inverted colors or uses indexed colors) inside PDF to recreate the image similar to how it is displayed in PDF. See Image object, /ImageMask and /Decode dictionaries in the official PDF Reference.
Not sure if PDFSharp is capable of finding Image Mask objects inside PDF but iTextSharp is able to access image mask objects (see PdfName.MASK object types).
Commercial tools like PDF Extractor SDK are able to extract images in both original form and in "as rendered" form.
I work for ByteScout, maker of PDF Extractor SDK
Maybe not directly answer the question but another option to extract images from PDF is to use FreeSpire.PDF which can extract the image from pdf easily. It is available as Nuget package https://www.nuget.org/packages/FreeSpire.PDF/. They handle all the image format and can export as PNG. Their sample code is
using System;
using System.Collections.Generic;
using System.Text;
using System.Drawing;
using Spire.Pdf;
namespace ExtractImagesFromPDF
{
class Program
{
static void Main(string[] args)
{
//Instantiate an object of Spire.Pdf.PdfDocument
PdfDocument doc = new PdfDocument();
//Load a PDF file
doc.LoadFromFile("sample.pdf");
List<Image> ListImage = new List<Image>();
for (int i = 0; i < doc.Pages.Count; i++)
{
// Get an object of Spire.Pdf.PdfPageBase
PdfPageBase page = doc.Pages[i];
// Extract images from Spire.Pdf.PdfPageBase
Image[] images = page.ExtractImages();
if (images != null && images.Length > 0)
{
ListImage.AddRange(images);
}
}
if (ListImage.Count > 0)
{
for (int i = 0; i < ListImage.Count; i++)
{
Image image = ListImage[i];
image.Save("image" + (i + 1).ToString() + ".png", System.Drawing.Imaging.ImageFormat.Png);
}
System.Diagnostics.Process.Start("image1.png");
}
}
}
}
(code taken from https://www.e-iceblue.com/Tutorials/Spire.PDF/Spire.PDF-Program-Guide/How-to-Extract-Image-From-PDF-in-C.html)