Develop Reference

Insufficient buffer size using WriteableBitmap?

I am modifying the ColorBasic Kinect example in order to display an image overlaid to the video stream. So what I've done is to load an image with transparent background (now a GIF but it may change), and write to the displayed bitmap.
The error I'm getting is that the buffer I'm writing to is too small.
I cannot see what the actual error is (I'm a complete newbie in XAML/C#/Kinect), but the WriteableBitmap is 1920x1080, and the bitmap I want to copy is 200x200, so why am I getting this error? I cannot see how a transparent background could be of any harm, but I am beginning to suspect that...
Note that without the last WritePixels, the code works and I see the webcam's output. My code follows.
The overlay image:
public BitmapImage overlay = new BitmapImage(new Uri("C:\\users\\user\\desktop\\something.gif"));
The callback function that displays the Kinect's webcam (see the default example ColorBasic) with my very small modifications:
private void Reader_ColorFrameArrived(object sender, ColorFrameArrivedEventArgs e)
{
// ColorFrame is IDisposable
using (ColorFrame colorFrame = e.FrameReference.AcquireFrame())
{
if (colorFrame != null)
{
FrameDescription colorFrameDescription = colorFrame.FrameDescription;
using (KinectBuffer colorBuffer = colorFrame.LockRawImageBuffer())
{
this.colorBitmap.Lock();
// verify data and write the new color frame data to the display bitmap
if ((colorFrameDescription.Width == this.colorBitmap.PixelWidth) && (colorFrameDescription.Height == this.colorBitmap.PixelHeight))
{
colorFrame.CopyConvertedFrameDataToIntPtr(
this.colorBitmap.BackBuffer,
(uint)(colorFrameDescription.Width * colorFrameDescription.Height * 4),
ColorImageFormat.Bgra);
this.colorBitmap.AddDirtyRect(new Int32Rect(0, 0, this.colorBitmap.PixelWidth, this.colorBitmap.PixelHeight));
}
if(this.overlay != null)
{
// Calculate stride of source
int stride = overlay.PixelWidth * (overlay.Format.BitsPerPixel / 8);
// Create data array to hold source pixel data
byte[] data = new byte[stride * overlay.PixelHeight];
// Copy source image pixels to the data array
overlay.CopyPixels(data, stride, 0);
this.colorBitmap.WritePixels(new Int32Rect(0, 0, overlay.PixelWidth, overlay.PixelHeight), data, stride, 0);
}
this.colorBitmap.Unlock();
}
}
}
}

Your overlay.Format.BitsPerPixel / 8 will be 1 (because it's a gif), but you're trying to copy it to something that is not a gif, probably BGRA (32 bit). Thus you got a huge difference in size (4x).
.WritePixels should take in the stride value of the destination buffer, but you past it the stride value of the overlay (this can cause weird problems as well).
And finally, even if it went 100% smooth your overlay will not actually "overlay" anything, it will replace -- since I don't see any alpha bending math in your code.
Switch your .gif to a .png (32bit) and see if that helps.
Also, if you're looking for an AlphaBltMerge type code: I wrote the entire thing here.. it's very easy to understand.
Merge 2 - 32bit Images with Alpha Channels

image quality is blurry

I have a situation where I want to convert some XAML to an image, so I created a RichTextBox and then took the image of it. Now problem is that words in image is blurred, any idea how I might be able to fix it?
public System.Drawing.Bitmap ConvertXamltoImage(string XamlString, int Width, int Height)
{
RichTextBox AdContentRichTextBox = new RichTextBox() { Width = Width, Height = Height };
AdContentRichTextBox.BorderThickness = new Thickness(0);
XmlReader _XmlReader = XmlReader.Create(new StringReader(XamlString));
AdContentRichTextBox.Document = XamlString;
var size = new Size(Width, Height);
AdContentRichTextBox.Measure(size);
AdContentRichTextBox.Arrange(new Rect(size));
RenderTargetBitmap bmp = new RenderTargetBitmap(Width, Height, 300, 300, PixelFormats.Pbgra32);
bmp.Render(AdContentRichTextBox);
DrawingVisual _drawingVisual = new DrawingVisual();
using (DrawingContext _drwaingContext = _drawingVisual.RenderOpen())
{
VisualBrush _visualBrush = new VisualBrush(AdContentRichTextBox);
}
PngBitmapEncoder _png = new PngBitmapEncoder();
_png.Frames.Add(BitmapFrame.Create(bmp));
System.Drawing.Bitmap _tempBitmap = null;
using (Stream _fileStream = new MemoryStream())
{
_png.Save(_fileStream);
_tempBitmap = new System.Drawing.Bitmap(_fileStream);
_fileStream.Flush();
}
return _tempBitmap;
}

Hmmmm..there could be lots of things all interacting here:
1st
"Grayscale fall back - if ClearType is disabled or one is rendering text in certain situations where the ClearType algorithm cannot be run, WPF will use a grayscale rendering algorithm to antialias the rendered text."
Rendering Text to a RenderTargetBitmap seems to be one of those situations....(the renderer switches from a hardware to a software path).
2nd
In addition NET 4 switched the default scaling algorithm from high-quality (Fant) to low-quality (Bi-Linear).....now that shouldn't come into play here as it doesn't look like you are scaling the bitmap in any way...but you never know what's going on inside. It's possible to switch the scaler back to the higher quality one.
http://www.olsonsoft.com/blogs/stefanolson/post/Workaround-for-low-quality-bitmap-resizing-in-WPF-4.aspx
3rd
You may need to take into account the parent container of the RichTextBox...see last link below, mentions it can distort the font rendering.
Problems with rendering text as bitmaps using WPF
Some ideas on how to work around this are:
render the RichTextBox at a higher resolution e.g. 600dpi, and then scale down the bitmap (probably will make no difference)
capture the screen....difficult or not practical if your visual is offscreen/obscured, etc.
See related links:
http://windowsclient.net/wpf/white-papers/wpftextclarity.aspx
WPF RenderTargetBitmap downscaling text ClearType to GreyScale
WPF RenderTargetBitmap downscaling TextRenderMode to GreyScale
WPF text rendering inconsistencies

Kinect and EMGU (OpenCV) - BitmapSource issue

Alright, so basically I am trying to use openCV with the Kinect (Microsoft's new Kinect 1.0 SDK). I am very new to both C# and Kinect. But what I want to do is use the kinect for facial recognition using EMGU (openCV wrapper for C#). So far I have successfully captured the video stream from the kinect, converted it into an EMGU Image<>, then converted it a Byte[] array so that I can use the BitmapSource to display my image on the screen.
While that works fine, problems seem to arise when I try to actually do some image processing with the Image<> class. It actually seems to be processing fine, but it is not very fast. This wouldn't necessarily be a problem for me, but now the BitmapSource isn't being displayed at all.
Here is an example of my code to detect faces:
img = new Image<Bgr, byte>(clone);
haar = new HaarCascade("directory");
Image<Gray, Byte> gray;
using (HaarCascade face = new HaarCascade("blablabla.xml"))
using (HaarCascade eye = new HaarCascade("blarg.xml"))
{
using ( gray = img.Convert<Gray, Byte>()) //Convert it to Grayscale
{
MCvAvgComp[] facesDetected = face.Detect(gray, 1.1, 1, mgu.CV.CvEnum.HAAR_DETECTION_TYPE.DO_CANNY_PRUNING, new System.Drawing.Size(img.Width / 8, img.Height / 8));
foreach (MCvAvgComp f in facesDetected)
{
img.Draw(f.rect, new Bgr(System.Drawing.Color.Blue), 2);
imgDoneProc = img.ToBitmap();
}
}
}
Then I use the BitmapSource.Create() :
BitmapSource bmapa = BitmapSource.Create(PImage.Width, PImage.Height, 96, 96, PixelFormats.Bgr32, null, bmpBytes, PImage.Width * PImage.BytesPerPixel);
image1.Source = bmapa;
(PImage is the stream from the Kinect; bmpBytes is a Byte[] converted from the Image<>)
So, if I comment out the code that does the image processing, all of the converting back and forth works fine. When I add the image proc code, I can write to the console some useful data, but the image is not displayed. I have also noticed that the 'bmapa' is not updated quickly. That is the only noticeable difference other than nothing being displayed in image1.
So, am I using BitmapSource incorrectly, or is there are way speed up my code or perhaps slow BitmapSource's "refresh rate"? Because when I am just converting between data structures, I get a steady stream from the kinect and all works fine.
Thanks,
Brent

Memory leak while asynchronously loading BitmapSource images

I have a fair few images that I'm loading into a ListBox in my WPF application. Originally I was using GDI to resize the images (the originals take up far too much memory). That was fine, except they were taking about 400ms per image. Not so fine. So in search of another solution I found a method that uses TransformedBitmap (which inherits from BitmapSource). That's great, I thought, I can use that. Except I'm now getting memory leaks somewhere...
I'm loading the images asynchronously using a BackgroundWorker like so:
BitmapSource bs = ImageUtils.ResizeBitmapSource(ImageUtils.GetImageSource(photo.FullName));
//BitmapSource bs = ImageUtils.GetImageSource(photo.FullName);
bs.Freeze();
this.dispatcher.Invoke(new Action(() => { photo.Source = bs; }));
GetImageSource just gets the Bitmap from the path and then converts to BitmapSource.
Here's the code snippet for ResizeBitmapSource:
const int thumbnailSize = 200;
int width;
int height;
if (bs.Width > bs.Height)
{
width = thumbnailSize;
height = (int)(bs.Height * thumbnailSize / bs.Width);
}
else
{
height = thumbnailSize;
width = (int)(bs.Width * thumbnailSize / bs.Height);
}
BitmapSource tbBitmap = new TransformedBitmap(bs,
new ScaleTransform(width / bs.Width,
height / bs.Height, 0, 0));
return tbBitmap;
That code is essentially the code from:
http://rongchaua.net/blog/c-wpf-fast-image-resize/
Any ideas what could be causing the leak?
edit:
Here's the code for GetImageSource, as requested
using (var stream = new FileStream(path, FileMode.Open, FileAccess.Read))
{
using (var bmp = Image.FromStream(stream, false, false))
{
// Use WPF to resize
var bitmapSource = ConvertBitmapToBitmapSource(bmp);
bitmapSource = ResizeBitmapSource(bitmapSource);
return bitmapSource;
}
}

I think you misunderstood how the TransformedBitmap works. It holds onto a reference to the source bitmap, and transforms it in memory. Maybe you could encode the transformed bitmap into a memory stream, and read it right back out. I'm not sure how fast this would be, but you wouldn't then be holding on to the full sized bitmap.
I found this blog post that returned a WriteableBitmap with the TransformedBitmap as the source. The WriteableBitmap will copy the pixel data to a memory buffer in the initializer, so it doesn't actually hold on to a reference to the TransformedBitmap, or the full sized image.

At a guess, from looking at your code you might need to dispose of the bitmap returned by the call to ImageUtils.GetImageSource(photo.FullName).
I have also noted on the blog you pointed out that the author has added an update (11th of March) about inserting a using statement to prevent memory leaks.

Image resizing - sometimes very poor quality?

I'm resizing some images to the screen resolution of the user; if the aspect ratio is wrong, the image should be cut.
My code looks like this:
protected void ConvertToBitmap(string filename)
{
var origImg = System.Drawing.Image.FromFile(filename);
var widthDivisor = (double)origImg.Width / (double)System.Windows.Forms.Screen.PrimaryScreen.Bounds.Width;
var heightDivisor = (double)origImg.Height / (double)System.Windows.Forms.Screen.PrimaryScreen.Bounds.Height;
int newWidth, newHeight;
if (widthDivisor < heightDivisor)
{
newWidth = (int)((double)origImg.Width / widthDivisor);
newHeight = (int)((double)origImg.Height / widthDivisor);
}
else
{
newWidth = (int)((double)origImg.Width / heightDivisor);
newHeight = (int)((double)origImg.Height / heightDivisor);
}
var newImg = origImg.GetThumbnailImage(newWidth, newHeight, null, IntPtr.Zero);
newImg.Save(this.GetBitmapPath(filename), System.Drawing.Imaging.ImageFormat.Bmp);
}
In most cases, this works fine. But for some images, the result has an extremely poor quality. It looks like the would have been resized to something very small (thumbnail size) and enlarged again.. But the resolution of the image is correct. What can I do?
Example orig image:
alt text http://img523.imageshack.us/img523/1430/naturaerowoods.jpg
Example resized image:
Note: I have a WPF application but I use the WinForms function for resizing because it's easier and because I already need a reference to System.Windows.Forms for a tray icon.

Change the last two lines of your method to this:
var newImg = new Bitmap(newWidth, newHeight);
Graphics g = Graphics.FromImage(newImg);
g.DrawImage(origImg, new Rectangle(0,0,newWidth,newHeight));
newImg.Save(this.GetBitmapPath(filename), System.Drawing.Imaging.ImageFormat.Bmp);
g.Dispose();

I cannot peek into the .NET source at the moment, but most likely the problem is in the Image.GetThumbnailImage method. Even MSDN says that "it works well when the requested thumbnail image has a size of about 120 x 120 pixels, but it you request a large thumbnail image (for example, 300 x 300) from an Image that has an embedded thumbnail, there could be a noticeable loss of quality in the thumbnail image". For true resizing (i.e. not thumbnailing), you should use the Graphics.DrawImage method. You may also need to play with the Graphics.InterpolationMode to get a better quality if needed.

If you're not creating a thumbnail, using a method called GetThumbnailImage probably isn't a good idea...
For other options, have a look at this CodeProject article. In particular, it creates a new image, creates a Graphics for it and sets the interpolation mode to HighQualityBicubic and draws the original image onto the graphics. Worth a try, at least.

As indicated on MSDN, GetThumbnailImage() is not designed to do arbitrary image scaling. Anything over 120x120 should be scaled manually. Try this instead:
using(var newImg = new Bitmap(origImg, newWidth, newHeight))
{
newImg.Save(this.GetBitmapPath(filename), System.Drawing.Imaging.ImageFormat.Bmp);
}
Edit
As a point of clarification, this overload of the Bitmap constructor calls Graphics.DrawImage, though you do not have any control over the interpolation.

instead of this code:
newImg.Save(this.GetBitmapPath(filename), System.Drawing.Imaging.ImageFormat.Bmp);
use this one :
System.Drawing.Imaging.ImageCodecInfo[] info = System.Drawing.Imaging.ImageCodecInfo.GetImageEncoders();
System.Drawing.Imaging.EncoderParameters param = new System.Drawing.Imaging.EncoderParameters(1);
param.Param[0] = new System.Drawing.Imaging.EncoderParameter(System.Drawing.Imaging.Encoder.Quality, 100L);
newImg.Save(dest_img, info[1], param);

For examples, the original image is JPG and the resized image is PNG. Are you converting between formats on purpose? Switching between different lossey compression schemes can cause quality loss.

Are you increasing or decreasing the size of the image when you resize it? If you are creating a larger image from a smaller one, this sort of degradation is to be expected.

Images will definitely be degraded if you enlarge them.

Some camera's put a resized thumbnail into the file itself presumably for preview purposes on the device itself.
The GetThumbnail method actually gets this Thumbnail image which is embedded within the image file instead of getting the higher res method.
The easy solution is to trick .Net into throwing away that thumbnail information before doing your resize or other operation. like so....
img.RotateFlip(System.Drawing.RotateFlipType.Rotate180FlipX);
//removes thumbnails from digital camera shots
img.RotateFlip(System.Drawing.RotateFlipType.Rotate180FlipX);
If you are attempting to resize constraining proportions I wrote an extension method on System.Drawing.Image that you might find handy.
/// <summary>
///
/// </summary>
/// <param name="img"></param>
/// <param name="size">Size of the constraining proportion</param>
/// <param name="constrainOnWidth"></param>
/// <returns></returns>
public static System.Drawing.Image ResizeConstrainProportions(this System.Drawing.Image img,
int size, bool constrainOnWidth, bool dontResizeIfSmaller)
{
if (dontResizeIfSmaller && (img.Width < size))
return img;
img.RotateFlip(System.Drawing.RotateFlipType.Rotate180FlipX);
img.RotateFlip(System.Drawing.RotateFlipType.Rotate180FlipX);
float ratio = 0;
ratio = (float)img.Width / (float)img.Height;
int height, width = 0;
if (constrainOnWidth)
{
height = (int)(size / ratio);
width = size;
}
else
{
width = (int)(size * ratio);
height = size;
}
return img.GetThumbnailImage(width, height, null, (new System.IntPtr(0)));
}

This is going to vary widely based on the following factors:
How closely the destination resolution matches a "natural" scale of the original resolution
The source image color depth
The image type(s) - some are more lossy than others