Develop Reference

C# cut rectangle blocks from image

I develop a screen sharing app and i would like to make it as efficient as posibble so im trying to send only the differences between the screen shots.
So, suppose we have this image for example:its a 32bpprgba image with transpert parts around.
I would like to store each one of the blocks here as a rectangle in a List and get them bounds. It may sounds very complex but actually it just requires a little logic.
This is my code so far:
private unsafe List<Rectangle> CodeImage(Bitmap bmp)
{
List<Rectangle> rec = new List<Rectangle>();
Bitmap bmpRes = new Bitmap(bmp.Width, bmp.Height);
BitmapData bmData = bmp.LockBits(new Rectangle(0, 0, bmp.Width, bmp.Height), System.Drawing.Imaging.ImageLockMode.ReadOnly, bmp.PixelFormat);
IntPtr scan0 = bmData.Scan0;
int stride = bmData.Stride;
int nWidth = bmp.Width;
int nHeight = bmp.Height;
int minX = int.MaxValue; ;
int minY = int.MaxValue;
int maxX = 0;
bool found = false;
for (int y = 0; y < bmp.Height; y++)
{
byte* p = (byte*)scan0.ToPointer();
p += y * stride;
for (int x = 0; x < bmp.Width; x++)
{
if (p[3] != 0) //Check if pixel is not transparent;
{
found = true;
if (x < minX)
minX = x;
if (x > maxX)
maxX = x;
if (y < minY)
minY = y;
}
else
{
if (found)
{
int height = getBlockHeight(stride, scan0, maxX, minY);
found = false;
Rectangle temp = new Rectangle(minX, minY, maxX - minX, height);
rec.Add(temp);
y += minY;
break;
}
}
p += 4;//add to the pointer 4 bytes;
}
}
return rec;
}
as you see im trying to scan the image using the height and width, and when i found a pixel i send it to GetBlockHeight function to get it's height:
public unsafe int getBlockHeight(int stride, IntPtr scan, int x, int y1)
{
int height = 0; ;
for (int y = y1; y < 1080; y++)
{
byte* p = (byte*)scan.ToPointer();
p += (y * stride) + (x * 4);
if (p[3] != 0) //Check if pixel is not transparent;
{
height++;
}
}
return height;
}
But im just not getting the result... i think there's somthing with the logic here... can anyone light my eyes? i know it requires a bit time and thinking but i would very very appreciate anyone who could help a little.

In your current algorithm, after successfully matching a rectangle, you increase y with its height and break out of the inner loop. This means you can only detect data for one rectangle per horizontal line.
If I were you I'd think about the following things, before jumping back into the code:
Save the complete image as a PNG file, and look at its size. Is further processing really required?
Are these rectangles accurate? Will there be scenario's in which you would be constantly sending the contents of the entire screen anyway?
If you're developing for Windows, you might be able to hook into the procedure that invalidates areas on the screen, in which case you wouldn't have to determine these rectangles yourself. I don't know about other OSes
Also I personally wouldn't try to solve the rectangle-detection algorithm in a "nesty" for-loop, but go with something like this:
public void FindRectangles(Bitmap bitmap, Rectangle searchArea, List<Rectangle> results)
{
// Find the first non-transparent pixel within the search area.
// Ensure that it is the pixel with the lowest y-value possible
Point p;
if (!TryFindFirstNonTransparent(bitmap, searchArea, out p))
{
// No non-transparent pixels in this area
return;
}
// Find its rectangle within the area
Rectangle r = GetRectangle(bitmap, p, searchArea);
results.Add(r);
// No need to search above the rectangle we just found
Rectangle left = new Rectangle(searchArea.Left, r.Top, r.Left - searchArea.Left, searchArea.Bottom - r.Top);
Rectangle right = new Rectangle(r.Right, r.Top, searchArea.Right - r.Right, searchArea.Bottom - r.Top);
Rectangle bottom = new Rectangle(r.Left, r.Bottom, r.Width, searchArea.Bottom - r.Bottom);
FindRectangles(bitmap, left, results);
FindRectangles(bitmap, right, results);
FindRectangles(bitmap, bottom, results);
}
public Rectangle GetRectangle(Bitmap bitmap, Point p, Rectangle searchArea)
{
int right = searchArea.Right;
for (int x = p.X; x < searchArea.Right; x++)
{
if (IsTransparent(x, p.Y))
{
right = x - 1;
break;
}
}
int bottom = searchArea.Bottom;
for (int y = p.Y; y < searchArea.Bottom; y++)
{
if (IsTransparent(p.X, y))
{
bottom = y - 1;
break;
}
}
return new Rectangle(p.X, p.Y, right - p.X, bottom - p.Y);
}
This approach, when fully implemented, should give you a list of rectangles (although it will occasionally split a rectangle in two).
(Of course instead of providing the bitmap, you'd pass the pointer to the pixel data with some metadata instead)

How can i use LockBits with a Bitmap to scan for white pixels and then write to a new bitmap all the non white pixels?

This is what i did in form1 constructor:
Bitmap bmp2 = new Bitmap(#"e:\result1001.jpg");
CropImageWhiteAreas.ImageTrim(bmp2);
bmp2.Save(#"e:\result1002.jpg");
bmp2.Dispose();
The class CropImageWhiteAreas:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Drawing;
using System.Drawing.Imaging;
using System.Runtime.InteropServices;
namespace Test
{
class CropImageWhiteAreas
{
public static Bitmap ImageTrim(Bitmap img)
{
//get image data
BitmapData bd = img.LockBits(new Rectangle(Point.Empty, img.Size),
ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb);
int[] rgbValues = new int[img.Height * img.Width];
Marshal.Copy(bd.Scan0, rgbValues, 0, rgbValues.Length);
img.UnlockBits(bd);
#region determine bounds
int left = bd.Width;
int top = bd.Height;
int right = 0;
int bottom = 0;
//determine top
for (int i = 0; i < rgbValues.Length; i++)
{
int color = rgbValues[i] & 0xffffff;
if (color != 0xffffff)
{
int r = i / bd.Width;
int c = i % bd.Width;
if (left > c)
{
left = c;
}
if (right < c)
{
right = c;
}
bottom = r;
top = r;
break;
}
}
//determine bottom
for (int i = rgbValues.Length - 1; i >= 0; i--)
{
int color = rgbValues[i] & 0xffffff;
if (color != 0xffffff)
{
int r = i / bd.Width;
int c = i % bd.Width;
if (left > c)
{
left = c;
}
if (right < c)
{
right = c;
}
bottom = r;
break;
}
}
if (bottom > top)
{
for (int r = top + 1; r < bottom; r++)
{
//determine left
for (int c = 0; c < left; c++)
{
int color = rgbValues[r * bd.Width + c] & 0xffffff;
if (color != 0xffffff)
{
if (left > c)
{
left = c;
break;
}
}
}
//determine right
for (int c = bd.Width - 1; c > right; c--)
{
int color = rgbValues[r * bd.Width + c] & 0xffffff;
if (color != 0xffffff)
{
if (right < c)
{
right = c;
break;
}
}
}
}
}
int width = right - left + 1;
int height = bottom - top + 1;
#endregion
//copy image data
int[] imgData = new int[width * height];
for (int r = top; r <= bottom; r++)
{
Array.Copy(rgbValues, r * bd.Width + left, imgData, (r - top) * width, width);
}
//create new image
Bitmap newImage = new Bitmap(width, height, PixelFormat.Format32bppArgb);
BitmapData nbd
= newImage.LockBits(new Rectangle(0, 0, width, height),
ImageLockMode.WriteOnly, PixelFormat.Format32bppArgb);
Marshal.Copy(imgData, 0, nbd.Scan0, imgData.Length);
newImage.UnlockBits(nbd);
return newImage;
}
}
}
I also tried before it Peter solution.
In both the result is(This is a screenshot of my facebook after uploaded the image) still the white areas around:
You can the rectangle around the image i just uploaded and see what i mean by white area around.

If I understand correctly, you have found a sample code snippet that uses LockBits(), but you are not sure how it works or how to modify it to suit your specific need. So I will try to answer from that perspective.
First, a wild guess (since you didn't include the implementation of the LockBitmap class you're using in the first example): the LockBitmap class is some kind of helper class that is supposed to encapsulate the work of calling LockBits() and using the result, including providing versions of GetPixel() and SetPixel() which are presumably much faster than calling those methods on a Bitmap object directly (i.e. access the buffer obtained by calling LockBits()).
If that's the case, then modifying the first example to suit your need is probably best:
public void Change(Bitmap bmp)
{
Bitmap newBitmap = new Bitmap(bmp.Width, bmp.Height, bmp.PixelFormat);
LockBitmap source = new LockBitmap(bmp),
target = new LockBitmap(newBitmap);
source.LockBits();
target.LockBits();
Color white = Color.FromArgb(255, 255, 255, 255);
for (int y = 0; y < source.Height; y++)
{
for (int x = 0; x < source.Width; x++)
{
Color old = source.GetPixel(x, y);
if (old != white)
{
target.SetPixel(x, y, old);
}
}
}
source.UnlockBits();
target.UnlockBits();
newBitmap.Save("d:\\result.png");
}
In short: copy the current pixel value to a local variable, compare that value to the white color value, and if it is not the same, go ahead and copy the pixel value to the new bitmap.
Some variation on the second code example should work as well. The second code example does explicitly what is (I've assumed) encapsulated inside the LockBitmap class that the first code example uses. If for some reason, the first approach isn't suitable for your needs, you can follow the second example.
In that code example you provide, most of the method there is just handling the "grunt work" of locking the bitmap so that the raw data can be accessed, and then iterating through that raw data.
It computes the oRow and nRow array offsets (named for "old row" and "new row", I presume) based on the outer y loop, and then accesses individual pixel data by computing the offset within a given row based on the inner x loop.
Since you want to do essentially the same thing, but instead of converting the image to grayscale, you just want to selectively copy all non-white pixels to the new bitmap, you can (should be able to) simply modify the body of the inner x loop. For example:
byte red = oRow[x * pixelSize + 2],
green = oRow[x * pixelSize + 1],
blue = oRow[x * pixelSize];
if (red != 255 || green != 255 || blue != 255)
{
nRow[x * pixelSize + 2] = red;
nRow[x * pixelSize + 1] = green;
nRow[x * pixelSize] = blue;
}
The above would entirely replace the body of the inner x loop.
One caveat: do note that when using the LockBits() approach, knowing the pixel format of the bitmap is crucial. The example you've shown assumes the bitmaps are in 24 bpp format. If your own bitmaps are in this format, then you don't need to change anything. But if they are in a different format, you'll need to adjust the code to suit that. For example, if your bitmap is in 32 bpp format, you need to pass the correct PixelFormat value to the LockBits() method calls, and then set pixelSize to 4 instead of 3 as the code does now.
Edit:
You've indicated that you would like to crop the new image so that it is the minimize size required to contain all of the non-white pixels. Here is a version of the first example above that should accomplish that:
public void Change(Bitmap bmp)
{
LockBitmap source = new LockBitmap(bmp);
source.LockBits();
Color white = Color.FromArgb(255, 255, 255, 255);
int minX = int.MaxValue, maxX = int.MinValue,
minY = int.MaxValue, maxY = int.MinValue;
// Brute-force scan of the bitmap to find image boundary
for (int y = 0; y < source.Height; y++)
{
for (int x = 0; x < source.Width; x++)
{
if (source.GetPixel(x, y) != white)
{
if (x < minX) minX = x;
if (x > maxX) maxX = x;
if (y < minY) minY = y;
if (y > maxY) maxY = y;
}
}
}
Bitmap newBitmap = new Bitmap(maxX - minx + 1, maxY - minY + 1, bmp.PixelFormat);
LockBitmap target = new LockBitmap(newBitmap);
target.LockBits();
for (int y = 0; y < target.Height; y++)
{
for (int x = 0; x < target.Width; x++)
{
target.SetPixel(x, y, source.GetPixel(x + minX, y + minY));
}
}
source.UnlockBits();
target.UnlockBits();
newBitmap.Save("d:\\result.png");
}
This example includes an initial scan of the original bitmap, after locking it, to find the minimum and maximum coordinate values for any non-white pixel. Having done that, it uses the results of that scan to determine the dimensions of the new bitmap. When copying the pixels, it restricts the x and y loops to the dimensions of the new bitmap, adjusting the x and y values to map from the location in the new bitmap to the given pixel's original location in the old one.
Note that since the initial scan determines where the non-white pixels are, there's no need to check again when actually copying the pixels.
There are more efficient ways to scan the bitmap than the above. This version simply looks at every single pixel in the original bitmap, keeping track of the min and max values for each coordinate. I'm guessing this will be fast enough for your purposes, but if you want something faster, you can change the scan so that it scans for each min and max in sequence:
Scan each row from y of 0 to determine the first row with a non-white pixel. This is the min y value.
Scan each row from y of source.Height - 1 backwards, to find the max y value.
Having found the min and max y values, now scan the columns from x of 0 to find the min x and from source.Width - 1 backwards to find the max x.
Doing it that way involves a lot more code and is probably harder to read and understand, but would involve inspecting many fewer pixels in most cases.
Edit #2:
Here is a sample of the output of the second code example:
Note that all of the white border of the original bitmap (shown on the left side) has been cropped out, leaving only the smallest subset of the original bitmap that can contain all of the non-white pixels (shown on the right side).

Why is SetPixel not working correctly with ColorDialog?

I'm trying to change an image color by using Bitmap.SetPixel method. And I have a problem when using it like this:
Bitmap bmp = new Bitmap(Project1.Properties.Resources.gray_square_button);
int Width = bmp.Width;
int Height = bmp.Height;
Bitmap Nbmp = new Bitmap(bmp);
ColorDialog ColorDialog = new ColorDialog();
ColorDialog.AllowFullOpen = true;
DialogResult result = ColorDialog.ShowDialog();
if (result == DialogResult.OK)
{
for (int y = 0; y < Height; y++)
{
for (int x = 0; x < Width; x++)
{
System.Drawing.Color BackColor = ColorDialog.Color;
System.Drawing.Color p = bmp.GetPixel(x, y);
int a = BackColor.A;
int r = BackColor.R;
int g = BackColor.G;
int b = BackColor.B;
Nbmp.SetPixel(x, y, System.Drawing.Color.FromArgb(a, r, g, b));
}
}
PictureBox1.Image = Nbmp;
}
It will only draw a square with the color that I choose like in this image:
But if I use it like this, using a manual color and the original image color references (p insted of BackColor which is the color defined by the ColorDialog):
if (result == DialogResult.OK)
{
for (int y = 0; y < Height; y++)
{
for (int x = 0; x < Width; x++)
{
System.Drawing.Color BackColor = ColorDialog.Color;
System.Drawing.Color p = bmp.GetPixel(x, y);
int a = p.A;
int r = p.R;
int g = p.G;
int b = p.B;
Nbmp.SetPixel(x, y, System.Drawing.Color.FromArgb(a, r, 0, 0));
}
}
PictureBox1.Image = Nbmp;
}
The color is applied correctly and the image is displayed correctly as well:
What i've tried is changing only 1 value of the RGB color. But then if you choose a color the color will not be the one you selected but a different one based on the one that is modified by the Color of the Color Dialog.
int a = p.A;
int r = BackColor.R;
int g = p.G;
int b = p.B;
Why are the ColorDialog RGB values not allowing the image to be displayed correctly but only a colored square?
This is the original image:

You're replacing the colour of a "shaded" image with a flat selected colour from a ColorDialog. So basically, you're replacing every pixel, regardless of ARGB, with a single ARGB from the dialog, thus completely overwriting any existing image information. You may as well just draw the image from scratch based on the old image's dimensions, without concern for the original image since it's being completely rewritten.
It sounds like what you are probably intending to do is blend the 2 colours together (the original image + the new colour). There are literally hundreds of ways to do this. One that comes to mind is to create an overlay with 50% transparency and apply it over the original image. You could also set the ARGB to an average:
int a = (BackColor.A + p.A) / 2;
int r = (BackColor.R + p.R) / 2;
int g = (BackColor.G + p.G) / 2;
int b = (BackColor.B + p.B) / 2;
This will give you a feel for how to consider the original colours while changing them versus flat out replacing them.

Where to place the coordinates of text in a C# graphic bitmap

I wrote a C# Render method that renders a heatmap onto a Grasshopper canvas. Grasshopper is a Rhino plugin that allows for a simple GUI programming interface.
protected override void Render(Grasshopper.GUI.Canvas.GH_Canvas canvas, Graphics graphics, Grasshopper.GUI.Canvas.GH_CanvasChannel channel) {
base.Render(canvas, graphics, channel);
if (channel == Grasshopper.GUI.Canvas.GH_CanvasChannel.Wires) {
var comp = Owner as KT_HeatmapComponent;
if (comp == null)
return;
List<HeatMap> maps = comp.CachedHeatmaps;
if (maps == null)
return;
if (maps.Count == 0)
return;
int x = Convert.ToInt32(Bounds.X + Bounds.Width / 2);
int y = Convert.ToInt32(Bounds.Bottom + 10);
for (int i = 0; i < maps.Count; i++) {
Bitmap image = maps[i].Image;
if (image == null)
continue;
Rectangle mapBounds = new Rectangle(x, y, maps[i].Width, maps[i].Height);
//Rectangle mapBounds = new Rectangle(x, y, maps[i].Width * 10, maps[i].Height * 10);
mapBounds.X -= mapBounds.Width / 2;
Rectangle edgeBounds = mapBounds;
edgeBounds.Inflate(4, 4);
GH_Capsule capsule = GH_Capsule.CreateCapsule(edgeBounds, GH_Palette.Normal);
capsule.Render(graphics, Selected, false, false);
capsule.Dispose();
graphics.InterpolationMode = System.Drawing.Drawing2D.InterpolationMode.NearestNeighbor;
graphics.PixelOffsetMode = System.Drawing.Drawing2D.PixelOffsetMode.Half;
graphics.DrawImage(image, mapBounds);
graphics.InterpolationMode = System.Drawing.Drawing2D.InterpolationMode.HighQualityBicubic;
graphics.PixelOffsetMode = System.Drawing.Drawing2D.PixelOffsetMode.Default;
graphics.DrawRectangle(Pens.Black, mapBounds);
y = edgeBounds.Bottom - (mapBounds.Height) - 4;
}
}
}
Currently, this render methods draws an image like this onto the canvas:
With that said, I would like to put some title text on top, and put in labels for the X and Y axis, like a standard heat map graph. However, my understanding of the graphics component is too limited, and I would like to request the assistance of you guys.
I did some research, and it seems the drawText() method could do what I want: c# write text on bitmap
But I am unsure where to specify the coordinates while at the same time leaving some space on the top of the displayed graph to put the title text.

The coordinate system that GDI+ uses starts from the topleft corner which is (0,0)
The bottom right corner (fullimagewidth,fullimageheight)
so if you need to draw on the topleft corner of the image use
//Position
PointF drawPoint = new PointF(0F, 0F);
// Draw string to screen.
e.Graphics.DrawString("hey", drawFont, drawBrush, drawPoint);

Draw image on top of another image with blending mode color

In Photoshop you can select "Color" (the second from the bottom) to set the blending mode to the next lower layer:
If you have just a gradient on top of an image the result could look like this:
The description of the color blending mode I found somewhere is:
Color changes the hue and saturation of the lower layer to the hue and saturation of the upper layer but leaves luminosity alone.
My code so far is:
using(var g = Graphics.FromImage(canvas))
{
// draw the lower image
g.DrawImage(lowerImg, left, top);
// creating a gradient and draw on top
using (Brush brush = new LinearGradientBrush(new Rectangle(0, 0, canvasWidth, canvasHeight), Color.Violet, Color.Red, 20))
{
g.FillRectangle(brush, 0, 0, canvasWidth, canvasHeight);
}
}
But that is - of course - just painting over the lower image.
So the question is:
How can I draw an image on top of another image using the blending mode "color" as available in Photoshop?
EDIT:
To make it a bit more clear of what I want to achieve:
And if someone wants to use the images for testing:

Here is my solution. I've used Rich Newman's HSLColor class to convert between RGB and HSL values.
using (Bitmap lower = new Bitmap("lower.png"))
using (Bitmap upper = new Bitmap("upper.png"))
using (Bitmap output = new Bitmap(lower.Width, lower.Height))
{
int width = lower.Width;
int height = lower.Height;
var rect = new Rectangle(0, 0, width, height);
BitmapData lowerData = lower.LockBits(rect, ImageLockMode.ReadOnly, PixelFormat.Format24bppRgb);
BitmapData upperData = upper.LockBits(rect, ImageLockMode.ReadOnly, PixelFormat.Format24bppRgb);
BitmapData outputData = output.LockBits(rect, ImageLockMode.WriteOnly, PixelFormat.Format24bppRgb);
unsafe
{
byte* lowerPointer = (byte*) lowerData.Scan0;
byte* upperPointer = (byte*) upperData.Scan0;
byte* outputPointer = (byte*) outputData.Scan0;
for (int i = 0; i < height; i++)
{
for (int j = 0; j < width; j++)
{
HSLColor lowerColor = new HSLColor(lowerPointer[2], lowerPointer[1], lowerPointer[0]);
HSLColor upperColor = new HSLColor(upperPointer[2], upperPointer[1], upperPointer[0]);
upperColor.Luminosity = lowerColor.Luminosity;
Color outputColor = (Color) upperColor;
outputPointer[0] = outputColor.B;
outputPointer[1] = outputColor.G;
outputPointer[2] = outputColor.R;
// Moving the pointers by 3 bytes per pixel
lowerPointer += 3;
upperPointer += 3;
outputPointer += 3;
}
// Moving the pointers to the next pixel row
lowerPointer += lowerData.Stride - (width * 3);
upperPointer += upperData.Stride - (width * 3);
outputPointer += outputData.Stride - (width * 3);
}
}
lower.UnlockBits(lowerData);
upper.UnlockBits(upperData);
output.UnlockBits(outputData);
// Drawing the output image
}

You will have to restructure your code so that you draw your gradient on a temporary bitmap, read each pixel from the temporary bitmap and canvas, and write a composed pixel to canvas. You should be able to find code converting between RGB and HSL colors, and once you can do that, setting the hue and saturation of pixels in canvas to the values from your temporary bitmap is trivial (though it's a bit harder if you want to use alpha values).

Here's a safe (and slower) version of the accepted answer for completeness.
using (var lower = new Bitmap(#"lower.png"))
using (var upper = new Bitmap(#"upper.png"))
using (var output = new Bitmap(lower.Width, lower.Height))
{
var width = lower.Width;
var height = lower.Height;
for (var i = 0; i < height; i++)
{
for (var j = 0; j < width; j++)
{
var upperPixel = upper.GetPixel(j, i);
var lowerPixel = lower.GetPixel(j, i);
var lowerColor = new HSLColor(lowerPixel.R, lowerPixel.G, lowerPixel.B);
var upperColor = new HSLColor(upperPixel.R, upperPixel.G, upperPixel.B) {Luminosity = lowerColor.Luminosity};
var outputColor = (Color)upperColor;
output.SetPixel(j, i, outputColor);
}
}
// Drawing the output image
}