I am trying to find coordinates of one image inside of another using AForge framework:
ExhaustiveTemplateMatching tm = new ExhaustiveTemplateMatching();
TemplateMatch[] matchings = tm.ProcessImage(new Bitmap("image.png"), new Bitmap(#"template.png"));
int x_coordinate = matchings[0].Rectangle.X;
ProcessImages takes about 2 minutes to perform.
Image's size is about 1600x1000 pixels
Template's size is about 60x60 pixels
Does anyone know how to speed up that process?
As addition to the other answers, I would say that for your case:
Image's size is about 1600x1000 pixels Template's size is about 60x60 pixels
This framework is not the best fit. The thing you are trying to achieve is more search-image-in-other-image, than compare two images with different resolution (like "Search Google for this image" can be used).
About this so
called pyramid search.
it's true that the algorithm works way faster for bigger images. Actually the image-pyramid is based on template matching. If we take the most popular implementation (I found and used):
private static bool IsSearchedImageFound(this Bitmap template, Bitmap image)
{
const Int32 divisor = 4;
const Int32 epsilon = 10;
ExhaustiveTemplateMatching etm = new ExhaustiveTemplateMatching(0.90f);
TemplateMatch[] tm = etm.ProcessImage(
new ResizeNearestNeighbor(template.Width / divisor, template.Height / divisor).Apply(template),
new ResizeNearestNeighbor(image.Width / divisor, image.Height / divisor).Apply(image)
);
if (tm.Length == 1)
{
Rectangle tempRect = tm[0].Rectangle;
if (Math.Abs(image.Width / divisor - tempRect.Width) < epsilon
&&
Math.Abs(image.Height / divisor - tempRect.Height) < epsilon)
{
return true;
}
}
return false;
}
It should give you a picture close to this one:
As bottom line - try to use different approach. Maybe closer to Sikuli integration with .Net. Or you can try the accord .Net newer version of AForge.
If this is too much work, you can try to just extend your screenshot functionality with cropping of the page element that is required (Selenium example).
2 minutes seems too much for a recent CPU with the image a template sizes you are using. But there are a couple of ways to speed up the process. The first one is by using a smaller scale. This is called pyramid search. You can try to divide the image and template by 4 so that you will have an image of 400x250 and a template of 15x15 and match this smaller template. This will run way faster but it will be also less accurate. You can then use the interesting pixels found with the 15x15 template and search the corresponding pixels in the 1600x1000 image using the 60x60 template instead of searching in the whole image.
Depending on the template details you may try at an even lower scale (1/8) instead.
Another thing to know is that a bigger template will run faster. This is counter-intuitive but with a bigger template you will have less pixel to compare. So if possible try to use a bigger template. Sometimes this optimization is not possible if your template is already as big as it can be.
Related
I am trying to compare two bitmaps to one another. One is premade, and the other one consists of a small image of the main screen being taken and filtered for everything besides full white. I now need a way to compare the amount of white pixels in the live Bitmap to the amount in the premade Bitmap (101 white pixels). A way I know of would be using the Bitmap.Get/SetPixel commands, but they are really slow and as this is used for a kind of time critical application, unfitting.
Especially since I could cut down the filtering process by a factor of 70 by following this guide.
https://www.codeproject.com/articles/617613/fast-pixel-operations-in-net-with-and-without-unsa
I also can't just compare the 2 Bitmaps, as the live one will usually not have the pixels in the same position, but will share the same amount of white pixels.
So yeah. It'd be great if one of you had a time effective solution to this problem.
Edit
Huge oversight on my part. When looking at the filtering method, it becomes apparent that one can just use a counter+=1 every time a pixel is not filtered out.
So I just changed this line of code in the filter function
row[rIndex] = row[bIndex] = row[gIndex] = distance > toleranceSquared ? unmatchingValue : matchingValue;
to this
if(distance > toleranceSquared)
{
row[rIndex] = row[bIndex] = row[gIndex] = unmatchingValue;
}
else
{
row[rIndex] = row[bIndex] = row[gIndex] = matchingValue;
WhitePixelCount += 1;
}
I have an application that generate graphs from some input files. The user has the option to generate an image from these graphs. The graph is created using an external library that draws into a custom visual object (inherits from Canvas).
The Functionality
I created a custom functionality to create the image out of this canvas and it has worked well so far. The problem I'm having is related to its performance when the need to generate an image out of VERY large graph.
The images generated by this functionality has always been big, to be able to generate these large images I generate smaller "chunks" (configurable, but currently at 2000x2000px, larger chunks can lose quality and smaller is slower). After these "chunks" are created I merge them using another external library (this last works fine and it is really fast, the issue does not lie here).
The Issue
The problem is happening that the larger the original visual is, each chunk rendering takes longer.
The problem is that the larger the image is, each "chunk" rendering takes longer. I'll post the code here later, but I've noticed that the "RenderTargetBitmap.Render()" method when using my "DrawingVisual" (with the chunk information). As an example, this method call takes 400ms+ on my local tests where the Canvas is 74052.75px x 1119.4px; the total image creation time takes about 20 secs.
This would not be that bad, but there was a requirement where a large (MUCH larger) image has to be created and each chunk is taking 5 minutes to generate, so my estimation is that it would take about 60 days.
The Code
This is the code I'm using at each "chunk" loop. The code is modified to remove some cancellation token and progress report calls. I also added a comment to highlight the call that takes 400+ms.
for (int i = 0; i < rowCount; i++)
{
for (int j = 0; j < colCount; j++)
{
int row = i, col = j; //this is done because task start could have a modified value on call
Rect rect = new Rect(
col * maxWidth,
row * maxHeight,
col + 1 < colCount ? maxWidth :
surfaceWidth - (colCount - 1) * maxWidth,
row + 1 < rowCount ? maxHeight :
surfaceHeight - (rowCount - 1) * maxHeight);
DrawingVisual visual = new DrawingVisual();
using (var dc = visual.RenderOpen())
{
VisualBrush brush = new VisualBrush(element)
{
Viewbox = rect,
ViewboxUnits = BrushMappingMode.Absolute,
Stretch = Stretch.None
};
dc.DrawRectangle(brush,
null,
new Rect(0, 0, rect.Width, rect.Height));
}
RenderTargetBitmap bitmap = new RenderTargetBitmap(
(int)(rect.Width * (imgdpi / DEFAULT_DPI)),
(int)(rect.Height * (imgdpi / DEFAULT_DPI)),
imgdpi,
imgdpi,
PixelFormat);
//***********************//
//THIS IS THE CALL THAT TAKES 400+ms
//***********************//
bitmap.Render(visual);
PngBitmapEncoder encoder = new PngBitmapEncoder();
encoder.Frames.Add(BitmapFrame.Create(bitmap));
MemoryStream ms = new MemoryStream();
encoder.Save(ms);
//... Some additional code to work with the chunk asynchronously
}
}
Things I've tried
I've tried several things:
Calling Dispatcher.DisableProcessing: I was thinking that it could have been that the UI kept trying to update the surface while processing the chunks (althought my understanding that if it has not been modified it shouldn't be much of a problem). But this did not change the processing time.
While creating the VisualBrush object to assign to it smaller Viewports (chunk size or bigger). The time actually increased.
Adding CacheMode to the Canvas: The time actually increased and the image looked low resolution.
Some additional desperate measures that made no sense haha.
Additional Information
Since I'm not completely sure how this Render functionality works internally (And I have searched for information on these, but what I've found is superficial at best) I'm not sure if the cause is the large surface or the number of controls in the surface (or a combination of both). So here is some additional information on the test case that is taking 400ms per render:
Surface Size is: 74052.75px x 1119.4px.
N. of Graph's Nodes: 1977.
N. of Connectors (Arrows): 2052.
Each Control visual is defined in the XAML using resources.
Each Graph Node is a Filled Rectangle with a Textblock at its center.
Each Connector is a straight line with a triangle at one of its edges.
Also, since the very large image generation is not something that will happen that often, and not to a regular user. I would not mind if the solution provided sacrifices UI responsiveness.
Notes added to clarify some concerns in the comments section
The graph is shown on screen to the user, this can be manipulated by it.
The "Store as Image" functionality is just an additional optional functionality to save the graph as an Image file. (This works under normal load, the speed issue happens with an unnusually large graph).
I'd appreciate any help you could provide.
Thanks!
I'm trying to find a digit within an image. To test my code I took an image of the digit and then used AForge's Exhaustive Template Matching algorithm to search for it in another image. But I think there is a problem in that the digit is obviously not rectangular whereas the image that contains it is. That means that there are a lot of pixels participating in the comparison which shouldn't be. Is there any way to make this comparison while ignoring those pixels? If not in AForge then maybe EMGU/OpenCV or Octave?
Here's my code:
Grayscale gray = new GrayscaleRMY();
Bitmap template = (Bitmap)Bitmap.FromFile(#"5template.png");
template = gray.Apply(template);
Bitmap image = (Bitmap)Bitmap.FromFile(filePath);
Bitmap sourceImage = gray.Apply(image);
ExhaustiveTemplateMatching tm = new ExhaustiveTemplateMatching(0.7f);
TemplateMatch[] matchings = tm.ProcessImage(sourceImage, template);
As mentioned above in the comment, you should preprocess your data to improve matching.
The first thing that comes to mind is morphological opening (erode then dilate) to reduce the background noise
Read in your image and invert it so that your character vectors are white:
Apply opening with smallest possible structuring element/window (3x3):
You could try slightly larger structuring elements (5x5):
Invert it back to the original:
See if that helps!
I've looked everywhere but there doesn't seem to be a standard (I could see) of how one would go about checking to see if an image is blank. In C#
I have a way of doing this, but would love to know what the correct way is of checking to see if an image is blank, so everyone could also know in the future.
I'm not going to copy paste a bunch of code in, if you want me to, it will be my pleasure, but I just first want to explain how i go about checking to see if an image is blank.
You take a .jpg image, Get the width of it. For example 500 pixels
Then you divide that by 2
giving you 250
Then you check what the colour of every pixel is in the location of (250 width, and i height) (where you iterate thought the hight of the image.
What this then do is only check the middle line of pixels of an image, vertically. It goes though all the pixels checking to see if the colour is anything Except white. I've done this so you wont have to search ALL 500*height of pixels and since you will almost always come across a colour in the middle of the page.
Its working... a bit slow...There must be a better way to do this? You can change it to search 2/3/4 lines vertically to increase your chance to spot a page that's not blank, but that will take even longer.
(Also note, using the size of the image to check if it contains something will not work in this case, since a page with two sentences on and a blank page's size is too close to one another)
After solution has been added.
Resources to help with the implementation and understanding of the solution.
Writing unsafe code - pointers in C
Using Pointers in C#
/unsafe (C# Compiler Options)
Bitmap.LockBits Method (Rectangle, ImageLockMode, PixelFormat)
(Note that on the first website, the stated Pizelformat is actually Pixelformat) - Small error i know, just mentioning, might cause some confusion to some.
After I implemented the method to speed up the pixel hunting, the speed didn't increase that much. So I would think I'm doing something wrong.
Old time = 15.63 for 40 images.
New time = 15.43 for 40 images
I saw with the great article DocMax quoted, that the code "locks" in a set of pixels. (or thats how i understood it)
So what I did is lock in the middle row of pixels of each page. Would that be the right move to do?
private int testPixels(String sourceDir)
{
//iterate through images
string[] fileEntries = Directory.GetFiles(sourceDir).Where(x => x.Contains("JPG")).ToArray();
var q = from string x in Directory.GetFiles(sourceDir)
where x.ToLower().EndsWith(".jpg")
select new FileInfo(x);
int holder = 1;
foreach (var z in q)
{
Bitmap mybm= Bitmap.FromFile(z.FullName) as Bitmap;
int blank = getPixelData2(mybm);
if (blank == 0)
{
holder = 0;
break;
}
}
return holder;
}
And then the class
private unsafe int getPixelData2(Bitmap bm)
{
BitmapData bmd = bm.LockBits(new System.Drawing.Rectangle((bm.Width / 2), 0, 1, bm.Height), System.Drawing.Imaging.ImageLockMode.ReadOnly, bm.PixelFormat);
int blue;
int green;
int red;
int width = bmd.Width / 2;
for (int y = 0; y < bmd.Height; y++)
{
byte* row = (byte*)bmd.Scan0 + (y * bmd.Stride);
blue = row[width * 3];
green = row[width * 2];
red = row[width * 1];
// Console.WriteLine("Blue= " + blue + " Green= " + green + " Red= " + red);
//Check to see if there is some form of color
if ((blue != 255) || (green != 255) || (red != 255))
{
bm.Dispose();
return 1;
}
}
bm.Dispose();
return 0;
}
If you can tolerate the chance of getting it wrong, the approach seems fine; I have done something very similar in my case, although I always had a visual confirmation to deal with errors.
For the performance, the key open question is how you are getting the pixels to test. If you are using Bitmap.GetPixel, you are bound to have performance problems. (Search for "Bitmap.GetPixel slow" in Google to see lots of discussion.)
Far better performance will come from getting all the pixels at once and then looping over them. I personally like Bob Powell's LockBits discussion for clarity and completeness. With that approach, checking all of the pixels may well be reasonable depending on your performance needs.
If you're using System.Drawing.Bitmap you can speed up things up (substantially), by:
Not using GetPixel to access the pixels, use LockBits and UnlockBits to copy the image bitmap to regular memory. See the examples on the MSDN documentation for usage.
Not calling the Width, Height or Size properties in for loop. Call Size once, store the values in a local variable and use those in the loop.
Notes:
When using System.Drawing.Bitmap your image may be in device memory and accessing it may be time consuming.
I don't remember whether loading an image into a Bitmap already converts it to RGB format as other formats are more difficult to work with, but if that is not the case you can create an RGB Bitmap of the same size as your original image, get it's Graphic object (Graphics.FromImage) and use DrawImage to draw the original image in the RGB bitmap.
Edit: Beat to the punch by DocMax.
In any case for speed you can also try using alternative libraries such as the excellent FreeImage which includes C# wrappers.
Scale the image to 1x1 then check one pixel
new Bitmap(previousImage, new Size(1, 1));
I need to take a full color JPG Image and remap it's colors to a Indexed palette. The palette will consist of specific colors populated from a database. I need to map each color of the image to it's "closest" value in the index. I am sure there are different algorithms for comparing and calculating the "closest" value. Looking for C#, .NET managed code libraries only.
(It will be used in a process where we have 120 or so specific colors of buttons, and we want to map any image to those 120 colors to make a collage).
Nothing will help you with GDI. It seems indexed images are too backward a technology for Microsoft to care. All you can do is read and write indexed image files.
There are usually two step when quantizing colors in an image:
1) Find the best palette for the image (Color Quantization)
2) Map the source solors to the found palette (Color Mapping)
From what I understand, you already have the palette in the database, that means the hardest part has been done for you. All you need to do is map the 24 bit colors to the provided palette colors. If you don't have the starting palette, then you will have to compute it yourself using a quantisation algorithm: Octrees or Median Cut are the most well known. Median Cut gives better results but is slower and harder to implement and fine tune.
To map the colors, the simplest algorithm in your case is to calculate the distance from your source color to all the palette colors and pick the nearest.
float ColorDistanceSquared(Color c1, Color c2)
{
float deltaR = c2.R - c1.R;
float deltaG = c2.G - c1.G;
float deltaB = c2.B - c1.B;
return deltaR*deltaR + deltaG*deltaG + deltaB*deltaB;
}
You can also ponderate the channels so that blue has less weight, don't go too overboard with it, else it will give horrible results, specifically 30/59/11 won't work at all:
float ColorDistanceSquared(Color c1, Color c2)
{
float deltaR = (c2.R - c1.R) * 3;
float deltaG = (c2.G - c1.G) * 3;
float deltaB = (c2.B - c1.B) * 2;
return deltaR*deltaR + deltaG*deltaG + deltaB*deltaB;
}
Call that thing for all source and palette colors and find the Min. If you cache your results as you go in a map, this will be very fast.
Also, the source color will rarely fit a palette color enough to not create banding and plain areas and loss of details in your image. To avoid that, you can use dithering. The simplest algorithm and the one that gives the best results is Error Diffusion Dithering.
Once you mapped your colors, you will have to manually lock a Bitmap and write the indices in there as .Net won't let you write to an indexed image.
This process is called Quantization. Since each color represents 3 packed values, you'll need to use Octrees to solve this problem.
Check out this article with example code.
The article focuses on getting the ultimate palette for the image, but your process it would be reverse for the second part, only reduce the most used colors that are close to the given palette.
I had to do this in a big .NET project. There's nothing in the framework for it, but this article quickly led me to a solution: http://codebetter.com/blogs/brendan.tompkins/archive/2004/01/26/6103.aspx
The JPEG word should ring alarm bells. The images are very likely to already be in a heavily quantised colour space, and further resampling will potentially introduce aliasing. If you can, work from uncompressed images to reduce this effect.
The answer to your question is yes - you can save the images in an alternate format - but I'm not sure if the native functionality is adequate for what sounds like a quite complex requirement. If you are able to define the colour palette from the collection of images, you will likely improve the quality of the output.
The already referenced blog entry entitled Use 'GDI+ to Save Crystal-Clear GIF Images with .NET' contains useful references to code.