Develop Reference

C# Monogame Minicraft get pixels Colors from png

I am struggling with understanding, how ColorHelper.get works in this project. Here is C# version from github.
For better understanding u will need that project from github
About project: The project is remake of THIS (in JAVA) but someone remakes this into C#. Its
a game in Monogame called Minicraft.
Problem: Because its open source, i wanna just play with that project and learn more. The main problem is, the color palette is limited that I can use sprite with only 4 Colors. That means, when I want to have colorful sprite I cant. The way how coloring works (for my opinion) is trough the ColorHelper.get function where function contains 4 ints each represent specific color. (Example HERE)
The upper (Human) is from the game and has 4 colors which are converted by values.
The bottom sprite is my sea stuff with 8 colors. With ColorHelper.get i was able to convert only 4 colors and my sprite have to be with those 4 specific colors as that Human on the top (it cant be colorfull).
Code:
ColorHelper.cs
public static int get(int a, int b, int c, int d)
{
return (get(d) << 24) + (get(c) << 16) + (get(b) << 8) + (get(a));
}
public static int get(int d)
{
if (d < 0) return 255;
int r = d / 100 % 10;
int g = d / 10 % 10;
int b = d % 10;
return r * 36 + g * 6 + b;
}
Screen.cs
public void render(int xp, int yp, int tile, int colors, int bits)
{
xp -= xOffset;
yp -= yOffset;
var mirrorX = (bits & BIT_MIRROR_X) > 0;
var mirrorY = (bits & BIT_MIRROR_Y) > 0;
var xTile = tile % 32;
var yTile = tile / 32;
var toffs = xTile * 8 + yTile * 8 * sheet.width;
for (int y = 0; y < 8; y++)
{
int ys = y;
if (mirrorY) ys = 7 - y;
if (y + yp < 0 || y + yp >= h) continue;
for (int x = 0; x < 8; x++)
{
if (x + xp < 0 || x + xp >= w) continue;
int xs = x;
if (mirrorX) xs = 7 - x;
int col = (colors >> (sheet.pixels[xs + ys * sheet.width + toffs] * 8)) & 255;
if (col < 255)
pixels[(x + xp) + (y + yp) * w] = col;
}
}
}
public void render(int xp, int yp, int tile, int colors, int bits) = **"int colors" represent
ColorHelper.get(-1,50,250,455)** (Those numbers are fictional only for example.)
Exmaple of rendering something:
//Hight of pixels 8 pixels * 2 from sprite so 16 pixels in x axis in sprite is rendered
int h = 2;
//Width of pixels 16 pixels * 13 in y axis in sprite is rendered
int w = 13;
//4 Colors sprite which i dont want !
int titleColor = ColorHelper.get(0, 010, 131, 551);
//Padding Left
int xo = (screen.w - w * 8) / 2;
//Padding top
int yo = 24;
for (int y = 0; y < h; y++)
{
for (int x = 0; x < w; x++)
{
//public void render(int xp, int yp, int tile, int colors, int bits) in Screen.cs 8x8 pixels are rendered
screen.render(xo + x * 8, yo + y * 8, x + (y + 6) * 32, titleColor, 0);
}
}
What I want to achieve
When i debug a "int colors" from "public void render" i get rly weird numbers which is unreadable for me. (You can try it on your own. Just create a Console Application and paste THIS.
So am I here to ask you. Is anyone who can help me remake this project, that I can get all pixels from sprite with colors as they are in png file without converting from ColorHelper.get function ?. I just want to render png as it is.
If not, its ok,I will try to continue with struggling :D. I am actully trying to solve this about 3 days straight. I think its way to hard to remake that.
Maybe I want a lot, but my brain is not capable to understand this. Only this thing is holding me up from continuing. I hope you understand that.
Not important info :
If someone says that i am jumping into big projects and I should try easier stuff. I am working in Unity since 2017 and i want to have full controll of my projects and obtain some better knowladge of codding. Learnin trough the games is for my best option, coz i like games. :) For those who doesnt understand me, I am rly sorry but I am not from America, i am from Czech.
Thank you very much.

I don't know how to comment a long block of code so I'll just put it in here. I think the .get(4 params) is trying to do this
//rgba ranges from 0 - 255
public static UInt32 get(int b, int g, int r, int a)
{
UInt32 rgba = ((UInt32)a << 24) | ((UInt32)r << 16)| ((UInt32)g << 8) |
((UInt32)b);
return rgba; //this will return a 32 bit integer like 3869934080 for
//example
}
So I don't think it is taking in 4 colors at all, it's trying to take in 1 color through it's rgba input (I don't understand why it would use 555 as a range and the point of get(1 parameters) ~ maybe someone can explain it)
So if you want to display the colors you want, replace parameters r,g,b,a which in turns gives you a 32 bit unsigned int (this would probably be passed into the render function.
Resource about bit shifting here

Dubre den,
To bypass the processing step and directly load the file.
To load a .png file to a Texture2D use:
Texture2D image = Texture2D.FromFile(graphicsDevice, #"C:\path\file.png");
Obviously, image must be used in the containing scope.

How are GDI+ functions so fast?

I am trying to recreate very simple GDI+ functions, such as scaling and rotating an image. The reason is that some GDI functions can't be done on multiple threads (I found a work around using processes but didn't want to get into that), and processing thousands of images on one thread wasn't nearly cutting it.
Also my images are grayscale, so a custom function would only have to worry about one value instead of 4.
No matter what kind of function I try to recreate, even when highly optimized, it is always SEVERAL times slower, despite being greatly simplified compared to what GDI is doing (I am operating on a 1D array of bytes, one byte per pixel)
I thought maybe the way I was rotating each point could be the difference, so I took it out completely, and basically had a function that goes through each pixel and just sets it to what it already is, and that was only roughly tied with the speed of GDI, even though GDI was doing an actual rotation and changing 4 different values per pixel.
What makes this possible? Is there a way to match it using your own function?

The GDI+ code is written in C/C++, or possibly even partially in assembly. Some GDI+ calls may use GDI, an old and well optimized API. You will find it difficult to match the performance, even if you know all the pixel manipulation tricks.

I am adding my own answer along with my code to help anyone else who may be looking to do this.
From a combination of pointers and using an approximation of Sine and Cosine instead of calling an outside function for the rotation, I have come pretty darn close to reaching GDI speeds. No outside functions are called at all.
It still takes about 50% more time than GDI, but my earlier implementation took over 10 times longer than GDI. And when you consider multi threading, this method can be 10 times faster than GDI. This function can rotate a 300x400 picture in 3 milliseconds on my machine.
Keep in mind that this is for grayscale images and each byte in the input array represents one pixel.
If you have any ideas to make it faster please share!
private unsafe byte[] rotate(byte[] input, int inputWidth, int inputHeight, int cx, int cy, double angle)
{
byte[] result = new byte[input.Length];
int
tx, ty, ix, iy, x1, y1;
double
px, py, fx, fy, sin, cos, v;
byte a, b;
//Approximate Sine and Cosine of the angle
if (angle < 0)
sin = 1.27323954 * angle + 0.405284735 * angle * angle;
else
sin = 1.27323954 * angle - 0.405284735 * angle * angle;
angle += 1.57079632;
if (angle > 3.14159265)
angle -= 6.28318531;
if (angle < 0)
cos = 1.27323954 * angle + 0.405284735 * angle * angle;
else
cos = 1.27323954 * angle - 0.405284735 * angle * angle;
angle -= 1.57079632;
fixed (byte* pInput = input, pResult = result)
{
byte* pi = pInput;
byte* pr = pResult;
for (int x = 0; x < inputWidth; x++)
for (int y = 0; y < inputHeight; y++)
{
tx = x - cx;
ty = y - cy;
px = tx * cos - ty * sin + cx;
py = tx * sin + ty * cos + cy;
ix = (int)px;
iy = (int)py;
fx = px - ix;
fy = py - iy;
if (ix < inputWidth && iy < inputHeight && ix >= 0 && iy >= 0)
{
//keep in array bounds
x1 = ix + 1;
y1 = iy + 1;
if (x1 >= inputWidth)
x1 = ix;
if (y1 >= inputHeight)
y1 = iy;
//bilinear interpolation using pointers
a = *(pInput + (iy * inputWidth + ix));
b = *(pInput + (y1 * inputWidth + ix));
v = a + ((*(pInput + (iy * inputWidth + x1)) - a) * fx);
pr = (pResult + (y * inputWidth + x));
*pr = (byte)(v + (((b + ((*(pInput + (y1 * inputWidth + x1)) - b) * fx)) - v) * fy));
}
}
}
return result;
}

How to select by color range?

In my application I have loaded a picture and I want to be able to detect similar colors. So if I select a color I want the application to be able to find all pixels with that same (or almost the same) color. This is what I wrote for a detection system that looks in a vertical direction between the point of the mouse click and the end of the bitmap.
for (int y = mouseY; y < m_bitmap.Height; y++)
{
Color pixel = m_bitmap.GetPixel(mouseX, y);
//check if there is another color
if ((pixel.R > curcolor.R + treshold || pixel.R < curcolor.R - treshold) ||
(pixel.G > curcolor.G + treshold || pixel.G < curcolor.G - treshold) ||
(pixel.B > curcolor.B + treshold || pixel.B < curcolor.B - treshold))
{ //YESSSSS!
if ((y - ytop > minheight)&&(curcolor != Color.White)) //no white, at least 15px height
{
colorlayers.Add(new ColorLayer(curcolor, y - 1, ytop));
}
curcolor = pixel;
ytop = y;
}
}
Would this be the best way? Somehow it looks like it doesn't work too good with yellowish colors.

RGB is a 3D space.
A color far away threshold in all directions is not so similar to original one (and what is similar according to numbers may not be so similar to human beings eyes).
I would make a check using HSL (for example) where hue value as a finite 1D range, just for example:
for (int y = mouseY; y < m_bitmap.Height; y++)
{
Color pixel = m_bitmap.GetPixel(mouseX, y);
if (Math.Abs(color.GetHue() - curcolor.GetHue()) <= threshold)
{
// ...
}
}
Moreover please note that using bitmaps in this way (GetPixel() is terribly slow, take a look to this post to see a - much - faster alternative).

It might be interesting to look at how the magic wand tool in Paint.NET works.
This is how they compare 2 colors:
private static bool CheckColor(ColorBgra a, ColorBgra b, int tolerance)
{
int sum = 0;
int diff;
diff = a.R - b.R;
sum += (1 + diff * diff) * a.A / 256;
diff = a.G - b.G;
sum += (1 + diff * diff) * a.A / 256;
diff = a.B - b.B;
sum += (1 + diff * diff) * a.A / 256;
diff = a.A - b.A;
sum += diff * diff;
return (sum <= tolerance * tolerance * 4);
}
Source

The reason why yellow colors give a problem might be that RGB is not a perceptually uniform colorspace. This means that, given a distance between two points/colors in the colorspace, the perception of this color distance/difference will in general not be the same.
That said, you might want to use another color space, like HSL as suggested by Adriano, or perhaps Lab.
If you want to stick to RGB, I would suggest to calculate the euclidian distance, like this (I think it's simpler):
float distance = Math.sqrt((pixel.R-curcolor.R)^2 + (pixel.G-curcolor.G)^2 + (pixel.B-curcolor.B)^2);
if(distance < threshold)
{
// Do what you have to.
}

How calcutale GraphicsStream Position of specific pixel on screen?

How calculate position of specific pixel inside of graphics stream? This stream is screenshot of my primary monitor.
Lets say I want position for pixel at:
0, 0
0, 10
10, 0
10, 10
Im using DirectX 9 SDK.
Here is part of code what Im using(from this tutorial) to calculate position:
const int Bpp = 4;
int o = 20;
int m = 8;
int screenWidthMinusM = Screen.PrimaryScreen.Bounds.Width - m;
int screenHeightMinusM = Screen.PrimaryScreen.Bounds.Height - m;
int bx = (screenWidthMinusM - m) / 3 + m;
int by = (screenHeightMinusM - m) / 3 + m;
int bx2 = (screenWidthMinusM - m) * 2 / 3 + m;
int by2 = (screenHeightMinusM - m) * 2 / 3 + m;
long x, y;
long pos;
y = m;
for (x = m; x < screenWidthMinusM; x += o)
{
pos = (y * Screen.PrimaryScreen.Bounds.Width + x) * Bpp;
if (x < bx) tlPos.Add(pos);
else if (x > bx && x < bx2) tPos.Add(pos);
else if (x > bx2) trPos.Add(pos);
}
but this returns collection with numbers starting about 53 000 to about 7 000 000(from another similar method). After that color is extracted
Surface s = sc.CaptureScreen();
GraphicsStream gs = s.LockRectangle(LockFlags.None);
byte[] bu = new byte[4];
foreach (long pos in positions)
{
gs.Position = pos;
gs.Read(bu, 0, 4);
r += bu[2];
g += bu[1];
b += bu[0];
i++;
}
I need to make my own collection containing these positions.

I assume you're having all pixels of your screenshot in this graphics stream (doc) and want to get the index within the stream of a specified pixelposition (x,y).
Therefore it should be easily: Index = (x + y * screenwidth) * SizeOf(Pixel)
For reading the data you should combine Seek with Read.

Ok,I found answer, its in my code, I dont exactly understand why,but:
//x - screen width
//y - screen height
Bpp = 4; // ?
gs.Position = (x * Screen.PrimaryScreen.Bounds.Width + y) * Bpp;

Autofocus routine detecting very small differences in blur

I'm developing an autofocus routine for positioning on the micrometer scale, so I need to find very small differences in focus/blur between images. Luckily, the image pattern will always be the same (these are 256x256 center crops of the original 2 MP images):
Perfect focus | 50 µm off
Finding the better focused image of the two above is not a problem, I guess most algorithms will do. But I really need to compare images with a lot less difference in focus, like the ones below:
5 µm off | 10 µm off
An alternative to stepping closer and closer to optimal focus is to find two images that have the same amount of blur on opposite sides of the focus plane. For example, one could save an image from -50 µm and then try to find an image around +50 µm where the blur is equal. Lets say that the image was found at +58 µm, then the focus plane should be positioned at +4 µm.
Any ideas for a suitable algorithm?

Surprisingly, many quite simple autofocus algorithms actually performed quite well on this problem. I implemented 11 of the 16 algorithms outlined in the paper Dynamic evaluation of autofocusing for automated microscopic analysis of blood smear and pap smear by Liu, Wang & Sun. Since I had trouble finding recommendations for setting the threshold values, I also added some variants without thresholds. I also added a simple but clever suggestion found here on SO: compare the file size of the JPEG compressed images (larger size = more detail = better focus).
My autofocus routine does the following:
Save 21 images at an interval of 2 µm focus distance, total range ±20 µm.
Calculate the focus value of each image.
Fit the result to a second degree polynomial.
Find the position that gives a maximum value of the polynomial.
All algorithms except Histogram Range gave good results. Some of the algorithms are slightly modified, for example they use the brightness difference in both X & Y directions. I also had to change the sign of the StdevBasedCorrelation, Entropy, ThresholdedContent, ImagePower and ThresholdedImagePower algorithms to get a maximum instead of a minimum at the focus position. The algorithms expect a 24 bit grayscale image where R = G = B. If used on a color image, only the blue channel will be calculated (easily corrected of course).
The optimal threshold values was found by running the algorithms with threshold values 0, 8, 16, 24 etc up to 255 and selecting the best value for:
Stable focus position
Large negative x² coefficient resulting in a narrow peak at the focus position
Low residual sum of squares from the polynomial fit
It's interesting to note that the ThresholdedSquaredGradient and ThresholdedBrennerGradient algorithms have an almost flat line of focus position, x² coefficient and residual sum of squares. They are very insensitive to changes in the threshold value.
Implementation of algorithms:
public unsafe List<Result> CalculateFocusValues(string filename)
{
using(Bitmap bmp = new Bitmap(filename))
{
int width = bmp.Width;
int height = bmp.Height;
int bpp = Bitmap.GetPixelFormatSize(bmp.PixelFormat) / 8;
BitmapData data = bmp.LockBits(new Rectangle(0, 0, width, height), ImageLockMode.ReadOnly, bmp.PixelFormat);
long sum = 0, squaredSum = 0;
int[] histogram = new int[256];
const int absoluteGradientThreshold = 148;
long absoluteGradientSum = 0;
long thresholdedAbsoluteGradientSum = 0;
const int squaredGradientThreshold = 64;
long squaredGradientSum = 0;
long thresholdedSquaredGradientSum = 0;
const int brennerGradientThreshold = 184;
long brennerGradientSum = 0;
long thresholdedBrennerGradientSum = 0;
long autocorrelationSum1 = 0;
long autocorrelationSum2 = 0;
const int contentThreshold = 35;
long thresholdedContentSum = 0;
const int pixelCountThreshold = 76;
long thresholdedPixelCountSum = 0;
const int imagePowerThreshold = 40;
long imagePowerSum = 0;
long thresholdedImagePowerSum = 0;
for(int row = 0; row < height - 1; row++)
{
for(int col = 0; col < width - 1; col++)
{
int current = *((byte *) (data.Scan0 + (row + 0) * data.Stride + (col + 0) * bpp));
int col1 = *((byte *) (data.Scan0 + (row + 0) * data.Stride + (col + 1) * bpp));
int row1 = *((byte *) (data.Scan0 + (row + 1) * data.Stride + (col + 0) * bpp));
int squared = current * current;
sum += current;
squaredSum += squared;
histogram[current]++;
int colDiff1 = col1 - current;
int rowDiff1 = row1 - current;
int absoluteGradient = Math.Abs(colDiff1) + Math.Abs(rowDiff1);
absoluteGradientSum += absoluteGradient;
if(absoluteGradient >= absoluteGradientThreshold)
thresholdedAbsoluteGradientSum += absoluteGradient;
int squaredGradient = colDiff1 * colDiff1 + rowDiff1 * rowDiff1;
squaredGradientSum += squaredGradient;
if(squaredGradient >= squaredGradientThreshold)
thresholdedSquaredGradientSum += squaredGradient;
if(row < bmp.Height - 2 && col < bmp.Width - 2)
{
int col2 = *((byte *) (data.Scan0 + (row + 0) * data.Stride + (col + 2) * bpp));
int row2 = *((byte *) (data.Scan0 + (row + 2) * data.Stride + (col + 0) * bpp));
int colDiff2 = col2 - current;
int rowDiff2 = row2 - current;
int brennerGradient = colDiff2 * colDiff2 + rowDiff2 * rowDiff2;
brennerGradientSum += brennerGradient;
if(brennerGradient >= brennerGradientThreshold)
thresholdedBrennerGradientSum += brennerGradient;
autocorrelationSum1 += current * col1 + current * row1;
autocorrelationSum2 += current * col2 + current * row2;
}
if(current >= contentThreshold)
thresholdedContentSum += current;
if(current <= pixelCountThreshold)
thresholdedPixelCountSum++;
imagePowerSum += squared;
if(current >= imagePowerThreshold)
thresholdedImagePowerSum += current * current;
}
}
bmp.UnlockBits(data);
int pixels = width * height;
double mean = (double) sum / pixels;
double meanDeviationSquared = (double) squaredSum / pixels;
int rangeMin = 0;
while(histogram[rangeMin] == 0)
rangeMin++;
int rangeMax = histogram.Length - 1;
while(histogram[rangeMax] == 0)
rangeMax--;
double entropy = 0.0;
double log2 = Math.Log(2);
for(int i = rangeMin; i <= rangeMax; i++)
{
if(histogram[i] > 0)
{
double p = (double) histogram[i] / pixels;
entropy -= p * Math.Log(p) / log2;
}
}
return new List<Result>()
{
new Result("AbsoluteGradient", absoluteGradientSum),
new Result("ThresholdedAbsoluteGradient", thresholdedAbsoluteGradientSum),
new Result("SquaredGradient", squaredGradientSum),
new Result("ThresholdedSquaredGradient", thresholdedSquaredGradientSum),
new Result("BrennerGradient", brennerGradientSum),
new Result("ThresholdedBrennerGradient", thresholdedBrennerGradientSum),
new Result("Variance", meanDeviationSquared - mean * mean),
new Result("Autocorrelation", autocorrelationSum1 - autocorrelationSum2),
new Result("StdevBasedCorrelation", -(autocorrelationSum1 - pixels * mean * mean)),
new Result("Range", rangeMax - rangeMin),
new Result("Entropy", -entropy),
new Result("ThresholdedContent", -thresholdedContentSum),
new Result("ThresholdedPixelCount", thresholdedPixelCountSum),
new Result("ImagePower", -imagePowerSum),
new Result("ThresholdedImagePower", -thresholdedImagePowerSum),
new Result("JpegSize", new FileInfo(filename).Length),
};
}
}
public class Result
{
public string Algorithm { get; private set; }
public double Value { get; private set; }
public Result(string algorithm, double value)
{
Algorithm = algorithm;
Value = value;
}
}
To be able to plot and compare the focus values of the different algorithms they were scaled to a value between 0 and 1 (scaled = (value - min)/(max - min)).
Plot of all algorithms for a range of ±20 µm:
0 µm | 20 µm
Things look quite similar for a range of ±50 µm:
0 µm | 50 µm
When using a range of ±500 µm things get more interesting. Four algorithms exhibit more of a fourth degree polynomial shape, and the others start to look more like Gaussian functions. Also, the Histogram Range algorithm start to perform better than for smaller ranges.
0 µm | 500 µm
Overall I'm quite impressed by the performance and consistency of these simple algorithms. With the naked eye, it's quite hard to tell that even the 50 µm image is out of focus but the algorithms have no problem comparing images just a few microns apart.

An extra answer to NindzAl's comment on the original answer:
I use the Extreme Optimization library to fit the sharpness values to a second degree polynomial. The distance of maximum sharpness is then extracted using the first derivative of the polynomial.
The Extreme Optimization library costs USD 999 for a single dev license, but I'm sure there are open source math libraries that can perform the fitting just as well.
// Distances (in µm) where the images were saved
double[] distance = new double[]
{
-50,
-40,
-30,
-20,
-10,
0,
+10,
+20,
+30,
+40,
+50,
};
// Sharpness value of each image, as returned by CalculateFocusValues()
double[] sharpness = new double[]
{
3960.9,
4065.5,
4173.0,
4256.1,
4317.6,
4345.4,
4339.9,
4301.4,
4230.0,
4131.1,
4035.0,
};
// Fit data to y = ax² + bx + c (second degree polynomial) using the Extreme Optimization library
const int SecondDegreePolynomial = 2;
Extreme.Mathematics.Curves.LinearCurveFitter fitter = new Extreme.Mathematics.Curves.LinearCurveFitter();
fitter.Curve = new Extreme.Mathematics.Curves.Polynomial(SecondDegreePolynomial);
fitter.XValues = new Extreme.Mathematics.LinearAlgebra.GeneralVector(distance, true);
fitter.YValues = new Extreme.Mathematics.LinearAlgebra.GeneralVector(sharpness, true);
fitter.Fit();
// Find distance of maximum sharpness using the first derivative of the polynomial
// Using the sample data above, the focus point is located at distance +2.979 µm
double focusPoint = fitter.Curve.GetDerivative().FindRoots().First();

As for the free library, Math.Net will work for that purpose