Align two images from a set of four points each - c#

I'm playing around in accord.net in C# and am trying to align two images. I'm new to computer vision and wondered if accord.net can do this or if I will have to write something myself.
I have two images and for each image I have four points per image. The points are assigned anti clockwise around the image starting with top left so (TL, TR, BR, BL)
Example:
Image 1
Point 1(221, 156)
Point 2(4740, 156)
Point 3(4740, 3347)
Point 4(221, 3347)
Image 2
Point 1(157, 213)
Point 2(4572, 32)
Point 3(4697, 3221)
Point 4(282, 3402)
In the two images image 1's point 1 correlates to image 2's point 1 same with the remaining points.
So what I would like to do is align the two images so the scale, rotation and alignment matches between the two images and I end up with two images when overlayed should look like this image:
So far I have this, which works for rotation and alignment but doesn't scale the image. from my understanding RANSAC seems to be over kill for this job as I'm already correlating the points? Plus I'd like the images output separately for further image processing.
// Images
var img1Path = Path.Combine(filePath, "image1.jpg");
var image1 = new[] { new PointF(221, 156), new PointF(4740, 156), new PointF(4740, 3347), new PointF(221, 3347) };
var img2Path = Path.Combine(filePath, "image2.jpg");
var image2 = new[] { new PointF(157, 213), new PointF(4572, 32), new PointF(4697, 3221), new PointF(282, 3402) };
// Create Bitmaps
var img1 = new Bitmap(img1Path);
var img2 = new Bitmap(img2Path);
var ransac = new RansacHomographyEstimator(0.001, 0.99);
var homographyMatrix = ransac.Estimate(image1, image2);
var blend = new Blend(homographyMatrix, img1) { Gradient = false };
var result = blend.Apply(img2);
result.Save("out.jpg", ImageFormat.Jpeg);
Thanks!

The answer turns out to be a couple of the points were wrong causing the non-scaling of the image content, a stupid but thankfully simple mistake.
and secondly to have accord.net save the output as separate images you use this code:
var filter = new Rectification(homographyMatrix);
var result = filter.Apply(image);
result.Save("out.jpg", ImageFormat.Jpeg);
Thanks to ezfn I also got this working in emguCV but the forum post isn't complete, I worked out this missing code and here it is completed:
var srcImagePath = Path.Combine(FilePath, "image2.jpg");
var dstImagePath = Path.Combine(FilePath, "image1.jpg");
var srcImage = new Image<Bgr, int>(srcImagePath);
var dstImage = new Image<Bgr, int>(dstImagePath);
float[,] srcPoints = { { 221, 156 }, { 4740, 156 }, { 4740, 3347 }, { 221, 3347 } };
float[,] dstPoints = { { 371, 356 }, { 4478, 191 }, { 4595, 3092 }, { 487, 3257 } };
var srcMat = new Matrix<float>(srcPoints);
var dstMat = new Matrix<float>(dstPoints);
var homogMat = new Matrix<float>(3, 3);
var invertHomogMat = new Matrix<float>(3, 3);
var maskMat = new IntPtr();
var s = new MCvScalar(0, 0, 0);
// .Ptr?
CvInvoke.cvFindHomography(srcMat, dstMat, homogMat, HOMOGRAPHY_METHOD.DEFAULT, 3.0, maskMat);
CvInvoke.cvInvert(homogMat, invertHomogMat, SOLVE_METHOD.CV_LU);
CvInvoke.cvWarpPerspective(srcImage, dstImage, invertHomogMat, (int)INTER.CV_INTER_NN, s);
dstImage.Save("2.jpg");
Thanks to everyone for the help!

Well,
Assuming you have the 4 points accurately corresponding - the solution is very simple. You have to calculate the homographic perspective transformation (In short - homography) between the images using the 4 correspondences. You can then use the calculated homography in order warp the second image so that is sits exactly on the first image.
You can do it easily using EmguCV. You have a great example of how to calculate the homography and apply it on an image:
http://www.emgu.com/forum/viewtopic.php?f=7&t=4122
Where you can treat image 2 as sourceImage, and image 1 as dest image.

Related

C# Emgu CV Trouble Predicting Face EigenFaceRecognizer

I'm trying to save face images and predict them using Emgu CV 4.4 with EigenFaceRecognizer using .Net5.
Basically I am getting a single image, resizing it (I read somewhere that smaller images have better accuracy),adding it to the List<Mat>and saving the trained data to see what is there out of curiosity.After that I'm using the same image and attempt to predict what it is (label).
Unfortunately it fails producing this error.
Emgu.CV.Util.CvException: 'OpenCV: Wrong shapes for given matrices. Was size(src) = (1,57600), size(W) = (19200,1).'
I also tried imageForTraining.Mat within Predict however the same error occurred.
// Get The GrayImage And Resize It.
Image<Bgr, byte> imageForTraining = grayImage.Resize(128, 150, Emgu.CV.CvEnum.Inter.Cubic);
// Create New cv::Mat Lists.
List<Mat> mats = new List<Mat>()
{
imageForTraining.Mat
};
// Dirty Solution For Label Testing.
List<int> labels = new List<int>()
{
0
};
// Create Face Recognizer And Tell It There Is Only 1 Item.
FaceRecognizer faceRecognizer = new EigenFaceRecognizer(1);
// Write Data To File To See If Something Is There.
faceRecognizer.Write("trainingData");
// Attempt To Train.
faceRecognizer.Train(new VectorOfMat(mats.ToArray()), new VectorOfInt(labels.ToArray()));
// Test Image That Was Just Trained.
FaceRecognizer.PredictionResult predictionResult = faceRecognizer.Predict(imageForTraining);

strategy of working with multiple image types and merging into 1 tiff

I don't have the slightest idea about graphics. I need to build a function that will take a list of different types of files (pngs, bmps, jpeg, pdf) and it will create 1 tiff file, merging all of the files together.
Can you guide me on the strategy in solving this? ( ...should i first convert every file to a tiff and then merge them together? ... should i simply use library XYZ to do this for me because this functionality already exists! should i first compress them somehow together and that will yield a tiff? )
It's doesn't matter what image format you want to use.You are working with bitmaps at the end. load all images and draw(merge) theme on single graphic object and at the end save the file as your own format :
static void Main(string[] args)
{
var imageList = Directory.GetFiles(#"C:\Users\ABC\Desktop\cars");
Bitmap destnationBitmap = new Bitmap(1000, 300);
Graphics g = Graphics.FromImage(destnationBitmap);
try
{
var drawPoint = new Point(0, 0);
foreach (string imagePath in imageList)
{
var tempBitmap = new Bitmap(imagePath);
g.DrawImage(tempBitmap, drawPoint);
drawPoint.X += tempBitmap.Width;
}
}
finally
{
var tiffVersion = ConverTo(destnationBitmap, ImageFormat.Tiff);
tiffVersion.Save("TiffVersion.tiff");
g.Dispose();
}
}
public static Image ConverTo(Bitmap bitmapImage, ImageFormat pFormat)
{
MemoryStream stream = new MemoryStream();
bitmapImage.Save(stream, pFormat);
return new Bitmap(stream);
}

C# Cropping image return wrong coordinates

I've been trying to crop a specific image using Selenium and different cropping methods for a few days.
An important note before my code - the following method used to work 2 weeks ago and for some reason it now returns an image with wrong coordinates
// Go to site
Driver.Navigate().GoToUrl("http://google.com");
Screenshot screenshot = driver.GetScreenshot();
using (var ms = new MemoryStream(screenshot.AsByteArray))
using (var imgShot = Image.FromStream(ms))
using (var src = new Bitmap(imgShot))
{
IWebElement element = driver.FindElement(By.XPath("//canvas"));
Rectangle cropRect = new Rectangle(element.Location.X, element.Location.Y, element.Size.Width, element.Size.Height);
var clone = src.Clone(cropRect, src.PixelFormat);
clone.Save(filePath);
}
Things I tried:
1) I usually use Firefox driver for this purpose, I tried using ChromeDriver instead and got the same result.
2) I checked for the element's coordiantes using the following console command: $0.getBoundingClientRect() and the position I got in my code matches it.
3) I tried 4 different cropping methods including this one:
IWebElement element = Driver.FindElement(By.XPath("//canvas"));
string filename = #"C:\Users\User\Desktop\test.png";
Screenshot screenshot = Driver.GetScreenshot();
screenshot.SaveAsFile(filename, ImageFormat.Png);
Rectangle cropRect = new Rectangle(element.Location.X, element.Location.Y,
element.Size.Width, element.Size.Height);
using (Image imgShot = Image.FromFile(filename))
using (Bitmap original = new Bitmap(imgShot))
using (Bitmap target = new Bitmap(original, new Size(cropRect.Width, cropRect.Height)))
using (Graphics g = Graphics.FromImage(target))
{
g.DrawImage(original, new Rectangle(0, 0, target.Width, target.Height),
cropRect,
GraphicsUnit.Pixel);
target.Save(#"C:\Users\User\Desktop\test1.png", ImageFormat.Png);
}
Just to be clear, the image I get is totally blank. In a different website the image I get is not blank so I can tell it's just in the wrong coordinates.
4) I tried a different website and different elements and they were all in the wrong coordinates.
5) I tried to Google it and found so many different approaches that didn't work. This answer however, says something about resolution which was my best guess. I tried playing with both the original and the target's resolution and saw no difference. The set resolution method was called either before or after the Graphics variable was created and still, zero change.
The funny thing is, it used to work 2 weeks ago but I never changed the code...
You are getting a blank image probably because the area is not yet rendered when GetScreenshot is called.
Try to wait to see if it's the case:
Thread.Sleep(3000);
Screenshot screenshot = ((ITakesScreenshot)element).GetScreenshot();
It could also be due to the implementation in the page preventing web scrapers, in which case there's nothing much you can do without digging in the code.
Note that you shouldn't use element.Location since it returns the coordinates relative to the document and not from the viewport.
You should also consider calling GetScreenshot directly on a IWebElement if the driver supports it.
Here's a working example to capture a footer:
var options = new ChromeOptions();
options.AddArgument("disable-infobars");
var driver = new ChromeDriver(options);
driver.Url = "https://stackoverflow.com/questions";
IWebElement element = driver.FindElement(By.CssSelector("#footer"));
string filePath = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Desktop), #"screenshot.png");
try {
Thread.Sleep(500);
Screenshot screenshot = ((ITakesScreenshot)element).GetScreenshot();
screenshot.SaveAsFile(filePath, ScreenshotImageFormat.Png);
}
catch (WebDriverException) {
var result = ((IJavaScriptExecutor)driver).ExecuteScript(
"var elm = arguments[0];" +
"elm.scrollIntoView(true);" +
"var rect = elm.getBoundingClientRect();" +
"return [rect.left, rect.top, rect.width, rect.height];"
, element);
int[] pts = Array.ConvertAll(((IReadOnlyCollection<object>)result).ToArray(), Convert.ToInt32);
var rect = new Rectangle(pts[0], pts[1], pts[2], pts[3]);
Screenshot screenshot = ((ITakesScreenshot)driver).GetScreenshot();
using (var mstream = new MemoryStream(screenshot.AsByteArray))
using (var bitmap = (Bitmap)Image.FromStream(mstream, false, false)) {
rect.Intersect(new Rectangle(0, 0, bitmap.Width, bitmap.Height));
if (rect.IsEmpty)
throw new ArgumentOutOfRangeException("Cropping rectangle is out of range.");
var clone = bitmap.Clone(rect, bitmap.PixelFormat);
clone.Save(filePath);
}
}
So apparently the thing that actually changed 2 weeks ago wasn't my code but the Firefox version installed on my machine. The current version - v52, return elements in wrong coordinates.
Uninstalling it and reinstalling the previous version - v47 solved the issue

Rotate PDF without truncating contents using PDFsharp

After rotating to portrait view, how might I ensure all content is visible (within page bounds)?
PDFsharp truncates my page when rotated by +90 degrees but not when rotated by -90 degrees.
PdfDocument outputDocument = new PdfDocument();
XPdfForm old = XPdfForm.FromFile(in_filename);
PdfPage newP = outputDocument.AddPage();
if (old.Page.Orientation == PageOrientation.Landscape)
{
old.Page.Rotate= (old.Page.Rotate - 90) % 360;
old.Page.Orientation = PageOrientation.Portrait;
}
newP.Height = old.Page.Height;
newP.Width = old.Page.Width;
// Get a graphics object for page1
XGraphics gfx = XGraphics.FromPdfPage(newP);
// Draw the page identified by the page number like an image
gfx.DrawImage(old, new XRect(0, 0, old.PointWidth, old.PointHeight));
The above works for a few pdf test cases, but I am skeptical it is coincidental/luck
I am using PDFsharp 1.50 beta.
There is a known problem with PDFsharp 1.50 beta with respect to importing rotated PDFs. That problem is still under investigation.
PDF files come in many different variations, therefore it is very difficult to ensure that code works in all cases.
I ended up doing the following:
(Note, this only had to work for a landscape to portrait)
var output = new PdfDocument();
var outputPage = output.AddPage();
using (var stream = new MemoryStream(Convert.FromBase64String(pdfBase64String)))
{
using (var input = XPdfForm.FromStream(stream))
{
outputPage.Height = input.PointWidth;
outputPage.Width = input.PointHeight;
using (var graphics = XGraphics.FromPdfPage(outputPage))
{
graphics.RotateAtTransform(90, new XPoint(input.PointHeight / 2, input.PointHeight / 2));
graphics.DrawImage(input, new XRect(0, 0, input.PointWidth, input.PointHeight));
}
}
}

EncodeParameters deinterlace bmp

I'm looking for a way to take in a 32 bit bitmap and save it again however deinterlacing the frames. When the image is taken two fields are visible but only the last one is necessary. Is this possible to do using EncoderParameters. This is what I've tried so far:
using (Image source = Image.FromFile(#"C:\Users\Martin vanPutten\Desktop\test.bmp"))
{
ImageCodecInfo codec = ImageCodecInfo.GetImageEncoders().First(c => c.MimeType == "image/bmp");
EncoderParameters parameters = new EncoderParameters(3);
parameters.Param[0] = new EncoderParameter(System.Drawing.Imaging.Encoder.Quality, 100L);
parameters.Param[1] = new EncoderParameter(System.Drawing.Imaging.Encoder.ScanMethod, (int)EncoderValue.LastFrame);
parameters.Param[2] = new EncoderParameter(System.Drawing.Imaging.Encoder.RenderMethod, (int)EncoderValue.RenderNonProgressive);
source.Save(#"C:\Users\Martin vanPutten\Desktop\test2.bmp", codec, parameters);
}
Is there another way to do this? All I need to do is remove the second overlapping frame in an image.
Quick update, its not that it has two frames, but 2 fields in 1 frame.

Categories

Resources