Develop Reference

C# Cropping image return wrong coordinates

I've been trying to crop a specific image using Selenium and different cropping methods for a few days.
An important note before my code - the following method used to work 2 weeks ago and for some reason it now returns an image with wrong coordinates
// Go to site
Driver.Navigate().GoToUrl("http://google.com");
Screenshot screenshot = driver.GetScreenshot();
using (var ms = new MemoryStream(screenshot.AsByteArray))
using (var imgShot = Image.FromStream(ms))
using (var src = new Bitmap(imgShot))
{
IWebElement element = driver.FindElement(By.XPath("//canvas"));
Rectangle cropRect = new Rectangle(element.Location.X, element.Location.Y, element.Size.Width, element.Size.Height);
var clone = src.Clone(cropRect, src.PixelFormat);
clone.Save(filePath);
}
Things I tried:
1) I usually use Firefox driver for this purpose, I tried using ChromeDriver instead and got the same result.
2) I checked for the element's coordiantes using the following console command: $0.getBoundingClientRect() and the position I got in my code matches it.
3) I tried 4 different cropping methods including this one:
IWebElement element = Driver.FindElement(By.XPath("//canvas"));
string filename = #"C:\Users\User\Desktop\test.png";
Screenshot screenshot = Driver.GetScreenshot();
screenshot.SaveAsFile(filename, ImageFormat.Png);
Rectangle cropRect = new Rectangle(element.Location.X, element.Location.Y,
element.Size.Width, element.Size.Height);
using (Image imgShot = Image.FromFile(filename))
using (Bitmap original = new Bitmap(imgShot))
using (Bitmap target = new Bitmap(original, new Size(cropRect.Width, cropRect.Height)))
using (Graphics g = Graphics.FromImage(target))
{
g.DrawImage(original, new Rectangle(0, 0, target.Width, target.Height),
cropRect,
GraphicsUnit.Pixel);
target.Save(#"C:\Users\User\Desktop\test1.png", ImageFormat.Png);
}
Just to be clear, the image I get is totally blank. In a different website the image I get is not blank so I can tell it's just in the wrong coordinates.
4) I tried a different website and different elements and they were all in the wrong coordinates.
5) I tried to Google it and found so many different approaches that didn't work. This answer however, says something about resolution which was my best guess. I tried playing with both the original and the target's resolution and saw no difference. The set resolution method was called either before or after the Graphics variable was created and still, zero change.
The funny thing is, it used to work 2 weeks ago but I never changed the code...

You are getting a blank image probably because the area is not yet rendered when GetScreenshot is called.
Try to wait to see if it's the case:
Thread.Sleep(3000);
Screenshot screenshot = ((ITakesScreenshot)element).GetScreenshot();
It could also be due to the implementation in the page preventing web scrapers, in which case there's nothing much you can do without digging in the code.
Note that you shouldn't use element.Location since it returns the coordinates relative to the document and not from the viewport.
You should also consider calling GetScreenshot directly on a IWebElement if the driver supports it.
Here's a working example to capture a footer:
var options = new ChromeOptions();
options.AddArgument("disable-infobars");
var driver = new ChromeDriver(options);
driver.Url = "https://stackoverflow.com/questions";
IWebElement element = driver.FindElement(By.CssSelector("#footer"));
string filePath = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Desktop), #"screenshot.png");
try {
Thread.Sleep(500);
Screenshot screenshot = ((ITakesScreenshot)element).GetScreenshot();
screenshot.SaveAsFile(filePath, ScreenshotImageFormat.Png);
}
catch (WebDriverException) {
var result = ((IJavaScriptExecutor)driver).ExecuteScript(
"var elm = arguments[0];" +
"elm.scrollIntoView(true);" +
"var rect = elm.getBoundingClientRect();" +
"return [rect.left, rect.top, rect.width, rect.height];"
, element);
int[] pts = Array.ConvertAll(((IReadOnlyCollection<object>)result).ToArray(), Convert.ToInt32);
var rect = new Rectangle(pts[0], pts[1], pts[2], pts[3]);
Screenshot screenshot = ((ITakesScreenshot)driver).GetScreenshot();
using (var mstream = new MemoryStream(screenshot.AsByteArray))
using (var bitmap = (Bitmap)Image.FromStream(mstream, false, false)) {
rect.Intersect(new Rectangle(0, 0, bitmap.Width, bitmap.Height));
if (rect.IsEmpty)
throw new ArgumentOutOfRangeException("Cropping rectangle is out of range.");
var clone = bitmap.Clone(rect, bitmap.PixelFormat);
clone.Save(filePath);
}
}

So apparently the thing that actually changed 2 weeks ago wasn't my code but the Firefox version installed on my machine. The current version - v52, return elements in wrong coordinates.
Uninstalling it and reinstalling the previous version - v47 solved the issue

How can I get good-quality text when using a RenderTargetBitmap?

I need to create a small, simple TIFF image that is just black text on a white background, as part of a support library we use across several of our products. (Think a simple receipt or deposit ticket, that kind of thing). I would prefer not to have to use any third-party libraries, to avoid requiring us to add additional dependencies to all of the existing products that use the library.
I have almost gotten the problem solved using just WPF, based on the code from this question. The TIFF is created and rendered properly, but the text quality is horrible. As far as I can determine, the problem is that RenderTargetBitmap, at least the way I'm using it, isn't doing any kind of anti-aliasing, isn't using ClearType, or otherwise is just really bad at rendering plain text, e.g.:
I've read similar questions, such as this one or this one, but they all seem to discuss how to capture text from a visual control or window handle without reducing the quality. I'm doing this entirely off-screen, so I'm not clear if/how I can apply those solutions to my problem.
What I'm currently doing is:
var visual = new DrawingVisual();
using (var draw = visual.RenderOpen())
{
// 'background' is a pre-loaded blank white bitmap
draw.DrawImage(background, new Rect(0, 0, IMG_WIDTH, IMG_HEIGHT));
var font = new Typeface("Segoe UI");
Func<string, FormattedText> format = s => new FormattedText(
s,
CultureInfo.CurrentUICulture,
FlowDirection.LeftToRight,
font,
16,
Brushes.Black);
draw.DrawText(format("Some text"), new Point(HEADER_LEFT, HEADER_TOP);
// Call draw.DrawText a whole bunch more times.
.
.
.
}
var render = new RenderTargetBitmap(IMG_WIDTH, IMG_HEIGHT, IMG_DPI, IMG_DPI, PixelFormats.Default);
render.Render(visual);
var encoder = new TiffBitmapEncoder()
{
Compression = TiffCompressOption.Ccitt4
};
encoder.Frames.Add(BitmapFrame.Create(render));
var stream = new MemoryStream();
encoder.Save(stream);
Is there something else I can put in there to get the text quality better than what it is?

Don't use Ccitt4, it's for black-and-white images only. Try Default instead.
UPDATE:
It's a TT font and you're trying to render it at a small size, no matter what you do it's going to look pretty crap. Still, if you insist then don't use a visual. Screen elements use hardware acceleration, when you use them with a render target as you are here it has to fall back to a software approach which isn't always pixel-for-pixel the same. And either way you always get at least some aliasing. If you really do want proper bitonal then you'll have to use a Graphics object which will use GDI:
var bitmap = new Bitmap(512, 256);
var graphics = Graphics.FromImage(bitmap);
graphics.Clear(System.Drawing.Color.White);
var font = new Font("Segoe UI", 16);
graphics.TextRenderingHint = System.Drawing.Text.TextRenderingHint.SingleBitPerPixelGridFit;
graphics.DrawString("Hello World", font, System.Drawing.Brushes.Black, 0, 0);
ImageCodecInfo myImageCodecInfo = GetEncoderInfo("image/tiff");
var myEncoderParameters = new EncoderParameters(1);
myEncoderParameters.Param[0] = new EncoderParameter(System.Drawing.Imaging.Encoder.Compression, (long)EncoderValue.CompressionCCITT4);
bitmap.Save(#"myimage.tif", myImageCodecInfo, myEncoderParameters);
graphics.Dispose();
private static ImageCodecInfo GetEncoderInfo(String mimeType)
{
int j;
ImageCodecInfo[] encoders;
encoders = ImageCodecInfo.GetImageEncoders();
for (j = 0; j < encoders.Length; ++j)
{
if (encoders[j].MimeType == mimeType)
return encoders[j];
}
return null;
}
Result (which SO is blurring a bit itself):

Sending screenshots taking too much memory

I am trying to make a small application to serve screenshot of entire screen through network. I want it to be able to serve one every 1-2 seconds through LAN. I thought it won't be a big problem, so I chose C# and nancy www self host (as the easiest option). Now, the speed is allright for my needs, but it seems to be taking way too much memory, and it grows with time. It seems to settle on taking about 3.5 GB RAM and from then no longer grows, but that's not an acceptable amount - even with big resolution (mine is 2560x1440).
Is my code bad, or is nancy not suitable for handling many big responses, or maybe C# method of capturing screen is poorly optimized and I should try pinvoke methods? Or maybe it's just a terrible way to do it and I should try something more advanced, like using VNC library?
Currently my code looks like this:
public class HomeModule : NancyModule
{
private Bitmap bitmapScreenCapture;
private Graphics graphics;
private Object lockMe = new object();
private MemoryStream memoryStream = new MemoryStream();
public HomeModule()
{
Get["image"] = parameters =>
{
lock (lockMe)
{
GC.Collect();
if (bitmapScreenCapture == null || bitmapScreenCapture.Width != Screen.PrimaryScreen.Bounds.Width || bitmapScreenCapture.Height != Screen.PrimaryScreen.Bounds.Height)
{
if (bitmapScreenCapture != null)
{
bitmapScreenCapture.Dispose();
}
bitmapScreenCapture = new Bitmap(Screen.PrimaryScreen.Bounds.Width, Screen.PrimaryScreen.Bounds.Height);
graphics = Graphics.FromImage(bitmapScreenCapture);
}
graphics.CopyFromScreen(Screen.PrimaryScreen.Bounds.X,
Screen.PrimaryScreen.Bounds.Y,
0, 0,
bitmapScreenCapture.Size,
CopyPixelOperation.SourceCopy);
bitmapScreenCapture.Save(memoryStream, ImageFormat.Png);
memoryStream.Position = 0;
return new Response()
{
Contents = stream => memoryStream.CopyTo(stream)
};
}
};
}
}

As much as possible, keep variables in the most local scope possible, and dispose of what you can.
Part of your issue may be that you're new'ing up a Graphics instance repeatedly, but never disposing of the old reference. The GC will collect it eventually, but you can place your code in a using block to let it know you're done with it.
I haven't tested this, but here I've made your instances local, and disposed of the Graphics and Bitmap instances. I didn't dispose of the MemoryStream since I'm not sure it will successfully return the value if you do, but you could play around with it.
var screen = Screen.PrimaryScreen;
using (var bitmap = new Bitmap(screen.Bounds.Width, screen.Bounds.Height))
{
using (var g = Graphics.FromImage(bitmap))
{
g.CopyFromScreen(screen.Bounds.Left, screen.Bounds.Top, 0, 0, screen.Bounds.Size);
}
var imageStream = new MemoryStream();
bitmap.Save(imageStream, ImageFormat.Png);
imageStream.Position = 0;
return new Response()
{
Contents = stream => memoryStream.CopyTo(stream)
};
}

Two Different GCHandle's refer to same array in memory

This is probably not a well-phrased question, because I am not sure what is happenning, so i don't know how to ask it pointedly. I am trying to learn, and i hope i can get some direction on this. Your patience with a neophyte is appreciated.
I have a piece of code i am modifying. It displays an image. I want to modify the image, and display it in a different window. I copy the code that displays the image, do the modification, and it displays the modified image for both the original and modified images.
It seems GCHandle keeps referring to the same memory? Am i not really making a new handle by changing the handle name? Sorry for the long piece of code, but i am just lost.
What is going wrong?
Most perplexing is that it was working, then i changed something, and now can't get back to the working version, tho i think my code is the same as the one that worked. Some setting some where?
System.Runtime.InteropServices.GCHandle gch3 = System.Runtime.InteropServices.GCHandle.Alloc(scaled, System.Runtime.InteropServices.GCHandleType.Pinned);
int pitch = mImageWidth;
if (pitch % 4 != 0)
pitch = ((pitch / 4) + 1) * 4;
System.Drawing.Bitmap bitmap = new Bitmap(mImageWidth, mImageHeight, pitch, System.Drawing.Imaging.PixelFormat.Format8bppIndexed, gch3.AddrOfPinnedObject());
gch3.Free();
if (pal == null)
{
System.Drawing.Imaging.ColorPalette cp = bitmap.Palette;
for (i = 0; i < cp.Entries.Length; ++i)
{
cp.Entries[i] = Color.FromArgb(i, i, i);
}
pal = cp;
}
bitmap.Palette = pal;
FirstImageDisplay.Image = bitmap;
//second image here
for (i = 0; i < frame.Length; ++i)
scaled[i] = (byte)(.5 * scaled[i]);
System.Runtime.InteropServices.GCHandle gch4 = System.Runtime.InteropServices.GCHandle.Alloc(scaled, System.Runtime.InteropServices.GCHandleType.Pinned);
int pitch1 = mImageWidth;
if (pitch1 % 4 != 0)
pitch1 = ((pitch1 / 4) + 1) * 4;
System.Drawing.Bitmap bitmap2 = new Bitmap(mImageWidth, mImageHeight, pitch, System.Drawing.Imaging.PixelFormat.Format8bppIndexed, gch4.AddrOfPinnedObject());
gch4.Free();
if (pal == null)
{
System.Drawing.Imaging.ColorPalette cp = bitmap.Palette;
for (i = 0; i < cp.Entries.Length; ++i)
{
cp.Entries[i] = Color.FromArgb(i, i, i);
}
pal = cp;
}
bitmap.Palette = pal;
SecondImageDisplay.Image = bitmap;
//end second image code

What you're doing definitely isn't safe. Why are you doing this? Is there a reason you're so comfortable leaving the safe, managed environment?
The bitmap is created around that byte[]. This is okay as long as you don't mind having a pinned byte[] in the managed memory (okay for a few moments, not really for the duration of the application etc.). However, on the very next line, you release the pointer!
Then you use the same byte[], modify it, and use it for another bitmap. Boom, it's still the same byte array. It shouldn't be surprising that both bitmaps have the same content - you asked for that.
The reason why it sometimes works and sometimes it doesn't is that if the handle isn't moved by the GC, both bitmaps will be correct. However, if the GC moves the byte array, the Bitmaps have no way of adjusting - they will still point to the same location in memory (which is now wrong).
What you have to understand is that a GCHandle doesn't create a new object. It just instructs the GC not to mess with the physical location (well, in virtual memory, but...) as long as the GCHandle exists. If you want to create a new object, do something like byte[].Clone(). However, you're still going to have to have the handle pinned for all the lifetime of the Bitmap, which is usually a bad idea. Instead, try creating the Bitmap the usual way, then doing Bitmap.LockBits, then use Marshal.Copy to copy the bitmap array to the unmanaged memory of the Bitmap, and you're done, nice and relatively safe.
Here's a code snippet that illustrates the whole concept:
byte[] data = new byte[320 * 200 * 1];
Bitmap bmp1 = new Bitmap(320, 200,
System.Drawing.Imaging.PixelFormat.Format8bppIndexed);
Bitmap bmp2 = new Bitmap(320, 200,
System.Drawing.Imaging.PixelFormat.Format8bppIndexed);
var bdata = bmp1.LockBits(new Rectangle(new Point(0, 0), bmp1.Size),
ImageLockMode.WriteOnly, bmp1.PixelFormat);
try
{
Marshal.Copy(data, 0, bdata.Scan0, data.Length);
}
finally
{
bmp1.UnlockBits(bdata);
}
// Do your modifications
bdata = bmp2.LockBits(new Rectangle(new Point(0, 0), bmp2.Size),
ImageLockMode.WriteOnly, bmp2.PixelFormat);
try
{
Marshal.Copy(data, 0, bdata.Scan0, data.Length);
}
finally
{
bmp2.UnlockBits(bdata);
}
This isn't the best code performance-wise (it does need some copying), but the only real alternative is to use unsafe code - which you really shouldn't be doing, given your current apparent knowledge about the managed environment. It can lead to some nasty issues if you don't use it properly. In any case, the performance gains might be quite negligible - find out if you actually care before you go the unsafe way.
For more information about the problem and the complexities of working with managed and unmanaged memory, you can have a look at my blog at http://www.luaan.cz/2014/07/a-quick-introduction-to-managed-and.html It's still rather high-level, but it explains more than this answer on its own.

OpenGL: glReadPixels "fails" although glGetError returns 0

Problem summary:
I use OpenGL glReadPixels to get a screenshot but the buffer is unchanged.
If I use glGetError to get the error from the last function, it returns 0, as if everything is fine.
I've researched online throughly and haven't found anyone facing a problem as this one.
In details:
I'm using the CsGL.dll, which basically just wraps OpenGL to C#, and I have made the required initializations for using GL:
DC from hWnd, ChoosePixelFormat for DC using PixelFormatDescriptor and setting the it as it's format, creating RC for the DC and calling wglMakeCurrent(RC, DC).
Note that I am using simple wrappers for the Windows API functions.
Here is the code for this initialization I have used:
public unsafe void Init(IntPtr hWnd)
{
this.DC = (IntPtr)User.GetDC(this.HWnd = hWnd);
var pfd = new PIXELFORMATDESCRIPTOR();
var sizeOf = Marshal.SizeOf(pfd);
Kernel.ZeroMemory(new IntPtr(&pfd), sizeOf);
pfd.nSize = (short)sizeOf;
pfd.nVersion = 1;
pfd.dwFlags = (int)(PixelFormatDescriptorFlagsEnum.PFD_DRAW_TO_WINDOW |
PixelFormatDescriptorFlagsEnum.PFD_SUPPORT_OPENGL |
PixelFormatDescriptorFlagsEnum.PFD_DOUBLEBUFFER);
pfd.iPixelType = PIXELFORMATDESCRIPTOR.PFD_TYPE_RGBA;
pfd.cColorBits = 24;
pfd.cDepthBits = 16;
pfd.iLayerType = PIXELFORMATDESCRIPTOR.PFD_MAIN_PLANE;
var iFormat = GDI.ChoosePixelFormat(this.DC, ref pfd);
GDI.SetPixelFormat(this.DC, iFormat, ref pfd);
this.RC = wglCreateContext(this.DC);
wglMakeCurrent(this.DC, this.RC);
}
I've sent User.GetForegroundWindow() as the hWnd.
After this initialization I try getting a screenshot into in image (and I have also tried to read it into a simple byte array)
Short psuedo code of glReadPixels usage:
var area = new Rectangle(0, 0, 100, 100);
var bmp = new Bitmap(area.Width, area.Height);
var data = bmp.LockBits(area, ILM.WriteOnly, PF.24bppRgb);
glReadBuffer(BACK);
glReadPixels(0, 0, area.Width, area.Height, BGR_EXT /*also tried RGB and RGBA*/, UNSIGNED_BYTE, data.Scan0);
bmp.UnlockBits(data);
bmp.Save(#"C:\Back.bmp");
data = bmp.LockBits(area, ILM.WriteOnly, PF.24bppRgb);
glReadBuffer(FRONT);
glReadPixels(0, 0, area.Width, area.Height, BGR_EXT /*also tried RGB and RGBA*/, UNSIGNED_BYTE, data.Scan0);
bmp.UnlockBits(data);
bmp.Save(#"C:\Front.bmp");
Trying it with a simple and small byte[] was done like this:
var bytes = new byte[10 * 10 * 3];
glReadPixels(0, 0, 10, 10, RGB, UNSIGNED_BYTE, bytes);
And the byte[] was all zero's.
I've also tried it with bigger size for the array (keeping the 0, 0, 10, 10), but still to no avail.
In both cases, the result is the same.
The buffer doesn't change at all, while a glGetError call returns 0 after each GL function call.
Both Back.bmp and Front.bmp are totally black.
Please tell me what am I doing wrong?
Thanks

Are you trying to create a screenshot of something you rendered, or a screenshot of the screen in general?
glReadPixels is assured to work only for contents of the OpenGL framebuffer. It's not suited for taking screenshots of anything else (there used to be a time, where this was possible, but ever since the advent of compositing window managers those times are gone).
So if you're trying to do general screen capture, glReadPixels is the wrong way to go.
If however you're trying to take a screenshot of what you rendered, then you must make sure that the context you rendered with is also active when calling glReadPixels.