Develop Reference

Fast way to convert large vectorial images to binary matrix

I have complex vectorial images I need to convert to binary matrix (a kind of rasterizing) with high precision, using necessarily c#. Currently I'm doing it by painting vectorial images in a bitmap and reading the internal bitmap array. Then I convert this array in a binary matrix where 1 indicates that the corresponding pixel is of a specific color, and 0 when is not. As I access the internal bitmap array directly, that's fast and allows using Parallel.For
The problem is that I need to obtain very high definition matrixes of the images (50000x25000 or more). Therefore I have to paint each vectorial image dividing it in several parts, as a bitmap of this size is not supported.
As I have a lot of images to convert, it is very slow.
I'm looking for a fast way of doing that without loosing precision (ideally even increasing it). And I need to integrate this feature in a c# application.

I finally used Gdal rasterize tool, passing polygons as shapefiles, and it works quite well and fast.

Comparing two base64 image strings and removing matches?

Not sure if what I'm trying to do will work out, or is even possible. Basically I'm creating a remote desktop type app which captures the screen as a jpeg image and sends it to the client app for displaying.
I want to reduce the amount of data sent each time by comparing the image to the older one and only sending the differences. For example:
var bitmap = new Bitmap(1024, 720);
string oldBase = "";
using (var stream = new MemoryStream())
using (var graphics = Graphics.FromImage(bitmap))
{
graphics.CopyFromScreen(bounds.X, bounds.Y, 0, 0, bounds.Size);
bitmap.Save(stream, ImageFormat.Jpeg);
string newBase = Convert.ToBase64String(stream.ToArray());
// ! Do compare/replace stuff here with newBase and oldBase !
// Store the old image as a base64 string.
oldBase = newBase;
}
Using something like this I could compare both base64 strings and replace any matches. The matched text could be replaced with something like:
[number of characters replaced]
That way, on the client side I know where to replace the old data and add the new. Again, I'm not sure if this would even work so anyones thoughts on this would be very appreciated. :) If it is possible, could you point me in the right direction? Thanks.

You can do this by comparing the bitmap bits directly. Look into Bitmap.LockBits, which will give you a BitmapData pointer from which you can get the pixel data. You can then compare the pixels for each scan line and encode them into whatever format you want to use for transport.
Note that a scan line's length in bytes is always a multiple of 4. So unless you're using 32-bit color, you have to take into account the padding that might be at the end of the scan line. That's what the Stride property is for in the BitmapData structure.
Doing things on a per-scanline basis is easier, but potentially not as efficient (in terms of reducing the amount of data sent) as treating the bitmap as one contiguous block of data. Your transport format should look something like:
<start marker>
// for each scan line
<scan line marker><scan line number>
<pixel position><number of pixels><pixel data>
<pixel position><number of pixels><pixel data>
...
// next scan line
<scan line marker><scan line number>
...
<end marker>
each <pixel position><number of pixels><pixel data> entry is a run of changed pixels. If a scan line has no changed pixels, you can choose not to send it. Or you can just send the scan line marker and number, followed immediately by the next scan line.
Two bytes will be enough for the <pixel position> field and for the <number of pixels> field. So you have an overhead of four bytes for each block. An optimization you might be interested in, after you have the simplest version working, would be to combine blocks of changed/unchanged pixels if there are small runs. For example, if you have uucucuc, where u is an unchanged pixel and c is a changed pixel, you'll probably want to encode the cucuc as one run of five changed pixels. That will reduce the amount of data you have to transmit.
Note that this isn't the best way to do things, but it's simple, effective, and relatively easy to implement.
In any case, once you've encoded things, you can run the data through the built-in GZip compressor (although doing so might not help much) and then push it down the pipe to the client, which would decompress it and interpret the result.
It would be easiest to build this on a single machine, using two windows to verify the results. Once that's working, you can hook up the network transport piece. Debugging the initial cut by having that transport step in the middle could prove very frustrating.

We're currently working on something very similar - basically, what you're trying to implement is video codec (very simple motion jpeg). There are some simple approaches and some very complicated.
The simplest approach is to compare consecutive frames and send only the differences. You may try to compare color differences between the frames in RGB space or YCbCr space and send only the pixels that changed with some metadata.
The more complicated solution is to compare the pictures after DCT transformation but before entropy coding. That would give you better comparisons and remove some ugly artifacts.
Check more info on JPEG, Motion JPEG, H.264 - you may use some methods these codecs are using or simply use the existing codec if possible.

This wont work for a JPEG. You need to use BMP, or possibly uncompressed TIFF.
I think if it were me I'd use BMP, scan the pixels for changes and construct a PNG where everything except the changes were transparent.
First, this would reduce your transmission size because the PNG conpression is quite good especially for repeating pixels.
Second, it makes dispay on the receiving end very easy since you can simply paint the new image overtop the old image.

Processing on large bitmaps (up to 3GB)

I'm working on some university project and got stuck with memory issue.
I load a bitmap which takes about 1,5GB on HDD with code below:
Bitmap bmp = new Bitmap(pathToFile);
The issue is that the newly created Bitmap object uses about 3,5GB of RAM which is something I can't understand (that's really BIG wrapper :E). I need to get to the pixel array, and the use of Bitmap class is really helpful (I use LockBits() method later, and process the array byte per byte) but in this case it's total blocker. So here is my question:
Is there any easy way to extract the pixel array without lending additional 2gb?
I'm using c# just to extract the needed array, which is later processed in c++ - maybe I can extract all needed data in c++ (but conversion issue appears here - I'm concentrating on 24bgr format)?
PS: I need to keep the whole bitmap in memory so splitting it into parts is no solution.
PS2: Just to clarify some issues: I know the difference between file extension and file format. The loaded file is uncompressed bitmap 3 bytes per pixel of size ~1.42GB (16k x 32k pixels), so why Bitmap object is more than two times bigger? Any decompressing issues and converting into other format aren't taking place.

Consider using Memory Mapped Files to access your HUGE data :).
An example focused on what you need can be found here: http://visualstudiomagazine.com/articles/2010/06/23/memory-mapped-files.aspx
It's in managed code but you might as well use it from equivalent native code.
Let me know if you need more details.

You can use this solution , Work with bitmaps faster in C#
http://www.codeproject.com/Tips/240428/Work-with-bitmap-faster-with-Csharp
Or you can use memory mapped files
http://visualstudiomagazine.com/articles/2010/06/23/memory-mapped-files.aspx

You can stop memory caching.
Instead of
Bitmap bmp = new Bitmap(pathToFile);
Use
var bmp = (Bitmap)Image.FromStream(sourceFileStream, false, false);
see https://stackoverflow.com/a/47424918/887092

Data reading and storing in c#, (Concept..no code)

I want to translate my Matlab code (least squares plane fitting) into C#.
I have many problems in understanding c#.
Let me ask here.
Reading a text file and storing data in xyz format in matrix (e.g., xyzdata= xyz) in Matlab is quite easy.
Translating it into CSharp?
How can I read [x y z] without knowing length of file and how can I store it in Matrix form?
Thank you very much for your help and If someone has plane fitting code / link, please guide me.

I don't know the content of your text file, but File.ReadAllLines is the easiest way to read a text file into a string array representing all lines in the file. No trouble with having to know the length of the file.
If the lines contain the entries of your matrix, the next step would be looping through the lines and for each line use String.Split to get the individual elements.
When you've got that far, you have all information for creating a matrix of the required size. To fill its elements you're going to need Int32.Parse or Decimal.Parse to convert the elements as string into numbers.
However, hard to tell from your post what kind of matrix you'll need (probably a multi dimensional array). Search "[matrix] [c#]" here at stack overflow. And try "[math] [.net]" to find posts on math libraries for .net.

How to read and modify the colorspace of an image in c#

I'm loading a Bitmap from a jpg file. If the image is not 24bit RGB, I'd like to convert it. The conversion should be fairly fast. The images I'm loading are up to huge (9000*9000 pixel with a compressed size of 40-50MB). How can this be done?
Btw: I don't want to use any external libraries if possible. But if you know of an open source utility class performing the most common imaging tasks, I'd be happy to hear about it. Thanks in advance.

The jpeg should start with 0xFF 0xD8. After that you will find various fields in the format:
Field identifier 2 bytes
Field length, excluding field identifier. 2 bytes.
Variable data.
Parse through the fields. The identifier you will be looking for is 0xFF 0xC0. This is called SOF0, and contains height, width, bit depth, etc. 0xFF 0xC0 will be followed by two bytes for the field length. Immediately following that will be a single byte showing the bit depth, which will usually be 8. Then there will be two bytes for height, two for width, and a single byte for the number of components; this will usually be 1 (for greyscale) or 3. (for color)

This isn't something I've tried myself, but I think you might need to acccess the picture's EXIF information as a start.
Check out Scott Hanselman's blog-entry on accessing EXIF information from pictures.

Standard .NET System.Drawing namespace should have all that you need,
but it probably won't be very efficient. It'll load the whole thing into RAM, uncompress it, convert it (probably by making a copy) and then re-compress and save it. If you aim for high performance, I'm afraid you might need to look into C/C++ libraries and make .NET wrappers for them.

As far as I know jpg is always 24 bpp. The only thing that could change would be that it's CMY(K?) rather then RGB. That information would be stored in the header. Unfortunately I don't have any means of creating a CMYK image to test whether loading into a Bitmap will convert it automatically.
The following line will read the file into memory:
Bitmap image = Image.FromFile(fileName);
image.PixelFormat will tell you the image format. However, I can't test what the file load does with files other than 24bpp RGB jpgs. I can only recommend that you try it out.