Not sure if what I'm trying to do will work out, or is even possible. Basically I'm creating a remote desktop type app which captures the screen as a jpeg image and sends it to the client app for displaying.
I want to reduce the amount of data sent each time by comparing the image to the older one and only sending the differences. For example:
var bitmap = new Bitmap(1024, 720);
string oldBase = "";
using (var stream = new MemoryStream())
using (var graphics = Graphics.FromImage(bitmap))
{
graphics.CopyFromScreen(bounds.X, bounds.Y, 0, 0, bounds.Size);
bitmap.Save(stream, ImageFormat.Jpeg);
string newBase = Convert.ToBase64String(stream.ToArray());
// ! Do compare/replace stuff here with newBase and oldBase !
// Store the old image as a base64 string.
oldBase = newBase;
}
Using something like this I could compare both base64 strings and replace any matches. The matched text could be replaced with something like:
[number of characters replaced]
That way, on the client side I know where to replace the old data and add the new. Again, I'm not sure if this would even work so anyones thoughts on this would be very appreciated. :) If it is possible, could you point me in the right direction? Thanks.
You can do this by comparing the bitmap bits directly. Look into Bitmap.LockBits, which will give you a BitmapData pointer from which you can get the pixel data. You can then compare the pixels for each scan line and encode them into whatever format you want to use for transport.
Note that a scan line's length in bytes is always a multiple of 4. So unless you're using 32-bit color, you have to take into account the padding that might be at the end of the scan line. That's what the Stride property is for in the BitmapData structure.
Doing things on a per-scanline basis is easier, but potentially not as efficient (in terms of reducing the amount of data sent) as treating the bitmap as one contiguous block of data. Your transport format should look something like:
<start marker>
// for each scan line
<scan line marker><scan line number>
<pixel position><number of pixels><pixel data>
<pixel position><number of pixels><pixel data>
...
// next scan line
<scan line marker><scan line number>
...
<end marker>
each <pixel position><number of pixels><pixel data> entry is a run of changed pixels. If a scan line has no changed pixels, you can choose not to send it. Or you can just send the scan line marker and number, followed immediately by the next scan line.
Two bytes will be enough for the <pixel position> field and for the <number of pixels> field. So you have an overhead of four bytes for each block. An optimization you might be interested in, after you have the simplest version working, would be to combine blocks of changed/unchanged pixels if there are small runs. For example, if you have uucucuc, where u is an unchanged pixel and c is a changed pixel, you'll probably want to encode the cucuc as one run of five changed pixels. That will reduce the amount of data you have to transmit.
Note that this isn't the best way to do things, but it's simple, effective, and relatively easy to implement.
In any case, once you've encoded things, you can run the data through the built-in GZip compressor (although doing so might not help much) and then push it down the pipe to the client, which would decompress it and interpret the result.
It would be easiest to build this on a single machine, using two windows to verify the results. Once that's working, you can hook up the network transport piece. Debugging the initial cut by having that transport step in the middle could prove very frustrating.
We're currently working on something very similar - basically, what you're trying to implement is video codec (very simple motion jpeg). There are some simple approaches and some very complicated.
The simplest approach is to compare consecutive frames and send only the differences. You may try to compare color differences between the frames in RGB space or YCbCr space and send only the pixels that changed with some metadata.
The more complicated solution is to compare the pictures after DCT transformation but before entropy coding. That would give you better comparisons and remove some ugly artifacts.
Check more info on JPEG, Motion JPEG, H.264 - you may use some methods these codecs are using or simply use the existing codec if possible.
This wont work for a JPEG. You need to use BMP, or possibly uncompressed TIFF.
I think if it were me I'd use BMP, scan the pixels for changes and construct a PNG where everything except the changes were transparent.
First, this would reduce your transmission size because the PNG conpression is quite good especially for repeating pixels.
Second, it makes dispay on the receiving end very easy since you can simply paint the new image overtop the old image.
Related
I use this link to add my program the capability to adjust the brightness of the image. This code is ok but it takes time to adjust the brightness(Image file size 1.8mb). When I try the lower quality image it instantly adjusts the image(Image file size 100KB). Is there any efficient way to adjust the brightness of the image.
The code seems to use GetPixel and SetPixel on regular Bitmaps. This is a bad idea because it is so slow.
To manipulate a single pixel of a Bitmap it must be locked (which Get/SetPixel do behind the scenes) and doing it on a pixel by pixel basis means that for a 1000x1000 sized image a million locking/unlocking operations must be performed. This creates an enormous overhead.
Method one
One way to avoid this is to lock the whole bitmap with the LockBits function. Now we can loop over the pixels and modify them.
Two notes about this method:
What we now access are the raw bytes of each pixel, that is each channel separately: either BGR or BGRA, depending on the pixel format. This means that the channels are physically reversed from the usual RGB/ARGB format of the Color methods.
To loop over the physical bitmap pixel rows we also need to add some stride to each row, which pads the rows to a multiple of 4 bytes. Also see here
For some examples you may want to browse over some of these posts. Note especially this one which uses a delegate to allow for flexible operations!
(Note that several of the posts use 2 or even 3 locked bitmaps because they aim at combining images..)
Method two
Another way to get around the overhead of locking pixels one by one are ready-made bitmap classes that help by locking themselves as a whole. Here and here are examples I didn't try myself.
Method three
Finally there is a very elegant method for image manipulation, which is both rather simple and really fast; also professionally crafted for best results: You can set up a ColorMatrix.
It will let you change brightness, gamma, hues and then some. Here is a very nice introduction.
The only drawback is, that is limited to some fixed operations, so you can't create custom filters for other fancy stuff, like photoshop-type layer modes or others, especially those that need to process neighbouring pixels e.g. for blurring..
But if all you want is changing brightness, this is what I would recommend!
I'm trying to make a screen sharing program, the program flows will be like this:
capture screen
slice to 9
compare new slice with old slice
replace the different slice
upload to web (with new slice)
But I've got some problems with replacing the slices (in replace function). From all the source I have searched I need to convert the bitmap image (the slice) to string, then I can replace. but there's no example for converting bitmap double array to strings.
Is there any possibility to replace the image without convert it to strings?
Why would you need to replace bitmap data using a string as intermediate? You can use bitmap manipulation functions just fine. Also, I'm having trouble understanding your algorithm. You get a bitmap of the whole screen. Then you cut it into 9 parts (are those the corners, edges and center?), compare each of the slices to their old versions one by one, replace the ones that changed, and then you upload the whole bitmap? Don't you want to upload each of the slices separately, only uploading the ones that changed? Otherwise it doesn't really make sense to do the slicing at all, or does it?
Now, it's true that converting the data to string lets you use the string comparison functions and other stuff like that, but that's an awful idea. The fastest way to compare two byte arrays would be using the memcmp function in msvcrt.dll. This answer gives you the solution to that - https://stackoverflow.com/a/2038515/3032289, including reading the data from the original bitmaps.
Then you just send the slices that aren't the same as their older versions and you're done, no replacing needed.
Probably the best way is to perform a Base64 encoding
Google for base64 C++ source code.
We have an application that show a large image file (satellite image) from local network resource.
To speed up the image rendering, we divide the image to smaller patches (e.g. 6x6 cm) and the app tiles them appropriately.
But each time the satellite image updated, the dividing pre-process should be done, which is a time consuming work.
I wonder how can we load the patches from the original file?
PS 1: I find the LeadTools library, but we need an open source solution.
PS 2: The app is in .NET C#
Edit 1:
The format is not a point for us, but currently it's JPG.
changing the format to a another could be consider, but BMP format is hardly acceptable, because of it large volume.
I wote a beautifull attempt of answer to your question, but my browser ate it... :(
Basically what I tried to say was:
1.- Since Jpeg (and most compression formats) uses a secuential compression, you'll always need to decode all the bits that are before the ones that you need.
2.- The solution I propose need to be done with each format you need to support.
3.- There are a lot of open source jpeg decoders that you could modify. Jpeg decoders need to decode blocks of bits (of variable size) that convert into pixel blocks of size 8x8. What you could do is modify the code to save in memory only the blocks you need and discard all the others as soon as they aren't needed any more (basically as soon as they are decoded). With those memory-saved blocks, create the image you need.
4.- Since Jpeg works with blocks of 8x8, your work could be easier if you work with patches of sizes multiples of 8 pixels.
5.- The modification done to the jpeg decoder could be used to substitute the preprocessing of the images you are doing if you save the patch and discard the blocks as soon as you complete them. It would be really fast and less memory consuming.
I know it needs a lot of work and there are a lot of details to be taken in consideration (specially if you work with color images), but if you need performance I belive you will always end fighting or playing (as you want to see it) with the bytes.
Hope it helps.
I'm not 100% sure what you're after but if you're looking for a way to go from string imagePath, Rectangle desiredPortion to a System.Drawing.Image object then perhaps something like this:
public System.Drawing.Image LoadImagePiece(string imagePath, Rectangle desiredPortion)
{
using (Image img = Image.FromFile(path))
{
Bitmap result = new Bitmap(desiredPortion.Width, desiredPortion.Height, PixelFormat.Format24bppRgb);
using (Graphics g = Graphics.FromImage((Image)result))
{
g.InterpolationMode = System.Drawing.Drawing2D.InterpolationMode.HighQualityBicubic;
g.SmoothingMode = System.Drawing.Drawing2D.SmoothingMode.HighQuality;
g.PixelOffsetMode = System.Drawing.Drawing2D.PixelOffsetMode.HighQuality;
g.CompositingQuality = System.Drawing.Drawing2D.CompositingQuality.HighQuality;
g.DrawImage(img, 0, 0, desiredPortion, GraphicsUnit.Pixel);
}
return result;
}
}
Note that for performance reasons you may want to consider building multiple output images at once rather than calling this multiple times - perhaps passing it an array of rectangles and getting back an array of images or similar.
If that's not what you're after can you clarify what you're actually looking for?
I'm loading a Bitmap from a jpg file. If the image is not 24bit RGB, I'd like to convert it. The conversion should be fairly fast. The images I'm loading are up to huge (9000*9000 pixel with a compressed size of 40-50MB). How can this be done?
Btw: I don't want to use any external libraries if possible. But if you know of an open source utility class performing the most common imaging tasks, I'd be happy to hear about it. Thanks in advance.
The jpeg should start with 0xFF 0xD8. After that you will find various fields in the format:
Field identifier 2 bytes
Field length, excluding field identifier. 2 bytes.
Variable data.
Parse through the fields. The identifier you will be looking for is 0xFF 0xC0. This is called SOF0, and contains height, width, bit depth, etc. 0xFF 0xC0 will be followed by two bytes for the field length. Immediately following that will be a single byte showing the bit depth, which will usually be 8. Then there will be two bytes for height, two for width, and a single byte for the number of components; this will usually be 1 (for greyscale) or 3. (for color)
This isn't something I've tried myself, but I think you might need to acccess the picture's EXIF information as a start.
Check out Scott Hanselman's blog-entry on accessing EXIF information from pictures.
Standard .NET System.Drawing namespace should have all that you need,
but it probably won't be very efficient. It'll load the whole thing into RAM, uncompress it, convert it (probably by making a copy) and then re-compress and save it. If you aim for high performance, I'm afraid you might need to look into C/C++ libraries and make .NET wrappers for them.
As far as I know jpg is always 24 bpp. The only thing that could change would be that it's CMY(K?) rather then RGB. That information would be stored in the header. Unfortunately I don't have any means of creating a CMYK image to test whether loading into a Bitmap will convert it automatically.
The following line will read the file into memory:
Bitmap image = Image.FromFile(fileName);
image.PixelFormat will tell you the image format. However, I can't test what the file load does with files other than 24bpp RGB jpgs. I can only recommend that you try it out.
I'm writing some map software for my smartphone and have hit a problem whereby I don't want to load all of the (large) image files into memory when only a portion will be displayed.
Is there a way to read only a subsection (the viewable portion) of a big image given that you know the x and y offsets and width? I know it's probably possibly to do it by reading the file a byte at a time but I'm not sure how to do this.
Thank you,
Nico
It's going to depend at least in part on what format(s) your images are saved in. If you have raw image files or bitmaps, it may be possible, but if your data is compressed in any manner, such as JPEG or PNG, it's going to be a lot more difficult to read just a subsection.
If you truly don't want to ever load the full data into memory, you'll have to write your own IO routine that reads the file. For anything more complex than BMP, your decompression algorithm could get complicated.
If it's a BMP file, it shouldn't be that hard.
First you read the header from the file, if I recall correctly it's 44 bytes, but you can find that out from searching the web for a specification.
The header contains information like how many bytes there are per pixel, total width and height, how many bytes per scan line. Normally the bitmap is stored upside down, so you would calculate where in the file the first pixel of the bottom line was and skip to that location. Then you read the pixels you want from that line and skip to the correct pixel on the next line.
The FileStream class has what you need; a Read method for reading and a Seek method to skip to a given position.
Couldn't you cut the image up into sections beforehand?
Splitting it into many 256x256 pixel images means you'd only have to load a couple of them and stitch them back together on the viewable canvas. To name one implementation - google maps uses this technique.
This is something I have done with bitmaps...
public BitmapCropBitmap(BitMap fullBitmap, Rectangle rectangle)
{
return proBitmap.clone(fullBitmap, rectangle, fullBitmap.PixelFormat);
}