Develop Reference

Allocating 5MB for a single variable within a Unity script

TL;DR below
Hi,
I'm trying to make an in-game screen, whose purpose is to show the contents of another window running on the computer (under Windows). For that matter, I had to get my hands dirty and play with that good ol' Windows legacy API, the one with external references and everything. After a couple days of struggle, I was able to make it work! The only problem is that Unity doesn't seem to support it...
I already asked this on Unity answers, but had no answers.
I have a WindowOperation class which wraps all of the API calls into static methods, and also performs a few basic tasks. One of them consists in capturing the screen pixels and saving them in a System.Drawing.Bitmap:
public static Bitmap CaptureApplication(string procName)
{
// ...
var bmp = new Bitmap(width, height, PixelFormat.Format24bppRgb);
// ...
return bmp;
}
Unity fails declaring bmp and outputs the following message in the console:
ArgumentException: Parameter is not valid.
System.Drawing.Bitmap..ctor (Int32 width, Int32 height, PixelFormat format)
(wrapper remoting-invoke-with-check) System.Drawing.Bitmap:.ctor (int,int,System.Drawing.Imaging.PixelFormat)
[More info about the stack trace, which leads to the declaration of bmp in the above code]
According to MSDN and other threads on SO, such exceptions are thrown when .NET is unable to allocate a single block of contiguous memory as big as requested.
Since I didn't have such problems when I tested my code outside of Unity, my best guess is that (correct me if I'm wrong) Unity sandboxes the execution of the script and somehow doesn't allow the allocation of the memory bmp needs.
My screen dimension is 1600*900, and pixel depth is 3 bytes ( PixelFormat.Format24bppRgb ), so the amount of memory needed is 4,320,000 bytes, or just a bit more than 4MB, which doesn't seem excessive to me.
To capture the screen, I'm using .NET's Graphics.CopyFromScreen() function. I don't know how it works, but let's say it uses the same amount of memory as bmp, so the biggest amount of memory being used at one point of time in this script according to a pessimistic estimation rounds up to 10MB. However, I don't think this changes the root of the problem, and bmp is declared before any usage of CopyFromScreen() anyway.
TL;DR, is there a way I can tell Unity that it should let me allocate tons of memory in my scripts if I want to? Or maybe I'm completely off tracks and the problem lies somewhere else?
All answers are more than welcome!
Thank you in advance. :)

Processing on large bitmaps (up to 3GB)

I'm working on some university project and got stuck with memory issue.
I load a bitmap which takes about 1,5GB on HDD with code below:
Bitmap bmp = new Bitmap(pathToFile);
The issue is that the newly created Bitmap object uses about 3,5GB of RAM which is something I can't understand (that's really BIG wrapper :E). I need to get to the pixel array, and the use of Bitmap class is really helpful (I use LockBits() method later, and process the array byte per byte) but in this case it's total blocker. So here is my question:
Is there any easy way to extract the pixel array without lending additional 2gb?
I'm using c# just to extract the needed array, which is later processed in c++ - maybe I can extract all needed data in c++ (but conversion issue appears here - I'm concentrating on 24bgr format)?
PS: I need to keep the whole bitmap in memory so splitting it into parts is no solution.
PS2: Just to clarify some issues: I know the difference between file extension and file format. The loaded file is uncompressed bitmap 3 bytes per pixel of size ~1.42GB (16k x 32k pixels), so why Bitmap object is more than two times bigger? Any decompressing issues and converting into other format aren't taking place.

Consider using Memory Mapped Files to access your HUGE data :).
An example focused on what you need can be found here: http://visualstudiomagazine.com/articles/2010/06/23/memory-mapped-files.aspx
It's in managed code but you might as well use it from equivalent native code.
Let me know if you need more details.

You can use this solution , Work with bitmaps faster in C#
http://www.codeproject.com/Tips/240428/Work-with-bitmap-faster-with-Csharp
Or you can use memory mapped files
http://visualstudiomagazine.com/articles/2010/06/23/memory-mapped-files.aspx

You can stop memory caching.
Instead of
Bitmap bmp = new Bitmap(pathToFile);
Use
var bmp = (Bitmap)Image.FromStream(sourceFileStream, false, false);
see https://stackoverflow.com/a/47424918/887092

StackOverFlowException - but oviously NO recursion/endless loop

I'm now blocked by this problem the entire day, read thousands of google results, but nothing seems to reflect my problem or even come near to it... i hope any of you has a push into the right direction for me.
I wrote a client-server-application (so more like 2 applications) - the client collects data about his system, as well as a screenshot, serializes all this into a XML stream (the picture as a byte[]-array]) and sends this to the server in regular intervals.
The server receives the stream (via tcp), deserializes the xml to an information-object and shows the information on a windows form.
This process is running stable for about 20-25 minutes at a submission interval of 3 seconds. When observing the memory usage there's nothing significant to see, also kinda stable. But after these 20-25 mins the server throws a StackOverflowException at the point where it deserializes the tcp-stream, especially when setting the Image property from the byte[]-array.
I thoroughly searched for recursive or endless loops, and regarding the fact that it occurs after thousands of sucessfull intervals, i could hardly imagine that.
public byte[] ImageBase
{
get
{
MemoryStream ms = new MemoryStream();
_screen.Save(ms, System.Drawing.Imaging.ImageFormat.Jpeg);
return ms.GetBuffer();
}
set
{
if (_screen != null) _screen.Dispose(); //preventing well-known image memory leak
MemoryStream ms = new MemoryStream(value);
try
{
_screen = Image.FromStream(ms); //<< EXCEPTION THROWING HERE
}
catch (StackOverflowException ex) //thx to new CLR management this wont work anymore -.-
{
Console.WriteLine(ex.Message + Environment.NewLine + ex.StackTrace);
}
ms.Dispose();
ms = null;
}
}
I hope that more code would be unnecessary, or it could get very complex...
Please help, i have no clue at all anymore
thx
Chris

I suspect that it's not the code you posted but the code that reads from the TCP stream that's growing the stack. The fact that the straw breaking the camel's back happens during Image.FromStream is probably irrelevant. I've oftentimes seen people write socket processing code containing self-calling code (sometimes indirectly, like A -> B -> A -> B). You should inspect that code and post it here for us to look at.

You might want to read this. Loading an image from a stream without keeping the stream open
It seem possible that streams are being maintained on the stack or some other object that is eventually blowing the stack.
My suggestion would be to then just hold onto the byte[] and wait until the last possible moment to decode it and draw it. then dispose of the Image immediately. Your get/set would just then set/get the byte[]. You would then implement a custom drawing routine that would decode the current byte[] and draw it making sure not to hold onto any more resources than necessary.
Update
If there is a way you can get us a full stack trace we might be able to help further. I'm beginning to think it isn't the problem like I described. I created a sample program that created 10,000 images just like you do in your setter and there wasn't a problem. If you send an image every 3 seconds, that's 30 images a minute times 20 minutes which is only 600 images.
I'm very interested in the solution to this. I'll come back to it later.
There are a few possibilities, however, remote.
Image.FromStream is trying to process an invalid/corrupt byte[] and that method somehow uses recursion to decode a bitmap. Highly unlikely.
The exception isn't being thrown where you think it is. A full stack trace, if possible would be very helpful. As you stated you cannot catch a StackOverflowException. I believe there are provisions for this if you are running it through the debugger though.

I'm not sure if it's relevant but the MSDN documentation for Image.FromStream states that
You must keep the stream open for the lifetime of the Image.

Compress image in .NET

How can i compress and image file(*bmp,*jpeg) in C#,
I have to display some images as background on my controls, i m using following code to scale my image
Bitmap orgBitmap = new Bitmap(_filePath);
Bitmap regBitmap = new Bitmap(reqSize.Width, reqSize.Height);
using (Graphics gr = Graphics.FromImage(reqBitmap))
{
gr.SmoothingMode = System.Drawing.Drawing2D.SmoothingMode.AntiAlias;
gr.DrawImage(bmp, new RectangleF(0, 0, reqSize.Width, reqSize.Height));
}
It giving me the required bitmap.
My problem is if orginal bitmap is to heavy(2 MB) then when i load 50 image it feed all my memory, i want to compress the image as much i can without losing the so much quality,How can i do the same in .NET ?

Do you definitely need the large images to be present at execution time? Could you resize them (or save them at a slightly lower quality) using an image editing program (Photoshop, Paintshop Pro etc) beforehand? There seems to be little point in doing the work every time you run - especially as editing tools are likely to do a better job anyway.
Of course, this won't work if it's something like the user picking arbitrary images from their hard disk.
Another point: are you disposing of the bitmaps when you're finished with them? You aren't showing that in your code... if you're not disposing of the original (large) bitmaps then you'll be at the mercy of finalizers to release the unmanaged resources. The finalizers will also delay garbage collection of those objects.

JPEG always lose something, PNG don't.
This is how you encode and decode PNG with C#:
http://msdn.microsoft.com/en-us/library/aa970062.aspx

Perhaps I'm misunderstanding things, but why not convert the bitmaps to jpg's before you import them into your project as control backgrounds?

Good luck compressing JPEG. :) It's compressed already. As for your BMPs, make them JPEGs.

How to load a specific patch/rectangle from an image?

We have an application that show a large image file (satellite image) from local network resource.
To speed up the image rendering, we divide the image to smaller patches (e.g. 6x6 cm) and the app tiles them appropriately.
But each time the satellite image updated, the dividing pre-process should be done, which is a time consuming work.
I wonder how can we load the patches from the original file?
PS 1: I find the LeadTools library, but we need an open source solution.
PS 2: The app is in .NET C#
Edit 1:
The format is not a point for us, but currently it's JPG.
changing the format to a another could be consider, but BMP format is hardly acceptable, because of it large volume.

I wote a beautifull attempt of answer to your question, but my browser ate it... :(
Basically what I tried to say was:
1.- Since Jpeg (and most compression formats) uses a secuential compression, you'll always need to decode all the bits that are before the ones that you need.
2.- The solution I propose need to be done with each format you need to support.
3.- There are a lot of open source jpeg decoders that you could modify. Jpeg decoders need to decode blocks of bits (of variable size) that convert into pixel blocks of size 8x8. What you could do is modify the code to save in memory only the blocks you need and discard all the others as soon as they aren't needed any more (basically as soon as they are decoded). With those memory-saved blocks, create the image you need.
4.- Since Jpeg works with blocks of 8x8, your work could be easier if you work with patches of sizes multiples of 8 pixels.
5.- The modification done to the jpeg decoder could be used to substitute the preprocessing of the images you are doing if you save the patch and discard the blocks as soon as you complete them. It would be really fast and less memory consuming.
I know it needs a lot of work and there are a lot of details to be taken in consideration (specially if you work with color images), but if you need performance I belive you will always end fighting or playing (as you want to see it) with the bytes.
Hope it helps.

I'm not 100% sure what you're after but if you're looking for a way to go from string imagePath, Rectangle desiredPortion to a System.Drawing.Image object then perhaps something like this:
public System.Drawing.Image LoadImagePiece(string imagePath, Rectangle desiredPortion)
{
using (Image img = Image.FromFile(path))
{
Bitmap result = new Bitmap(desiredPortion.Width, desiredPortion.Height, PixelFormat.Format24bppRgb);
using (Graphics g = Graphics.FromImage((Image)result))
{
g.InterpolationMode = System.Drawing.Drawing2D.InterpolationMode.HighQualityBicubic;
g.SmoothingMode = System.Drawing.Drawing2D.SmoothingMode.HighQuality;
g.PixelOffsetMode = System.Drawing.Drawing2D.PixelOffsetMode.HighQuality;
g.CompositingQuality = System.Drawing.Drawing2D.CompositingQuality.HighQuality;
g.DrawImage(img, 0, 0, desiredPortion, GraphicsUnit.Pixel);
}
return result;
}
}
Note that for performance reasons you may want to consider building multiple output images at once rather than calling this multiple times - perhaps passing it an array of rectangles and getting back an array of images or similar.
If that's not what you're after can you clarify what you're actually looking for?