I have to combine lets say about 100 images (png-files).
The problem is not combining them, that runs quick enough.
But loading them from storage takes up to 4sec.
That is too much time.
So I can use TPL or multiple threads but it is still too slow.
How can I speed it up? To hold all the images in main storage is not an option unfortunately.
The images are quite small: from 4KByte to 10KByte
I'm loading the images that way:
Image img = Image.FromFile(file);
Creating multiple threads does not improve I/O speed in your case. That is about your harddisk's read-write speed.
Loading 100 high quality images in 4 seconds seems normal.
Two ideas:
If the bulk of the lag really is from IO, compress the files. Depending on their contents, even a simple ZIP compression could reduce their size, thus less bytes to read. The work will be to decompress them in memory. I don't know if that's applicable for your case
Lazy load them. Do you need all 100 images loaded all the time? Perhaps you can just load the first ones, or the most important ones first, let the software do the other stuff while it finishes loading the remaining images in the background.
How do you load your image files? Please share a piece of your code. My guess is that you read not whole file at once but something like byte by byte until the EOF... One of ways to optimize file loading is to load whole file into preallocated memory buffer.
Related
I have 2 memory streams which are representing wav files in my Windows Phone 8.1 app. I want to play them but avoiding gaps between them, is there any way to do that without using Sleep methods or something like that?
I've tried already the Thread.Sleep() but I think it is making lot of gaps as my 2 files are 20ms duration each.
Assuming the audio is the same format and the wave headers have been stripped you could just concatenate the memory streams.
streamTwo.CopyTo(streamOne);
If the wave headers are still embedded then you'd need to skip over the one in the second stream - generally 44 bytes. If the formats are different then you'll need to find another technique.
I am writing a 32 bit app and so only 4GB, the files I process can be very huge upto 3.5 GB, but what size should I consider before loading the file for processing?
I mean, C# .net I suppose only have limited RAM for framework, what should be the cutoff for a file (though it depends on how much memory the application takes, just to arrive at a ballpark figure) ? ( i dont have a file of that magnitude, just want to handle it before a memory error)
I suppose, what I need is actual file size than file size on disk ? and is it possible find that without opening the file?
You can use FileInfo.Length property.
But instead consider using stream classes (see *Reader classes in System.IO), they allow you to read parts of a file, analyse it and discard. In this way you don't care about the size of a file at all.
I'm working with kinect and I want to save the video and audio stream do a file (it doesn't matter if I can play it or not, I want to save the raw data).
My question is, if I'm constantly writing to disk (25fps) the computer may lag right? so what I'm trying to do is save to file in an efficient way. I thought of having like a list of images (like 5 seconds) and then write it all to file. What do you think? Is this a correct way?
Or is there another way to do this without losing performance?
Thank you
Writing to disk is a low intensitivity task for the CPU, it mostly just uses a memory buffer and some memory bandwidth. However, if you have to access the disk while you are writing to it, you will experience an increased delay.
As for how to do it; I have never worked with video before but I am thinking it is most easily done by using a buffer to hold the captured frames, and then writing from that buffer to the disk.
Saving the frames into arrays of 125 images (5s*25fps) sounds like an inefficient way to buffer the frames.
As for avoiding losing performance there really is no a way to do this; however I cannot see you losing much performance as the bitrate of the captured video and audio is comparatively low.
At the end of my process, I need to upload several paged .tiff file images to a website. The files need to be very small, 500kb or less when i upload them.
The problem is, even with me resizing them a lot but at the same time being able to read a few lines of text that are in some of them, they are around 1mb each or so.
I first resize all images going into the tiff files but it's not enough. I need a way to change the quality of them to decrease their size as well.
Can C# do this or would I need a third party software to do it?
The files being uploaded MUST be .tiff.
You don't provide much detail about your data, so can only make some guesses as to what you might need to look at.
First, can you loose some resolution? Can you make the images smaller?
Second, can you loose some color depth? Are you saving the files in a color format when bilevel or greyscale images would suffice?
Third, how clean are these images? Are they photos, scanned documents, what? If they are scanned documents of text or drawings, then some pre-processing to remove noise can make a significant difference in size.
Lastly, what compression method are you saving the file with? Only a lossy format is going to give you the highest degree of compression is most circumstances.
Based on your follow-up:
1) If you can make smaller, this of course saves significant storage space. Determine what is the minimum acceptable resolution that they need to be and standardize on that.
2) If you need to persist color, then this step might not be as effective, since you would have to algorithmically decrease the dynamic range of colors used in the image to an acceptable level before compressing. If you are not sure what this means, then you would probably best skip considering this completely unless you can spend time learning more about image processing and/or using a image processing library that will simplify this for you.
3) I don't think you addressed this in your comments. If you want more precise help, you should update your original question and add much more detail about what you are trying to accomplish. Provide some explanations of what/why you need to do in order to help determine what tradeoffs make sense.
4) Yes, JPG is a lossy format, but I think you may be confusing a few different things (or I may not be understanding your intent from your description). If you are first resizing your original images down into a new JPG file (an intermediate image file), then you are building a TIFF file and inserting the resized JPG as a source image into a multi-page TIFF and saving that, then you need to realize that the process of how the files are compressed in the intermediate files do not necessarily have any correlation with the compression format used in the TIFF file. Depending on what you are using to build and create the TIFF file, the compression format used in the TIFF is done separately and you probably need to specify those parameters when you save that file. If this is what you are doing, then the intermediary process of saving the JPG files may be increasing the size a bit.
I need to edit(Increase the height) the Image on the fly.
The file is mostly 5000*4000 in dimension. I see the memory shoots up to peak level when I create a bmp of large dimensions and call Graphics.DrawImage method on the bmp instance.
How do I get rid of the Out Of Memory exception? Is there a way to work with large bitmaps in c# ?
The problem is the Huge amount of Memory required for the operation. Yours is taking about some GigaBytes, so the solution could be to use a Stream and process the file in chunks.
Or the the best option would be to use some Third party library for it. Below are some for .Net
AForge
Image Resizer
Also have a look at this SO question.
https://stackoverflow.com/questions/158756/what-is-the-best-image-manipulation-library
It's depends on you application specific requeirements, it's not very clear from yuor post, but generaly, working with big media files (images, sounds, videos) I think really good solution is
Memory Mapped Files
Save yuor image on the disk in memory mapped file and resize it having on disk, by free yuor RAM as much as possible from a lot of data that you, probably, don't need to have a fast access(in that moment at least)
Hope this helps.
Regards.