I'm looking to send bitmap (or image) over ethernet in order to perform some image processing, then send them back.
The 'client' is running c# code, extracting frames from a video using emgucv. The 'server' will be running c/c++ on an arm cpu, although at the moment is x86 on my laptop using elementary os. So I need to avoid using things like opencv for the image processing itself, but that's another point.
I looked into sockets etc and can send some data to/from the server, just text typed into a console at the moment.
From initial research, it seems like I'll need to convert the bitmap into a byte array in order to then send it, which I've done (I think) using the following code:
Stream stm = tcpclnt.GetStream();
int l;
using (MemoryStream ms = new MemoryStream())
{
bmpFrame.Save(ms, ImageFormat.Bmp);
byte[] byteFrame = ms.ToArray();
l = byteFrame.Length;
stm.Write(byteFrame, 0, byteFrame.Length);
stm.Flush();
}
Then on the server side trying to read it using:
char buff[10000];
int n;
n = read(conn_desc, buff, sizeof(buff)-1);
if ( n > 0)
{
//MemoryStream ms = new MemoryStream(receivedBytes);
//Bitmap bmpReceived = new Bitmap(ms, System.Drawing.Imaging.ImageFormat.Bmp);
printf("Received %s\n", buff);
}
else
{
printf("Failed receiving\n");
}
You can see the commented code where I thought I'd be able to change it back into a bitmap, but I'm not sure if I want/need to anymore, if I can just edit the images by accessing the bytes directly, and also I don't know how/if bitmaps work in C rather than C#.
Am I going along the right lines? Ideally I want to send all the data for a single frame, then do stuff to it, then send it back, but I've no idea if that's what I'm doing. Finding it more difficult than usual as on the server I'm just writing in scratch/gedit and using gcc to compile, having never coded on linux before so I'm missing things like intellisense/debugging.
Any help/recommendations would be appreciated
Yes, in short, I would say your approach for doing what you say you want to do is correct.
I would ask though, why are you sending it to the server? I'm guessing it has to do with parallel processing or something? Unless you're doing very heavy processing, working with a single frame is probably faster than transferring it to the server (as far as I know, EmguCV in C# isn't considerably slower than OpenCV in C/C++).
The System.Drawing.Bitmap class is part of the .Net framework, and you might not find a direct correspondance in C. You might want to consider first converting the image to some known format, such as a .png file. Then you send the raw bytes of that file. On the server side, you receive the raw byte array, and use whatever C struct to load that image as a .png file. The point is to remove some abstraction and have more precise knowledge of exactly what is being sent.
Perhaps it'd be easier to use something like the System.Net.WebRequest class? Again, it depends a bit on exactly what problem you're trying to solve.
I hope this helps at all - this response is a bit vague (your question is, too :P), so feel free to ask for clarification on specific parts :)
Related
I'm using the hololens and I'm trying to save a video stream with the world/projection matrices avaiable.
I've been trying to just take a sequence of pictures and save the data, but I can't find a way to save the image and the matrices.
When saving to disk, there is no option to get the photocaptureframe (which contains the matrix data), when saving to memory, I seem to not be able to save the image to disk.
I tried using the following methode, but this seemed to crash my unity program:
List<byte> imageBufferList = new List<byte>();
photoCaptureFrame.CopyRawImageDataIntoBuffer(imageBufferList);
byte[] myArrayImage = imageBufferList.ToArray();
And then use this to convert the byte array:
using (MemoryStream mStream = new MemoryStream(byteArrayIn))
return Image.FromStream(mStream);
After which I save the result.
When I remove the memorystream thing, the program doesn't crash (but it doesn't save my image either).
I've been looking all over the internet but there are a lot of vague statements about it
a) not beeing possible
b) using the memorystream (but that crashes)
Any suggestions?
If anyone knows a way to save all the the matrix (projection and world) data per frame and the corresponding frame for a video stream, it would be a great help.
Edit: I also tried to look into https://github.com/VulcanTechnologies/HoloLensCameraStream but this seems to give problems with newer Unity versions. Any remarks about this?
To clarify my end goal:
When filming, the program should save all frames and the corresponding matrices, for example:
Frame_01, Frame_02, Frame_03, ... (.jpg/png/...)
World_matrix_01, World_matrix_02, ... (.txt)
Projection_matrix_01, Projection_matrix_02,... (.txt)
Edit: I also tried to look into https://github.com/VulcanTechnologies/HoloLensCameraStream but this seems to give problems with newer Unity versions. Any remarks about this?
I used it and it worked very well in Unity 2018.3.13f1. You can only test in Hololens out of debug mode.
I have a service that takes a pdf document, resizes all the images, and replaces it in the pdf. The problem that I'm getting at, is the compression.
Some documents are scanned and saved with a Compression.CCITTFAX3 compression and some are saved with a Compression.CCITTFAX4 compression. I am using iTextSharp and convert the stream bytes to a Tiff, otherwise the image becomes funky because of stride or something.
Below is the code I'm currently making use of to check for the correct filter, and then convert to tiff image.
if (filter == "/CCITTFaxDecode")
{
byte[] data = PdfReader.GetStreamBytesRaw((PRStream)stream);
using (MemoryStream ms = new MemoryStream())
{
using (Tiff myTiff = Tiff.ClientOpen("in-memory", "w", ms, new TiffStream()))
{
myTiff.SetField(TiffTag.IMAGEWIDTH, UInt32.Parse(dict.Get(PdfName.WIDTH).ToString()));
myTiff.SetField(TiffTag.IMAGELENGTH, UInt32.Parse(dict.Get(PdfName.HEIGHT).ToString()));
myTiff.SetField(TiffTag.COMPRESSION, Compression.CCITTFAX3);
myTiff.SetField(TiffTag.BITSPERSAMPLE, UInt32.Parse(dict.Get(PdfName.BITSPERCOMPONENT).ToString()));
myTiff.SetField(TiffTag.SAMPLESPERPIXEL, 1);
myTiff.WriteRawStrip(0, data, data.Length);
myTiff.Flush();
using (System.Drawing.Image img = new Bitmap(ms))
{
if (img == null) continue;
ReduceResolution(stream, img, quality);
}
myTiff.Close();
}
}
}
Just to make sure that you understand my question...
I want to find out how I know when to use G3 compression and when to use G4 compression.
Keep in mind that I've tried every code sample I could find.
This is quite important, as we interface with banking systems, and the files uploaded are sent to them as FICA documents.
Please help...
You need to go low level and inspect the image dictionary. The /DecodeParms entry is a dictionary that contains several keys related to CCITT compression. The /K key specifies the compression type: -1 is G4, 0 is G3 1D and 1 is G3 2D.
Update: to be more exact a negative value, usually -1, is G4, 0 is G3 1D and a positive value, usually 1, is G3 2D. To answer your question in the comment, the /K entry is optional and if it is missing the default value is considered to be 0.
I would not advise inserting the data direct. I base this assertion on many years of practical experience of PDFs and TIFF in products like ABCpdf .NET (on which I work).
While in theory you should be able to move the data over direct, minor differences between the formats of the compressed data are likely to lead to occasional mismatches.
The fact that some Fax TIFFs contain data which will display correctly in a TIFF viewer but not in a PDF one leads me to suspect that the same kind of problem is likely to operate in the other direction too.
I'm not going to say this kind of problem is common but it is the kind of thing I wouldn't rely on if I was in a bank. Unless you are very sure your data source will be uniform I would suggest it is much safer to decompress and recompress.
I would also note that sometimes images are held inline in the content stream rather than in a separate XObject. Again this is something you will need to cope with unless your data source produces a standard format which you can be sure will not contain this kind of structure.
Thank you for the replies above. The solution from Mihai seems viable if you do have all the information from the stream. I found that iTextSharp does not do this properly, so I ended up buying pdf4net. Much simpler than trying to figure out whats the better solution, besides, it ended up cheaper than my time I spent on this.
OnceUponATime.... Thank you for the information given above.
PDF4Net has a built in method that you get all the images per page... This sorted my issues, whereas I tried to do this myself using iTextSharp and the examples that were given to me.
Not sure if what I'm trying to do will work out, or is even possible. Basically I'm creating a remote desktop type app which captures the screen as a jpeg image and sends it to the client app for displaying.
I want to reduce the amount of data sent each time by comparing the image to the older one and only sending the differences. For example:
var bitmap = new Bitmap(1024, 720);
string oldBase = "";
using (var stream = new MemoryStream())
using (var graphics = Graphics.FromImage(bitmap))
{
graphics.CopyFromScreen(bounds.X, bounds.Y, 0, 0, bounds.Size);
bitmap.Save(stream, ImageFormat.Jpeg);
string newBase = Convert.ToBase64String(stream.ToArray());
// ! Do compare/replace stuff here with newBase and oldBase !
// Store the old image as a base64 string.
oldBase = newBase;
}
Using something like this I could compare both base64 strings and replace any matches. The matched text could be replaced with something like:
[number of characters replaced]
That way, on the client side I know where to replace the old data and add the new. Again, I'm not sure if this would even work so anyones thoughts on this would be very appreciated. :) If it is possible, could you point me in the right direction? Thanks.
You can do this by comparing the bitmap bits directly. Look into Bitmap.LockBits, which will give you a BitmapData pointer from which you can get the pixel data. You can then compare the pixels for each scan line and encode them into whatever format you want to use for transport.
Note that a scan line's length in bytes is always a multiple of 4. So unless you're using 32-bit color, you have to take into account the padding that might be at the end of the scan line. That's what the Stride property is for in the BitmapData structure.
Doing things on a per-scanline basis is easier, but potentially not as efficient (in terms of reducing the amount of data sent) as treating the bitmap as one contiguous block of data. Your transport format should look something like:
<start marker>
// for each scan line
<scan line marker><scan line number>
<pixel position><number of pixels><pixel data>
<pixel position><number of pixels><pixel data>
...
// next scan line
<scan line marker><scan line number>
...
<end marker>
each <pixel position><number of pixels><pixel data> entry is a run of changed pixels. If a scan line has no changed pixels, you can choose not to send it. Or you can just send the scan line marker and number, followed immediately by the next scan line.
Two bytes will be enough for the <pixel position> field and for the <number of pixels> field. So you have an overhead of four bytes for each block. An optimization you might be interested in, after you have the simplest version working, would be to combine blocks of changed/unchanged pixels if there are small runs. For example, if you have uucucuc, where u is an unchanged pixel and c is a changed pixel, you'll probably want to encode the cucuc as one run of five changed pixels. That will reduce the amount of data you have to transmit.
Note that this isn't the best way to do things, but it's simple, effective, and relatively easy to implement.
In any case, once you've encoded things, you can run the data through the built-in GZip compressor (although doing so might not help much) and then push it down the pipe to the client, which would decompress it and interpret the result.
It would be easiest to build this on a single machine, using two windows to verify the results. Once that's working, you can hook up the network transport piece. Debugging the initial cut by having that transport step in the middle could prove very frustrating.
We're currently working on something very similar - basically, what you're trying to implement is video codec (very simple motion jpeg). There are some simple approaches and some very complicated.
The simplest approach is to compare consecutive frames and send only the differences. You may try to compare color differences between the frames in RGB space or YCbCr space and send only the pixels that changed with some metadata.
The more complicated solution is to compare the pictures after DCT transformation but before entropy coding. That would give you better comparisons and remove some ugly artifacts.
Check more info on JPEG, Motion JPEG, H.264 - you may use some methods these codecs are using or simply use the existing codec if possible.
This wont work for a JPEG. You need to use BMP, or possibly uncompressed TIFF.
I think if it were me I'd use BMP, scan the pixels for changes and construct a PNG where everything except the changes were transparent.
First, this would reduce your transmission size because the PNG conpression is quite good especially for repeating pixels.
Second, it makes dispay on the receiving end very easy since you can simply paint the new image overtop the old image.
I'm working on some university project and got stuck with memory issue.
I load a bitmap which takes about 1,5GB on HDD with code below:
Bitmap bmp = new Bitmap(pathToFile);
The issue is that the newly created Bitmap object uses about 3,5GB of RAM which is something I can't understand (that's really BIG wrapper :E). I need to get to the pixel array, and the use of Bitmap class is really helpful (I use LockBits() method later, and process the array byte per byte) but in this case it's total blocker. So here is my question:
Is there any easy way to extract the pixel array without lending additional 2gb?
I'm using c# just to extract the needed array, which is later processed in c++ - maybe I can extract all needed data in c++ (but conversion issue appears here - I'm concentrating on 24bgr format)?
PS: I need to keep the whole bitmap in memory so splitting it into parts is no solution.
PS2: Just to clarify some issues: I know the difference between file extension and file format. The loaded file is uncompressed bitmap 3 bytes per pixel of size ~1.42GB (16k x 32k pixels), so why Bitmap object is more than two times bigger? Any decompressing issues and converting into other format aren't taking place.
Consider using Memory Mapped Files to access your HUGE data :).
An example focused on what you need can be found here: http://visualstudiomagazine.com/articles/2010/06/23/memory-mapped-files.aspx
It's in managed code but you might as well use it from equivalent native code.
Let me know if you need more details.
You can use this solution , Work with bitmaps faster in C#
http://www.codeproject.com/Tips/240428/Work-with-bitmap-faster-with-Csharp
Or you can use memory mapped files
http://visualstudiomagazine.com/articles/2010/06/23/memory-mapped-files.aspx
You can stop memory caching.
Instead of
Bitmap bmp = new Bitmap(pathToFile);
Use
var bmp = (Bitmap)Image.FromStream(sourceFileStream, false, false);
see https://stackoverflow.com/a/47424918/887092
I'm now blocked by this problem the entire day, read thousands of google results, but nothing seems to reflect my problem or even come near to it... i hope any of you has a push into the right direction for me.
I wrote a client-server-application (so more like 2 applications) - the client collects data about his system, as well as a screenshot, serializes all this into a XML stream (the picture as a byte[]-array]) and sends this to the server in regular intervals.
The server receives the stream (via tcp), deserializes the xml to an information-object and shows the information on a windows form.
This process is running stable for about 20-25 minutes at a submission interval of 3 seconds. When observing the memory usage there's nothing significant to see, also kinda stable. But after these 20-25 mins the server throws a StackOverflowException at the point where it deserializes the tcp-stream, especially when setting the Image property from the byte[]-array.
I thoroughly searched for recursive or endless loops, and regarding the fact that it occurs after thousands of sucessfull intervals, i could hardly imagine that.
public byte[] ImageBase
{
get
{
MemoryStream ms = new MemoryStream();
_screen.Save(ms, System.Drawing.Imaging.ImageFormat.Jpeg);
return ms.GetBuffer();
}
set
{
if (_screen != null) _screen.Dispose(); //preventing well-known image memory leak
MemoryStream ms = new MemoryStream(value);
try
{
_screen = Image.FromStream(ms); //<< EXCEPTION THROWING HERE
}
catch (StackOverflowException ex) //thx to new CLR management this wont work anymore -.-
{
Console.WriteLine(ex.Message + Environment.NewLine + ex.StackTrace);
}
ms.Dispose();
ms = null;
}
}
I hope that more code would be unnecessary, or it could get very complex...
Please help, i have no clue at all anymore
thx
Chris
I suspect that it's not the code you posted but the code that reads from the TCP stream that's growing the stack. The fact that the straw breaking the camel's back happens during Image.FromStream is probably irrelevant. I've oftentimes seen people write socket processing code containing self-calling code (sometimes indirectly, like A -> B -> A -> B). You should inspect that code and post it here for us to look at.
You might want to read this. Loading an image from a stream without keeping the stream open
It seem possible that streams are being maintained on the stack or some other object that is eventually blowing the stack.
My suggestion would be to then just hold onto the byte[] and wait until the last possible moment to decode it and draw it. then dispose of the Image immediately. Your get/set would just then set/get the byte[]. You would then implement a custom drawing routine that would decode the current byte[] and draw it making sure not to hold onto any more resources than necessary.
Update
If there is a way you can get us a full stack trace we might be able to help further. I'm beginning to think it isn't the problem like I described. I created a sample program that created 10,000 images just like you do in your setter and there wasn't a problem. If you send an image every 3 seconds, that's 30 images a minute times 20 minutes which is only 600 images.
I'm very interested in the solution to this. I'll come back to it later.
There are a few possibilities, however, remote.
Image.FromStream is trying to process an invalid/corrupt byte[] and that method somehow uses recursion to decode a bitmap. Highly unlikely.
The exception isn't being thrown where you think it is. A full stack trace, if possible would be very helpful. As you stated you cannot catch a StackOverflowException. I believe there are provisions for this if you are running it through the debugger though.
I'm not sure if it's relevant but the MSDN documentation for Image.FromStream states that
You must keep the stream open for the lifetime of the Image.