Develop Reference

Decode H264 frame to Bitmap

I am receiving H264 encoded frame but when I convert it into bitmap I just get a black screen. The resolution is right. I have tried a lot of things and couldnt find a working way. Thank you!
Here is my code
public System.Drawing.Bitmap CopyDataToBitmap(byte[] data)
{
//Here create the Bitmap to the know height, width and format
System.Drawing.Bitmap bmp = new System.Drawing.Bitmap((int)2592, (int)1936, System.Drawing.Imaging.PixelFormat.Format24bppRgb);
//Create a BitmapData and Lock all pixels to be written
System.Drawing.Imaging.BitmapData bmpData = bmp.LockBits(
new System.Drawing.Rectangle(0, 0, bmp.Width, bmp.Height),
System.Drawing.Imaging.ImageLockMode.WriteOnly, bmp.PixelFormat);
//Copy the data from the byte array into BitmapData.Scan0
Marshal.Copy(data, 0, bmpData.Scan0, data.Length);
//Unlock the pixels
bmp.UnlockBits(bmpData);
//Return the bitmap
return bmp;
}
public async void ListenVideo()
{
byte[] data = new byte[1024];
IPEndPoint ipep = new IPEndPoint(IPAddress.Any, 11111);
UdpClient newsock = new UdpClient(ipep);
IPEndPoint sender = new IPEndPoint(IPAddress.Any, 11111);
data = newsock.Receive(ref sender);
string message = Encoding.UTF8.GetString(data, 0, data.Length);
while (true)
{
data = newsock.Receive(ref sender);
message = Encoding.UTF8.GetString(data, 0, data.Length);
MemoryStream stream = new MemoryStream(data);
panel1.BackgroundImage = CopyDataToBitmap(data);
await Task.Delay(2000);
}
}

H.264 is encoding for elementary video stream, it is not an encoding for a separate image.
This means that decoding process involves a setup for video encoding. There is no single function (at least it is not supposed to work this way) to take bitstream and output a video frame.
You would typically setup stream decoding context, then feed input and pull output when it is ready.
Windows comes with a stock H.264 encoder and a typical API to consume such decoding services is Media Foundation. It is not the API for .NET so hence Media Foundation with C#
There is also Media Foundation .NET project with C# wrappers over native Media Foundation API.

WCF: using memorystream to transfer bitmaps in C#

I am using WCF for communication between client and server program on the same machine. Earlier I used Bitmaps to transfer video frames to the client. It was successful with WCF, but I only have 6 megabytes / sec LAN. Transferring 1.2Mb every second would block the LAN. I decided to JPEG encode the captured Bitmaps from the camera. It is possible to save a JPEG on the disk from a bitmap, or to a Memorystream. I created a
[MessageContract]
public class RemoteResponse
{
[MessageBodyMember(Order = 1)]
public System.IO.MemoryStream jpegImage;
}
I return this back from the (I)LiveImageService.
As I get it out in the client: RemoteResponse ret = client.GetLiveImage();
MemoryStream returnedImage = returnedResponse.jpegImage;
returnedImage.Position = 0L;
returnedImage.Seek(0, SeekOrigin.Begin);
args.image = returnedImage;
I do this in the event handler:
args.image.Position = 0L;
args.image.Seek(0, SeekOrigin.Begin);
//bitmap = new Bitmap(args.image);
bitmap = (Bitmap)Image.FromStream(args.image);
It gives me a "parameter is not valid" exception.
I have read that this is a buggy solution of Microsoft. Actually the
retreived stream is believed to be 21 bytes long because there is a trailing
zero \0 at the end of the leading "System.Drawing.Bitmap"\0. That is why the
processing of the image is stopped.
I have read a solution that avoids this:
byte[] arr = PdfReader.FlateDecode(PdfReader.GetStreamBytesRaw((PRStream)obj), true);
System.Drawing.Bitmap bmp = new System.Drawing.Bitmap(width, height, pixelFormat);
System.Drawing.Imaging.BitmapData bmd = bmp.LockBits(new System.Drawing.Rectangle(0, 0, width, height), System.Drawing.Imaging.ImageLockMode.WriteOnly, pixelFormat);
Marshal.Copy(arr, 0, bmd.Scan0, arr.Length);
bmp.UnlockBits(bmd);
using (MemoryStream ms = new MemoryStream())
{
bmp.Save(ms, System.Drawing.Imaging.ImageFormat.Png); // here it is the trick
arr = ms.GetBuffer(); // now I replace my incorrect image format bytes for the correct ones
System.Drawing.Image image = System.Drawing.Image.FromStream(ms);
image.Save(path, System.Drawing.Imaging.ImageFormat.Png);
}
How does the last arr = ... have any effect? It is not used anymore later?
So I don't understand.
update:
On a Microsoft website: https://social.msdn.microsoft.com/Forums/vstudio/en-US/f9e39595-04ca-42ae-a353-eb1a08602631/resolved-creating-image-file-parameter-is-not-valid?forum=netfxbcl
I rewrote the example so:
public static Bitmap DecodeJPEGFromMemoryStream(MemoryStream stream, int width, int height, PixelFormat pixelFormat)
{
string content = System.Text.Encoding.UTF8.GetString(stream.ToArray());
byte[] arr = Encoding.UTF8.GetBytes(content);
//byte[] arr = PdfReader.FlateDecode(PdfReader.GetStreamBytesRaw((PRStream)obj), true);
System.Drawing.Bitmap bmp = new System.Drawing.Bitmap(width, height, pixelFormat);
System.Drawing.Imaging.BitmapData bmd = bmp.LockBits(new
System.Drawing.Rectangle(0, 0, width, height),
System.Drawing.Imaging.ImageLockMode.WriteOnly, pixelFormat);
Marshal.Copy(arr, 0, bmd.Scan0, arr.Length);
bmp.UnlockBits(bmd);
using (MemoryStream ms = new MemoryStream())
{
bmp.Save(ms, System.Drawing.Imaging.ImageFormat.Jpeg); // here it is the trick
arr = ms.GetBuffer(); // now I replace my incorrect image format bytes for the correct ones
System.Drawing.Image image = System.Drawing.Image.FromStream(ms);
Bitmap ret = new Bitmap(image);
return ret;
}
}
I can see the image in the Picturebox, but it has only 2 lines, flickering.
If I obscure the camera I see that it darkens. So the data comes through now. But I cannot reconstruct my JPEG live data.
I was able to decode the JPEG data, only with saving it to disk.
So I've got the image without error. I tried to save the image only
temporarily and delete after.
public static Bitmap DecodeJPEGFromMemoryStream(MemoryStream stream, int width, int height, PixelFormat pixelFormat)
{
try
{
if(File.Exists("c:\\imagecache.jpg") == true)
File.Delete("c:\\imagecache.jpg");
//using (FileStream file = new FileStream("c:\\imagecache.jpg", FileMode.CreateNew, System.IO.FileAccess.ReadWrite))
using (FileStream file = File.Open("c:\\imagecache.jpg", FileMode.CreateNew, FileAccess.Write, FileShare.ReadWrite))
{
stream.WriteTo(file);
//JpegBitmapDecoder decoder = new JpegBitmapDecoder(file, BitmapCreateOptions.PreservePixelFormat, BitmapCacheOption.Default);
file.Close();
file.Dispose();
}
System.GC.Collect();
System.GC.WaitForPendingFinalizers();
//BitmapSource bitmapSource = decoder.Frames[0];
Bitmap jpeg;
{
{
FileInfo info = new FileInfo("c:\\imagecache.jpg");
if (info.Length == 0)
{
File.Delete("c:\\imagecache.jpg");
return null;
}
}
System.GC.Collect();
System.GC.WaitForPendingFinalizers();
jpeg = (Bitmap)Image.FromFile("c:\\imagecache.jpg"); //the process cannot access the file...
}
System.GC.Collect();
System.GC.WaitForPendingFinalizers();
if (File.Exists("c:\\imagecache.jpg") == true)
File.Delete("c:\\imagecache.jpg");
return jpeg;
}
catch(Exception ex)
{
return null;
}
}
I get the exception: The process cannot access the file "imagecache.jpg",
because it is being used by another process. Exception comes at the FromFile.
System.GC.Collect();
System.GC.WaitForPendingFinalizers();
Bitmap retImage = (Bitmap)jpeg.Clone();
jpeg.Dispose();
if (File.Exists("c:\\imagecache.jpg") == true)
{
System.IO.FileInfo fi = new System.IO.FileInfo("c:\\imagecache.jpg");
fi.Delete();
}
return retImage;
}
catch (Exception ex)
{
return null;
}
finally
{
stream.Dispose();
}
It is fixed so. Image.FromFile: the File remains locked until the Image is disposed.
So I cloned my Image and disposed the old one.
Before I've seen my Image, now nothing is visible.

Unable to read QR using Zxing

I am using zxing library to generate and decode the QR codes. I my application I am generating QR code dynamically and sending the file containing QR by fax API. If I get this fax message from the api and decode it, Qr code is read successfully, but when I send a scanned copy of this file by fax and then receive and read it I am unable to do that. But if I try to read this file using my mobile Qr application it properly reads the Qr code. I am unable to find a solution how to read this file.
Methods used for encoding:
public static System.Drawing.Image GenerateJSONQrCode(QRJsonFax model)
{
var jsonString = JsonConvert.SerializeObject(model);
//encrypt json string
jsonString = Hugo.BLL.Utilities.EncryptionHelper.EncryptQR(jsonString, FaxSetting.IsUseHashing);
var bw = new ZXing.BarcodeWriter();
var encOptions = new ZXing.Common.EncodingOptions() { Width = 200, Height = 200, Margin = 0 };
bw.Options = encOptions;
bw.Format = ZXing.BarcodeFormat.QR_CODE;
var image = new Bitmap(bw.Write(Compress(jsonString)));
return image;
}
private static string Compress(string text)
{
byte[] buffer = Encoding.UTF8.GetBytes(text);
var ms = new MemoryStream();
using (var zip = new GZipStream(ms, CompressionMode.Compress, true))
{
zip.Write(buffer, 0, buffer.Length);
}
ms.Position = 0;
byte[] compressed = new byte[ms.Length];
ms.Read(compressed, 0, compressed.Length);
byte[] gzBuffer = new byte[compressed.Length + 4];
System.Buffer.BlockCopy(compressed, 0, gzBuffer, 4, compressed.Length);
System.Buffer.BlockCopy(BitConverter.GetBytes(buffer.Length), 0, gzBuffer, 0, 4);
return Convert.ToBase64String(gzBuffer);
}
Methods used for encoding and decoding
public static FaxReceiver.QrFinder DecodeQrCode(string imagePathToDecode)
{
long userId = 0;
Bitmap bitmapImage = (Bitmap)Image.FromFile(imagePathToDecode);
ZXing.BarcodeReader barcodeReader = new BarcodeReader() { AutoRotate = true, TryHarder = true }; ;
Result decode = barcodeReader.Decode(bitmapImage);
var scanResult = string.Empty;
if (decode != null)
{
scanResult= Decompress(decode.Text);
}
if (!string.IsNullOrWhiteSpace(scanResult))
{
//decrypt Qr received
var decryptedString = DecryptionHelper.Decrypt(scanResult, FaxSetting.IsUseHashing);
//deserialize JSON received
var resultJson = JsonConvert.DeserializeObject<QRJsonFax>(decryptedString);
if (resultJson != null)
{
long.TryParse(resultJson.UserID.ToString(), out userId);
return new QrFinder()
{
FilePath = imagePathToDecode,
UserId = userId,
PageNo = 0,
DataSourceID = resultJson.DataSourceID,
InboundFaxTypeID = resultJson.InboundFaxTypeID
};
}
}
return null;
}
private static string Decompress(string compressedText)
{
byte[] gzBuffer = Convert.FromBase64String(compressedText);
using (var ms = new MemoryStream())
{
int msgLength = BitConverter.ToInt32(gzBuffer, 0);
ms.Write(gzBuffer, 4, gzBuffer.Length - 4);
byte[] buffer = new byte[msgLength];
ms.Position = 0;
using (var zip = new GZipStream(ms, CompressionMode.Decompress))
{
zip.Read(buffer, 0, buffer.Length);
}
return Encoding.UTF8.GetString(buffer);
}
}
File containing Qr code

The problem is that the QR Decoder is getting confused by the gaps between the pixels in your faxed image. If we zoom into a corner of it, this is what we see.
The scanner is looking for solid black squares to identify the QR code.
If we shrink the image by 50%, it becomes readable.
See for yourself at http://zxing.org/w/decode?u=http%3A%2F%2Fi.stack.imgur.com%2FSCYsd.png
I would suggest that after receiving the faxed image, you should either shrink it, or apply a filter to ensure that the QR codes are solid black. You could also look at sending it at a smaller resolution to see if that helps.

How to set length of byteBuffer for incoming data so it is enough big?

I am sending jpg via tcp:
On the sender side we got:
private void sendResponse() {
BitmapImage bmi = new BitmapImage(new Uri(#"C:\a.jpg", UriKind.Absolute));
byte[] data;
JpegBitmapEncoder encoder = new JpegBitmapEncoder();
encoder.Frames.Add(BitmapFrame.Create(bmi));
using (MemoryStream ms = new MemoryStream()) {
encoder.Save(ms);
data = ms.ToArray();
}
clientStream.Write(data, 0, data.Length);
}
but how should I set buffer size on the other side(receiver side)?
I got buffer set for 4096 bytes which is obviously wrong:
public string Receive() {
string response = "Operation Timeout";
if (connection != null) {
connectArgs.RemoteEndPoint = connection.RemoteEndPoint;
connectArgs.SetBuffer(new Byte[4096], 0, 4096);
connectArgs.Completed += new EventHandler<SocketAsyncEventArgs>(delegate(object s, SocketAsyncEventArgs e) {
if (e.SocketError == SocketError.Success) {
BITMAP DECODING
} else {
response = e.SocketError.ToString();
}
_clientDone.Set();
});
_clientDone.Reset();
connection.ReceiveAsync(connectArgs);
_clientDone.WaitOne(3000);
} else {
response = "Socket is not initialized";
}
return response;
}

If you know the exact size of the data, set the buffer to that size. If not, set it to something reasonable and read from the stream until it's empty.
See this question as an example.

In addition to Sebastian Negraszus. You can add your own data formatting (application protocol).
For example you can first send the size of the data you're trying to transmit. Say, it will be 4 bytes (size of the int type). The other end will know that it has to read 4 bytes from network to know the amount of data to expect from the stream.
After data size is known at the other end - memory allocation should not be a problem.

Parameter is not valid - error when sending a bitmap via TCP with C#

I have a file transfer program. The program (Client) does following operations to send a bitmap via TCP socket: get screenshot -> grab Bitmap from memory -> convert to stream -> send
MemoryStream Fs = new MemoryStream();
//////////////11111
Bitmap bmp = TakeScreen();
///////////////2222
//Bitmap bmp = new Bitmap(#"C:\temp\001.bmp");
bmp.Save(Fs, ImageFormat.Bmp);
Byte[] buffer = Fs.ToArray();
Fs.Read(buffer, 0, buffer.Length);
TcpClient socket = new TcpClient("192.168.0.131", 1095);
NetworkStream nw = socket.GetStream();
nw.Write(buffer, 0, buffer.Length);
nw.Close();
Fs.Dispose();
socket.Close();
bmp.Dispose();
If I choose to transfer image directly from memory - no errors.
If I try to load Bitmap from file first - getting "Parameter is not valid" error on a server's side.
here is the server side:
NetworkStream Nw = new NetworkStream(handlerSocket.Client);
int thisRead = 0;
int Blocksize = 1024;
Byte[] dataByte = new Byte[Blocksize];
Bitmap screen = getScreen(Nw, dataByte, thisRead, Blocksize);
Nw.Close();
and
private Bitmap getScreen(NetworkStream Nw, Byte[] dataByte, int thisRead, int Blocksize)
{
Bitmap bitmap;
using (var strm = new MemoryStream())
{
while (true)
{
thisRead = Nw.Read(dataByte, 0, Blocksize);
strm.Write(dataByte, 0, thisRead);
if (thisRead == 0)
break;
}
bitmap = new Bitmap(strm); // Error here
}
Bitmap bm3 = new Bitmap(bitmap);
bitmap.Dispose();
return bm3;
}
What's causing this error? I guess it has something to do with the MemoryStream.
edit: simplified the question

You nee to seek to the beginning of the stream before you can create the Bitmap in getScreen.
private Bitmap getScreen(NetworkStream Nw, Byte[] dataByte, int thisRead, int Blocksize)
{
Bitmap bitmap;
using (var strm = new MemoryStream())
{
while (true)
{
thisRead = Nw.Read(dataByte, 0, Blocksize);
strm.Write(dataByte, 0, thisRead);
if (thisRead == 0)
break;
}
stream.Seek(0, SeekOrigin.Begin; // <-- Go Back to beginning of stream
bitmap = new Bitmap(strm); // Error here
}
Bitmap bm3 = new Bitmap(bitmap);
bitmap.Dispose();
return bm3;
}
EDIT
Detailed explanation: After writing the last byte to the stream, the stream's current position is a the end of the stream. Creating a Bitmap from the screen now tries to read the bitmap from the stream, which doesn't work, as there is no more data after the current position (= the end of the stream).
So what you need to do is tell the stream to set the current position back to the beginning of the stream. Then, the bitmap information can be read.