I have been trying to convert a captured VideoFrame object to a byte array with little success. It is clear from the documentation that each frame can be saved to a SoftwareBitmap object, e.g.
SoftwareBitmap bitmap = frame.SoftwareBitmap;
I have been able to save this bitmap as an image but I would like to obtain it's data and store it in a byte array. Many SO questions already deal with this but the SoftwareBitmap belongs to the Windows.Graphics.Imaging namespace (not the more typical Xaml.Controls.Image which the other SO posts address, such as this one) so traditional methods like image.Save() are unavailable.
It seems that each SoftwareBitmap has a CopyToBuffer() method but the documentation on this is very terse with regards to how to actually use this. And I'm also not sure if that's the right way to go?
Edit:
Using Alan's recommendation below I've managed to get this working. I'm not sure if it's useful but here's the code I used if anyone else comes across this:
private void convertFrameToByteArray(SoftwareBitmap bitmap)
{
byte[] bytes;
WriteableBitmap newBitmap = new WriteableBitmap(bitmap.PixelWidth, bitmap.PixelHeight);
bitmap.CopyToBuffer(newBitmap.PixelBuffer);
using (Stream stream = newBitmap.PixelBuffer.AsStream())
using (MemoryStream memoryStream = new MemoryStream())
{
stream.CopyTo(memoryStream);
bytes = memoryStream.ToArray();
}
// do what you want with the acquired bytes
this.videoFramesAsBytes.Add(bytes);
}
By using the CopyToBuffer() method, you can copy pixel data to the PixelBuffer of a WriteableBitmap.
Then I think you can refer to the answer in this question to convert it to byte array.
For anyone looking to access an encoded byte[] array from the SoftwareBitmap (e.g. jpeg):
private async void PlayWithData(SoftwareBitmap softwareBitmap)
{
var data = await EncodedBytes(softwareBitmap, BitmapEncoder.JpegEncoderId);
// todo: save the bytes to a DB, etc
}
private async Task<byte[]> EncodedBytes(SoftwareBitmap soft, Guid encoderId)
{
byte[] array = null;
// First: Use an encoder to copy from SoftwareBitmap to an in-mem stream (FlushAsync)
// Next: Use ReadAsync on the in-mem stream to get byte[] array
using (var ms = new InMemoryRandomAccessStream())
{
BitmapEncoder encoder = await BitmapEncoder.CreateAsync(encoderId, ms);
encoder.SetSoftwareBitmap(soft);
try
{
await encoder.FlushAsync();
}
catch ( Exception ex ){ return new byte[0]; }
array = new byte[ms.Size];
await ms.ReadAsync(array.AsBuffer(), (uint)ms.Size, InputStreamOptions.None);
}
return array;
}
Related
I'm trying to get a byte array from the InkCanvas control, but the method i've come up with so far seems a bit long winded.
Currently I use the following:
StorageFolder folder = await Windows.Storage.ApplicationData.Current.LocalFolder.GetFolderAsync("Temp");
StorageFile file = await folder.CreateFileAsync(GenerateString(5)+".zzx", Windows.Storage.CreationCollisionOption.GenerateUniqueName);
using (IRandomAccessStream stream = await file.OpenAsync(FileAccessMode.ReadWrite))
{
await SignatureCanvas.InkPresenter.StrokeContainer.SaveAsync(stream);
var array = await IRandomAccessStreamToByteArray(stream);
}
The custom stream reader is as follows.
private async Task<byte[]> IRandomAccessStreamToByteArray(IRandomAccessStream stream)
{
var reader = new DataReader(stream.GetInputStreamAt(0));
var bytes = new byte[stream.Size];
await reader.LoadAsync((uint)stream.Size);
reader.ReadBytes(bytes);
return bytes;
}
This works, and gives me the byte array that i need, but also leaves me with unwanted images. Was having some access issues due to files still being written when another call wanted to replace the file so decided to go down the multiple images route. Is there a way to skip the image file entirely? It's not too much of an issue to clear out a temp folder, but if it can be avoided that would be preferable.
I had read somewhere already that InkCanvas doesn't support direct to array dumps, so any suggestions would be appreciated!
If you want to save your InkCanvas strokes to bytes Array without create the file, you should be able to use the Win2D.uwp.
To install Win2D.uwp, run the "Install-Package Win2D.uwp" command in the Package Manager Console.
There is a CanvasDrawingSession.DrawInk method to draw a collection of ink strokes. That we should be able to use CanvasBitmap.GetPixelBytes method to get an array of raw byte data for the entire bitmap.
For example:
private byte[] ConvertInkCanvasToByteArray()
{
var canvasStrokes = SignatureCanvas.InkPresenter.StrokeContainer.GetStrokes();
if (canvasStrokes.Count > 0)
{
var width = (int)SignatureCanvas.ActualWidth;
var height = (int)SignatureCanvas.ActualHeight;
var device = CanvasDevice.GetSharedDevice();
device.DeviceLost += DeviceOnDeviceLost;
var renderTarget = new CanvasRenderTarget(device, width, height, 96);
using (var ds = renderTarget.CreateDrawingSession())
{
ds.Clear(Windows.UI.Colors.White);
ds.DrawInk(SignatureCanvas.InkPresenter.StrokeContainer.GetStrokes());
}
return renderTarget.GetPixelBytes();
}
else
{
return null;
}
}
private void DeviceOnDeviceLost(CanvasDevice sender, object args)
{
Debug.WriteLine("DeviceOnDeviceLost");
}
Also if we want to convert the bytes array to image, we should be able to use following code:
WriteableBitmap bitmap = new WriteableBitmap((int)SignatureCanvas.ActualWidth, (int)SignatureCanvas.ActualHeight);
await bitmap.PixelBuffer.AsStream().WriteAsync(mybytes, 0, mybytes.Length);
I'm working with Microsoft's OCR library and am having problems converting the BitmapImage to a pixel array.
I'm making this application for Windows Phone 8, and WriteableBitmap.PixelBuffer.ToArray() isn't an option so I have a static function that'll change a normal BitmapImage into a byte array to feed into the OCR engine.
Well, every time I feed it in the application crashes. What's wrong here?
Here is my static class with the bitmap converter
public static class ByteArrayChange
{
public static byte[] ConvertToBytes(this BitmapImage bitmapImage)
{
byte[] data = null;
using (MemoryStream stream = new MemoryStream())
{
WriteableBitmap wBitmap = new WriteableBitmap(bitmapImage);
wBitmap.SaveJpeg(stream, wBitmap.PixelWidth, wBitmap.PixelHeight, 0, 100);
stream.Seek(0, SeekOrigin.Begin);
data = stream.GetBuffer();
}
return data;
}
}
Here is the piece of code in the OCR method that's causing the application to crash.
byte[] pa = ByteArrayChange.ConvertToBytes(bitmap);
//Here Is he problem
var ocrResult = await ocrEngine.RecognizeAsync((uint)bitmap.PixelHeight, (uint)bitmap.PixelWidth, pa);
What am I doing wrong here?
Thanks!
You're saving your image as JPEG, but I'm fairly certain that OCR library accept RGB/BGRA as an input.
So why don't you use Pixels property? It represents image as BGRA array, so the only thing you need is to convert it to byte[] array.
I have a problem converting byte array to InMemoryRandomAccessStream or IRandomAccessStream in windows 8?
This is my code, but It doesn't work :
internal static async Task<InMemoryRandomAccessStream> ConvertTo(byte[] arr)
{
InMemoryRandomAccessStream randomAccessStream = new InMemoryRandomAccessStream();
Stream stream = randomAccessStream.AsStream();
await stream.WriteAsync(arr, 0, arr.Length);
await stream.FlushAsync();
return randomAccessStream;
}
And I created the RandomAccessStreamReference and set the requst datapack in order to share the image to other app
private static async void OnDeferredImageStreamRequestedHandler(DataProviderRequest Request)
{
DataProviderDeferral deferral = Request.GetDeferral();
InMemoryRandomAccessStream stream = await ConvertTo(arr);
RandomAccessStreamReference referenceStream =
RandomAccessStreamReference.CreateFromStream(stream);
Request.SetData(referenceStream);
}
There's no exception but I can't use the result image byte array
I think there's an error when converting byte[] to InMemoryRandomAccessStream, but it doesn't throw an exception.
Anybody know how to implement it?
if anyone knows how to convert the byte array to IRandomAccessStream, it's also appreciated
On Windows 8.1 it's even easier as we added the AsRandomAccessStream extension method:
internal static IRandomAccessStream ConvertTo(byte[] arr)
{
MemoryStream stream = new MemoryStream(arr);
return stream.AsRandomAccessStream();
}
Add the using statement at the top of the document.
using System.Runtime.InteropServices.WindowsRuntime;
internal static async Task<InMemoryRandomAccessStream> ConvertTo(byte[] arr)
{
InMemoryRandomAccessStream randomAccessStream = new InMemoryRandomAccessStream();
await randomAccessStream.WriteAsync(arr.AsBuffer());
randomAccessStream.Seek(0); // Just to be sure.
// I don't think you need to flush here, but if it doesn't work, give it a try.
return randomAccessStream;
}
In one line:
internal static IRandomAccessStream ConvertTo(byte[] arr)
{
return arr.AsBuffer().AsStream().AsRandomAccessStream();
}
I have WriteableBitmap. I know that in variable wp1 exists image, because I saved this picture, and it was all fine. I need to encode the image into a byte[] array.
WriteableBitmap wp1 = new WriteableBitmap(1, 1); ;
wp1.SetSourceAsync(memStream);
using (Stream stream = wp1.PixelBuffer.AsStream())
{
if (stream.CanWrite)
{
byte[] pixelArray = new byte[stream.Length];
await stream.ReadAsync(pixelArray, 0, pixelArray.Length);
}
}
After all the pixelArray is empty. Length of the array pixelArray equals to the length of stream, but all bytes are zero. What should I do?
I think your problem is this line of code:
wp1.SetSourceAsync(memStream);
That's an asynchronous method, so you'll have to wait until it's done before proceeding. Try changing it to:
await wp1.SetSourceAsync(memStream);
If you're using WriteableBitmap Extensions, there's an easier method that does that for you, something like :var pixelDataArray = wp1.ToByteArray();
I have been trying get a stream from a byte array in metro style app using the following code.
InMemoryRandomAccessStream memoryStream = new InMemoryRandomAccessStream();
memoryStream.AsStreamForWrite().Write(byteArray, 0, byteArray.Length);
memoryStream.Seek(0);
It executes with no errors but stream size is zero (0). Can anybody tell me why is its size is zero?
You can use the DataWriter and DataReader classes. For example ...
// put bytes into the stream
var ms = new InMemoryRandomAccessStream();
var dw = new Windows.Storage.Streams.DataWriter(ms);
dw.WriteBytes(new byte[] { 0x00, 0x01, 0x02 });
await dw.StoreAsync();
// read them out
ms.Seek(0);
byte[] ob = new byte[ms.Size];
var dr = new Windows.Storage.Streams.DataReader(ms);
await dr.LoadAsync((uint)ms.Size);
dr.ReadBytes(ob);
You can also use the BinaryWriter/BinaryReader to read and write from and to byte[] and Streams.
private Stream ConvertToStream(byte[] raw)
{
Stream streamOutput = new MemoryStream();
using (BinaryWriter writer = new BinaryWriter(streamOutput))
{
writer.Write(raw);
}
return streamOutput;
}
Another option is to use built in extension methods as Marc Gravell already mentioned:
private Stream ConvertToStream(byte[] raw)
{
return raw.AsBuffer().AsStream();
}
The extension methods are commented with [Security Critical] which may indicate a later change. However, after looking around a bit I couldn't find any additional information on the security code comment.
I know this is a very old question, but I was running into this issue myself today and figured it out so I'll leave this here for others.
I realized that the stream wasn't being written if it was too small. To fix this, I explicitly set the length of the stream like this:
ms.AsStreamForWrite(imageBytes.Length).Write(imageBytes, 0, imageBytes.Length);
That should be all you need.