Develop Reference

How to resize VideoFrame or ImageFeatureValue to specific size to match input shape requirements?

My goal is to use tinyYolov3 model to perform object detection in real-time through the HoloLens. I want to incorporate the model as an ONNX file directly in the project and to compute the predictions inside the HoloLens itself. In order to do so, I am planning to use Windows.media and Windows.AI.MachineLearning libraries as a pipeline between the camera and my predictions.
Following this tutorial, I am able to capture the frames as VideoFrame and I can convert them in ImageFeatureValue to match my input type requirement. My issue now is about the shape requirement. Yolo models need a 3x416x416 frame as input and I can't find any docs online about resizing VideoFrame or ImageFeatureValue.
Thank you very much for your help.
using (var frameReference = CameraFrameReader.TryAcquireLatestFrame())
using (var videoFrame = frameReference?.VideoMediaFrame?.GetVideoFrame())
await ModelHelper.EvaluateVideoFrameAsync(videoFrame).ConfigureAwait(false);
public async Task EvaluateVideoFrameAsync(VideoFrame frame)
{
if (frame != null)
{
try
{
ModelInput inputData = new ModelInput();
inputData.image = ImageFeatureValue.CreateFromVideoFrame(frame);
//TODO: CHANGE SIZE FRAME
var output = await Model.EvaluateAsync(inputData).ConfigureAwait(false);
}
}
}

I have no experience using the Windows Machine Learning API and ImageFeatureValue class. But when I tried to resize frames from the HoloLens, I had to use the SoftwareBitmap instead of VideoFrame. Then, I use BitmapEncoder to resize them, and convert back to VideoFrame:
private async Task<SoftwareBitmap> ResizeBitmap(SoftwareBitmap softwareBitmap, uint width, uint height)
{
using (InMemoryRandomAccessStream stream = new InMemoryRandomAccessStream())
{
BitmapEncoder encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.BmpEncoderId, stream);
encoder.SetSoftwareBitmap(softwareBitmap);
encoder.BitmapTransform.ScaledWidth = width;
encoder.BitmapTransform.ScaledHeight = height;
encoder.BitmapTransform.InterpolationMode = BitmapInterpolationMode.NearestNeighbor;
await encoder.FlushAsync();
BitmapDecoder decoder = await BitmapDecoder.CreateAsync(stream);
return await decoder.GetSoftwareBitmapAsync(softwareBitmap.BitmapPixelFormat, softwareBitmap.BitmapAlphaMode);
}
}
var inputBitmap = frameReference.VideoMediaFrame.SoftwareBitmap;
var outputBitmap = ResizeBitmap(inputBitmap, your_width, your_height);
var outputVideoFrame = VideoFrame.CreateWithSoftwareBitmap(SoftwareBitmap);

Convert a System.Drawing.Bitmap to Windows.Graphics.Imaging.SoftwareBitmap

I have a WPF project and captured an image from a usb camera to a System.Drawing.Bitmap (I also can capture System.Windows.Media.Imaging.BitmapSource) and need to convert it to a Windows.Graphics.Imaging.SoftwareBitmap to make a "VideoFrame" to compare to an Onnx model.
The camera driver is a .net assembly and will not bind to a uwp project. I've tried creating a .net standard assembly to bridge the gap with no success. I simply need a bitmap converted to a SoftwareBitmap. Please help!
I'm using this code for the basis of the compassion of the Bitmap image from the camera - https://github.com/Azure-Samples/cognitive-services-onnx12-customvision-sample

There is no direct conversion. You'll need to extract the image data from the System.Drawing.Bitmap and then create the new SoftwareBitmap from that data.
For example, you could use Save(Stream, ImageFormat) method to save this image to the specified stream in the specified format.
Then, you could try to call BitmapDecoder.CreateAsync method to create the decoder from the stream.
After that you could call GetSoftwareBitmapAsync to get a SoftwareBitmap object.
The following is a simple code sample:
Bitmap bitmap = getyourbitmap();
using (var stream = new Windows.Storage.Streams.InMemoryRandomAccessStream())
{
bitmap.Save(stream.AsStream(),ImageFormat.Jpeg);//choose the specific image format by your own bitmap source
Windows.Graphics.Imaging.BitmapDecoder decoder = await Windows.Graphics.Imaging.BitmapDecoder.CreateAsync(stream);
SoftwareBitmap softwareBitmap = await decoder.GetSoftwareBitmapAsync();
}

I found that you can use Windows.Security.Cryptography to create a IBuffer from the image array bytes. Then you can copy the IBuffer to the SoftwareBitmap.
using Windows.Security.Cryptography;
IBuffer buffer = CryptographicBuffer.CreateFromByteArray(ImageByteArray);
SoftwareBitmap softwareBitmap = new SoftwareBitmap(BitmapPixelFormat.Gray8, 800, 600);
softwareBitmap.CopyFromBuffer(buffer);
VideoFrame inputImage = VideoFrame.CreateWithSoftwareBitmap(softwareBitmap);

This worked for me:
Bitmap bitmap = ...;
var memoryStream = new MemoryStream();
using (var graphics = Graphics.FromImage(bitmap))
{
bitmap.Save(memoryStream, ImageFormat.Bmp);
}
var decoder = await Windows.Graphics.Imaging.BitmapDecoder.CreateAsync(stream);
var bitmap = await decoder.GetSoftwareBitmapAsync();

Load Image from Stream

Is there any way of converting Stream to Image?
I tried Bitmap, but it states that I don't have System.Drawing... so I tried this:
var bitMap = new BitmapImage();
bitMap.SetSourceAsync(stream);
image.Source = bitMap;
EDIT:
I am trying to build UWP app + using VS 2015.
2 - It just states that System.Drawing does not exist in the namespace.
EDIT2:
Ok, I might have explained it wrong. The idea is: I have an Image, and I want to change its source to something different and then for it to reload, so I can see the image.
The image is effectively a "Stream", so I assume I need to convert it to Bitmap and then load somehow.
EDIT3:
Ok, so I think it will be easier to describe and then use the code above:
There is a picture box and I am using:
var stream = new InMemoryRandomAccessStream();
await mediaCapture.CapturePhotoToStreamAsync(ImageEncodingProperties.CreateJpeg(), stream);
Now I would like this Captured Photo to be displayed as an Image. ( Image "box" is created at the start, so the idea is to change source).

Right, so I managed to fix it:
var bitMap = new BitmapImage();
stream.seek(0); // LINE ADDED
bitMap.SetSourceAsync(stream);
image.Source = bitMap;
I turned out that the error that was being produced was : "The component cannot be found.", so I managed to fix it by using this trick.

I am not sure if this is what you are looking for but if you would like to use stream with BitMapImage you should use:
var image = new BitmapImage();
await image.SetSourceAsync(stream);
For instance when you have your photo stored as a byte[] array you can use the stream to convert it to image:
using (InMemoryRandomAccessStream stream = new InMemoryRandomAccessStream())
{
DataWriter writer = new DataWriter(stream.GetOutputStreamAt(0))
writer.WriteBytes(<<here your byte[] array>>);
await writer.StoreAsync();
var image = new BitmapImage();
await image.SetSourceAsync(stream);
}
Is that what you need?

Crop and save image (UIElement) in Windows Store App using c# with

I already can take a print screen from the actual content of my application.
I choose az UIElement (e.g. a grid), and I render it into a bmp file.
But how can I crop this image as i feel?
The code is below works, just the cropping missing. I work for windows 8.1.
public async void SaveVisualElementToFile(UIElement element, StorageFile file)
{
var renderTargetBitmap = new RenderTargetBitmap();
await renderTargetBitmap.RenderAsync(element);
var pixels = await renderTargetBitmap.GetPixelsAsync();
using (IRandomAccessStream stream = await
file.OpenAsync(FileAccessMode.ReadWrite))
{
var encoder = await BitmapEncoder.CreateAsync(
BitmapEncoder.JpegEncoderId, stream);
byte[] bytes = pixels.ToArray();
encoder.SetPixelData(BitmapPixelFormat.Bgra8,
BitmapAlphaMode.Ignore,
(uint)renderTargetBitmap.PixelWidth,
(uint)renderTargetBitmap.PixelHeight,
96, 96, bytes);
await encoder.FlushAsync();
}
}

There are a couple of ways to do this. There's the traditional way as espoused by MSFT themself. You can also use some extensions such as WinRTXamlToolkit and WriteableBitmapEx. The latter two make it quite easy. Check in their source codes on Codeplex for their sample applications, which will have samples for how to use cropping.

Download and Save image in Pictures Library through Windows 8 Metro XAML App

I am trying to develop a simple Windows 8 Metro app which simply downloads an image file from a given URL (say http://sample.com/foo.jpg) and then save it to Pictures Library.
I have an image control in the UI to display the downloaded image.
I'm also facing difficulty in setting the image source for the image control to the newly downloaded image (actually I'm not even able to download it).
Also, is it possible to store the image file in a particular folder in the Pictures library (if it doesn't exist, then the app should create it)?
I'm really stuck here.
Please help me.

Here's some rough code that I believe accomplishes what you want. It assumes you have two image controls (Image1 and Image2) and that you have the Pictures Library capability checked in the manifest. Take a look at the XAML images sample as well
Uri uri = new Uri("http://www.picsimages.net/photo/lebron-james/lebron-james_1312647633.jpg");
var fileName = Guid.NewGuid().ToString() + ".jpg";
// download pic
var bitmapImage = new BitmapImage();
var httpClient = new HttpClient();
var httpResponse = await httpClient.GetAsync(uri);
byte[] b = await httpResponse.Content.ReadAsByteArrayAsync();
// create a new in memory stream and datawriter
using (var stream = new InMemoryRandomAccessStream())
{
using (DataWriter dw = new DataWriter(stream))
{
// write the raw bytes and store
dw.WriteBytes(b);
await dw.StoreAsync();
// set the image source
stream.Seek(0);
bitmapImage.SetSource(stream);
// set image in first control
Image1.Source = bitmapImage;
// write to pictures library
var storageFile = await KnownFolders.PicturesLibrary.CreateFileAsync(
fileName,
CreationCollisionOption.ReplaceExisting);
using (var storageStream = await storageFile.OpenAsync(FileAccessMode.ReadWrite))
{
await RandomAccessStream.CopyAndCloseAsync(stream.GetInputStreamAt(0), storageStream.GetOutputStreamAt(0));
}
}
}
// read from pictures library
var pictureFile = await KnownFolders.PicturesLibrary.GetFileAsync(fileName);
using ( var pictureStream = await pictureFile.OpenAsync(FileAccessMode.Read) )
{
bitmapImage.SetSource(pictureStream);
}
Image2.Source = bitmapImage;
}