Please help me.
I'm trying to get photo from camera to buffer in order to make image processing later.
I found that I should use MediaCapture element to work with camera, but I didn't find how to save frame into buffer.
You can save frame to bitmap and process it later. Here is the OCR sample code that provided by Microsoft on GitHub:
// Create the video frame to request a SoftwareBitmap preview frame.
var videoFrame = new VideoFrame(BitmapPixelFormat.Bgra8, videoFrameWidth, videoFrameHeight);
// Capture the preview frame.
using (var currentFrame = await mediaCapture.GetPreviewFrameAsync(videoFrame))
{
// Collect the resulting frame.
SoftwareBitmap bitmap = currentFrame.SoftwareBitmap;
...
var ocrResult = await ocrEngine.RecognizeAsync(bitmap);
}
one more example: CaptureElement https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CameraManualControls
Saving the photo:
private async Task TakePhotoAsync()
{
var stream = new InMemoryRandomAccessStream();
try
{
await _mediaCapture.CapturePhotoToStreamAsync(ImageEncodingProperties.CreateJpeg(), stream);
var photoOrientation = ConvertOrientationToPhotoOrientation(GetCameraOrientation());
await ReencodeAndSavePhotoAsync(stream, photoOrientation);
}
catch (Exception ex)
{
}
}
Related
My goal is to use tinyYolov3 model to perform object detection in real-time through the HoloLens. I want to incorporate the model as an ONNX file directly in the project and to compute the predictions inside the HoloLens itself. In order to do so, I am planning to use Windows.media and Windows.AI.MachineLearning libraries as a pipeline between the camera and my predictions.
Following this tutorial, I am able to capture the frames as VideoFrame and I can convert them in ImageFeatureValue to match my input type requirement. My issue now is about the shape requirement. Yolo models need a 3x416x416 frame as input and I can't find any docs online about resizing VideoFrame or ImageFeatureValue.
Thank you very much for your help.
using (var frameReference = CameraFrameReader.TryAcquireLatestFrame())
using (var videoFrame = frameReference?.VideoMediaFrame?.GetVideoFrame())
await ModelHelper.EvaluateVideoFrameAsync(videoFrame).ConfigureAwait(false);
public async Task EvaluateVideoFrameAsync(VideoFrame frame)
{
if (frame != null)
{
try
{
ModelInput inputData = new ModelInput();
inputData.image = ImageFeatureValue.CreateFromVideoFrame(frame);
//TODO: CHANGE SIZE FRAME
var output = await Model.EvaluateAsync(inputData).ConfigureAwait(false);
}
}
}
I have no experience using the Windows Machine Learning API and ImageFeatureValue class. But when I tried to resize frames from the HoloLens, I had to use the SoftwareBitmap instead of VideoFrame. Then, I use BitmapEncoder to resize them, and convert back to VideoFrame:
private async Task<SoftwareBitmap> ResizeBitmap(SoftwareBitmap softwareBitmap, uint width, uint height)
{
using (InMemoryRandomAccessStream stream = new InMemoryRandomAccessStream())
{
BitmapEncoder encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.BmpEncoderId, stream);
encoder.SetSoftwareBitmap(softwareBitmap);
encoder.BitmapTransform.ScaledWidth = width;
encoder.BitmapTransform.ScaledHeight = height;
encoder.BitmapTransform.InterpolationMode = BitmapInterpolationMode.NearestNeighbor;
await encoder.FlushAsync();
BitmapDecoder decoder = await BitmapDecoder.CreateAsync(stream);
return await decoder.GetSoftwareBitmapAsync(softwareBitmap.BitmapPixelFormat, softwareBitmap.BitmapAlphaMode);
}
}
var inputBitmap = frameReference.VideoMediaFrame.SoftwareBitmap;
var outputBitmap = ResizeBitmap(inputBitmap, your_width, your_height);
var outputVideoFrame = VideoFrame.CreateWithSoftwareBitmap(SoftwareBitmap);
I'm trying to get a byte array from the InkCanvas control, but the method i've come up with so far seems a bit long winded.
Currently I use the following:
StorageFolder folder = await Windows.Storage.ApplicationData.Current.LocalFolder.GetFolderAsync("Temp");
StorageFile file = await folder.CreateFileAsync(GenerateString(5)+".zzx", Windows.Storage.CreationCollisionOption.GenerateUniqueName);
using (IRandomAccessStream stream = await file.OpenAsync(FileAccessMode.ReadWrite))
{
await SignatureCanvas.InkPresenter.StrokeContainer.SaveAsync(stream);
var array = await IRandomAccessStreamToByteArray(stream);
}
The custom stream reader is as follows.
private async Task<byte[]> IRandomAccessStreamToByteArray(IRandomAccessStream stream)
{
var reader = new DataReader(stream.GetInputStreamAt(0));
var bytes = new byte[stream.Size];
await reader.LoadAsync((uint)stream.Size);
reader.ReadBytes(bytes);
return bytes;
}
This works, and gives me the byte array that i need, but also leaves me with unwanted images. Was having some access issues due to files still being written when another call wanted to replace the file so decided to go down the multiple images route. Is there a way to skip the image file entirely? It's not too much of an issue to clear out a temp folder, but if it can be avoided that would be preferable.
I had read somewhere already that InkCanvas doesn't support direct to array dumps, so any suggestions would be appreciated!
If you want to save your InkCanvas strokes to bytes Array without create the file, you should be able to use the Win2D.uwp.
To install Win2D.uwp, run the "Install-Package Win2D.uwp" command in the Package Manager Console.
There is a CanvasDrawingSession.DrawInk method to draw a collection of ink strokes. That we should be able to use CanvasBitmap.GetPixelBytes method to get an array of raw byte data for the entire bitmap.
For example:
private byte[] ConvertInkCanvasToByteArray()
{
var canvasStrokes = SignatureCanvas.InkPresenter.StrokeContainer.GetStrokes();
if (canvasStrokes.Count > 0)
{
var width = (int)SignatureCanvas.ActualWidth;
var height = (int)SignatureCanvas.ActualHeight;
var device = CanvasDevice.GetSharedDevice();
device.DeviceLost += DeviceOnDeviceLost;
var renderTarget = new CanvasRenderTarget(device, width, height, 96);
using (var ds = renderTarget.CreateDrawingSession())
{
ds.Clear(Windows.UI.Colors.White);
ds.DrawInk(SignatureCanvas.InkPresenter.StrokeContainer.GetStrokes());
}
return renderTarget.GetPixelBytes();
}
else
{
return null;
}
}
private void DeviceOnDeviceLost(CanvasDevice sender, object args)
{
Debug.WriteLine("DeviceOnDeviceLost");
}
Also if we want to convert the bytes array to image, we should be able to use following code:
WriteableBitmap bitmap = new WriteableBitmap((int)SignatureCanvas.ActualWidth, (int)SignatureCanvas.ActualHeight);
await bitmap.PixelBuffer.AsStream().WriteAsync(mybytes, 0, mybytes.Length);
In WP 8 I used PhotoCamera to make a camera app and to save the image in camera roll I used this method:
private void cam_CaptureImageAvailable(object sender, ContentReadyEventArgs e)
{
string fileName = "photo.jpg";
MediaLibrary library = new MediaLibrary();
library.SavePictureToCameraRoll(fileName, e.ImageStream);
}
In WPSL 8.1 I use MediaCapture and I use the same style to save image in camera roll but I don't know how to retrieve ImageStream from MediaCapture like in e.ImageStream. I am open to suggestions even with other programming style for saving to camera roll.
var file = await Windows.Storage.KnownFolders.PicturesLibrary.CreateFileAsync(IMAGECAPTURE_FILENAME, Windows.Storage.CreationCollisionOption.ReplaceExisting);
await _exceptionHandler.Run(async () =>
{
await _mediaCapture.CapturePhotoToStorageFileAsync(_imageEncodingProperties, file);
var photoStream = await file.OpenAsync(Windows.Storage.FileAccessMode.Read);
await bitmap.SetSourceAsync(photoStream);
});
The above is taken from a UWP app to save an image to storage, then read it from disk as a stream. I've never been able to capture the image directly as a stream.
Can any of you provide an actual working sample of how to take and save a photo using the MediaCapture element. I've tried looking for an actual solution in MSDN but none of those explanations or code actually describe the process in a simple way.
I need to take a picture and save it to my library (i need to show the correct preview for this), however right now it is rotated 90 degrees and i can't adjust it. I've tried setting the rotation of the video preview and it works for the preview however when i do this the aspect ratio its all wrong and the saved image its not correct.
The examples from channel 9 kind of suck too. I just need a simple implementation...
Im using a Runtime app NOT a silverlight app for Windows Phone 8.1.
I have had the same issue, SetRecordRotation doesn't work for me. I found workaround - take photo and rotate an image, it works great. I use method like that:
private async void CapturePhoto()
{
string photoPath = string.Empty;
ImageEncodingProperties format = ImageEncodingProperties.CreateJpeg();
using (var imageStream = new InMemoryRandomAccessStream())
{
await MediaCapture.CapturePhotoToStreamAsync(format, imageStream);
BitmapDecoder dec = await BitmapDecoder.CreateAsync(imageStream);
BitmapEncoder enc = await BitmapEncoder.CreateForTranscodingAsync(imageStream, dec);
enc.BitmapTransform.Rotation = BitmapRotation.Clockwise90Degrees;
await enc.FlushAsync();
StorageFolder folder = ApplicationData.Current.LocalFolder;
StorageFile capturefile = await folder.CreateFileAsync("photo.jpg", CreationCollisionOption.GenerateUniqueName);
photoPath = capturefile.Name;
using (var fileStream = await capturefile.OpenAsync(FileAccessMode.ReadWrite))
{
try
{
await RandomAccessStream.CopyAsync(imageStream, fileStream);
}
catch {}
}
}
}
I modified sample of code from article How to capture a photo in your Windows Phone 8.1 Runtime app by Marco Siccardi
http://dotnet.dzone.com/articles/how-capture-photo-your-windows-0
There are two samples posted on the Microsoft github page that are relevant, although they target Windows 10. Still, the APIs should work on 8/8.1.
GetPreviewFrame: This sample will not lock the page rotation, and apply a corrective rotation to the preview stream. It does not use SetPreviewRotation, as that method is more resource-heavy than using the metadata approach. This sample doesn't capture photos (just preview frames)
UniversalCameraSample: This one does capture photos, and supports portrait and landscape orientations. Here is the relevant part:
var stream = new InMemoryRandomAccessStream();
try
{
await _mediaCapture.CapturePhotoToStreamAsync(ImageEncodingProperties.CreateJpeg(), stream);
var photoOrientation = ConvertOrientationToPhotoOrientation(GetCameraOrientation());
await ReencodeAndSavePhotoAsync(stream, photoOrientation);
}
catch (Exception ex)
{
Debug.WriteLine("Exception when taking a photo: {0}", ex.ToString());
}
With:
private static async Task ReencodeAndSavePhotoAsync(IRandomAccessStream stream, PhotoOrientation photoOrientation)
{
using (var inputStream = stream)
{
var decoder = await BitmapDecoder.CreateAsync(inputStream);
var file = await KnownFolders.PicturesLibrary.CreateFileAsync("SimplePhoto.jpeg", CreationCollisionOption.GenerateUniqueName);
using (var outputStream = await file.OpenAsync(FileAccessMode.ReadWrite))
{
var encoder = await BitmapEncoder.CreateForTranscodingAsync(outputStream, decoder);
var properties = new BitmapPropertySet { { "System.Photo.Orientation", new BitmapTypedValue(photoOrientation, PropertyType.UInt16) } };
await encoder.BitmapProperties.SetPropertiesAsync(properties);
await encoder.FlushAsync();
}
}
}
Have a closer look at the sample to see how to get the orientation of the camera in the first place (a call to it is being made in the first snippet I posted).
Or, if you prefer a video, you can watch the camera session from the recent //build/ conference, which includes a little bit of a walkthrough through some camera samples.
you can change the aspect ratio for your video preview & captured photo by setting in the MediaCapture.VideoDeviceController.
Also, you can set your video preview upright by using the following code.
MediaCapture.SetPreviewRotation(VideoRotation.Clockwise90Degrees);
I have answered a similar questions in the another post in the link below. Hope it helps.
https://stackoverflow.com/a/29875992/4672579
I am trying to develop a simple Windows 8 Metro app which simply downloads an image file from a given URL (say http://sample.com/foo.jpg) and then save it to Pictures Library.
I have an image control in the UI to display the downloaded image.
I'm also facing difficulty in setting the image source for the image control to the newly downloaded image (actually I'm not even able to download it).
Also, is it possible to store the image file in a particular folder in the Pictures library (if it doesn't exist, then the app should create it)?
I'm really stuck here.
Please help me.
Here's some rough code that I believe accomplishes what you want. It assumes you have two image controls (Image1 and Image2) and that you have the Pictures Library capability checked in the manifest. Take a look at the XAML images sample as well
Uri uri = new Uri("http://www.picsimages.net/photo/lebron-james/lebron-james_1312647633.jpg");
var fileName = Guid.NewGuid().ToString() + ".jpg";
// download pic
var bitmapImage = new BitmapImage();
var httpClient = new HttpClient();
var httpResponse = await httpClient.GetAsync(uri);
byte[] b = await httpResponse.Content.ReadAsByteArrayAsync();
// create a new in memory stream and datawriter
using (var stream = new InMemoryRandomAccessStream())
{
using (DataWriter dw = new DataWriter(stream))
{
// write the raw bytes and store
dw.WriteBytes(b);
await dw.StoreAsync();
// set the image source
stream.Seek(0);
bitmapImage.SetSource(stream);
// set image in first control
Image1.Source = bitmapImage;
// write to pictures library
var storageFile = await KnownFolders.PicturesLibrary.CreateFileAsync(
fileName,
CreationCollisionOption.ReplaceExisting);
using (var storageStream = await storageFile.OpenAsync(FileAccessMode.ReadWrite))
{
await RandomAccessStream.CopyAndCloseAsync(stream.GetInputStreamAt(0), storageStream.GetOutputStreamAt(0));
}
}
}
// read from pictures library
var pictureFile = await KnownFolders.PicturesLibrary.GetFileAsync(fileName);
using ( var pictureStream = await pictureFile.OpenAsync(FileAccessMode.Read) )
{
bitmapImage.SetSource(pictureStream);
}
Image2.Source = bitmapImage;
}