I'm trying to get a byte array from the InkCanvas control, but the method i've come up with so far seems a bit long winded.
Currently I use the following:
StorageFolder folder = await Windows.Storage.ApplicationData.Current.LocalFolder.GetFolderAsync("Temp");
StorageFile file = await folder.CreateFileAsync(GenerateString(5)+".zzx", Windows.Storage.CreationCollisionOption.GenerateUniqueName);
using (IRandomAccessStream stream = await file.OpenAsync(FileAccessMode.ReadWrite))
{
await SignatureCanvas.InkPresenter.StrokeContainer.SaveAsync(stream);
var array = await IRandomAccessStreamToByteArray(stream);
}
The custom stream reader is as follows.
private async Task<byte[]> IRandomAccessStreamToByteArray(IRandomAccessStream stream)
{
var reader = new DataReader(stream.GetInputStreamAt(0));
var bytes = new byte[stream.Size];
await reader.LoadAsync((uint)stream.Size);
reader.ReadBytes(bytes);
return bytes;
}
This works, and gives me the byte array that i need, but also leaves me with unwanted images. Was having some access issues due to files still being written when another call wanted to replace the file so decided to go down the multiple images route. Is there a way to skip the image file entirely? It's not too much of an issue to clear out a temp folder, but if it can be avoided that would be preferable.
I had read somewhere already that InkCanvas doesn't support direct to array dumps, so any suggestions would be appreciated!
If you want to save your InkCanvas strokes to bytes Array without create the file, you should be able to use the Win2D.uwp.
To install Win2D.uwp, run the "Install-Package Win2D.uwp" command in the Package Manager Console.
There is a CanvasDrawingSession.DrawInk method to draw a collection of ink strokes. That we should be able to use CanvasBitmap.GetPixelBytes method to get an array of raw byte data for the entire bitmap.
For example:
private byte[] ConvertInkCanvasToByteArray()
{
var canvasStrokes = SignatureCanvas.InkPresenter.StrokeContainer.GetStrokes();
if (canvasStrokes.Count > 0)
{
var width = (int)SignatureCanvas.ActualWidth;
var height = (int)SignatureCanvas.ActualHeight;
var device = CanvasDevice.GetSharedDevice();
device.DeviceLost += DeviceOnDeviceLost;
var renderTarget = new CanvasRenderTarget(device, width, height, 96);
using (var ds = renderTarget.CreateDrawingSession())
{
ds.Clear(Windows.UI.Colors.White);
ds.DrawInk(SignatureCanvas.InkPresenter.StrokeContainer.GetStrokes());
}
return renderTarget.GetPixelBytes();
}
else
{
return null;
}
}
private void DeviceOnDeviceLost(CanvasDevice sender, object args)
{
Debug.WriteLine("DeviceOnDeviceLost");
}
Also if we want to convert the bytes array to image, we should be able to use following code:
WriteableBitmap bitmap = new WriteableBitmap((int)SignatureCanvas.ActualWidth, (int)SignatureCanvas.ActualHeight);
await bitmap.PixelBuffer.AsStream().WriteAsync(mybytes, 0, mybytes.Length);
Related
I am having the issue of "Canvas Drawing too large bitmaps". After a quick search, I found the following thread, which promptly helped me know what is the issue.
The solution is to put the image in drawable-xxhdpi/ instead of simply drawable/. And here lies the issue: the image is not static, it is imported when I need it. As such, I do not chose where the image ends up stored. It store itself in drawable. Is there 1) A solution to chose which folder to use, or 2) a way to tell it not get the image if it's too heavy?
var file = new SmbFile(path, auth);
try
{
if (file.Exists())
{
// Get readable stream.
var readStream = file.GetInputStream();
//Create reading buffer.
MemoryStream memStream = new MemoryStream();
//Get bytes.
((Stream)readStream).CopyTo(memStream);
var stream1 = new MemoryStream(memStream.ToArray());
if (stream1.Length < 120188100)
{
//Save image
ProductImage = ImageSource.FromStream(() => stream1);
//Dispose readable stream.
readStream.Dispose();
InfoColSpan = 1;
}
else
{
Common.AlertError("Image trop lourde pour l'affichage");
}
}
}
My goal is to use tinyYolov3 model to perform object detection in real-time through the HoloLens. I want to incorporate the model as an ONNX file directly in the project and to compute the predictions inside the HoloLens itself. In order to do so, I am planning to use Windows.media and Windows.AI.MachineLearning libraries as a pipeline between the camera and my predictions.
Following this tutorial, I am able to capture the frames as VideoFrame and I can convert them in ImageFeatureValue to match my input type requirement. My issue now is about the shape requirement. Yolo models need a 3x416x416 frame as input and I can't find any docs online about resizing VideoFrame or ImageFeatureValue.
Thank you very much for your help.
using (var frameReference = CameraFrameReader.TryAcquireLatestFrame())
using (var videoFrame = frameReference?.VideoMediaFrame?.GetVideoFrame())
await ModelHelper.EvaluateVideoFrameAsync(videoFrame).ConfigureAwait(false);
public async Task EvaluateVideoFrameAsync(VideoFrame frame)
{
if (frame != null)
{
try
{
ModelInput inputData = new ModelInput();
inputData.image = ImageFeatureValue.CreateFromVideoFrame(frame);
//TODO: CHANGE SIZE FRAME
var output = await Model.EvaluateAsync(inputData).ConfigureAwait(false);
}
}
}
I have no experience using the Windows Machine Learning API and ImageFeatureValue class. But when I tried to resize frames from the HoloLens, I had to use the SoftwareBitmap instead of VideoFrame. Then, I use BitmapEncoder to resize them, and convert back to VideoFrame:
private async Task<SoftwareBitmap> ResizeBitmap(SoftwareBitmap softwareBitmap, uint width, uint height)
{
using (InMemoryRandomAccessStream stream = new InMemoryRandomAccessStream())
{
BitmapEncoder encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.BmpEncoderId, stream);
encoder.SetSoftwareBitmap(softwareBitmap);
encoder.BitmapTransform.ScaledWidth = width;
encoder.BitmapTransform.ScaledHeight = height;
encoder.BitmapTransform.InterpolationMode = BitmapInterpolationMode.NearestNeighbor;
await encoder.FlushAsync();
BitmapDecoder decoder = await BitmapDecoder.CreateAsync(stream);
return await decoder.GetSoftwareBitmapAsync(softwareBitmap.BitmapPixelFormat, softwareBitmap.BitmapAlphaMode);
}
}
var inputBitmap = frameReference.VideoMediaFrame.SoftwareBitmap;
var outputBitmap = ResizeBitmap(inputBitmap, your_width, your_height);
var outputVideoFrame = VideoFrame.CreateWithSoftwareBitmap(SoftwareBitmap);
I've been struggling with this implementation for a few hours now and can't seem to find any solutions wherever I look (SO, Xamarin Forums, Google etc)...
In this current scenario I have a few images in .Droid.Resources.Drawable which I wish to access and convert into a byte[] from my shared code. This is due to the fact that I wish to test the full span of my CRUD functionality on a REST API I've set up as an end-point for our server.
The images show up fine in the application, but for some reason I simply can't seem to warp my head around the process of converting these images to a byte[] in Xamarin. I've done it countless times in 'normal' C#...
Sorry if the code is a bit messy, but I'm sure you get the idea.
I want to get an image from .Droid storage (will be ported for iOS later)
Convert said image into a byte[]
Send that byte[] representation to my API.
In the code's current state I'm getting this error:
C#: An instance of an abstract class can not be created
Where I'm attempting to create a new Stream (new Stream(sauce))
The below example is based on snippets found here and full credit goes to Sten and Vincent.
/*
* Takes an arbitrary string as a token, updates a record with dummy data and a placeholder_image.
*/
public async Task<string> PostUpdateFoundation(string arbitrary, Image img)
{
ImageSource sauce = ImageSource.FromFile("abc.png");
byte[] byte_img = FromStreamToByte(new Stream(sauce)); //error occurs here
Debug.WriteLine("I'm in!");
var client = new System.Net.Http.HttpClient();
client.DefaultRequestHeaders.Add("Accept", "application/json");
var content = new StringContent(arbitrary);
var response = await client.PostAsync(String.Format("http://some.api.to.test.com?s={0}&img={1}", arbitrary, byte_img), content);
var result = response.Content.ReadAsStringAsync().Result;
return result;
}
/*
* Attempts to convert an stream (based on image source) into a byte[].
*/
public static byte[] FromStreamToByte (Stream input)
{
using (MemoryStream ms = new MemoryStream())
{
input.CopyTo(ms);
return ms.ToArray();
}
}
Try using Plugin.Media
byte BImageSource = ReadFully(file.GetStream());
var bytes = new byte[file.GetStream().Length]; //file is from the plugin and contains your image
file.GetStream().Position = 0;
file.GetStream().Read(bytes, 0, (int)file.GetStream().Length);
BImageSource = ReadFully(file.GetStream()); //BImageSource is your resource in bytes
byte[] ReadFully(Stream input)
{
using (MemoryStream ms = new MemoryStream())
{
input.CopyTo(ms);
return ms.ToArray();
}
}
Hope this helps!
I have been trying to convert a captured VideoFrame object to a byte array with little success. It is clear from the documentation that each frame can be saved to a SoftwareBitmap object, e.g.
SoftwareBitmap bitmap = frame.SoftwareBitmap;
I have been able to save this bitmap as an image but I would like to obtain it's data and store it in a byte array. Many SO questions already deal with this but the SoftwareBitmap belongs to the Windows.Graphics.Imaging namespace (not the more typical Xaml.Controls.Image which the other SO posts address, such as this one) so traditional methods like image.Save() are unavailable.
It seems that each SoftwareBitmap has a CopyToBuffer() method but the documentation on this is very terse with regards to how to actually use this. And I'm also not sure if that's the right way to go?
Edit:
Using Alan's recommendation below I've managed to get this working. I'm not sure if it's useful but here's the code I used if anyone else comes across this:
private void convertFrameToByteArray(SoftwareBitmap bitmap)
{
byte[] bytes;
WriteableBitmap newBitmap = new WriteableBitmap(bitmap.PixelWidth, bitmap.PixelHeight);
bitmap.CopyToBuffer(newBitmap.PixelBuffer);
using (Stream stream = newBitmap.PixelBuffer.AsStream())
using (MemoryStream memoryStream = new MemoryStream())
{
stream.CopyTo(memoryStream);
bytes = memoryStream.ToArray();
}
// do what you want with the acquired bytes
this.videoFramesAsBytes.Add(bytes);
}
By using the CopyToBuffer() method, you can copy pixel data to the PixelBuffer of a WriteableBitmap.
Then I think you can refer to the answer in this question to convert it to byte array.
For anyone looking to access an encoded byte[] array from the SoftwareBitmap (e.g. jpeg):
private async void PlayWithData(SoftwareBitmap softwareBitmap)
{
var data = await EncodedBytes(softwareBitmap, BitmapEncoder.JpegEncoderId);
// todo: save the bytes to a DB, etc
}
private async Task<byte[]> EncodedBytes(SoftwareBitmap soft, Guid encoderId)
{
byte[] array = null;
// First: Use an encoder to copy from SoftwareBitmap to an in-mem stream (FlushAsync)
// Next: Use ReadAsync on the in-mem stream to get byte[] array
using (var ms = new InMemoryRandomAccessStream())
{
BitmapEncoder encoder = await BitmapEncoder.CreateAsync(encoderId, ms);
encoder.SetSoftwareBitmap(soft);
try
{
await encoder.FlushAsync();
}
catch ( Exception ex ){ return new byte[0]; }
array = new byte[ms.Size];
await ms.ReadAsync(array.AsBuffer(), (uint)ms.Size, InputStreamOptions.None);
}
return array;
}
I have spent the last 10-12 hours trying to figure out how to correctly make a downloaded web image smaller in size and pixels in C# in a Windows Store app under development.
Whatever I do, I keep getting artifacts on the final images such as a "half picture", gray/same-colored areas and likewise. Like if the stream has not been flushed correctly although I believe to have done so (not done so in the code below since it works without it...)
This is my approach for retrieving the image - this part works, but is included here to make sure all info is here (see code below):
Get URL of image
Use HttpWebRequest to get response
Create stream to get response stream
Create empty StorageFile and open for writing
Copy the response stream to the storage file.
Close everything
From there, I need to do the following:
Determine the size (e.g. using BitmapDecoder)
If the width of the image is above a certain amount (e.g. 700 px), it must be resized.
No matter what, the files are always too big and need to be compressed further
The image need to be saved as a jpg with image quality set to a medium/semi-high setting
I have tried many things including messing pretty much around with BitmapEncoder/BitmapDecoder, but no matter what I am still getting half-processed images.
Can somebody please help me find the correct way to compress and resize images?
My code in the current state:
using (var response = await HttpWebRequest.CreateHttp(internetUri).GetResponseAsync())
{
using (var stream = response.GetResponseStream())
{
var imageFolder = await localFolder.CreateFolderAsync(
CachedImagesFolderEndFolderPath, CreationCollisionOption.OpenIfExists);
string fileName = string.Format("{0}.jpg",
Path.GetFileNameWithoutExtension(Path.GetRandomFileName()));
var file = await imageFolder.CreateFileAsync(fileName,
CreationCollisionOption.ReplaceExisting);
using (var filestream = await file.OpenStreamForWriteAsync())
{
await stream.CopyToAsync(filestream);
}
}
}
The following solution was provided by StefanDK in this edit:
It seems that the problem with my former solution was that I did not properly close the streams and that I did not have the correct settings.
Basically the solution incorporates elements from these articles:
How to resize Image in C# WinRT/winmd?
https://stackoverflow.com/questions/15481126/windows-store-app-resize-bitmapimage-c-sharp
http://msdn.microsoft.com/en-us/library/windows/apps/jj709942.aspx
http://msdn.microsoft.com/en-us/library/windows/apps/hh465076.aspx
From the main part of the code I make these calls for each image that needs downloading, resizing and compressing:
Main code
Note that I am well aware of the "not best practice" in assigning a string value and then setting it again. This is prototype code that has not been fine-tuned yet.
var img = await ArticleStorage.GetLocalImageAsync(src);
img = await ArticleStorage.ResizeAndCompressLocalImage(img);
Source code of the methods in ArticleStorage
public const string CachedImagesFolderFullPath = "ms-appdata:///local/cache/";
public const string CachedImagesFolderEndFolderPath = "cache";
public const string OfflinePhotoImgPath = "ms-appx:///Assets/OfflinePhoto.png";
public const int MaximumColumnWidth = 700;
public static async Task<string> GetLocalImageAsync(string internetUri)
{
if (string.IsNullOrEmpty(internetUri))
{
return null;
}
// Show default image if local folder does not exist
var localFolder = ApplicationData.Current.LocalFolder;
if (localFolder == null)
{
return OfflinePhotoImgPath;
}
// Default to offline photo
string src = OfflinePhotoImgPath;
try
{
using (var response = await HttpWebRequest.CreateHttp(internetUri)
.GetResponseAsync())
{
using (var stream = response.GetResponseStream())
{
// New random filename (e.g. x53fjtje.jpg)
string fileName = string.Format("{0}.jpg",
Path.GetFileNameWithoutExtension(Path.GetRandomFileName()));
var imageFolder = await localFolder.CreateFolderAsync(
CachedImagesFolderEndFolderPath,
CreationCollisionOption.OpenIfExists);
var file = await imageFolder.CreateFileAsync(fileName,
CreationCollisionOption.ReplaceExisting);
// Copy bytes from stream to local file
// without changing any file information
using (var filestream = await file.OpenStreamForWriteAsync())
{
await stream.CopyToAsync(filestream);
// Send back the local path to the image
// (including 'ms-appdata:///local/cache/')
return string.Format(CachedImagesFolderFullPath + "{0}",
fileName);
}
}
}
}
catch (Exception)
{
// Is implicitly handled with the setting
// of the initilized value of src
}
// If not succesfull, return the default offline image
return src;
}
public static async Task<string> ResizeAndCompressLocalImage(string imgSrc)
{
// Remove 'ms-appdata:///local/cache/' from the path ...
string sourcepathShort = imgSrc.Replace(
CachedImagesFolderFullPath,
string.Empty);
// Get the cached images folder
var folder = await ApplicationData.Current
.LocalFolder
.GetFolderAsync(
CachedImagesFolderEndFolderPath);
// Get a new random name (e.g. '555jkdhr5.jpg')
var targetPath = string.Format("{0}.jpg",
Path.GetFileNameWithoutExtension(
Path.GetRandomFileName()));
// Retrieve source and create target file
var sourceFile = await folder.GetFileAsync(sourcepathShort);
var targetFile = await folder.CreateFileAsync(targetPath);
using (var sourceFileStream = await sourceFile.OpenAsync(
Windows.Storage.FileAccessMode.Read))
{
using (var destFileStream = await targetFile.OpenAsync(
FileAccessMode.ReadWrite))
{
// Prepare decoding of the source image
BitmapDecoder decoder = await BitmapDecoder.CreateAsync(
sourceFileStream);
// Find out if image needs resizing
double proportionWidth = (double)decoder.PixelWidth /
LayoutDimensions.MaximumColumnWidth;
double proportionImage = decoder.PixelHeight /
(double)decoder.PixelWidth;
// Get the new sizes of the image whether it is the same or should be resized
var newWidth = proportionWidth > 1 ?
(uint)(MaximumColumnWidth) :
decoder.PixelWidth;
var newHeight = proportionWidth > 1 ?
(uint)(MaximumColumnWidth * proportionImage) :
decoder.PixelHeight;
// Prepare set of properties for the bitmap
BitmapPropertySet propertySet = new BitmapPropertySet();
// Set ImageQuality
BitmapTypedValue qualityValue = new BitmapTypedValue(0.75,
PropertyType.Single);
propertySet.Add("ImageQuality", qualityValue);
//BitmapEncoder enc = await BitmapEncoder.CreateForTranscodingAsync(
destFileStream, decoder);
BitmapEncoder enc = await BitmapEncoder.CreateAsync(
BitmapEncoder.JpegEncoderId,
destFileStream, propertySet);
// Set the new dimensions
enc.BitmapTransform.ScaledHeight = newHeight;
enc.BitmapTransform.ScaledWidth = newWidth;
// Get image data from the source image
PixelDataProvider pixelData = await decoder.GetPixelDataAsync();
// Copy in all pixel data from source to target
enc.SetPixelData(
decoder.BitmapPixelFormat,
decoder.BitmapAlphaMode,
decoder.PixelWidth,
decoder.PixelHeight,
decoder.DpiX,
decoder.DpiY,
pixelData.DetachPixelData()
);
// Make the encoder process the image
await enc.FlushAsync();
// Write everything to the filestream
await destFileStream.FlushAsync();
}
}
try
{
// Delete the source file
await sourceFile.DeleteAsync();
}
catch(Exception)
{
}
// Return the new path
// including "ms-appdata:///local/cache/"
return string.Format(CachedImagesFolderFullPath + "{0}",
targetPath);
}