Display videoframe and RenderTargetBitmap position - c#

I'm trying to crop an image from webcam and display right next to the camera preview.
Cropping an image should be carried with 3 considerations.
Output of cropped image should be in form of VideoFrame
The above output, VideoFrame, needs to be displayed (on XAML)
The target crop image is in the middle of original image
I found RenderTargetBitmap would help me to get an cropped image.
But still I have no idea how to display VideoFrame (without saving an image), set the position where to crop.
I got stuck below...
public async Task<VideoFrame> CroppingImage(Grid grid)
{
RenderTargetBitmap renderBitmap = new RenderTargetBitmap();
await renderBitmap.RenderAsync(grid);
var buffer = await renderBitmap.GetPixelsAsync();
var softwareBitmap = SoftwareBitmap.CreateCopyFromBuffer(buffer, BitmapPixelFormat.Bgra8, renderBitmap.PixelWidth, renderBitmap.PixelHeight, BitmapAlphaMode.Ignore);
buffer = null;
renderBitmap = null;
VideoFrame vf = VideoFrame.CreateWithSoftwareBitmap(softwareBitmap);
await CropAndDisplayInputImageAsync(vf);
return cropped_vf;
}
private async Task CropAndDisplayInputImageAsync(VideoFrame inputVideoFrame)
{
//some cropping algorithm here
//i have a rectangle on a canvas(camera preview is on CaptureElement)
//I know the left top position and width and height but no idea how to use
}
Any help?
This is what i found and done :)
(assume that there is a videoframe which name is croppedface)
croppedFace = new VideoFrame(BitmapPixelFormat.Bgra8, (int)width, (int)height, BitmapAlphaMode.Ignore);
await inputVideoFrame.CopyToAsync(croppedFace, cropBounds, null);
SoftwareBitmap asdf = croppedFace.SoftwareBitmap;
asdf = SoftwareBitmap.Convert(asdf, BitmapPixelFormat.Bgra8, BitmapAlphaMode.Ignore);
var qwer = new SoftwareBitmapSource();
await qwer.SetBitmapAsync(asdf);
CroppedFaceImage.Source = qwer;

But still I have no idea how to display VideoFrame(without saving an image), set the position where to crop.
If you want to show the frame on the xaml, you need to convert the frame to a displayable format and rendering it to the screen. Please check the FrameRender class in the official Camera frames sample. It has a ConvertToDisplayableImage method that should be what you want.
Then, you could show it in Image control. After that, you could use Image.Clip to set the position where you want to crop.

Related

Crop a svg image and convert it to png

I have a big svg image.
I would like to crop it to a rectangle, using coordinates, and convert it to png image.
I have to say that I'm not used to drawing with c#.
Surface, Canvas and other notions are new to me.
I have figured out how to load the svg, using SkiaShark and SkiaShark.Svg:
var svg = new SkiaSharp.Extended.Svg.SKSvg();
svg.Load(tmpPath);
And found a gist that saves a png. But this is "Chinese" for me.
var imageInfo = new SKImageInfo(100, 100);
using (var surface = SKSurface.Create(imageInfo))
using (var canvas = surface.Canvas)
{
// calculate the scaling need to fit to screen
var scaleX = 100 / svg.Picture.CullRect.Width;
var scaleY = 100 / svg.Picture.CullRect.Height;
var matrix = SKMatrix.CreateScale((float)scaleX, (float)scaleY);
// draw the svg
canvas.Clear(SKColors.Transparent);
canvas.DrawPicture(svg.Picture, ref matrix);
canvas.Flush();
using (var data = surface.Snapshot())
using (var pngImage = data.Encode(SKEncodedImageFormat.Png, 100))
{
tmpPath = Path.ChangeExtension(tmpPath, "PNG");
using var imageStream = new FileStream(tmpPath, FileMode.Create);
pngImage.SaveTo(imageStream);
}
}
If someone could show me the directions, it would be much appreciated.
EDIT
I've come to this implentation myself, though it is not working... The result bitmap is transparent and empty.
private string ConvertSVGToPNGRectangleWithSkiaSharpExtended(string path, double left, double top, double right, double bottom)
{
var svg = new SkiaSharp.Extended.Svg.SKSvg();
var picture = svg.Load(path);
// Get the initial map size
var source = new SKRect(picture.CullRect.Left, picture.CullRect.Top, picture.CullRect.Width, picture.CullRect.Height);
// Cropping Rect
var cropRect = new SKRect((int)left, (int)top, (int)right, (int)bottom);
var croppedBitmap = new SKBitmap((int)cropRect.Width, (int)cropRect.Height);
using var canvas = new SKCanvas(croppedBitmap);
canvas.Clear(SKColors.Transparent);
canvas.DrawBitmap(croppedBitmap, source, cropRect);
var data = croppedBitmap.Encode(SKEncodedImageFormat.Png, 100);
path = Path.ChangeExtension(path, "PNG");
using var imageStream = new FileStream(path, FileMode.Create);
data.SaveTo(imageStream);
return path;
}
EDIT 2:
I'm sorry , if it is not clear. You can refer to this question for doing so on Android.
Disclaimer: I'm not an expert with SkiaSharp, but I do know about drawing and canvases and I looked up the documentation at https://learn.microsoft.com/en-us/dotnet/api/skiasharp.skcanvas.drawpicture
I think the problem is this line:
canvas.DrawBitmap(croppedBitmap, source, cropRect);
I think you need:
canvas.DrawPicture(picture, -cropRect.Left, -cropRect.Top);
canvas.Flush();
I might have the coordinates wrong, but the problem behind your failure is that you are never drawing picture to the canvas.
To help with your confusion about what a canvas is: SKCanvas(croppedBitmap) is making it so that every time you draw to canvas, what you are actually drawing on is the bitmap. So in order to draw your picture onto the bitmap, you have to draw picture onto canvas. Using negative coordinates make it so the top and left of the svg picture are above and to the left of the top-left corner of the bitmap, which is the equivalent of having the bitmap start somewhere in the middle of the svg picture.
Hope this helps!

How to add contents of CanvasControl to a MediaClip

I'm creating a uwp app where text a user types in is converted into a video overlay. The user loads the video, types in their text, gets a preview of the video with the text, then they can save their video with the overlay.
Right now it works with an image overlay, but not text. I'm trying to use win2d to format the text. However, I can't figure out how to get the win2d canvas control into a windows.media.editing media clip.
I thought I could go CanvasControl > Bitmap of some sort > Image > MediaClip, but media clips can only be created through storage files or direct 3d surfaces. Anything would be helpful! This is my first time trying to write something outside of a game engine.
The part that currently converts the canvas control to a bitmap:
private async void btnAddOverlay_Click(object sender, RoutedEventArgs e)
{
var bitmap = new RenderTargetBitmap();
await bitmap.RenderAsync(canvasControl);
IBuffer pixelBuffer = await bitmap.GetPixelsAsync();
byte[] pixels = pixelBuffer.ToArray();
SoftwareBitmap outputBitmap = SoftwareBitmap.CreateCopyFromBuffer
(
pixelBuffer,
BitmapPixelFormat.Bgra8,
bitmap.PixelWidth,
bitmap.PixelHeight
);
}
The old part that added the image overlay to the video. (overlayImageFile is the storage file. I just don't know how to convert outputBitmap to a storage file to make a media clip from it.)
private async void CreateOverlays()
{
// Create a clip from video and add it to the composition
var baseVideoClip = await MediaClip.CreateFromFileAsync(pickedFile);
composition = new MediaComposition();
composition.Clips.Add(baseVideoClip);
// Create a clip from image
var overlayImageClip = await MediaClip.CreateFromImageFileAsync(overlayImageFile,timeSpan);
// Put image in upper left corner, retain its native aspect ratio
Rect imageOverlayPosition;
imageOverlayPosition.Height = mediaElement.ActualHeight / 3;
imageOverlayPosition.Width = (double)imageBitmap.PixelWidth / (double)imageBitmap.PixelHeight * imageOverlayPosition.Height;
imageOverlayPosition.X = 0;
imageOverlayPosition.Y = 0;
// Make the clip from the image an overlay
var imageOverlay = new MediaOverlay(overlayImageClip);
imageOverlay.Position = imageOverlayPosition;
imageOverlay.Opacity = 0.8;
// Make a new overlay layer and add the overlay to it
var overlayLayer = new MediaOverlayLayer();
overlayLayer.Overlays.Add(imageOverlay);
composition.OverlayLayers.Add(overlayLayer);
//Render to MediaElement
mediaElement.Position = TimeSpan.Zero;
mediaStreamSource = composition.GeneratePreviewMediaStreamSource((int)mediaElement.ActualWidth, (int)mediaElement.ActualHeight);
mediaElement.SetMediaStreamSource(mediaStreamSource);
txbNotification.Text = "Overlay creaeted";
}
Windows MediaClip Class: https://learn.microsoft.com/en-us/uwp/api/windows.media.editing.mediaclip?view=winrt-19041
I just don't know how to convert outputBitmap to a storage file to make a media clip from it.
After getting the RenderTargetBitmap, there is no need to convert to SoftwareBitmap. You can store the pixel data as a file in a specified format through BitmapEncoder. This is the method:
private async void btnAddOverlay_Click(object sender, RoutedEventArgs e)
{
RenderTargetBitmap renderTargetBitmap = new RenderTargetBitmap();
await renderTargetBitmap.RenderAsync(canvasControl);
var pixelBuffer = await renderTargetBitmap.GetPixelsAsync();
var pixels = pixelBuffer.ToArray();
var displayInformation = DisplayInformation.GetForCurrentView();
// Create the file in local folder
var file = await ApplicationData.Current.LocalFolder.CreateFileAsync("TempCanvas.png", CreationCollisionOption.ReplaceExisting);
// Write data
using (var stream = await file.OpenAsync(FileAccessMode.ReadWrite))
{
var encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.PngEncoderId, stream);
encoder.SetPixelData(BitmapPixelFormat.Bgra8, BitmapAlphaMode.Premultiplied, (uint)renderTargetBitmap.PixelWidth, (uint)renderTargetBitmap.PixelHeight, displayInformation.RawDpiX, displayInformation.RawDpiY, pixels);
await encoder.FlushAsync();
}
}
When needed, you can use the following code to get the saved picture:
var file = await ApplicationData.Current.LocalFolder.GetFileAsync("TempCanvas.png");
it's just another possible way
If you are using win2d than you can use
MediaClip.CreateFromSurface(IDirect3DSurface, TimeSpan) function and
you can pass a canvas render target directly to this method
this way you don't have to save bitmap to file
so you have to create an offline canvas device
then a canvas render target
then draw text over it
then pass that render target to function
and it will generate the media clip for overlay

how to extend draw area in Graphics.DrawImage c#

I have a Rectangle (rec) that contains the area in which a smaller image is contained within a larger image. I want to display this smaller image on a Picturebox. However, what I really am doing is using the smaller image as a picture detector for a larger image that is 333x324. So what I want to do is use the coordinates of the smaller image rectangle, and then draw to the Picturebox, starting from lefthand side of the rectangle, going outwards by 333 width and 324 height.
Currently my code works but it only displays the small image that was being used for detection purposes. I want it to display the smaller image + 300 width and + 300 height.
I fiddled with this code for hours and I must be doing something extremely basic wrong. If anyone can help me I would appreciate it so much!
My code for the class:
public static class Worker
{
public static void doWork(object myForm)
{
//infinitely search for maps
for (;;)
{
//match type signature for Threading
var myForm1 = (Form1)myForm;
//capture screen
Bitmap currentBitmap = new Bitmap(CaptureScreen.capture());
//detect map
Detector detector = new Detector();
Rectangle rec = detector.searchBitmap(currentBitmap, 0.1);
//if it actually found something
if(rec.Width != 0)
{
// Create the new bitmap and associated graphics object
Bitmap bmp = new Bitmap(rec.X, rec.Y);
Graphics g = Graphics.FromImage(bmp);
// Draw the specified section of the source bitmap to the new one
g.DrawImage(currentBitmap, 0,0, rec, GraphicsUnit.Pixel);
// send to the picture box &refresh;
myForm1.Invoke(new Action(() =>
{
myForm1.getPicturebox().Image = bmp;
myForm1.getPicturebox().Refresh();
myForm1.Update();
}));
// Clean up
g.Dispose();
bmp.Dispose();
}
//kill
currentBitmap.Dispose();
//do 10 times per second
System.Threading.Thread.Sleep(100);
}
}
}
If I understand correctly, the rec variable contains a rectangle with correct X and Y which identifies a rectangle with Width=333 and Height=324.
So inside the if statement, start by setting the desired size:
rec.Width = 333;
rec.Height = 324;
Then, note that the Bitmap constructor expects the width and height, so change
Bitmap bmp = new Bitmap(rec.X, rec.Y);
to
Bitmap bmp = new Bitmap(rec.Width, rec.Height);
and that's it - the rest of the code can stay the way it is now.

Printing bitmap from Silverlight - image blurry

I'm trying to print image from Silverlight application. I have pretty good quality scans (TIFF) with resolution 1696x2200
When I print - I get PrintableArea from PrintDocument and it's 816x1056
What I do - I resize bitmap to Printable area (to fit document to page) and result I get is blurry image. I understand this is scaling problem (most likely), but how do I scale properly so it looks good? When I display document inside Image and just set image size - it looks good.
For resizing I'm using WriteableBitmapEx extensions and tried both types of resize (Nearest neighbor and bilinear)
Code:
var printDocument = new PrintDocument();
printDocument.PrintPage += (s, ea) =>
{
var printableArea = ea.PrintableArea;
var bitmap = this.currentPreviewPage.FullBitmap.Resize((int)printableArea.Width, (int)printableArea.Height, WriteableBitmapExtensions.Interpolation.Bilinear);
var image = new Image { Source = bitmap };
var canvas = new Canvas { Width = bitmap.PixelWidth, Height = bitmap.PixelHeight };
canvas.Children.Add(image);
ea.PageVisual = canvas;
ea.HasMorePages = false;
};
printDocument.PrintBitmap("Silverlight Bitmap Print");
How document looks on screen (inside Image)
And this is printed:
Rather than using the WriteableBitmapEx extensions, when declaring your Image element, try setting the Stretch property so that it stretches based on your maximum specified dimensions:
var image = new Image { Source = bitmap, Stretch = Stretch.UniformToFill };
Blilinear filter tends to blur images.You may want to try WriteableBitmapExtensions.Interpolation.NearestNeighbor instead to see if you get better results
In my case it was enough to set UseLayoutRounding="True".

Rotating Cursor According to Rotated TextBox

I have a TextBox that I allow my users to rotate. But what I would LOVE for my users is to have their Cursor rotate to the same angle that the TextBox was rotated at. For example, if they rotated the TextBox to 28°, then when the Cursor enters that TextBox the Cursor should also rotate itself to 28°.
You can rotate your cursor using the System.Drawing.Icon class from WinForms in combination with WPF's bitmap rotation ability.
The way to do this is to load the icon, convert it to a BitmapSource, use Image and RenderTargetBitmap to rotate it, convert it back to an Icon, save it, and finally update bytes 2, 10, and 11 that make it a .cur instead of a .ico.
Here's what the code looks like:
public Cursor GetRotatedCursor(byte[] curFileBytes, double rotationAngle)
{
// Load as Bitmap, convert to BitmapSource
var origStream = new MemoryStream(curFileBytes);
var origBitmap = new System.Drawing.Icon(origStream).ToBitmap();
var origSource = System.Windows.Interop.Imaging.CreateBitmapSourceFromHBitmap(origBitmap.GetHBitmap());
// Construct rotated image
var image = new Image
{
BitmapSource = origSource,
RenderTransform = new RotateTransform(rotationAngle)
};
// Render rotated image to RenderTargetBitmap
var width = origBitmap.Width;
var height = origBitmap.Height;
var resultSource = new RenderTargetBitmap(width, height, 96, 96, PixelFormats.Pbgra32);
resultSource.Render(image);
// Convert to System.Drawing.Bitmap
var pixels = new int[width*height];
resultSource.CopyPixels(pixels, width, 0);
var resultBitmap = new System.Drawing.Bitmap(width, height, System.Drawing.Imaging.PixelFormat.Format32bppPargb);
for(int y=0; y<height; y++)
for(int x=0; x<width; x++)
resultBitmap.SetPixel(x, y, Color.FromArgb(pixels[y*width+x]));
// Save to .ico format
var resultStream = new MemoryStream();
new System.Drawing.Icon(resultBitmap.GetHIcon()).Save(resultStream);
// Convert saved file into .cur format
resultStream.Seek(2); resultStream.WriteByte(curFileBytes, 2, 1);
resultStream.Seek(10); resultStream.WriteByte(curFileBytes, 10, 2);
resultStream.Seek(0);
// Construct Cursor
return new Cursor(resultStream);
}
If you want to avoid the loop, you can replace it with a small bit of usafe code to call the System.Drawing.Bitmap constructor that takes initialization data:
fixed(int* bits = pixels)
{
resultBitmap = new System.Drawing.Bitmap(width, height, width, System.Drawing.Imaging.PixelFormat.Format32bppPargb, new IntPtr(bits));
}
You'll need to call this every time your TextBox rotation changes. This can be done either from the code that rotates your TextBox, or from a PropertyChangedCallback on a value that is bound to the TextBox's rotation.
mmm I'm not sure... but since the cursor is managed by Windows.. I guess you would need to hide the cursor when it enters the textbox and draw your own (which would be easy to rotate since you are rotating the other controls).
Heh, Googling for a way to do this, the first result was naturally from SO, you might wanna check the accepted answer (if you are using wpf):
Custom cursor in WPF?

Categories

Resources