Develop Reference

Draw rectangle over SwapChainPanel

I'm receiving a live video from a DJI drone in a SwapChainPanel and I'm using template matching from OpenCV to find an image inside the frames of the video.
I want to draw a rectangle when the subgimage is detected, but there are white spaces in both sides of the frames and I don't know how to get the size of those or align the image to the left and top.
Here is the XAML:
<SwapChainPanel x:Name="swapChainPanel" Grid.Column="8" Grid.ColumnSpan="4" Grid.Row="2" Grid.RowSpan="11">
<Canvas>
<Rectangle x:Name="rectFiducial" Visibility="Collapsed" Stroke="Red"></Rectangle>
<Rectangle Stroke="Blue" Width="400" Height="400"></Rectangle>
</Canvas>
</SwapChainPanel>
Here is the code when I try to draw the rectangle with the results of the Template Matching:
SoftwareBitmap inputBitmap = SoftwareBitmap.CreateCopyFromBuffer(CryptographicBuffer.CreateFromByteArray(imageData),
BitmapPixelFormat.Rgba8,
imageWidth,
imageHeight);
if (inputBitmap.BitmapPixelFormat != BitmapPixelFormat.Bgra8
|| inputBitmap.BitmapAlphaMode != BitmapAlphaMode.Premultiplied)
{
inputBitmap = SoftwareBitmap.Convert(inputBitmap, BitmapPixelFormat.Bgra8, BitmapAlphaMode.Premultiplied);
}
int x = 0, y = 0, matchMethod = 0;
double matchResult = 0;
Stopwatch testWatch = new Stopwatch();
testWatch.Start();
await txtMatchMethod.Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>
{
int.TryParse(txtMatchMethod.Text, out matchMethod);
});
OpenCV.TemplateMatching(inputBitmap, template, matchMethod, out x, out y, out matchResult);
testWatch.Stop();
await txtMatchTime.Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>
{
txtMatchTime.Text = testWatch.ElapsedMilliseconds.ToString();
txtMatchResult.Text = matchResult.ToString();
double swapX = x * swapChainPanel.ActualWidth / imageWidth;
double swapY = y * swapChainPanel.ActualHeight / imageHeight;
double swapWidth = template.PixelWidth * swapChainPanel.ActualWidth / imageWidth;
double swapHeight = template.PixelHeight * swapChainPanel.ActualHeight / imageHeight;
rectFiducial.SetValue(Canvas.LeftProperty, swapX);
rectFiducial.SetValue(Canvas.TopProperty, swapY);
rectFiducial.Width = swapWidth;
rectFiducial.Height = swapHeight;
rectFiducial.Visibility = Visibility.Visible;
});
}
There is an event where I can get the bytes of the image and I tried to display it in an Image inside a Canvas, but the image didn't stretch to fit the canvas size.
Edit
I realized that I'm calculating the position based on the width and height of the SwapChainPanel, but I have to calculate it with the width and height of the image inside.
So I would need to get the data of the resized image that is inside the SwapChainPanel and the padding of the left side.

Well, I couldn't find a solution for the SwapChainPanel, but I found a solution with the image and the canvas.
I was trying to make something like this:
<Canvas x:Name="canvas" Grid.Column="8" Grid.ColumnSpan="4" Grid.Row="4" Grid.RowSpan="11">
<Image x:Name="imgDrone" HorizontalAlignment="Left"/>
<Rectangle x:Name="rectFiducial" Visibility="Collapsed" Stroke="Red"></Rectangle>
<Rectangle Stroke="Blue" Width="400" Height="400"></Rectangle>
</Canvas>
But I couldn't stretch the image, it was displayed with it's real size.
So instead of declaring the image inside the Canvas I just took it out and set it's column and row with the same values of the canvas.
Like this:
<Image x:Name="imgDrone" HorizontalAlignment="Left" Grid.Column="8" Grid.ColumnSpan="2" Grid.Row="2" Grid.RowSpan="11"/>
<Canvas x:Name="canvas" Grid.Column="8" Grid.ColumnSpan="4" Grid.Row="4" Grid.RowSpan="11">
<Rectangle x:Name="rectFiducial" Visibility="Collapsed" Stroke="Red"></Rectangle>
<Rectangle Stroke="Blue" Width="400" Height="400"></Rectangle>
</Canvas>

Get coordinates on mouse click in viewport wpf

I want to return the coordinates of mouse click in viewport. When I click inside the viewport, it is showing me screen coordinates (for example 480,650) but the range for my viewport is say (0,0) to say (10,10). How do I get the coordinates of mouse clicked to viewport coordinate system.
My xaml code:
<Window x:Class="PathOptimization.MainWindow"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:local="clr-namespace:Geometries"
xmlns:tool3d="clr-namespace:_3DTools;assembly=3DTools"
Title="MainWindow" Height="350" Width="1000" WindowState="Maximized">
<Grid x:Name="grdView" >
<Grid.ColumnDefinitions >
<ColumnDefinition Width="200"></ColumnDefinition>
<ColumnDefinition></ColumnDefinition>
<ColumnDefinition Width="200"></ColumnDefinition>
</Grid.ColumnDefinitions>
<Viewport3D x:Name="viewport" Grid.Column="1" MouseDown="button_MouseDown" Margin="0,35,0,0">
<ContainerUIElement3D>
<ModelUIElement3D >
..............................
</ModelUIElement3D>
</ContainerUIElement3D>
<Viewport3D.Camera>
<PerspectiveCamera Position="0,1,9"
LookDirection="0,0,-2"
UpDirection="0,1,0"/>
</Viewport3D.Camera>
</Viewport3D>
When I click on the screen inside the viewport, I am getting screen coordinates. How can I get coordinates relative to viewport.
private void button_MouseDown(object sender, MouseButtonEventArgs e)
{
Point pointToWindow = e.GetPosition(viewport);
Point pointToScreen = PointToScreen(pointToWindow);
double x = (pointToScreen.X);
double y = (pointToScreen.Y);
MessageBox.Show(x + "," + y);
}

Click and drag box over image - WPF

I'm having trouble getting the selectionBox not to center. In the image below, I tried to draw a box around the word Final Bill (accented ms paint box) by clicking and dragging but the resulting selectionBox (dashed line in red) that outputs always starts in the center. The rectangle values that are calculated and saved in the mouseUp event are all correct, suggesting perhaps a XAML display issue?
I am very knew to WPF/XAML and front end stuff in general.
EDIT: By placing just the selectionBox in a <Canvas> tag I was able to get it almost working. It no longer centers but the start point appears to be twice as far from the left and top borders as the mouse is when clicked.
XAML
<DockPanel Width="Auto" Margin="225,65,5,5">
<Border x:Name="img_Border" ClipToBounds="True" Height="Auto" Width="Auto" Margin="0,0,0,0" VerticalAlignment="Top">
<Grid>
<ScrollViewer>
<Image x:Name="img_Box" ClipToBounds="True" MouseMove="img_Box_MouseMove" MouseWheel="img_Box_MouseWheel">
</Image>
</ScrollViewer>
<Rectangle x:Name="selectionBox" Visibility="Collapsed" Stroke="Red" StrokeThickness="3" StrokeDashArray="3,1">
</Rectangle>
</Grid>
</Border>
</DockPanel>
I can get the selectionBox to work correctly if I use <Canvas> tags. but that causes the image to not fit inside the <DockPanel>.
c#
public MainWindow()
{
InitializeComponent();
img_Box.MouseLeftButtonDown += img_Box_MouseLeftButtonDown;
img_Box.MouseLeftButtonUp += img_Box_MouseLeftButtonUp;
img_Box.MouseMove += img_Box_MouseMove;
Point mouseDownPos;
}
private void img_Box_MouseLeftButtonDown(object sender, MouseButtonEventArgs e)
{
img_Box.CaptureMouse();
var tt = (TranslateTransform)((TransformGroup)img_Box.RenderTransform).Children.First(tr => tr is TranslateTransform);
double BoxX = (e.GetPosition(img_Box).X / img_Box.ActualWidth) * 1;
double BoxY = (e.GetPosition(img_Box).Y / img_Box.ActualHeight) * 1;
double xValue = Math.Round((BoxX * img_Box.Source.Width), 0);
double yValue = Math.Round((BoxY * img_Box.Source.Height), 0);
StartDrag = new System.Windows.Point(xValue, yValue);
mouseDownPos.X = (int)xValue;
mouseDownPos.Y = (int)yValue;
Canvas.SetLeft(selectionBox, xValue);
Canvas.SetTop(selectionBox, yValue);
selectionBox.Width = 0;
selectionBox.Height = 0;
selectionBox.Visibility = Visibility.Visible;
}
private void img_Box_MouseMove(object sender, MouseEventArgs e)
{
double x = e.GetPosition(img_Box).X;
double y = e.GetPosition(img_Box).Y;
double BoxX = (x / img_Box.ActualWidth) * 1;
double BoxY = (y / img_Box.ActualHeight) * 1;
double xValue = Math.Round((BoxX * img_Box.Source.Width), 0);
double yValue = Math.Round((BoxY * img_Box.Source.Height), 0);
if (mouseDownPos.X < xValue)
{
Canvas.SetLeft(selectionBox, mouseDownPos.X);
selectionBox.Width = xValue - mouseDownPos.X;
}
else
{
Canvas.SetLeft(selectionBox, xValue);
selectionBox.Width = mouseDownPos.X - xValue;
}
if (mouseDownPos.Y < yValue)
{
Canvas.SetTop(selectionBox, mouseDownPos.Y);
selectionBox.Height = yValue - mouseDownPos.Y;
}
else
{
Canvas.SetTop(selectionBox, yValue);
selectionBox.Height = mouseDownPos.Y - yValue;
}
}

Not sure if I fully understand what exactly you are trying to accomplish here but if you just need a border around that text and not to allow the user to move it around with the mouse, set the image to be the background of the Grid and then partition the Grid so that one of the grid cells is directly above that text you want to place the border around. Place the border within that cell and If all the bill images are exactly the same size and have the text in exactly the same location it will always be directly over that text as long as you do not allow users to resize the image... If you want the scrolling just put the entire grid within the ScrollViewer control.
Something like this:
<Grid>
<Grid.Background>
<VisualBrush TileMode="None" >
<VisualBrush.Visual>
<Image Source="{Binding ImageSource}"/>
</VisualBrush.Visual>
</VisualBrush>
</Grid.Background>
<Grid.ColumnDefinitions>
<ColumnDefinition Width="80"/> //Distance from the left edge of the image to the right edge of the border around the "Final Bill" text.
</Grid.ColumnDefinitions>
<Grid.RowDefinitions>
<RowDefinition Height="150"/> //Distance from the top of the image to the top of the desired location of the border around the "Final Bill" text.
<RowDefinition Height="50"/> //Desired height of the border around the "Final Bill" text.
</Grid.RowDefinitions>
<Border Grid.Column="0" Grid.Row="1" BorderThickness="1"/>
</Grid>

Why my coordinate (1,1) start at (0, 1)?

I overriden Border control and in my overriden OnRender I do:
protected override void OnRender(System.Windows.Media.DrawingContext dc)
{
this.SnapsToDevicePixels = true;
this.VisualEdgeMode = EdgeMode.Aliased;
var myPen = new Pen(new SolidColorBrush(Colors.LightGray), 1);
dc.DrawLine(myPen, new Point(1, 1), new Point(1, RenderSize.Height - 1));
return;
Which give me this result:
Question:
Is anybody can tell me why my code draw a line that start at (0,1) while it is suppose to start at (1, 1) like written in code ?
My DPI are 96, 96.
For ref, this is my xaml:
<Window xmlns:MyControls="clr-namespace:MyControls;assembly=MyControls" x:Class="TestControls.MainWindow"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
Title="MainWindow" Height="350" Width="525">
<Grid>
<Grid.RowDefinitions>
<RowDefinition Height="Auto"></RowDefinition>
<RowDefinition></RowDefinition>
<RowDefinition></RowDefinition>
</Grid.RowDefinitions>
<Grid.ColumnDefinitions>
<ColumnDefinition Width="Auto"></ColumnDefinition>
<ColumnDefinition></ColumnDefinition>
<ColumnDefinition></ColumnDefinition>
</Grid.ColumnDefinitions>
<MyControls:Border3D BorderThickness="3" BorderBrush="Aqua">
<Rectangle Width="74" Height="3" HorizontalAlignment="Left">
<Rectangle.Fill>
<SolidColorBrush Color="#40FF0000">
</SolidColorBrush>
</Rectangle.Fill>
</Rectangle>
</MyControls:Border3D>
<Rectangle Grid.Row="1" Grid.Column="0" Width="80" Grid.ColumnSpan="2" HorizontalAlignment="Left">
<Rectangle.Fill>
<SolidColorBrush Color="LightGray"></SolidColorBrush>
</Rectangle.Fill>
</Rectangle>
</Grid>
</Window>

Keep in mind that (0, 0) is not the center of the upper left pixel. Instead it is the upper left corner of that pixel. In order to draw a line with a stroke thickness of 1 in the middle of the second pixel column (with index 1) you would have to draw from (1.5, 1) to (1.5, RenderSize.Height - 1):
dc.DrawLine(myPen, new Point(1.5, 1), new Point(1.5, RenderSize.Height - 1));
Setting SnapsToDevicePixels = true made your line snap to the left by half a pixel.
If you would use PenLineCap.Square for both the StartLineCap and EndLineCap properties of the line's Pen, you could draw from exactly one pixel center to the other:
var myPen = new Pen(Brushes.LightGray, 1);
myPen.StartLineCap = PenLineCap.Square;
myPen.EndLineCap = PenLineCap.Square;
dc.DrawLine(myPen, new Point(1.5, 1.5), new Point(1.5, RenderSize.Height - 1.5));

Using VisualBrush as OpacityMask

I want to set OpacityMask to a control, but I need to build that mask dynamically.
Here is how it should look:
The width and height of the whole (red) rectangle is dynamic, based on width and height of parent control. But I need to place two small rectangles (static width and height) in top left and top right corner, as shown on image. So how can I make this happen?
I tried this code, but it doesn't work: (nothing is displayed at all)
<Border BorderBrush="#80FFFFFF" BorderThickness="1" CornerRadius="5">
<Border.OpacityMask>
<VisualBrush>
<VisualBrush.Visual>
<DockPanel>
<StackPanel DockPanel.Dock="Top" Orientation="Horizontal" Height="2">
<Border Background="Transparent" Width="12" VerticalAlignment="Stretch" HorizontalAlignment="Left" />
<Border Background="Black" VerticalAlignment="Stretch" HorizontalAlignment="Stretch" />
<Border Background="Transparent" Width="12" VerticalAlignment="Stretch" HorizontalAlignment="Right" />
</StackPanel>
<Border Background="Black" />
</DockPanel>
</VisualBrush.Visual>
</VisualBrush>
</Border.OpacityMask>
</Border>
Is it even valid to use VisualBrush this way (as a OpacityMask)?

If I understand your question correctly you want those Black squares in you image to be transparent?
Update: Uploaded sample project here: http://www.mediafire.com/?5tfkd1cxwfq0rct
I think the problem is that the Panel inside the VisualBrush won't stretch. You could get the desired effect by Binding the Width and Height of whatever Panel you use to the ActualWidth and ActualHeight of the Border
<Border Name="border" BorderBrush="Red" BorderThickness="1" CornerRadius="5">
<Border.OpacityMask>
<VisualBrush>
<VisualBrush.Visual>
<Grid Width="{Binding ElementName=border, Path=ActualWidth}"
Height="{Binding ElementName=border, Path=ActualHeight}">
<Grid.ColumnDefinitions>
<ColumnDefinition Width="20"/>
<ColumnDefinition Width="*"/>
<ColumnDefinition Width="20"/>
</Grid.ColumnDefinitions>
<Grid.RowDefinitions>
<RowDefinition Height="20"/>
<RowDefinition Height="*"/>
</Grid.RowDefinitions>
<Rectangle Fill="Transparent" Grid.Column="0"/>
<Rectangle Fill="Black" Grid.Column="1"/>
<Rectangle Fill="Transparent" Grid.Column="2"/>
<Rectangle Fill="Black" Grid.Row="1" Grid.ColumnSpan="3"/>
</Grid>
</VisualBrush.Visual>
</VisualBrush>
</Border.OpacityMask>
<Grid>
<TextBlock Text="Testing OpacityMask with a rather long string................." Grid.ZIndex="3"/>
<Rectangle Fill="Green"/>
</Grid>
</Border>
Update Again
The DropShadowEffect for the Decorator Child of the Border seems to push the OpacityMask for the Border both Verticaly and Horizontaly. And what's even worse is that it seems to stack, so in your example when you have three DropShadowEffects for three nested Decorators, the sum of the BlurRadius is 45 (20+15+10) so the OpacityMask is pushed by a value of 45 (at least it looks like this is whats going on, but it's a little hard to tell..). You could try to compensate for this by increasing the ColumnDefinition Widths and RowDefinition Heights but I think it'll be hard to find a dynamic solution.
A better approach to your problem may be to use Border.Clip but that doesn't come easy either.
Point1: 0, 2
Point2: 12, 2
Point3: 12, 0
Point4: Width of Border - 12, 0
Point5: Width of Border - 12, 2
Point5: Width of Border, 2
Point6: Width of Border, Height of Border
Point7: 0, Height of Border
Update 3
Came up with a better solution that doesn't require so many Bindings. Create a custom class that derives from Border and override GetLayoutClip. This works both in Designer and Runtime. To increase flexibility of ClippedBorder you could introduce some Dependency Properties to use instead of the hardcoded 2 and 12. New sample app here: http://www.mediafire.com/?9i13rrqpbmzdbvs
public class ClippedBorder : Border
{
protected override Geometry GetLayoutClip(Size layoutSlotSize)
{
PathGeometry pathGeometry = new PathGeometry();
pathGeometry.Figures = new PathFigureCollection();
//Point1: 0, 2
PathFigure pathFigure = new PathFigure();
pathFigure.StartPoint = new Point(0, 2);
//Point2: 12, 2
LineSegment lineSegment1 = new LineSegment();
lineSegment1.Point = new Point(12, 2);
//Point3: 12, 0
LineSegment lineSegment2 = new LineSegment();
lineSegment2.Point = new Point(12, 0);
//Point4: Width of Border - 12, 0
LineSegment lineSegment3 = new LineSegment();
lineSegment3.Point = new Point(this.ActualWidth-12, 0);
//Point5: Width of Border - 12, 2
LineSegment lineSegment4 = new LineSegment();
lineSegment4.Point = new Point(this.ActualWidth-12, 2);
//Point5: Width of Border, 2
LineSegment lineSegment5 = new LineSegment();
lineSegment5.Point = new Point(this.ActualWidth, 2);
//Point6: Width of Border, Height of Border
LineSegment lineSegment6 = new LineSegment();
lineSegment6.Point = new Point(this.ActualWidth, this.ActualHeight);
//Point7: 0, Height of Border
LineSegment lineSegment7 = new LineSegment();
lineSegment7.Point = new Point(0, this.ActualHeight);
pathFigure.Segments.Add(lineSegment1);
pathFigure.Segments.Add(lineSegment2);
pathFigure.Segments.Add(lineSegment3);
pathFigure.Segments.Add(lineSegment4);
pathFigure.Segments.Add(lineSegment5);
pathFigure.Segments.Add(lineSegment6);
pathFigure.Segments.Add(lineSegment7);
pathGeometry.Figures.Add(pathFigure);
return pathGeometry;
}
}