Develop Reference

Determine which rectangle on picturebox is clicked

I have a c# WinForms project with a picture box that contains a document with text. I am gathering the OCR data for the document using the Google Cloud Vision API, which works great. Using the bounding rectangles returned from the Google API, I am drawing rectangles around each word using DrawRectangle, and in the process I am associating that rectangle with the underlying word. What do I need to do to be able to just click on any given rectangle and know exactly which rectangle it is without having to take the point clicked and loop through all the coordinates of all the rectangles until I find it.

Four options for ya OP
Just loop
Take the point clicked and loop through all the coordinates of all the rectangles until I find it
This is actually the simplest answer and possibly the best performing answer for a relatively small (<1000) rectangles. If your rectangles might overlap, be sure to store and loop through them in z-order from front to back.
Assisted lookup
If you have a crap ton of rectangles, you could create an additional data structure to assist with lookups. For example, you could define a 10x10 array where each element contains a list of the rectangles that overlap a portion of the screen. That way you can narrow the search. Of course there is additional overhead of maintaining the list, so it may not be worth it, depending on your usage characteristics.
Custom controls
As an alternative, you could change your approach completely and render each rectangle as its own custom control. As a custom control, it will have a click event handler just like any Win32 window. However there is considerable overhead in instantiating and managing all those controls, so this is not recommended for a large number of rectangles. Also, under the covers I'm pretty sure it'll end up using the same lookup algorithms described above, so it won't perform any better.
Bindable class
A final option is to create a class specifically for the rectangle and "bind" it to the PictureBox (register as a consumer for its events). Then every rectangle will handle the click event and raise its own event if the click was within its boundaries. Here is an example to get you started:
class ClickableRectangle
{
private Rectangle _box;
public event EventHandler Click;
public ClickableRectangle(Rectangle coordinates)
{
_box = coordinates;
}
public void BindToControl(Control control)
{
control.MouseUp += Control_MouseUp;
control.Paint += Control_Paint;
}
private void Control_Paint(object sender, PaintEventArgs e)
{
e.Graphics.DrawRectangle(Pens.Black, _box);
}
private void Control_MouseUp(object sender, MouseEventArgs e)
{
if (!_box.Contains(e.X, e.Y)) return;
if (Click != null) Click(this, e);
}
}
Then to display a new rectange in MyPictureBox, and to handle them with a method called MyClickHandler, just call
var r = new ClickableRectangle(myRectangle);
r.BindToControl(MyPictureBox);
r.Click += this.MyClickHandler;
Voila.
See also this related question.

KinectRegion HandPointer Cursor as mouse cursor in Awesomium Browser

I would like to use the kinect hand cursor as 'normal' mouse cursor. In the specific I want to be able to interact with the Awesomium browser object.
The problem is that no Awesomium Browser event is raised when the kinect hand cursor is (for example) over a link, or I do a click, or any other typical mouse event.
I modified the Control Basics-WPF example program that you can find in the example directory of the Kinect SDK
I am using c# visual studio 2012, Kinect SDK 1.7, Awesomium 1.7.1.

It's been a month since this question's been asked, so perhaps you've already found your own solution.
In any case, I found myself in this scenario as well, and here was my solution:
Inside MainWindow.xaml, you'll need the Awesomium control inside a KinectRegion (from the SDK).
You'll have to somehow tell the SDK that you want a control to also handle hand events. You can do this by adding this inside MainWindow.xaml.cs on the Window_Loaded handler:
KinectRegion.AddHandPointerMoveHandler(webControl1, OnHandleHandMove);
KinectRegion.AddHandPointerLeaveHandler(webControl1, OnHandleHandLeave);
Elsewhere in MainWindow.xaml.cs, you can define the hand handler events. Incidentally, I did it like this:
private void OnHandleHandLeave(object source, HandPointerEventArgs args)
{
// This just moves the cursor to the top left corner of the screen.
// You can handle it differently, but this is just one way.
System.Drawing.Point mousePt = new System.Drawing.Point(0, 0);
System.Windows.Forms.Cursor.Position = mousePt;
}
private void OnHandleHandMove(object source, HandPointerEventArgs args)
{
// The meat of the hand handle method.
HandPointer ptr = args.HandPointer;
Point newPoint = kinectRegion.PointToScreen(ptr.GetPosition(kinectRegion));
clickIfHandIsStable(newPoint); // basically handle a click, not showing code here
changeMouseCursorPosition(newPoint); // this is where you make the hand and mouse positions the same!
}
private void changeMouseCursorPosition(Point newPoint)
{
cursorPoint = newPoint;
System.Drawing.Point mousePt = new System.Drawing.Point((int)cursorPoint.X, (int)cursorPoint.Y);
System.Windows.Forms.Cursor.Position = mousePt;
}
For me, the tricky parts were:
1. Diving into the SDK and figuring out which handlers to add. Documentation wasn't terribly helpful on this.
2. Mapping the mouse cursor to the kinect hand. As you can see, it involves dealing with System.Drawing.Point (separate from another library's Point) and System.Windows.Forms.Cursor (separate from another library's Cursor).

C# ToolTip blocking execution?

I have no clue how to ask this question but this is a last ditch effort as I am stumped.
I have this bit of c# code that will send off to a server (using JSON) a request to draw a wms layer to a map. That server goes and does its duty and bam the wms layer appears on the map.
IN this bit of code I sometimes wanna output the times they are clicking on to render this map (its a timeline based render), so using a UserControl in c# that contains a picture box, I render this timeline and then put up a tooltip that shows the various times as the mouse moves along this pictureBox.
The Tooltip is defined as globally:
ToolTip myToolTip;
Then when I use it in my bit of code:
private void pictureBox1_MouseClick(object sender, MouseEventArgs e)
{
//...blah blah blah stuff to calculate time where user clicked based on width of timeline drawn in the picturebox etc.
prisms.callServer("Timetable", "setSelection", selection, null); //call to server to tell it to render the map at the time defined by selection
}
private void pictureBox1_MouseMove(object sender, MouseEventArgs e)
{
//...blah blah..code to calculate mTheDate showing what the time is where they are where they are hovering
myToolTip.Show(mTheDate, this, toolTipX, toolTipY, 100);
}
So when the scenario above happens, the map doesn't render until i move the mouse off the userControl that has the picturebox on it.
BUT if i take out the myToolTip.show in the MouseMove then minute I click the map renders (i realize it could have lots to do with the WMS server I am hitting etc etc, but it leads me to believe it is something with the tooltips, since removing it makes things work well enough that the map renders without having to move my mouse off the picturebox control).
So I realize this is a vague, hazy question but I am stuck and just throwing this out there for any ideas. Maybe there is something more to the tooltips I don't understand or I need a different approach that does not use the tool tips?
I should also state that the prisms.Call server does get executed (using breakpoint figured this out) even with tool tip there so not sure what is going on.

You could always hide the tooltip (if its showing) while you run the WMS call, then resume showing the tooltip.
I do not have any tooltip-specific knowledge.

Clickable Image Map c#

I'm making a C# inventory application. However, I want there to be a map where the user can click several areas and interact with the inventory on that certain area.
Currently I photoshopped the map and made it the background of the form. I'm planning on putting pictureboxes over the different areas and code manually the mouse over, click, and mouse down events to give the button appearance.
Anyway, my question is, is this a good idea? Should I just load the map into a picturebox, get rid of the buttonish visual effects and track the coordinates of the click?

While I don't think this is a bad idea, one alternative would be to use Rectangles and have an onclick function consisting of a series of Rectangle.Contains(Point) to find out if the point clicked by the mouse is contained in any of the clickable areas.
E.G.
Rectangle R1 = new Rectangle(/*Passed Variables*/);
Rectangle R2 = new Rectangle(/*Passed Variables*/);
//...
OnClick(object sender, MouseEventArgs e)
{
If (R1.Contains(e.Location))
{
//Stuff
}
else
{
If (R2.Contains(e.Location))
{
//Stuff
}
}
}
If you have a larger list of Rectangle objects, though, you could always use an array of Rectangles and a for loop for a more efficient way of checking if the clicked location is inside any Rectangle.

At CodeProject, someone has created an ImageMap Control for this very purpose. I imagine others are available.

How to draw on an unfilled box on a video stream using the mouse

I am using dshownet(first time) and C#. I have got a sample to take the web cam input and display it on a form. I now need to draw a rectangle on top of the video stream using the mouse. (the intent is to track what is inside the box from there onwards).
I heard that there is something called VMR. So I went to the dshownet samples and went through them. I didnt find any samples that use the mouse to overlay a shape on the video stream. Someone on here suggested to use colorkey. Another person said to use GDI+ and mouse handling. I tried to compile the DXLogo sample but got this error :
Error 1 Cannot create an instance of the abstract class or interface 'System.Drawing.Image' C:\Documents and Settings\TLNA\Desktop\Final Year Project\Libraries\DirectShow library 2\DirectShowSamples-2010-February\Samples\Capture\DxLogo\Capture.cs 128 32 DxLogo-2008
for the code section:
if (fileName.Length > 0)
{
m_Bitmap = new Image(fileName); // error happened here
Rectangle r = new Rectangle(0, 0, m_Bitmap.Width, m_Bitmap.Height);
m_bmdLogo = m_Bitmap.LockBits(r, ImageLockMode.ReadOnly, PixelFormat.Format24bppRgb);
}
I know that I must go through the Bitmap9 Interface. But I really dont know where to start. Should I read the Directshow API docs.
btw I also have the book Programming Microsoft Directshow for digital video and television. I started reading that book and stopped after a few pages since the code is mainly in C++. Should I continue to read this is book ? How can I accomplish the certain mentioned tasks in C# ?
Any suggestions as how to draw on the video. Some useful links(tutorials) would be helpful.
Many Thanks
Tlna

I'm not sure why the DirectShow sample doesn't compile, but you may be able to change the problem line:
m_Bitmap = new Image(fileName);
to this:
m_Bitmap = new Bitmap(fileName);
and get it to work.
You're actually facing a pretty difficult problem here. DirectShow renders a video by drawing a series of still frames to a device context (like a PictureBox or a Form, or even the screen) many times a second (depending on whatever the frame rate is). You (as a programmer) can also (easily) render graphics directly to this same device context.
However, in order to make your drawn box appear over top of a running video, your code needs to draw the rectangle immediately after DirectShow draws each frame of the video; otherwise, the next frame will obliterate your rectangle. DirectShow objects probably have some sort of frame rendering event that you can handle, and then inside the event handler you just re-draw your box (based on the initial and current mouse coordinates, which you can get from the MouseDown and MouseMove events of whatever control you're drawing on).
Update: I just took a look at my code from when I was playing around with DirectShow.NET, and it looks like there is an event (DsEvCode.Repaint) that you could hook into and use to draw your box.
I haven't looked at the code sample you're working with, but do a search and see if you can find an OnGraphNotify() method in your code, you should be able to add something like this:
if (code == DsEvCode.Repaint)
{
// draw the box here
}
Presumably, this event is fired after each frame of the video is rendered, so if you redraw your box here every time it will appear as if the box is persisting.