I'm working on a project which is by C# language. I want to produce a Dictionary which is similar to Narcis or Easylingo that able the user to use the mouse and just move the cursor on the WORD in any application then the program detect the word and will find the meaning for that particular word.
My question is How I can detect the word which is under the Mouse Cursor on the screen in any application such as Microsoft office or Web browsers, etc.
I'm looking for any tips and guide or any Codes which can help me.
Your cooperation is highly appreciated.
You will need to fiddle with Win API capture the GDI calls that output text to the screen.
As suggested in this answer on SO,
Usually for GDI controls you can get the position and size of the control, and you can usually get the font info. For example, with static text controls you'd use WM_GETFONT. Then once you have that you can get the position of the mouse relative to the position of the control and use one of the font functions, perhaps something like GetTextExtentPoint32 to figure out what is under the cursor.
So try finding something in that direction.
You could possibly mimic the same features as Spy++ and capture WM_MOUSEMOVE events, pre-filter them and detect whenever the mouse is over a textbox or similar control.
It certainly won't be easy, and will take a lot of work and research!
Related
I need to write a text recognition (from image) application. The main idea is that while my application is running, I may have the need to transform some text of an image into manageble text. So, in this case with the mouse I need to draw a square around the area that i need to capture and the software must convert the content of the extracted picture into text.
I solved the problem of image recognition. I also find a very easy way to capture from screen.
What I need to do now is be able to select with the mouse the interesting area that is over an other running opplication (for example over a webpage or over an image opened in Paint). That must be like the screencapture on windows7, you create a selection of the screen and this is saved like a picture.
By looking around, I didn't find anything and I don't knwo where to start.
Many thanks
You can achieve that using either a 'tricky' easy way or a real but difficult approach.
The tricky way
Screen Recorder applications usually use this approach:
Whenever user wants to select an area, you display a full screen border less Window with 0% opacity, then user attempts to select the screen area, and he is actually selecting your Window area, so you can receive mouse events and display/draw a rectangular shape to show the selection area to the user.
In this approach, the program needs to know when to display the Window and when to hide it. This can be done by for example defining Hotkeys for capturing:
Program registers a hot key using RegisterHotKey to Windows.
User presses and holds that hot key
Program displays the tricky Window
User selects the interested area, program receives that area using mouse events of the tricky Window
User released the hot key and program hides the Tricky window.
The real way
Using this way, you need to set a message hook in order to receive mouse and keyboard events while user is interacting with desktop not your program. This is not an easy to accomplish approach and I recommend you the first one.
Is there a way to know the universal coordinates of the point?
I mean the following:
we have the button which could be pressed by clicking (500;500) when application maximized;
if it possible to know which point to click to press the button when the window of application is 600x600;
And so on. Is there any function to calculate such a point?
Thanks in advance!
I don't think there is a general function, since any developer could change the appearance based on screen size.
However, different approaches may work:
Somehow get a handle to the button/element of interest, then just call its clicked command, or use that handle to get the coordinate position on the screen.
For instance, with Web pages, you can get the specific HTMLElement by name (or id) and then work backward from there.
If it is a Windows Form application, you can actually get a handle onto the window of that application, and then get the desired element/component by walking through them all.
Create a list of the button locations under different maximization/restorings/movings and calculate the function from that. (Perhaps make a computer program to help with this).
As a last-resort, like for applications which randomly move their buttons around to confuse bots (I can't think of any real life examples), you might need to do image analysis on a screenshot.
The spy++ tool in/for Visual Studio has a finder tool that can help get the window/element names of windows applications.
I have C# add-in for Word and would like to implement an interactive tutorial like games usually have. I'd like to somehow highlight (maybe by circling) certain visual elements and display text boxes that describe what the element does. For example, say the add-in is a generic workflow editor. I'd like to show to the user, step by step, what needs to be done by visually selecting elements and explaining what they do and what options (s)he has. My first question is: can this be done in C#? My second question is how? :) I suppose I'd have to get the positions of said visual elements and then draw an image on top but I don't know how that could be done.
I'm a bit disappointed that not even a single member of the Stack Overflow community took the the time to at least give a hint about this. But such is life and I'm just going to share my findings here because I'm certain someone else will benefit from them.
To be able to do an interactive tutorial you need three things:
a method to find where a control is located in terms of screen coordinates
a method to point the user to the control by highlighting it or
surrounding it with a red line for example.
a method to do its default action (ie: click a button).
In other words, the idea is to have some sort of window with text describing a control and some graphical way of indicating the control in the app. Optionally, the window could provide something like a ShowMe button which shows the user how to use the control by taking control of the mouse and keyboard.
This is trivial when the controls are made by yourself and thus have source code access. However, when doing such a thing for a black box app such as Word you can use the Windows IAccessible interface. Searching for that with your favorite search engine will yield everything you need to understand it and use it. IAccessible allows one to do many things but most importantly it can get a control's position and can also do the default action.
Having sorted out these things the next step is to figure out how to graphically point out the control. There are many ways to do this but in my case I chose to surround it with a red rectangle. I did this by creating an invisible, borderless form with an empty red rectangle on it. Having the control's position and size, I had no problems placing the aforesaid form over the control.
So there you have it. I laid out the building blocks that one needs to make an interactive tutorial for any app.
How to track the mouse position on the screen regardless of application.i.e. Whenever the user clicks or select something with mouse in any application, i want to display my own menu at that point itself.
Is there any way to get mouse position on the screen using c#?
To do this, you'd need to P/Invoke to user32.dll and use SetWindowsHookEx().
Have a look here:
SetWindowsHookEx (user32)
How to set a Windows hook in Visual C# .NET
That sounds like a bad idea. I'm sure you could force it with a bit of effort, but what about the other applications that may want to show their own menus when the mouse is clicked?
I have solved this issue, I used Low Level Mouse hooks and Low Level Keyboard hooks to implement the solution.
I am yet to add the Menu part of it.
Background:
I'm going to start studying/coding at the local university's library. Since I'm not a student, I won't be able to utilize their wireless internet access. Since StackOverflow is such a great resource, I want to be able to take it with me, so I'm building a small desktop application to load/search/display the most recent data dumps.
Problem:
I want to display code blocks in the same sort of rectangular block as this site does, so I played with the RichTextBox control to try to create this effect. Unfortunately, the RichTextBox.SelectedBackColor property only colors the actual text, when what I want is a rectangle reaching to the outer limits of the selection.
Example:
This is what I am able to produce with the RichTextBox:
This is what I would like to create:
Questions:
Is there any way to produce this effect using the RichTextBox?
If not, are there any other controls I could use to create this effect?
I don't think so, there is no mention of controlling this behavior on MSDN.