I was asking myself how the browsers are working. How does the browser tell to the OS to change the mouse pointer from arrow to hand(IDC_HAND) for example. In desktop application I know that are used windows messages(right) but how it is happening in browsers? Spy++ doesn't seems to catch any of the mouse pointer messages in this case. Can you help me with an explanation?
I'm trying to build a C# application which will detect the type of the mouse pointer.
You can define a specific cursor for each window class. Consult the documentation for the function RegisterClassEx and structure WNDCLASSEX
HTH.
The browser viewport is a simple window with hardly any standard events. A page is rendered by pixel and treated later as a bitmap. A browser builds an hierarchy of web page controls and display elements and keeps it in memory. Whenever mouse moves across the page, the browser algorithms search through this hierarchy to identify whether these particular coordinates belong to, say, a button or a link and then change the cursor to pointer. In short, it's what a browser engine is all about. Parse HTML to an hierarchy of controls, then parse CSS and update properties of these elements then render the controls under consideration of their properties to a viewport, then process user input and when required initiate a request. Browser engine also executes JavaScript code and performs manipulation on the document structure.
Remember also that FireFox exists for Linux as well in which case it would make no sense for browser developers to work with standard windows events. Some basic initialization code is definitely platform dependent but after the window is prepared and user input is forwarded through some abstraction layer to the core, then the browser engine leads the play with no concern for the underlying operating system and its event system whatsoever.
Related
Google Chrome does not refresh accessibility elements (AutomationElement) when a user scrolls down in the browser.
To reproduce it:
Enable renderer accessibility with : "chrome --force-render-accessibility" or by setting on Global Accessibility at "chrome://accessibility".
Go to http://en.wikipedia.org/wiki/Google
Open inspect.exe in UI Automation Mode (from Windows Kits), look for "Links to related articles" element.
Come back to Chrome, Scroll down until "Links to related articles" at the bottom is visible
"Links to related articles" element is marked off screen
I found some manual solutions that can force Chrome to refresh it:
Set Zoom to 90% then set it back to 100 % (very very ugly way)
Switch accessibility off then switch on in chrome://accessibility/
What I'm looking for is the ability to do one of these operations programatically, or any operation that can make Chrome refresh its cache tree.
What I've tried:
Resize window with PInvoke/MoveWindow
Redraw Window with PInvoke/Redrawwindow
Build a chrome extension and force zoom to 100% on demand: chrome.tabs.setZoom(null, 0); (working but blink and slow down the window)
None of these are working properly.
EDIT: Tested with Google Chrome 40.XX, 41.XX, 42.XX, 43.XX, 44.XX, 45.XX, 46.XX, 47.XX.Dev, 48.XX.Dev under Windows 7.
Scrolling in simple pages is optimized to not require computation from the renderer. Only the compositor and the GPU are needed to scroll therefore the render tree which is only updated from the renderer is still the same.
Requiring the renderer to traverse the DOM and update the Accessibility tree during a scroll runs contrary with the several years effort of having smooth scrolling, specially for touch devices so I don't think you are going to get traction on a bug fix.
Your idea of an extension I think is the best (although ugly) compromise. But rather that changing zoom, doing a small mutation of the page (or DOM) might be a better solution. Try for example adding a invisible (or nearly so) element with a low z-order. You will also need to rate control the mutation so it only happens 1 times per second or even less often.
Chrome's multi-process architecture is different from that of any other browser. For security, the main browser UI is in one process, and web pages are run in separate renderer processes (typically one per tab). The Renderer processes are the only ones with a representation of the webpage's DOM and therefore all of the accessibility information, but the renderer processes are specifically not allowed to interact with the operating system (sending or receiving events or messages) - in particular, the renderer processes cannot send or receive accessibility events.
I've spent two months creating an immaculate WPF browser app using the CefSharp embedded Chromium web browser. Now it was revealed that this browser must work with IME input methods, which it does not. Essentially, the WPF browser is rendered off-screen, with screen images and input events marshaled back-and-forth.
When the IME is invoked, text entry occurs in a popup tool-window outside of the app, typically in the upper left corner of the desktop. Once input is accepted, the input is not being marshaled back to the browser.
Is it possible to make the IME appear, as normal, next to the input-field?
I would appreciate some pointers on where to start reading or coding. If this takes a month to implement, I'm sure it will happen.
update -
It looks like the keys OemAuto and ImeProcessed are being previewed by the WPF browser control (but not passed to the off-screen browser). Passing those keys to the off-screen browser prevents the IME window from appearing. I'm not sure if this is progress or not. :)
update -
I think I'm going to roll my own window message loop to forward IME and input events to the browser. Maybe I can get the browser to handle IME events on its own?
update -
Off-screen IME support does not seems to be implemented on Windows (only Mac)
CEF3: Windows: Enable IME when Off Screen Rendering is enabled. I'm continuing to explore my options.
I threw together a quick hack posted as part of the discussion in https://github.com/cefsharp/CefSharp/issues/583 ... but as the proper place to resolve this really is in the CEF codebase I'm hesitating to add the hack the the CefSharp codebase (I'm afraid of side effects for those not needing IME support).
Anyway I hope the CefSharp issue #583 is helpful in bringing you towards a solution - either via the short term hack - or preferably leading to resolution of the CEF issue you already dug up yourself!
I need to connect to a website programmatically, but without rendering the browser view, and then query the "headless" browser for certain page element's coordinate positions and send JavaScript events to simulate the movement of a mouse across the page form its current location to the new location.
I already have found a method that "realistically" moves the mouse programmatically between two points. All I need to do now is find a library that will allow me to connect to a page without the GUI but includes the positions of the page elements, but still query the JavaScript of the page and send JavaScript to the page's browser instance.
I know this very specific, but it will allow us to simulate our pages and test PHP heat mapping of the mouse location. It is also vital that this is for C#. Does such a library exist for C#?
Have you looked at Selenium? This can be run in headless mode, and has a very powerful API. I've used in C# for automated testing (including Javascript manipulation), and it works very well. It should be fine for scraping too though.
I am using a picture box in my C# application, I want it to cover a certain portion of the web browser (the username part on youtube).
I have a 21.5" monitor and this is what it looks like to me:
But then this is what it looks like to one of my users with a 24" monitor:
As you can see the position of the picture box has moved up due to that persons screen size (I believe)
Is there a way to make sure that it will always be over that section of the web browser or moving it to that section of the web browser?
Thanks.
I am convinced your approach is wrong and would break anytime either for screen resolution or size changes, or for using the mouse-wheel to zoom in/out the page or whatever. it is just unreliable and patching this by overlapping another UI control like a picture box or a panel on top of what you want to hide is simply insecure and unreliable.
I think the tow real options you have are these:
You try to interpret the page content and remove from the page's DOM the information you do not want to show to the user (eventually HTML Agility Pack could help for this DOM parsing and manipulation but I am not sure if you can read what the WebBrowser control is showing and inject changes into it);
use the YouTube APIs and Tools - .NET APIs to load the videos and details you want to load and show but rendering this information with your specific UI elements in your windows forms application, without using a browser to show the normal YouTube site.
Probably the second option takes more work but is more secure, I am not sure 100%, as I said, if the first option is viable at all. You could search for HTML Agility Pack and web browser control to see if anybody has done this before already :)
I have a WinForms program written on .NET 2 which hosts a webbrowser control and renders asp.net pages from a known server.
I would like to be able to drag, say, a tree node from a treeview in my winforms app into a specific location in the hosted web page and have it trigger a javascript event there.
Currently, I can implement the IDocHostUIHandler interface and getting drag\drop events on the browser control, then call Navigate("javascript:fire_event(...)") on the control to execute a script on the page. However, I want this to work only when I drop data on a specific part of the page.
One solution, I suppose, would be to bite the bullet and write a custom browser plugin in the form of an activex control, embed that in the location I want to drop to and let that implement the needed drag\drop interfaces.
Would that work?
Is there a cleaner approach? Can I take advantage of the fact that the browser control is hosted in my app and provide some further level of interaction?
Take a look at the BrowserPlus project at Yahoo.
It looks like they have built a toolkit so that you don't have to do the gritty work of writing the browser plugin yourself.
If you can find out the on screen position of the part of the page you are interested in, you could compare this with the position of the mouse when you receive the drop event. I'm not sure how practical this is if you can get the info out of the DOM or whatnot.
As an alternative could you implement the mouse events on the bit of the page using javascript?