Google Chrome does not refresh accessibility elements (AutomationElement) when a user scrolls down in the browser.
To reproduce it:
Enable renderer accessibility with : "chrome --force-render-accessibility" or by setting on Global Accessibility at "chrome://accessibility".
Go to http://en.wikipedia.org/wiki/Google
Open inspect.exe in UI Automation Mode (from Windows Kits), look for "Links to related articles" element.
Come back to Chrome, Scroll down until "Links to related articles" at the bottom is visible
"Links to related articles" element is marked off screen
I found some manual solutions that can force Chrome to refresh it:
Set Zoom to 90% then set it back to 100 % (very very ugly way)
Switch accessibility off then switch on in chrome://accessibility/
What I'm looking for is the ability to do one of these operations programatically, or any operation that can make Chrome refresh its cache tree.
What I've tried:
Resize window with PInvoke/MoveWindow
Redraw Window with PInvoke/Redrawwindow
Build a chrome extension and force zoom to 100% on demand: chrome.tabs.setZoom(null, 0); (working but blink and slow down the window)
None of these are working properly.
EDIT: Tested with Google Chrome 40.XX, 41.XX, 42.XX, 43.XX, 44.XX, 45.XX, 46.XX, 47.XX.Dev, 48.XX.Dev under Windows 7.
Scrolling in simple pages is optimized to not require computation from the renderer. Only the compositor and the GPU are needed to scroll therefore the render tree which is only updated from the renderer is still the same.
Requiring the renderer to traverse the DOM and update the Accessibility tree during a scroll runs contrary with the several years effort of having smooth scrolling, specially for touch devices so I don't think you are going to get traction on a bug fix.
Your idea of an extension I think is the best (although ugly) compromise. But rather that changing zoom, doing a small mutation of the page (or DOM) might be a better solution. Try for example adding a invisible (or nearly so) element with a low z-order. You will also need to rate control the mutation so it only happens 1 times per second or even less often.
Chrome's multi-process architecture is different from that of any other browser. For security, the main browser UI is in one process, and web pages are run in separate renderer processes (typically one per tab). The Renderer processes are the only ones with a representation of the webpage's DOM and therefore all of the accessibility information, but the renderer processes are specifically not allowed to interact with the operating system (sending or receiving events or messages) - in particular, the renderer processes cannot send or receive accessibility events.
Related
I've spent two months creating an immaculate WPF browser app using the CefSharp embedded Chromium web browser. Now it was revealed that this browser must work with IME input methods, which it does not. Essentially, the WPF browser is rendered off-screen, with screen images and input events marshaled back-and-forth.
When the IME is invoked, text entry occurs in a popup tool-window outside of the app, typically in the upper left corner of the desktop. Once input is accepted, the input is not being marshaled back to the browser.
Is it possible to make the IME appear, as normal, next to the input-field?
I would appreciate some pointers on where to start reading or coding. If this takes a month to implement, I'm sure it will happen.
update -
It looks like the keys OemAuto and ImeProcessed are being previewed by the WPF browser control (but not passed to the off-screen browser). Passing those keys to the off-screen browser prevents the IME window from appearing. I'm not sure if this is progress or not. :)
update -
I think I'm going to roll my own window message loop to forward IME and input events to the browser. Maybe I can get the browser to handle IME events on its own?
update -
Off-screen IME support does not seems to be implemented on Windows (only Mac)
CEF3: Windows: Enable IME when Off Screen Rendering is enabled. I'm continuing to explore my options.
I threw together a quick hack posted as part of the discussion in https://github.com/cefsharp/CefSharp/issues/583 ... but as the proper place to resolve this really is in the CEF codebase I'm hesitating to add the hack the the CefSharp codebase (I'm afraid of side effects for those not needing IME support).
Anyway I hope the CefSharp issue #583 is helpful in bringing you towards a solution - either via the short term hack - or preferably leading to resolution of the CEF issue you already dug up yourself!
I am using the WebView extensively in a project but find its shortcomings to be a showstopper for the project. Are there any alternatives that can render HTML in windows store apps?
Edit: As requested, some of the problems we have with this control:
It has no dependency property support, you need to call the methods (workaround needed: wrap it in a custom control, or attached properties...)
It cannot render local html files (workaround with navigatetostring)
If local html files contain extra files, like css or js, you need more workarounds to make it work
It is a IE10 rendered above your application so it covers other controls (again you need that awful - sorry - webbrush workaround)
Animations do not work well when put in a Listview (scrolls faster than other controls, also stretches weird), there is no workaround
Animations do not work at all when put in an animated flyout, there is no workaround
It breaks semantic zooming in portrait mode (besides needing the webbrush workaround for it to work) for unknown reasons if you zoom in. It works in landscape and in portrait as long as you do not navigate anything, but do not never anytime navigate a website and it breaks the zooming.
There are probably more issues we had, these are the ones that made us consider it a showstopper
We had the same problem but solved it with the much more, not "shortcoming" alternative in this XAML Toolkit for WinRT.
There is a WebBrowser control you must have a look at.
I am using a picture box in my C# application, I want it to cover a certain portion of the web browser (the username part on youtube).
I have a 21.5" monitor and this is what it looks like to me:
But then this is what it looks like to one of my users with a 24" monitor:
As you can see the position of the picture box has moved up due to that persons screen size (I believe)
Is there a way to make sure that it will always be over that section of the web browser or moving it to that section of the web browser?
Thanks.
I am convinced your approach is wrong and would break anytime either for screen resolution or size changes, or for using the mouse-wheel to zoom in/out the page or whatever. it is just unreliable and patching this by overlapping another UI control like a picture box or a panel on top of what you want to hide is simply insecure and unreliable.
I think the tow real options you have are these:
You try to interpret the page content and remove from the page's DOM the information you do not want to show to the user (eventually HTML Agility Pack could help for this DOM parsing and manipulation but I am not sure if you can read what the WebBrowser control is showing and inject changes into it);
use the YouTube APIs and Tools - .NET APIs to load the videos and details you want to load and show but rendering this information with your specific UI elements in your windows forms application, without using a browser to show the normal YouTube site.
Probably the second option takes more work but is more secure, I am not sure 100%, as I said, if the first option is viable at all. You could search for HTML Agility Pack and web browser control to see if anybody has done this before already :)
I was asking myself how the browsers are working. How does the browser tell to the OS to change the mouse pointer from arrow to hand(IDC_HAND) for example. In desktop application I know that are used windows messages(right) but how it is happening in browsers? Spy++ doesn't seems to catch any of the mouse pointer messages in this case. Can you help me with an explanation?
I'm trying to build a C# application which will detect the type of the mouse pointer.
You can define a specific cursor for each window class. Consult the documentation for the function RegisterClassEx and structure WNDCLASSEX
HTH.
The browser viewport is a simple window with hardly any standard events. A page is rendered by pixel and treated later as a bitmap. A browser builds an hierarchy of web page controls and display elements and keeps it in memory. Whenever mouse moves across the page, the browser algorithms search through this hierarchy to identify whether these particular coordinates belong to, say, a button or a link and then change the cursor to pointer. In short, it's what a browser engine is all about. Parse HTML to an hierarchy of controls, then parse CSS and update properties of these elements then render the controls under consideration of their properties to a viewport, then process user input and when required initiate a request. Browser engine also executes JavaScript code and performs manipulation on the document structure.
Remember also that FireFox exists for Linux as well in which case it would make no sense for browser developers to work with standard windows events. Some basic initialization code is definitely platform dependent but after the window is prepared and user input is forwarded through some abstraction layer to the core, then the browser engine leads the play with no concern for the underlying operating system and its event system whatsoever.
Facing an issue where in the user objects goes more that 10000 in windows app and the app crashes.
After much analysis we realized that we need to get rid of the panels that we use to align the controls and may be reduce the possibility of user objects reaching 10000.
Our App UI is dynamically generated driven by a configuration and it can vary. So all the UI generation is happening dynamically.
Any help would be much appreciated
This is an unfounded suggestion, but remember to make sure that unneeded Controls always detach themselves from events they are be subscribed to. A Control that's still subscribed to an event of an "active" (what's the right term?) object can't be cleaned up.
Just as a note, the Chrome development team hit this problem too, and the scroll bar arrows (among other things) weren't drawing anymore when some internal gdi limit was hit. It is quite possible to hit this limit in a complex enough gdi app.
You might want to do some research and see how they fixed it.
As an alternative, you could consider using a different platform, either gtk or wpf would do fine and they don't use gdi handles to draw.
from here,
If your program runs haywire, you will
find that it manages to create about
10,000 window manager objects and then
the system won't let it have any more.
Why stop at 10,000?
The first answer is "If you have to
ask, you're probably doing something
wrong." Programs shouldn't be creating
anywhere near ten thousands window
manager objects in the first place.
There is no need for that many handles. I think you need a new solution.
I'm guessing this from your question, but you're probably putting this large number of controls on a scrollable panel or a tab control with multiple tab pages, which means that most of these controls aren't actually visible to the user at any given point in time (because they couldn't possibly all be visible at once).
If you have all of these controls on a scrollable panel, one possible solution is to only load and display the controls that are on the visible portion as the user scrolls around in the panel. As the user scrolls, you would unload and dispose the controls that are no longer visible.
If you have all of these controls in a multi-page tab control, you can use a similar strategy and only load the controls on a tab page when that page is made visible (and unload the controls from the previous top-most tab page at the same time).
Another general strategy is to break up your one monster form into a large number of UserControls, and only show one of these UserControls at a time.