Using HitTest to remote control WPF - c#

I am developing a mobile app to remotely control a WPF UI.
The UI is managed by a third party and that third party can add\remove buttons any time.
I am streaming the UI to the client Windows store app and it is sending the touch coordinates to server.
Server then use HitTest to get the element on WPF UI and run the action.
It works most of the time, but the filter callback is not getting called some times.
This is a dummy code.
private void Button_Click_1(object sender, RoutedEventArgs e)
{
Point p = new Point(77, 45);
// Coordinates received on server comes here
VisualTreeHelper.HitTest(this,
new HitTestFilterCallback(FilterCallback),
new HitTestResultCallback(ResultCallback),
new PointHitTestParameters(p));
}
private HitTestFilterBehavior FilterCallback( DependencyObject target)
{
System.Diagnostics.Debug.WriteLine(target.GetType());
if (typeof(Button) == target.GetType())
{
Button b = (Button)target;
Button_Click(b, null);
return HitTestFilterBehavior.Stop;
}
else
{
return HitTestFilterBehavior.Continue;
}
}
private HitTestResultBehavior ResultCallback(HitTestResult result)
{
System.Diagnostics.Debug.WriteLine(result.VisualHit.GetType());
return HitTestResultBehavior.Stop;
}

You have to return Continue from your HitTestResultCallback. Otherwise hit testing won't traverse the complete visual tree, and may stop before it has reached the Button control.
private HitTestResultBehavior ResultCallback(HitTestResult result)
{
return HitTestResultBehavior.Continue;
}
See the Hit Testing in the Visual Layer article on MSDN for details.

Related

Querying if Office 365 is installed on button event, then executing code based on return value (WPF/C# app)

Background: I'm creating an app that allows our other techs to quickly install a dozen or so programs within one simple GUI. I am embedding the installers of those apps within my main program. (not that it matters)
I created a separate class file (IsOfficeInstalled.cs) that gets called when a user clicks on a button to install Office. Here's the code in that file:
class IsOfficeInstalled
{
public static bool check()
{
RegistryKey key = Registry.LocalMachine.OpenSubKey(#"SOFTWARE\Microsoft\Windows\CurrentVersion\App Paths\Winword.exe");
if (key != null)
{
key.Close();
}
return key != null; }}
In my MainWindow.xaml file, I am trying to call the "IsOfficeInstalled" function and perform one of two actions based on the return value. This is where I am confused. The code in my button event looks like the following:
private void btn_InstallOffice_Click(object sender, RoutedEventArgs e)
if (IsOfficeInstalled.check())
{
//If yes, then perform some code
}
else
{
//If no, then perform some different code
}
Where I need help is, getting the true/false result back from the "IsOfficeInstalled" function so my code in the "btn_InstallOffice_Click" function will know which condition to execute.
I'm not sure if I'm understanding your question. Are you looking for a simple if statement?
private void btn_InstallOffice_Click(object sender, RoutedEventArgs e)
{
if (IsOfficeInstalled.check())
{
//If yes, then perform some code
}
else
{
//If no, then perform some different code
}
}

IHTMLChangeSink UnregisterForDirtyRange throwing ComException HRESULT E_FAIL

We have a WPF with a tab control being the primary UI for displaying customer details, each open customer gets its own tab.
Within the customer tabs we have another tab control that allows switching between various subsets of information, 2 of these use the Webbrowser control with IHTMLChangeSink functionality to monitor for hidden divs meant to trigger logic in the app.
Previously we were experiencing a very large memory leak when a Customer tab was closed, the cause of this was found to be the event handler created by RegisterForDirtyRange. To resolve the memory leak the Dispose methods were modified to call UnregisterForDirtyRange, using AutoIT to rapidly open and close customer tabs we were able to prove that the memory leak was fixed; this was done on a developer class machine.
Once this change was rolled out to testers we started seeing the application crash, in the event log we saw that the call to UnregisterForDirtyRange was throwing a ComException with HRESULT E_FAIL. Since we never saw this come up on the developer hardware and on the testers machines there was no guaranteed way to produce the crash I am thinking that there is some kind of race condition that is amplified when run on less powerful hardware.
Given this information my question is with regards to the internal workings of the Unregister call, can anyone think of what might be causing this exception?
My initial thought was that maybe the Notify method was running at the time of dispose so I tried introducing a lock between the dispose and notify but this didn't change anything.
Here is a stripped down version of the tab control that wraps the Web Browser:
public partial class BrowserTabWidget : BrowserWidget, IHTMLChangeSink
{
private static Guid _markupContainer2Guid = typeof(IMarkupContainer2).GUID;
private IMarkupContainer2 _container;
private uint _cookie;
public BrowserTabWidget()
{
InitializeComponent();
if (!System.ComponentModel.DesignerProperties.GetIsInDesignMode(this))
{
Loaded += OnLoaded;
}
}
protected override void DisposeControls()
{
if (_container != null)
{
_container.UnRegisterForDirtyRange(_cookie);
Marshal.ReleaseComObject(_container);
}
WebBrowser.LoadCompleted -= OnWebBrowserLoadCompleted;
WebBrowser.Dispose();
}
public override string CurrentUri
{
get { return (string)GetValue(CurrentUriProperty); }
set
{
NavigateTo(value);
SetValue(CurrentUriProperty, value);
}
}
private void NavigateTo(string value)
{
WebBrowser.Navigate(new Uri(value));
}
public static readonly DependencyProperty CurrentUriProperty = DependencyProperty.Register("CurrentUri", typeof(string), typeof(BrowserTabWidget), new FrameworkPropertyMetadata(CurrentUriChanged));
public static void CurrentUriChanged(DependencyObject d, DependencyPropertyChangedEventArgs e)
{
var widget = (BrowserTabWidget)d;
d.Dispatcher.BeginInvoke(
DispatcherPriority.Normal,
new Action(() => widget.NavigateTo(e.NewValue.ToString())));
}
private void InitializeWebBrowser()
{
WebBrowser.LoadCompleted += OnWebBrowserLoadCompleted;
WebBrowser.Navigate(new Uri(viewModel.InitialUrl));
}
void OnWebBrowserLoadCompleted(object sender, System.Windows.Navigation.NavigationEventArgs e)
{
_container = GetMarkupContainer();
_container.RegisterForDirtyRange(this, out _cookie);
}
private void OnLoaded(object sender, RoutedEventArgs e)
{
Loaded -= OnLoaded;
Load();
}
private void Load()
{
InitializeWebBrowser();
}
private IMarkupContainer2 GetMarkupContainer()
{
var oDocument = WebBrowser.Document as IHTMLDocument2;
var pDocument = Marshal.GetIUnknownForObject(oDocument);
IntPtr pMarkupContainer;
Marshal.QueryInterface(pDocument, ref _markupContainer2Guid, out pMarkupContainer);
var oMarkupContainer = Marshal.GetUniqueObjectForIUnknown(pMarkupContainer);
Marshal.Release(pDocument);
Marshal.Release(pMarkupContainer);
return (IMarkupContainer2)oMarkupContainer;
}
public void Notify()
{
var document = WebBrowser.Document as HTMLDocument;
if (document != null)
{
//Parse Dom for hidden elements and trigger appropriate event handler
}
}
}
Hmya, E_FAIL, the curse of COM. Useless to ever diagnose anything, it is just a teacher's grade for the quality of the error reporting. I wrote the same code in Winforms to get something to testable, no repro. It is nevertheless very easy to force a repro. Given that the method takes only one argument, there's only one thing that can go wrong:
if (_container != null)
{
_container.UnRegisterForDirtyRange(_cookie);
_container.UnRegisterForDirtyRange(_cookie); // Kaboom!!!
Marshal.ReleaseComObject(_container);
}
Bad cookie. Criminal that they don't return E_INVALIDARG btw.
I could not test your exact code of course, it does have problems. Most severe one I see and the repro case is that there is no protection against calling DisposeControls() more than once. In general it is never wrong to dispose objects more than once, I have no insight if that's a realistic failure mode in your project. Otherwise very simple to protect yourself against this. Including catch-and-swallow code:
protected override void DisposeControls()
{
if (_container == null) return;
try {
_container.UnRegisterForDirtyRange(_cookie);
}
catch (System.Runtime.InteropServices.COMException ex) {
if (ex.ErrorCode != unchecked((int)0x80004005)) throw;
// Log mishap...
}
finally {
Marshal.ReleaseComObject(_container);
_container = null;
_cookie = 0;
WebBrowser.LoadCompleted -= OnWebBrowserLoadCompleted;
WebBrowser.Dispose();
}
}
Another thing I noticed in my test version of your code is that you don't appear to have any protection against the browser navigating to another page by any other means than WebBrowser.Navigate(). Or the LoadCompleted event firing more than once, it does for the stackoverflow.com home page for example. Or any web page that uses frames. That's a leak. Make it resilient by having your OnWebBrowserLoadCompleted() event handler also unregister the cookie if it is set.

Serial IO Async Issues

I've got serial data coming in to my application and by definition, it's async, so I'm running into troubles when trying to update a label to show what the incoming data is. Every now and then, I get an error on the lblRx.AsyncUpdate line, telling me the object is in use elsewhere.
At present, I use the following code;
private void IODataReceived(object sender, IODataEventArgs e)
{
lblRx.AsyncUpdate(() => lblRx.Text = string.Format("{0}:\t{1}", e.Timestamp, e.Data));
SetBackColors(false, eIODirection.In);
}
public static void AsyncUpdate(this Control ctrl, ActionCallback action)
{
if (ctrl != null)
{
if (!ctrl.IsHandleCreated && ctrl.IsDisposed)
ctrl.CreateControl(); // MSDN says CreateControl() is preferred over CreateHandle().
if (!ctrl.IsDisposed)
AsyncInvoke(ctrl, action);
}
}
The AsyncUpdate method isn't an issue (AFAIK...works well in other situations).
I think I need to put a lock on the control before calling AsyncUpdate. Or is there a better way to handle this situation?

WP7 Text-To-Speech Playing Whole Collection At Once

I working on a Windows Phone 7 app with Text-To-Speech capabilities. I'm using Text-To-Speech with Microsoft Translator Service and the following C# code...
// Text-To-Speech with Microsoft Translator Service (http://translatorservice.codeplex.com/)
private void TextToSpeech_Play(object sender, EventArgs e)
{
SpeechSynthesizer speech = new SpeechSynthesizer(CLIENT_ID, CLIENT_SECRET);
//string content = "This is a beautiful day!";
string language = "en";
//speech.SpeakAsync(content, language);
foreach (UIElement element in todayschapter.Children)
{
if (element is TextBlock)
{
string content = (element as TextBlock).Text;
speech.SpeakAsync(content, language);
}
}
}
In this instance, todayschapter is a StackPanel and its Children are TextBlocks. I'm wanting to simply play audio of each TextBlock, in succession. The problem is that it is play the audio of EVERY TextBlock at the same time.
I have a sneaking suspicion that the problem is SpeakAsync(), but I'm not sure. The documentation shows Speak(), but that isn't available (maybe a different version) in the Visual Studio methods helper dropdown (little thing that shows as you type - not sure what it's called).
Is there a way to make it wait for each play to finish before playing the next? Is foreach not the right choice, for this?
As always, if my code just looks stupid, please recommend better ways. I'm very much a beginner programmer.
Just use the Speak instead of the async call, since you want to have it one after another anyway.
The SpeakAsync call is indeed the problem. Unfortunately, since SpeakAsync doesn't return a Task, you can't just convert this to await SpeakAsync() (which would be the most straightforward conversion).
And, looking at the source code, it doesn't fire an event to tell when it's done. So let's add one (in SpeechSynthesizer.cs):
public event EventHandler<SpeechEventArgs> SpeakCompleted;
public void SpeakAsync(string text, string language)
{
this.GetSpeakStreamAsyncHelper(text, language, result =>
{
if (result.Error == null)
{
SoundEffect effect = SoundEffect.FromStream(result.Stream);
FrameworkDispatcher.Update();
effect.Play();
this.OnSpeakCompleted(new SpeechEventArgs(result.Error)); // added to call completion handler
}
else
{
this.OnSpeakFailed(new SpeechEventArgs(result.Error));
}
});
}
// new function
private void OnSpeakCompleted(SpeechEventArgs e)
{
Deployment.Current.Dispatcher.BeginInvoke(() =>
{
if (SpeakCompleted != null)
SpeakCompleted(this, e);
});
}
Now, you'll need to handle the SpeakCompleted event and start speaking the next string.
Something like this (I haven't even compiled this, so be warned):
private Queue<string> utterances;
private SpeechSynthesizer speech;
private void TextToSpeech_Play(object sender, EventArgs e)
{
speech = new SpeechSynthesizer(CLIENT_ID, CLIENT_SECRET);
speech.SpeechCompleted += new EventHandler<SpeechEventArgs>(TextToSpeech_Completed);
foreach (UIElement element in todayschapter.Children)
{
if (element is TextBlock)
{
string content = (element as TextBlock).Text;
utterances.Enqueue(content);
}
}
TextToSpeech_Completed(null, null); // start speaking the first one
}
private void TextToSpeech_Completed(object sender, SpeechEventArgs e)
{
if (utterances.Any())
{
string contents = utterances.Dequeue();
speech.SpeakAsync(contents);
}
}

Trying to mitage WebBrowser Control delay

I use a WebBrowser control as a "preview" for a special type of markup. I process the markup, and then set the browser's DocumentText property to display it. This works okay.
However, sometimes the control takes upwards of 5-10 seconds for this operation to complete. Since I do this at application start up (blanking the window with DocumentText = ""), and it still takes forever to do this, I want to somehow Thread this or something so that the rest of the application can start up while the control... does whatever it does.
For reference, the function in question is:
private void btnRefresh_Click(object sender, EventArgs e) {
try {
scrolltop = html.Document.Body.ScrollTop;
scrollleft = html.Document.Body.ScrollLeft;
} catch (NullReferenceException) { }
html.DocumentText = HtmlProcessing.ProcessCode(txtCode.Text); //takes 5-10 seconds
}
I know I can't just fire up a thread to handle this, since I assume the WebBrowser control will puke if I try to access it from the other thread. Are there any other techniques I could use to do this?
Edit:
Turns out, my delay was not being caused by html.DocumentText, nor by HtmlProcessing.ProcessCode, but by a bit of code that I omitted from the sample, because I thought it was irrelevant:
private void btnRefresh_Click(object sender, EventArgs e) {
try {
scrolltop = html.Document.Body.ScrollTop;
scrollleft = html.Document.Body.ScrollLeft;
} catch (NullReferenceException) { }
html.DocumentText = HtmlProcessing.ProcessCode(txtCode.Text, GetImageList());
}
That call to GetImageList() was taking forever. The contents of the function looks like:
List<string> GetImageList() {
List<string> ret = new List<string>();
ret.AddRange(Directory.GetFiles(settings.LocalImageFolder).Where((f) => { return extensions.Contains(Path.GetExtension(f)); }));
ret.AddRange(Directory.GetFiles(settings.RemoteImageFolder).Where((f) => { return extensions.Contains(Path.GetExtension(f)); }));
return ret;
}
And, settings.RemoteImageFolder is, in fact, a network path that was taking forever. I ended up turning this into an IEnumerable<string>. That way, I only hit the network when ProcessCode actually needs files from there.
The problem was a slow network access that was hidden behind a property accessor, and had nothing to do with the web browser control.

Categories

Resources