Performace of multi-thread webbrowser control in ASP.NET Application - c#

We have a .aspx page to programmatically load page content of up to 5 urls and check each for a keyword. The following code is working but need to improve cpu usage and time performance and I am not an expert in this area
First: use of async web form to wait for the urls
protected void Page_Load(object sender, EventArgs e)
{
//the page won't load till the registered task is completed
RegisterAsyncTask(new PageAsyncTask(ProcessURLsKeywords));
}
Second: create task for each url-keyword pair and wait for them all to finish then proceed to return page response
private async Task ProcessURLsKeywords()
{
List<Task> availableTasks = new List<Task>();
failedURLs = new ConcurrentBag<string>(); //thread-safe collection of unordered items
//start checking each keyword-url pair
if (key1 != null && !string.IsNullOrWhiteSpace(url1))
{
availableTasks.Add(CheckPageContent(url1, key1));
}
//do the same for url2,key2 and url3,key3...etc
await Task.WhenAll(availableTasks.ToArray());
//return response
}
Third: Function get content
private async Task CheckPageContent(string url, string key)
{
try
{
string content;
var browser = new WebProcessor();
content = await browser.GetGeneratedHTML(url);
if (content != null)
{
if (!content.ToLower().Contains(key.ToLower()))
{
failedURLs.Add(url);
}
}
content = null;
}
catch (Exception ex)
{
}
}
WebProcessor Class creates STA thread for each url and return the result
public class WebProcessor
{
private string GeneratedSource { get; set; }
private string URL { get; set; }
private MyWebBrowser wb { get; set; }
private static Mutex mutex = new Mutex();
public async Task<string> GetGeneratedHTML(string url)
{
URL = url;
await Task.Run(() =>
{
Thread t = new Thread(new ThreadStart(WebBrowserThread));
t.SetApartmentState(ApartmentState.STA);
t.Start();
t.Join();
});
return GeneratedSource;
}
private void WebBrowserThread()
{
mutex.WaitOne();
wb = new MyWebBrowser();
try
{
wb.AllowNavigation = true;
wb.ScriptErrorsSuppressed = true;
wb.Navigate(URL);
wb.DocumentCompleted +=
new WebBrowserDocumentCompletedEventHandler(
wb_DocumentCompleted);
while (wb.ReadyState != WebBrowserReadyState.Complete)
Application.DoEvents();
//Added this line, because the final HTML takes a while to show up
GeneratedSource = wb.DocumentText;
//GeneratedSource = wb.Document.Body.InnerText;
}
catch (Exception ex)
{
}
finally
{
mutex.ReleaseMutex();
wb.Dispose();
}
}
private void wb_DocumentCompleted(object sender,
WebBrowserDocumentCompletedEventArgs e)
{
try
{
MyWebBrowser wb = (MyWebBrowser)sender;
if (wb.ReadyState != System.Windows.Forms.WebBrowserReadyState.Complete)
return;
if (wb.Document == null)
return;
GeneratedSource = wb.Document.Body.InnerText;
//load frame content within page - web browser doesn't do that automatically
foreach (var frame in wb.Document.Window.Frames)
{
HtmlWindow winframe = frame as HtmlWindow;
WebProcessor webFrame = new WebProcessor();
try
{
System.Threading.SynchronizationContext.Current.Post(async delegate
{
try
{
GeneratedSource += await webFrame.GetGeneratedHTML(winframe.Url.AbsoluteUri);
}
catch (Exception ex)
{
}
}, null);
}
catch (Exception ex)
{
}
}
}
catch (Exception ex)
{
}
}
/*********************************************************************************************/
}
I tried integrating Application.Run() and message pump features but the code blocks on Application.Run()

Related

How to call a function in a backgroundworker thread that is to be completed on the main UI thread? [duplicate]

So, first I have read a ton of threads on this particular problem and I still do not understand how to fix it. Basically, I am trying to communicate with a websocket and store the message received in an observable collection that is bound to a listview. I know that I am getting a response back properly from the socket, but when it tries to add it to the observable collection it gives me the following error:
The application called an interface that was marshalled for a different thread. (Exception from HRESULT: 0x8001010E (RPC_E_WRONG_THREAD))
I've read some information on "dispatch" as well as some other things, but I am just massively confused! Here is my code:
public ObservableCollection<string> messageList { get; set; }
private void MessageReceived(MessageWebSocket sender, MessageWebSocketMessageReceivedEventArgs args)
{
string read = "";
try
{
using (DataReader reader = args.GetDataReader())
{
reader.UnicodeEncoding = Windows.Storage.Streams.UnicodeEncoding.Utf8;
read = reader.ReadString(reader.UnconsumedBufferLength);
}
}
catch (Exception ex) // For debugging
{
WebErrorStatus status = WebSocketError.GetStatus(ex.GetBaseException().HResult);
// Add your specific error-handling code here.
}
if (read != "")
messageList.Add(read); // this is where I get the error
}
And this is the binding:
protected override async void OnNavigatedTo(NavigationEventArgs e)
{
//await Authenticate();
Gameboard.DataContext = Game.GameDetails.Singleton;
lstHighScores.ItemsSource = sendInfo.messageList;
}
How do I make the error go away while still binding to the observable collection for my listview?
This solved my issue:
Windows.ApplicationModel.Core.CoreApplication.MainView.CoreWindow.Dispatcher.RunAsync(CoreDispatcherPriority.Normal,
() =>
{
// Your UI update code goes here!
}
);
Correct way to get the CoreDispatcher in a Windows Store app
Try replacing
messageList.Add(read);
with
Dispatcher.Invoke((Action)(() => messageList.Add(read)));
If you're calling from outside your Window class, try:
Application.Current.Dispatcher.Invoke((Action)(() => messageList.Add(read)));
Slight modification for task based async methods but the code in here will not be awaited.
await Windows.ApplicationModel.Core.CoreApplication.MainView.CoreWindow.Dispatcher.RunAsync(CoreDispatcherPriority.Normal,
() =>
{
// Your UI update code goes here!
}
).AsTask();
This code WILL await, and will allow you to return a value:
private async static Task<string> GetPin()
{
var taskCompletionSource = new TaskCompletionSource<string>();
CoreApplication.MainView.CoreWindow.Dispatcher.RunAsync(CoreDispatcherPriority.Normal,
async () =>
{
var pin = await UI.GetPin();
taskCompletionSource.SetResult(pin);
}
);
return await taskCompletionSource.Task;
}
And on Android:
private async Task<string> GetPin()
{
var taskCompletionSource = new TaskCompletionSource<string>();
RunOnUiThread(async () =>
{
var pin = await UI.GetPin();
taskCompletionSource.SetResult(pin);
});
return await taskCompletionSource.Task;
}
Maby this is not a "good" practice, but it works.. I leave a message from webSocket, to mainBody instance, where I have a timered reader...
public class C_AUTHORIZATION
{
public Observer3.A_MainPage_cl parentPageInstance; //еще одни экземпляр родителя
public WebSocket x_Websocket;
private string payload = "";
private DateTime nowMoment = DateTime.Now;
public void GET_AUTHORIZED()
{
bitfinex_Websocket= new WebSocket("wss://*****.com/ws/2");
var apiKey = "";
var apiSecret = "";
DateTime nowMoment = DateTime.Now;
payload = "{}";
x_Websocket.Opened += new EventHandler(websocket_Opened);
x_Websocket.Closed += new EventHandler(websocket_Closed);
}
void websocket_Opened(object sender, EventArgs e)
{
x_Websocket.Send(payload);
parentPageInstance.F_messager(payload);
}
void websocket_Closed(object sender, EventArgs e)
{
parentPageInstance.F_messager("L106 websocket_Closed!");
GET_AUTHORIZED();
}
}
public sealed partial class A_MainPage_cl : Page
{
DispatcherTimer ChartsRedrawerTimer;
public bool HeartBeat = true;
private string Message;
public A_MainPage_cl()
{
this.InitializeComponent();
ChartsRedrawerTimer = new DispatcherTimer() { Interval = new TimeSpan(0, 0, 0, 0, 100) };
ChartsRedrawerTimer.Tick += Messager_Timer;
ChartsRedrawerTimer.Start();
}
private void Messager_Timer(object sender, object e)
{
if(Message !=null) //
{
F_WriteLine(Message);
Message = null; //
}
}
public void F_messager(string message) //
{
Message = message;
}
In Xamarin, I got around this by using:
Device.BeginInvokeOnMainThread(() => {
// code goes here
});

Handling CancellationToken from different class

I have a class like so:
public class FtpTaskVideo : IFtpTask
{
//some fields
public CancellationTokenSource tokenSource = new CancellationTokenSource();
private Panel CreatePanel(string text, int count, int value)
{
Panel pnlOutput = new Panel();
pnlOutput.Name = "pnlInfo";
pnlOutput.AutoSize = true;
pnlOutput.BorderStyle = BorderStyle.FixedSingle;
//adding some controls
Button btnUserCancel = new Button();
btnUserCancel.Name = "btnUserCancel";
btnUserCancel.AutoSize = true;
btnUserCancel.Text = "Stop";
btnUserCancel.Click += new EventHandler(btnUserCancel_Click);
pnlOutput.Controls.Add(btnUserCancel);
btnUserCancel.BringToFront();
return pnlOutput;
}
public void btnUserCancel_Click(object sender, EventArgs e)
{
tokenSource.Cancel();
}
public void Start()
{
//some code
while(somethingToDownload)
{
var task = Task<SharedConstants.downloadFtpFileStatus>.Factory.StartNew(() => dff.Download(tokenSource.Token), tokenSource.Token);
try
{
downloadStatus = task.Result;
}
catch (System.AggregateException exc)
{
//do something
}
//some code
}
}
And in the second class (dff):
public Shared.Classes.SharedConstants.downloadFtpFileStatus Download(CancellationToken token)
{
if (token.IsCancellationRequested)
{
return Shared.Classes.SharedConstants.downloadFtpFileStatus.CANCELLED;
}
else //do some stuff
}
Now, I have another class, which dff is an instance of and Download is it's method. One of the things dff does is update and redraw the panel according to the data it gets during Download method operation. How, after it draws a button and I press it can I send the cancel token back to original class to stop it from downloading?

.NET BackgroundWorker RunWorkerAsync() oddly gets called twice

CODE UPDATED TO REFLECT ANSWER: SAME PROBLEM STILL OCCURS
This class is supposed to run all tasks in the list, sleep and then wake up and repeat the process infinitely. For some reason though, after the first sleep, the sleepThread.RunWorkerAsync() call gets called twice for some reason. I can obviously solve this by:
if (!sleepThread.IsBusy) { sleepThread.RunWorkerAsync(); }
but that feels like a work around.
Here is the main routine class:
public class ServiceRoutine
{
private static volatile ServiceRoutine instance;
private static object instanceLock = new object();
private static object listLock = new object();
private static readonly List<Task> taskList = new List<Task>()
{
new UpdateWaferQueueTask(),
new UpdateCommentsTask(),
new UpdateFromTestDataTask(),
new UpdateFromTestStationLogsTask(),
new UpdateFromWatchmanLogsTask(),
new UpdateStationsStatusTask()
};
private List<Task> runningTasks;
private BackgroundWorker sleepThread;
private Logger log;
private ServiceRoutine()
{
log = new Logger();
runningTasks = new List<Task>();
sleepThread = new BackgroundWorker();
sleepThread.WorkerReportsProgress = false;
sleepThread.WorkerSupportsCancellation = false;
sleepThread.DoWork += (sender, e) =>
{
int sleepTime = ConfigReader.Instance.GetSleepTime();
log.Log(Logger.LogType.Info,
"service sleeping for " + sleepTime / 1000 + " seconds");
Thread.Sleep(sleepTime);
};
sleepThread.RunWorkerCompleted += (sender, e) => { Run(); };
}
public static ServiceRoutine Instance
{
get
{
if (instance == null)
{
lock (instanceLock)
{
if (instance == null)
{
instance = new ServiceRoutine();
}
}
}
return instance;
}
}
public void Run()
{
foreach (Task task in taskList)
{
lock (listLock)
{
runningTasks.Add(task);
task.TaskComplete += (completedTask) =>
{
runningTasks.Remove(completedTask);
if (runningTasks.Count <= 0)
{
sleepThread.RunWorkerAsync();
}
};
task.Execute();
}
}
}
}
this is called like this:
ServiceRoutine.Instance.Run();
from the service start method. Here is the Task class as well:
public abstract class Task
{
private Logger log;
protected BackgroundWorker thread;
public delegate void TaskPointer(Task task);
public TaskPointer TaskComplete;
public Task()
{
log = new Logger();
thread = new BackgroundWorker();
thread.WorkerReportsProgress = false;
thread.DoWork += WorkLoad;
thread.RunWorkerCompleted += FinalizeTask;
}
protected abstract string Name { get; }
protected abstract void WorkLoad(object sender, DoWorkEventArgs e);
private string GetInnerMostException(Exception ex)
{
string innerMostExceptionMessage = string.Empty;
if (ex.InnerException == null) { innerMostExceptionMessage = ex.Message; }
else
{
while (ex.InnerException != null)
{
innerMostExceptionMessage = ex.InnerException.Message;
}
}
return innerMostExceptionMessage;
}
protected void FinalizeTask(object sender, RunWorkerCompletedEventArgs e)
{
try
{
if (e.Error != null)
{
string errorMessage = GetInnerMostException(e.Error);
log.Log(Logger.LogType.Error, this.Name + " failed: " + errorMessage);
}
else
{
log.Log(Logger.LogType.Info, "command complete: " + this.Name);
}
}
catch (Exception ex)
{
string errorMessage = GetInnerMostException(ex);
log.Log(Logger.LogType.Error, this.Name + " failed: " + errorMessage);
}
finally { TaskComplete(this); }
}
public void Execute()
{
log.Log(Logger.LogType.Info, "starting: " + this.Name);
thread.RunWorkerAsync();
}
}
The question is, why is sleepThread.RunWorkerAsync() getting called twice and is there a better way to get this work without checking if the thread is busy before calling it?
You are facing a race condition here. The problem is in the TaskComplete callback. Last two tasks remove themselves from the runningTasks list before executing the if condition. When it is executed, the list count is zero. You should lock the list before changing its. The lock needs to be taken in the TaskComplete callback:
runningTasks.Add(task);
task.TaskComplete += (completedTask) =>
{
lock (runningTasks)
{
runningTasks.Remove(completedTask);
if (runningTasks.Count <= 0)
{
sleepThread.RunWorkerAsync();
}
}
};
task.Execute();
SOLVED
I tried several different locking techniques on the runningTasks list but nothing worked. After changing runningTasks to a BlockingCollection, everything worked perfectly.
Here is the new add/remove implementation using a BlockingCollection instead of a List:
foreach (Task task in taskList)
{
runningTasks.Add(task);
task.TaskComplete += (completedTask) =>
{
runningTasks.TryTake(out completedTask);
if (runningTasks.Count <= 0 && completedTask != null)
{
sleepThread.RunWorkerAsync();
}
};
task.Execute();
}

Best way to do a task looping in Windows Service

I have a method that send some SMS to our customers that look like below:
public void ProccessSmsQueue()
{
SmsDbContext context = new SmsDbContext();
ISmsProvider provider = new ZenviaProvider();
SmsManager manager = new SmsManager(context, provider);
try
{
manager.ProcessQueue();
}
catch (Exception ex)
{
EventLog.WriteEntry(ex.Message, EventLogEntryType.Error);
}
finally
{
context.Dispose();
}
}
protected override void OnStart(string[] args)
{
Task.Factory.StartNew(DoWork).ContinueWith( ??? )
}
So, I have some issues:
I don´t know how long it takes for the method run;
The method can throw exceptions, that I want to write on EventLog
I want to run this method in loop, every 10 min, but only after last execution finish.
How I can achieve this? I thought about using ContinueWith(), but I still have questions on how to build the entire logic.
You should have an async method that accepts a CancellationToken so it knows when to stop, calls ProccessSmsQueue in a try-catch block and uses Task.Delay to asynchronously wait until the next time it needs to run:
public async Task DoWorkAsync(CancellationToken token)
{
while (true)
{
try
{
ProccessSmsQueue();
}
catch (Exception e)
{
// Handle exception
}
await Task.Delay(TimeSpan.FromMinutes(10), token);
}
}
You can call this method when your application starts and Task.Wait the returned task before existing so you know it completes and has no exceptions:
private Task _proccessSmsQueueTask;
private CancellationTokenSource _cancellationTokenSource;
protected override void OnStart(string[] args)
{
_cancellationTokenSource = new CancellationTokenSource();
_proccessSmsQueueTask = Task.Run(() => DoWorkAsync(_cancellationTokenSource.Token));
}
protected override void OnStop()
{
_cancellationTokenSource.Cancel();
try
{
_proccessSmsQueueTask.Wait();
}
catch (Exception e)
{
// handle exeption
}
}
Sample Worker Class that I have used in Windows Services. It supports stopping in a 'clean' way by using a lock.
You just have to add your code in DoWork, set your timer in the StartTimerAndWork method (in milliseconds), and use this class in your service.
public class TempWorker
{
private System.Timers.Timer _timer = new System.Timers.Timer();
private Thread _thread = null;
private object _workerStopRequestedLock = new object();
private bool _workerStopRequested = false;
private object _loopInProgressLock = new object();
private bool _loopInProgress = false;
bool LoopInProgress
{
get
{
bool rez = true;
lock (_loopInProgressLock)
rez = _loopInProgress;
return rez;
}
set
{
lock (_loopInProgressLock)
_loopInProgress = value;
}
}
#region constructors
public TempWorker()
{
}
#endregion
#region public methods
public void StartWorker()
{
lock (_workerStopRequestedLock)
{
this._workerStopRequested = false;
}
_thread = new Thread(new ThreadStart(StartTimerAndWork));
_thread.Start();
}
public void StopWorker()
{
if (this._thread == null)
return;
lock (_workerStopRequestedLock)
this._workerStopRequested = true;
int iter = 0;
while (LoopInProgress)
{
Thread.Sleep(100);
iter++;
if (iter == 60)
{
_thread.Abort();
}
}
//if (!_thread.Join(60000))
// _thread.Abort();
}
#endregion
#region private methods
private void StartTimerAndWork()
{
this._timer.Elapsed += new ElapsedEventHandler(timer_Elapsed);
this._timer.Interval = 10000;//milliseconds
this._timer.Enabled = true;
this._timer.Start();
}
#endregion
#region event handlers
private void timer_Elapsed(object sender, ElapsedEventArgs e)
{
if (!LoopInProgress)
{
lock (_workerStopRequestedLock)
{
if (this._workerStopRequested)
{
this._timer.Stop();
return;
}
}
DoWork();
}
}
private void DoWork()
{
try
{
this.LoopInProgress = true;
//DO WORK HERE
}
catch (Exception ex)
{
//LOG EXCEPTION HERE
}
finally
{
this.LoopInProgress = false;
}
}
#endregion
}

Accessing WebBrowsers From Threadpool

I'm trying to access information on a web browser from another thread. When trying to access the browser.DocumentTitle, I get this error:
The name DocumentTitle does not exist in the current context
I can successfully navigate to webpages inside the DoWork or ProcessWebPage methods but I cannot access the GetTitle function without crashing. I have been working on this part alone for days and simply cannot figure it out.
Here is the problem code:
BROWSER CODE
class BrowserInterface : Form
{
WebBrowser browser;
Thread thread;
State state;
public State State { get { return state; } }
public BrowserInterface()
{
Initialize();
}
void Initialize()
{
browser = new WebBrowser();
state = State.Null;
state = State.Initializing;
thread = new Thread(StartThread);
thread.SetApartmentState(ApartmentState.STA);
thread.Start();
while (state == State.Initializing) Thread.Sleep(20);
}
void StartThread()
{
browser = new WebBrowser();
browser.Dock = DockStyle.Fill;
browser.Name = "webBrowser";
browser.ScrollBarsEnabled = false;
browser.TabIndex = 0;
browser.DocumentCompleted +=
new WebBrowserDocumentCompletedEventHandler(this.Web_Completed);
Form form = new Form();
form.Controls.Add(browser);
form.Name = "Browser";
state = State.Null;
Application.Run(form);
}
public void Navigate(string url)
{
state = State.Navigating;
if (browser.IsDisposed)
Initialize();
browser.Navigate(url);
}
public string GetTitle()
{
if (InvokeRequired)
{
BeginInvoke(new MethodInvoker(() => GetTitle()));
}
return browser.DocumentTitle;
}
private void Web_Completed(object sender, WebBrowserDocumentCompletedEventArgs e)
{
var br = sender as WebBrowser;
if (br.Url == e.Url)
state = State.Completed;
}
}
enum State
{
Initializing,
Null,
Navigating,
Completed
}
OTHER THREAD
class Controller
{
public int ThreadsAllowed;
private ManualResetEvent[] resetEvent;
private BrowserInterface[] browser;
static Thread mainThread;
bool run;
bool exit;
public Controller(int threadsAllowed)
{
ThreadsAllowed = threadsAllowed;
resetEvent = new ManualResetEvent[ThreadsAllowed];
browser = new BrowserInterface[ThreadsAllowed];
for (int i = 0; i < ThreadsAllowed; i++)
{
resetEvent[i] = new ManualResetEvent(true);
browser[i] = new BrowserInterface();
}
ThreadPool.SetMaxThreads(ThreadsAllowed, ThreadsAllowed);
mainThread = new Thread(RunThread);
mainThread.Start();
run = false;
exit = false;
}
public void Run()
{
run = true;
}
void RunThread()
{
while (true)
{
while (!run) Thread.Sleep(20);
while (mode == ScoutMode.Off) Thread.Sleep(100);
//wait for the last set to complete
WaitHandle.WaitAll(resetEvent);
if (exit)
break;
for (int i = 0; i < ThreadsAllowed; i++)
ThreadPool.QueueUserWorkItem(DoWork, i);
}
}
void DoWork(object o)
{
int i = (int)o;
if(browser[i].state == State.null)
{
…
… navigation code that works …
…
return;
}
else if(browser[i].state == State.Completed)
ProcessWebPage(i);
}
void ProcessWebPage(int i)
{
string title;
try
{
title = browser[i].GetTitle();
}
catch { return; }
}
}
What hurts my eye is your GetTitle function. When using MethodInvoker, you're dealing with methods of void type, that is, you cannot get return value from the function. That's why you need a delegate which will return you the value.
Also, you have to have else statement, so to not try to return the value when invoking is in fact required.
class BrowserInterface : Form
{
/* ... */
private delegate string StringDelegate();
public string GetTitle()
{
/*
if (InvokeRequired)
{
BeginInvoke(new MethodInvoker(() => GetTitle()));
}
return browser.DocumentTitle;
*/
if (InvokeRequired)
{
object result = Invoke(new StringDelegate(GetTitle));
return (string)result;
}
else
return browser.DocumentTitle;
}
/* ... */
}
At first, use browsers invoke instead of forms one. And the main problem that after invokation you will return to code and try to access browser.DocumentTitle as background thread. To avoid this, add else construction.
public string GetTitle()
{
if (this.browser.InvokeRequired)
{
this.browser.Invoke(new MethodInvoker(() => GetTitle()));
}
else
{
return browser.DocumentTitle;
}
}

Categories

Resources