Iam Unable to do this from past one week. I want to click on multiple links n multiple web pages using webBrowser in C# Following is the code please help me in this regard.
public void DoDelete()
{
int count = 0;
if (corruptList.Count > 0)
{
foreach (string listItem in corruptList)
{
var th = new Thread(() =>
{
try
{
WebBrowser webBrowser = new WebBrowser();
webBrowser.DocumentCompleted += new WebBrowserDocumentCompletedEventHandler(webBroswer_DocumentCompleted);
webBrowser.Navigate(listItem);
Thread.Sleep(100);
webBrowser.Dispose();
}
catch (Exception ex)
{
throw ex;
}
this.Invoke(new MethodInvoker(delegate
{
dataGridView_CorruptLinks.Rows[count].Cells[2].Value = "Deleted";
}));
});
th.SetApartmentState(ApartmentState.STA);
th.Start();
Thread.Sleep(100);
}
count++;
}
}
void webBroswer_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
{
try
{
WebBrowser webBrowser = sender as WebBrowser;
HtmlElementCollection ec = webBrowser.Document.GetElementsByTagName("a");
foreach (HtmlElement item in ec)
{
if (item.InnerHtml == "Delete this invalid field")
{
item.InvokeMember("Click");
break;
}
}
}
catch (Exception exp)
{
}
}
Navigate is an asynchronous action and you're only giving it 1/10 of a second to complete before you call Dispose on the web browser object. Your navigation and clicks are probably taking longer than that to complete and so there is no web browser to act against... You're also "swallowing" all exceptions in the document complete handler. This is a very bad thing to do. You should at the very least be doing some debug logging there to help yourself diagnose the problem.
But, to keep the similar logic you should create a collection of web browsers at class level. Something like:
private List<WebBrowser> _myWebBrowsers;
Then add to this list in your loop but do not call Dispose. You should only dispose of the browser when you're done with it.
That should get you closer though there are a few other potential issues with your code. You're allocating a borser object and thread for every time through a loop. This could quickly become unwieldy. You should use a thread management mechanism to throttle this process.
Simplified class:
class WebRunner
{
private List<string> _corruptList = new List<string>();
private List<WebBrowser> _browsers = new List<WebBrowser>();
public void Run()
{
_corruptList.Add("http://google.com");
_corruptList.Add("http://yahoo.com");
_corruptList.Add("http://bing.com");
DoDelete();
Console.ReadKey();
}
public void DoDelete()
{
if (_corruptList.Count < 1) return;
int counter = 1;
foreach (string listItem in _corruptList)
{
WebBrowser webBrowser = new WebBrowser();
_browsers.Add(webBrowser);
webBrowser.DocumentCompleted += new WebBrowserDocumentCompletedEventHandler(webBroswer_DocumentCompleted);
webBrowser.Navigated += new WebBrowserNavigatedEventHandler(webBrowser_Navigated);
webBrowser.Navigate(listItem);
if (counter % 10 == 0) Thread.Sleep(3000); // let app catch up every so often
counter++;
}
}
void webBrowser_Navigated(object sender, WebBrowserNavigatedEventArgs e)
{
Console.WriteLine("NAVIGATED: " + e.Url);
}
void webBroswer_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
{
Console.WriteLine("COMPLETED!");
try
{
WebBrowser webBrowser = sender as WebBrowser;
HtmlDocument doc = webBrowser.Document;
var button = doc.Body.Document.GetElementById("button");
button.InvokeMember("Click");
_browsers.Remove(webBrowser);
}
catch (Exception exp)
{
Console.WriteLine(exp.StackTrace);
MessageBox.Show(exp.Message);
}
}
}
You can access the WebBrowser document content using the following (you are missing body and need to type document to dynamic).
dynamic doc = browser.Document;
var button = doc.body.document.getElementById("button");
button.Click();
I found the solution very next day. Sorry for the late post by processing threads one by one by putting the statement after thread.sleep()
if (th.ThreadState == ThreadState.Aborted || th.ThreadState == ThreadState.Stopped)
Related
Hello i created a program which reads serial data from a scoreboard, then depending on the string the program separates the data into different boxes on the form and then to different txt files.The purpose of this, is to use those txt files for livestreaming purposes in basketball games.
It's the first time i work with serial data and i am not a very experienced programmer.
My problem, as the title of this post suggests is that every now and then without any reason i am loosing some packages. This is happening randomly , for example in 10 second period i could 1 package while the next one none or 4.
private void ReadData()
{
Thread MyThread = null;
{
try
{
ThreadStart ThreadMethod = new ThreadStart(ReadFromPort);
MyThread = new Thread(ThreadMethod);
}
catch (Exception e)
{
Console.WriteLine("Failed to create thread! " + e.Message);
return;
}
try
{
MyThread.Start();
}
catch (Exception e)
{
Console.WriteLine("The thread failed to start! " + e.Message);
}
}
}
//Recieves data and write them on textbox (optionally on a txt)
private void ReadFromPort()
{
while (Receiver == true)
{
try
{
int count = ComPort.BytesToRead;
System.Windows.Forms.Application.DoEvents();
byte[] data = new byte[count];
ComPort.Read(data, 0, data.Length);
currentMessage = Combine(currentMessage, data);
ReceivedData = (BitConverter.ToString(data));
if (count > 0)
{
if (chBoxUpdate.Checked)
{
DataType = count;
tempData = ReceivedData;
this.Invoke(new EventHandler(DisplayText));
if (chboxTxt.Checked)
{
this.Invoke(new EventHandler(ExportData));
}
}
else if (chBoxPrevious.Checked)
{
DataType = count;
tempData = ReceivedData;
this.Invoke(new EventHandler(ClearData));
this.Invoke(new EventHandler(DisplayText));
if (chboxTxt.Checked)
{
this.Invoke(new EventHandler(ExportData));
}
}
}
}
catch (Exception e)
{
}
}
}
//Displays Text
private void DisplayText(object sender, EventArgs e)
{
string temp;
Console.WriteLine(tempData+ " (" + tempData.Length.ToString() + ")");
try
{
if (tempData.Length == 38)// && ReceivedData.Substring(12, 5) == "03-02")
{
if (tempData.Substring(12, 5) == "03-02")
{
DataText.AppendText(tempData.Substring(24, 8));
DataText.AppendText("\n");
Blink.Text = "Reading...";
timer1.Start();
timer1.Enabled = true;
}
}
else
if (tempData.Length == 35)
{
if (tempData.Substring(12, 5) == "45-02")
{
AttackTime.AppendText(tempData.Substring(24, 5));
Blink.Text = "Reading...";
AttackTime.AppendText("\n");
timer1.Start();
timer1.Enabled = true;
}
}
else
if (tempData.Length == 29)
{
if (tempData.Substring(12, 5) == "03-36")
{
HomeScore.AppendText(tempData.Substring(21, 2));
Blink.Text = "Reading...";
HomeScore.AppendText("\n");
timer1.Start();
timer1.Enabled = true;
}
else
if (tempData.Substring(12, 5) == "03-46")
{
AwayScore.AppendText(tempData.Substring(21, 2));
Blink.Text = "Reading...";
AwayScore.AppendText("\n");
timer1.Start();
timer1.Enabled = true;
}
}
}
catch (ArgumentOutOfRangeException)
{
}
}
Keep in mind that tempData and ReceivedData are the same here and as the programs appears now there's not any particular reason to set tempData=ReceivedData.This is a part of a different,older code i used in the begining which i never changed but it doesn't effect the program.
At first i am using a thread to enable the ReadFromPort then if the program finds that there are available data in line it displays them with the DisplayText and it's using the ExportData to export the data to a txt.I think the problem is somewhere there but as i am not very experienced i can't tell where.
Are there any suggestions on how to improve my code? If you more details or information i can provide them.
You are using Invoke, this blocks the caller thread until the event has been processed, and since you are updating the UI this can take some time. This probably causes some buffer to overflow and data to be discarded. So you should do as little work as possible on the reading thread.
There are a few alternatives
Put the data on a concurrentQueue, and let the UI thread have a timer that periodically triggers a method that reads from the queue and updates the UI.
Put the data on a concurrentQueue, wrapped in a blocking collection, have another background thread iterate over the blocking collection, using GetConsumingEnumerable, and post the result to the main thread once done. This would be suitable if there is significant processing to be done.
Use BeginInvoke to post the received data to the main-thread, this does not wait for the call to complete, but probably have slightly higher overhead that the previous alternatives. I would recommend not accessing any UI properties from the background thread, since by default no property of a UI class is threadsafe. Even if you might get away with it if you are only reading, I would not take the chance.
I'm doing a post on a page that works as follows:
Click on a link and opens a text box, then accurately loaded to another post to save the contents of the input text.
Follows the code I'm using.
private void btnRobo_Click(object sender, EventArgs e)
{
// string[] paginasArray = new string[] { txtPaginas.Text };
string[] paginasArray = txtPaginas.Text.Split('\n');
foreach (string s in paginasArray)
{
// webBrowser1.Navigate(s);
if (s.Trim() != "")
runBrowserThread(new Uri(s));
}
}
private void runBrowserThread(Uri url)
{
var th = new Thread(() =>
{
var br = new WebBrowser();
br.DocumentCompleted += browser_DocumentCompleted;
br.Navigate(url);
Application.Run();
});
th.SetApartmentState(ApartmentState.STA);
th.Start();
}
void browser_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
{
var br = sender as WebBrowser;
if (br.Url == e.Url)
{
Console.WriteLine("Natigated to {0}", e.Url);
webBrowser1.Navigate(e.Url);
//ExecutaPostagem();
HtmlElementCollection elements = br.Document.GetElementsByTagName("input");
foreach (HtmlElement currentElement in elements)
{
if ((currentElement.GetAttribute("type") == "submit") && (currentElement.Name == "view_post"))
{
string postagem = txtPublicacao.Text;
HtmlElement elea = br.Document.GetElementById("u_0_0");
if (elea != null) elea.SetAttribute("value", postagem);
currentElement.InvokeMember("click");
Thread.Sleep(1000);
fullyLoaded = false;
}
}
Application.ExitThread(); // Stops the thread
}
}
After the page was loaded by the browser_DocumentCompleted need to run the code below, but how will I run the code if I am using thread and I'm no longer using the event browser_DocumentCompleted
var links = br.Document.GetElementsByTagName("A");
// var links = webBrowser1.Document.All;
foreach (HtmlElement link in links)
{
// if (link.InnerText != null)
if ((link.InnerText != null) && (link.InnerText.Contains("comentários")))
{
Thread.Sleep(1000);
MessageBox.Show(link.InnerText.ToString()); //.Contains("comentários").ToString());
//MessageBox.Show(link.GetAttribute("InnerText"));
// MessageBox.Show(link.GetAttribute("className"));
link.InvokeMember("click");
break;
}
}
Don't use WebClient or Threads at all.
Instead use http://restsharp.org/ to make the call to your website. Restsharp is super easy to work with.
And instead of using Threads use async/await (restsharp also supports this) to modify the UI later.
I need to browse a set of url adresses (yes another web scraper..).
I want to use tasks. But I have problem returning AFTER the browser is finished.
To be sure the site is fully loaded I have to jump to document_completed and from there I call the Navigate method with another url.
Something like this:
private WebBrowser browser;
private List<string> urlsToVisit;
int urlCounter = 0;
public PageBrowser(List<string> urls) //constructor
{
urlCounter = 0;
urlsToVisit = urls;
browser = new WebBrowser(); //one instance of browser for all urls
browser.ScriptErrorsSuppressed = true;
browser.DocumentCompleted += browser_DocumentCompleted;
}
//this I want to call from somewhere else and return true AFTER it opens all sites
public bool Run()
{
VisitPages();
return true;
}
private void VisitPages()
{
if (urlCounter < urlsToVisit.Count)
{
browser.Navigate(urlsToVisit[urlCounter]);
urlCounter++;
}
else
{
browser.Dispose();
}
}
private void browser_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
{
if (e.Url.AbsolutePath != (sender as WebBrowser).Url.AbsolutePath) return;
System.Threading.Thread.Sleep(3000); //random interval between requests
VisitPages();
}
I am pretty sure, the solution is very simple but I just don't see it..
Thank you
Petr
I would suggest you to use a WebClient instead of a WebBrowser UI component since you don't seem to need to render anything. It seems as you just want to send requests to your list of urls and return true if everything went well, and return false if any type of problem occured.
private async Task<bool> VisitUrls()
{
var urls = new List<string>
{
"http://stackoverflow.com",
"http://serverfault.com/",
"http://superuser.com/"
};
var success = await BrowseUrls(urls);
return success;
}
private async Task<bool> BrowseUrls(IEnumerable<string> urls)
{
var timeoutWebClient = new WebClient();
foreach (var url in urls)
{
try
{
var data = await timeoutWebClient.DownloadDataTaskAsync(url);
if (data.Length == 0) return false;
}
catch (Exception ex)
{
return false;
}
}
//Everything went well, returning true
return true;
}
I am currently writing a simple WPF 3.5 application that utilizes the SharePoint COM to make calls to SharePoint sites and generate Group and User information. Since this process takes awhile I want to show a ProgressBar while the groups are being generated. The desired process is as follows:
User enters url and clicks button to fetch site data.
ProgressBar begins animation
Groups are generated and names are added to a ListView
Upon completion ProgressBar animation ends
The problem I am running into is that the UI is never updated. Neither the ProgressBar or the ListView makes any changes. If anyone has any ideas to help with the code below it would be greatly appreciated.
private void GetGroupsAndUsersButton_Click(object sender, RoutedEventArgs e)
{
siteUrl = "";
if (SiteURLTextBox.Text.Length > 0)
{
FetchDataProgressBar.IsIndeterminate = true;
mWorker = new BackgroundWorker();
mWorker.DoWork += new DoWorkEventHandler(worker_DoWork);
mWorker.WorkerSupportsCancellation = true;
mWorker.RunWorkerCompleted += new RunWorkerCompletedEventHandler(worker_RunWorkerCompleted);
mWorker.RunWorkerAsync();
}
else
{
System.Windows.MessageBox.Show("Please enter a URL for the SharePoint site you wish to retrieve data");
}
}
private void worker_DoWork(object sender, System.ComponentModel.DoWorkEventArgs e)
{
siteUrl = SiteURLTextBox.Text;
GroupListView.ItemsSource = null;
try
{
using (SPSite site = new SPSite(siteUrl))
{
SPWeb web = site.OpenWeb();
SPGroupCollection collGroups = web.SiteGroups;
if (GroupNames == null)
GroupNames = new List<string>();
foreach (SPGroup oGroup in collGroups)
{
GroupListView.Items.Add(new ListViewItem() { Content = oGroup.Name });
}
foreach (ListViewItem item in GroupListView.Items)
{
item.MouseLeftButtonUp += item_MouseLeftButtonUp;
}
}
}
catch (Exception ex)
{
System.Windows.MessageBox.Show("Unable to locate a SharePoint site at: " + siteUrl);
}
}
private void worker_RunWorkerCompleted(object sender, System.ComponentModel.RunWorkerCompletedEventArgs e)
{
FetchDataProgressBar.Dispatcher.Invoke(System.Windows.Threading.DispatcherPriority.Normal,
new Action(
delegate()
{
FetchDataProgressBar.IsIndeterminate = false;
}
));
}
At first you need to support ProgressChanged events.
Update your BackgroundWorker initialization to:
GroupListView.ItemSource = null;
mWorker = new BackgroundWorker();
mWorker.DoWork += new DoWorkEventHandler(worker_DoWork);
mWorker.WorkerSupportsCancellation = true;
mWorker.WorkerReportsProgress = true;
mWorker.ProgressChanged += OnProgressChanged;
mWorker.RunWorkerCompleted +=
new RunWorkerCompletedEventHandler(worker_RunWorkerCompleted);
mWorker.RunWorkerAsync(SiteURLTextBox.Text);
After that you have to add a OnProgressChanged handler:
private void OnProgressChanged(object sender, ProgressChangedEventArgs e)
{
FetchDataProgressBar.Value = e.ProgressPercentage;
ListViewItem toAdd = (ListViewItem)e.UserState;
toAdd.MouseLeftButtonUp += item_MouseLeftButtonUp;
GroupListView.Items.Add(toAdd);
}
Therefore you have to change your DoWork:
private void worker_DoWork(object sender, System.ComponentModel.DoWorkEventArgs e)
{
BackgroundWorker worker = (BackgroundWorker)sender;
try
{
using (SPSite site = new SPSite((String)e.Argument))
{
SPWeb web = site.OpenWeb();
SPGroupCollection collGroups = web.SiteGroups;
if(GroupNames == null)
GroupNames = new List<string>();
int added = 0;
foreach(SPGroup oGroup in collGroups)
{
added++;
ListViewItem tmp = new ListViewItem() {
Content = oGroup.Name
};
worker.ReportProgress((added * 100)/collGroups.Count,tmp);
}
}
}
catch (Exception ex)
{
MessageBox.Show("Unable to locate a SharePoint site at: " + siteUrl);
}
}
That's because you're not allowed to change GUI on DoWork.
After that, each ListViewItem is added separately to your ListView. I would also recommend, that your URL is passed as an argument to RunWorkerAsync.
Edit: Add percentage to OnProgressChanged.
In your DoWork method, you are manipulating WPF controls in code on a background thread, which you are not supposed to do. Actually, you should receive errors like "Cannot access control from other thread". Probably those exceptions are caught by your catch-all error handler, and maybe even the MessageBox doesn't work from the background thread.
As a quick fix, you would have to make siteURL and collGroups class fields, move everything before the using block to your GetGroupsAndUsersButton_Click method, and everything starting with the first foreach loop to the RunworkerCompleted event, so that all code which accesses controls runs on the UI thread.
Another thing you should change is that you should not create ListViewItems in code, but use a DataTemplate instead... this is not connected to your problem, though.
You'll need:
mWorker.WorkerReportsProgress = true;
mWorker.ProgressChanged +=
new ProgressChangedEventHandler(worker_ProgressChanged);
Then in your DoWork you'll need to call:
var worker = (BackgroundWorker)sender;
worker.ReportProgress(progressAmount);
Good worked example here: http://msdn.microsoft.com/en-us/library/cc221403(v=vs.95).aspx
In C#, how do you make a program only process one thing at a time? I've been working on a patching system, and I think I have the coding all correct but can't test it because a lot of the functions are trying to process all at once when they need to be processing in an order. The program doesn't even let the display shown up before it starts trying to process everything. Because none of them return a value other then the main function all the functions are set to void. I thought about maybe using a return value inside of a loop to make sure the program is finished with that step first before moving on but it still leaves the problem of the program not even showing up until everything is done processing which its suppose to show the progress of everything. Any suggestions of tips?
Edit: I don't know what to post in the code, so im posting all the main functions:
public void DSP_Load(object sender, EventArgs e)
{
if (v1 >= v2)
{
File_Progress_Title.Text = "100%";
Update_Status.Text = "Divine Shadows is currently up to date.";
Application.DoEvents();
Process.Start("Divine Shadows.exe");
Close();
}
else
{
Update_Status.Text = "Checking For Updates...";
Application.DoEvents();
if (!Directory.Exists(tempFilePath))
{
Directory.CreateDirectory(tempFilePath);
}
using (SqlCon = new MySqlConnection(connString))
{
SqlCon.Open();
string command = "SELECT * FROM version where version > '" + v1 + "' ORDER BY version LIMIT 1";
MySqlCommand GetLatestVersion = new MySqlCommand(command, SqlCon);
using (MySqlDataReader DR = GetLatestVersion.ExecuteReader())
{
while(DR.Read())
{
do
{
string LatestVersion = Convert.ToString(DR.GetValue(1));
string WebURL = Convert.ToString(DR.GetValue(2));
update.DownloadFileAsync(new Uri(WebURL), tempFilePath + "patch" + LatestVersion + ".zip");
update.DownloadProgressChanged += new DownloadProgressChangedEventHandler(download);
update.DownloadFileCompleted += new AsyncCompletedEventHandler(extration);
Application.Restart();
}
while (v1 < v2);
Process.Start("Divine Shadows.exe");
Close();
}
}
}
}
}
public void download(object sender, DownloadProgressChangedEventArgs e)
{
if (v1 >= v2)
{
File_Progress_Title.Text = "100%";
Update_Status.Text = "Divine Shadows is currently up to date.";
Application.DoEvents();
Process.Start("Divine Shadows.exe");
Close();
}
else
{
Update_Status.Text = "Downloading Updates...";
Application.DoEvents();
File_Progress_Display.Value = e.ProgressPercentage;
File_Progress_Title.Text = Convert.ToString(e.ProgressPercentage) + "%";
}
}
public void extration(object sender, AsyncCompletedEventArgs e)
{
if (v1 >= v2)
{
File_Progress_Title.Text = "100%";
Update_Status.Text = "Divine Shadows is currently up to date.";
Application.DoEvents();
Process.Start("Divine Shadows.exe");
Close();
}
else
{
Update_Status.Text = "Installing Updates, Please Wait...";
Application.DoEvents();
UnzipFile(extactFile, extractLocation);
}
}
public static void UnzipFile(string extactFile, string extractLocation)
{
try
{
FastZip fastZip = new FastZip();
fastZip.CreateEmptyDirectories = false;
fastZip.ExtractZip(extactFile, extractLocation, FastZip.Overwrite.Always, null, null, null, false);
}
catch (Exception ex)
{
throw new Exception("Error unzipping file \"" + extactFile + "\"", ex);
}
File.Delete(extactFile);
}
Your problem is not WebClient() specific, its about how your application is working with threads.
In general, winforms applications have one GUI Thread. This thread is used to executed your methods and also updating the user interface. If you start a long term process, the gui thread gets locked till the operation is finished. Thats the reason why your display is not shown.
You can solve that problem by implementing the BackgroundWorker. On that website you can also find an example how to implement it. Let the BackgroundWorker do your patching process and use events inside the BackgroundWorker.RunWorkerAsync() method to update your GUI.
If you are using c#4 or newer you can use the Task Parallel Library to perform tasks asynchronously, thus leaving your UI response while thing are being downloaded. First of all you need a reference:
using System.Threading.Tasks;
And some code:
public void YourMainFunction()
{
var urls = new List<string>();
urls.Add("http://google.com");
urls.Add("http://yahoo.com");
foreach(var url in urls)
{
Task.Factory.StartNew<DownloadResult>(() =>
DownloadIt(url))
.ContinueWith(WorkDone, TaskScheduler.FromCurrentSynchronizationContext());
}
}
private class DownloadResult
{
public string Url {get; set;}
public string Result {get; set;}
}
private DownloadResult DownloadIt(string url)
{
var downloadResult = new DownloadResult{ Url = url };
var client = new WebClient();
downloadResult.Result = client.DownloadString(url);
return downloadResult;
}
private void WorkDone(Task<DownloadResult> task)
{
if(task.IsFaulted)
{
//An exception was thrown
MessageBox.Show(task.Exception.ToString());
return;
}
//Everything went well
var downloadResult = task.Result;
//Here you can update your UI to reflect progress.
MessageBox.Show(downloadResult.Result);
}