Develop Reference

C# live data stream from MySQL to DataGridView

I've been struggling to find efficient solution to get live data from MySQL database to datagridview. In my database values are changing every second or even faster. I need to get live feed directly to my windows app. The only solution I could found is to use async, but then app get laggy when trying to move it arround or do other tasks. My attempt:
public async void refreshOutput()
{
while (true)
{
DataTable dbResults = Connect_ToDB.executeQuery("SELECT * FROM mysql.test_table;");
dataGridView2.DataSource = dbResults;
await Task.Delay(200);
}
}
Also I tried:
private void button24_Click(object sender, EventArgs e)
{
DataTable data = Connect_ToDB.executeQuery("SELECT * FROM mysql.test_table;");
BindingSource bSource = new BindingSource();
bSource.DataSource = data;
dataGridView2.DataSource = bSource;
}
In any case i always need to push the button again to get new data. Is there a way to get it live without disturbing the application background processes? Thank you!

Add a progress bar while the a table is being load it to the DataGridView In C#

I need some help on adding a progress bar while the datagridview is being load it. I already have my code that loads the datagridview but as we know the loading takes time to finish loading depending of the records. So, I would like to add a progress bar loading and a label having a the count from 1 to 100 to complete.
I know there is a way using the background work handle event, but not sure how that make it work. I would like something simple but can do the work I need.
my code works great fills the datagridview as I want. but I need to add the progress bar while loading the datagridview.
change the code please review and let me know if I missed something.
So I made the changes and seems to work now, but there is an issue the progress bar does not work immediately takes a few seconds and then I can see the progress bar to move to 100%. Why it does that?
second issue after loading the datagridview the progress bar color goes away after I click the message MessageBox.Show("Successful Completion.");
here is a test image after my combo box select the value we want and display the datagridview
here I made the new changes to the program, but for some reason after I select the combobox the datagridview populates correctly but then I try again sometimes it fails and gives me this error
namespace DatagridViewProgressBar
{
public partial class Form1 : Form
{
//datagridview, bindingsource, data_apapter global objects variables
private DataGridView dataGridView = new DataGridView();
private BindingSource bindingSource = new BindingSource();
private SqlDataAdapter dataAdapter = new SqlDataAdapter();
DataTable dt = new DataTable();
//class objects
Databases lemars = new Databases();
Databases schuyler = new Databases();
Databases detroitlakeskc = new Databases();
public Form1()
{
InitializeComponent();
// To report progress from the background worker we set this property
dbWorker = new BackgroundWorker();
dbWorker.DoWork += new DoWorkEventHandler(dbWorker_DoWork);
dbWorker.ProgressChanged += new ProgressChangedEventHandler(dbWorker_ProgressChanged);
dbWorker.RunWorkerCompleted += new RunWorkerCompletedEventHandler(dbWorker_RunWorkerCompleted);
dbWorker.WorkerReportsProgress = true;
dbWorker.WorkerSupportsCancellation = true;
}
private void btn_Exit_Click(object sender, EventArgs e)
{
this.Close();
}
private void comboBox_Database_SelectedIndexChanged(object sender, EventArgs e)
{
if (comboBox_Database.SelectedItem.ToString() == "LeMars21St")
{
if (dbWorker.IsBusy != true)
{
dbWorker.RunWorkerAsync();
}
}
}
private void GetTableToDataGridView()
{
//prgBar_DataGridViewLoading
DatabaseColumns Obj = new DatabaseColumns();
String SqlcmdString = #"SELECT invoice, shipment, Project, invoiceDateTB, CreatedDate, typeName, exportedDate, statusName, total, import_status, Time_Completed, ERROR_DESCRIPTION FROM dbo.AllInvoicesInReadyStatus";
SqlDataReader reader;
int progress;
using (SqlConnection conn = new SqlConnection(lemars._LeMarsConnectionString))
{
reader = null;
SqlCommand Sqlcmd = new SqlCommand(SqlcmdString, conn);
conn.Open();
reader = Sqlcmd.ExecuteReader();
if (reader.HasRows)
{
try
{
dt.Load(reader);
for (int i = 0; i < dt.Rows.Count; i++)
{
Obj.Invoice = dt.Rows[i]["invoice"].ToString();
Obj.Shipment = dt.Rows[i]["shipment"].ToString();
Obj.Project = dt.Rows[i]["Project"].ToString();
Obj.InvoiceDateTB = Convert.ToDateTime(dt.Rows[i]["invoiceDateTB"]);
Obj.CreatedDate = Convert.ToDateTime(dt.Rows[i]["CreatedDate"]);
Obj.TypeName = dt.Rows[i]["typeName"].ToString();
Obj.ExportedDate = Convert.ToDateTime(dt.Rows[i]["exportedDate"]);
Obj.StatusName = dt.Rows[i]["statusName"].ToString();
Obj.Total = Convert.ToDecimal(dt.Rows[i]["total"]);
Obj.ImportStatus = dt.Rows[i]["import_status"].ToString();
if (!Convert.IsDBNull(dt.Rows[i]["Time_Completed"]))
{
Obj.TimeCompleted = Convert.ToDateTime(dt.Rows[i]["Time_Completed"]);
}
Obj.ErrorDescription = dt.Rows[i]["ERROR_DESCRIPTION"].ToString();
progress = i * 100 / dt.Rows.Count;
dbWorker.ReportProgress(progress);
Thread.Sleep(500);
}
}
finally
{
conn.Close();
}
}
}
}
private void dbWorker_DoWork(object sender, DoWorkEventArgs e)
{
GetTableToDataGridView();
dbWorker.ReportProgress(100);
}
private void dbWorker_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
progressBar_GetTasks.Value = e.ProgressPercentage;
// eg: Set your label text to the current value of the progress bar
lbl_PercentageCount.Text = (progressBar_GetTasks.Value.ToString() + "%");
}
private void dbWorker_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
dataGridView_ShowAllData.DataSource = dt;
if (e.Cancelled)
{
MessageBox.Show("Process Cancelled.");
}
else if (e.Error != null)
{
MessageBox.Show("Error occurred: " + e.Error.Message);
}
else
{
MessageBox.Show("Successful Completion.");
}
//progressBar_GetTasks.Value = 0;
}
private void btn_CancelOperation_Click(object sender, EventArgs e)
{
if (dbWorker.IsBusy)
{
dbWorker.CancelAsync();
}
}
}
}

Firstly, SELECT * is a bad idea, regardless of how many columns you have, or need. Explicitly stating which columns you want opens up possibilities for using indices and reduces maintainability issues with your code.
Secondly, your main question. I have done something similar recently, and can give some pointers. I am going not going to immediately apply this to your code-snippet, because I think that will complicate things.
EDIT
For thread safety purposes, the code inside dbWorker_DoWork() should not try to access form elements which were created in the main thread. There are obviously ways around this, and once you get to the dbWorker_RunWorkerCompleted() function, you are back in the main thread and you have full access to the necessary form elements.
END EDIT
1) Your form. You need a backgroundworker to do the work, as well as three callback functions to handle what is going on. A progress bar is assumed to be on the form as well (System.Windows.Forms.ProgressBar).
...
using System.ComponentModel;
...
public partial class YourForm : Form
{
BackgroundWorker dbWorker;
...
public YourForm()
{
...
dbWorker = new BackgroundWorker();
dbWorker.DoWork += new DoWorkEventHandler(dbWorker_DoWork);
dbWorker.ProgressChanged += new ProgressChangedEventHandler(dbWorker_ProgressChanged);
dbWorker.RunWorkerCompleted += new RunWorkerCompletedEventHandler(dbWorker_RunWorkerCompleted);
dbWorker.WorkerReportsProgress = true;
dbWorker.WorkerSupportsCancellation = true;
...
}
...
public void dbWorker_DoWork(object sender, DoWorkEventArgs e)
{
// This is where you put your GetTableToDataGridView() code.
// Add a line inside the loop, for reporting on progress:
// dbWorker.ReportProgress((int)(currentIteration * 100 / totalIterations));
// At the end of the process, set the progress bar to 100% (optional)
dbWorker.ReportProgress(100);
}
public void dbWorker_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
progressBar.Value = e.ProgressPercentage;
// Here you can also do other things, which depend on the progress.
// eg: Set your label text to the current value of the progress bar.
}
public void dbWorker_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
if (e.Cancelled)
{
MessageBox.Show("Process Cancelled.");
}
else if (e.Error != null)
{
MessageBox.Show("Error occurred: "+ e.Error.Message.");
}
else
{
MessageBox.Show("Successful Completion.");
}
progressBar.Value = 0;
}
}
2) Starting the process (this could go in your form load function, or if you start the process manually, in an OnClick event):
...
if (!dbWorker.IsBusy)
{
dbWorker.RunWorkerAsync();
}
...
3) Cancelling the process (if you have a button for cancelling, then this would go in the OnClick event code):
...
if (dbWorker.IsBusy)
{
dbWorker.CancelAsync();
}
...

A couple obvious problems:
Since you have the connection inside a using block, the explicit conn.Close() is unnecessary. The using mechanism will automatically close it even if an exception occurs. In fact, unless you have a need to handle exceptions at this level, you can remove the try block altogether.
Never use SELECT *. You are retrieving a ton of data you don't need and this is probably why it's going slow in the first place.
To the meat of your question: BackgroundWorker is your friend!
I'm more familiar with doing data binding with WPF, so you'll need to do some legwork on your own. But the basic idea is that you have your progressbar's visibility bound to a variable, then you update that variable:
when you launch the thread, to make it visible (using the IsIndeterminate property is useful since there's no real way to do percentages for a SQL query)
when the thread finishes, to hide the progress bar.
In this way, you get the animated progress bar while the query is running.
You can set the mouse cursor to busy/arrow in the same way.

C # memory management and performance

The criteria is simple. I am developing a winform application using C# 4.0 and have a win form having a grid view. On the load event, I retrieve data from database and then assign the datasource to the gridview. I have used BackGroundWorker object for database retrieval. This is nice. My GUI is responsive. The records are more than about 10 lac. So i fill my datatable using background worker and then assign the datatable to gridview. The problem is, my system gets slower. When i minimize mdi form of my application and try to do other things like opening internet browser and stuff, my pc gets slower. I have core i3 2ith 2GB RAM. A grid having 10 lac records makes my system slow. How to manage memory in this case? Here is the code:
BackgroundWorker bWorker;
DataTable dt;
public Form1()
{
InitializeComponent();
bWorker = new BackgroundWorker();
bWorker.DoWork += new DoWorkEventHandler(m_oWorker_DoWork);
bWorker.ProgressChanged += new ProgressChangedEventHandler(m_oWorker_ProgressChanged);
bWorker.RunWorkerCompleted += new RunWorkerCompletedEventHandler(m_oWorker_RunWorkerCompleted);
bWorker.WorkerReportsProgress = true;
bWorker.WorkerSupportsCancellation = true;
}
void m_oWorker_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
dataGridView1.DataSource = dt;
pictureBox1.Visible = false;
//progressBar1.Style = ProgressBarStyle.Blocks;
//label1.Text = "Data Loaded Successfully!";
this.Hide();
this.Show();
}
void m_oWorker_DoWork(object sender, DoWorkEventArgs e)
{
dt = getDataTable();
//bWorker.ReportProgress(100);
}
void m_oWorker_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
//Here you play with the main UI thread
//progressBar1.Value = e.ProgressPercentage;
}
private void Form1_Load(object sender, EventArgs e)
{
bWorker.RunWorkerAsync();
}
private DataTable getDataTable()
{
string conStr = ConString;
SqlConnection con = new SqlConnection(conStr);
SqlCommand cmd = new SqlCommand("Select *from testtable",con);
SqlDataAdapter da = new SqlDataAdapter(cmd);
dt = new DataTable();
try
{
da.Fill(dt);
return dt;
}
catch (Exception ex)
{
MessageBox.Show(ex.ToString());
return null;
}
}
Please guide me how to manage memory so that system should not get slower. If i dispose datatable after assigning it to grid, would that help?
I am actually a beginner. And want a complete guidance about memory management. Application should be fast. Thanx

Most likely your problem is that your DataGridView creates a row for each row in the result set. You should read about what the virtual display mode is, the VirtualMode property and how to implement it.

You can't expect to load a million rows of data into a GUI control and expect everything to behave well, can you?
The problem is here:
SqlCommand cmd = new SqlCommand("Select *from testtable",con);
There's no way you should be SELECTing the entire table. You'll want to LIMIT your query to only those rows that the GUI can reasonably display. Shooting completely in the dark, I'd say somewhere on the order of a thousand rows at maximum.

My assumation is you try to load all those rocords one time in getDataTable. Why not load by demand? I.e., only load the first level load. If all those are first level, you can separate into multi pages whose size may be a screen width.

You can use the DataGrid control with setting all appropriate select, insert, update, delete statements, add paging rules and this will also to the trick. The grid will not load all the records in once, but instead, it will use the clever internal functionality to select from the db and display only the appropriate data.
Hope this helps also.

Getting increasingly slow to write into textbox using textBox_TextChanged event and if statement together.

I am making a dictionary using C# and Windows forms. In my dictionary I have a textBox where the user can search for a word to get the meaning. I also have some options in a comboBox where the user can choose a language to see the meaning for that language. Because I am making the dictionary for different languages.
My code looks like:
private void textBox1_TextChanged(object sender, EventArgs e)
{
string word = textBox1.Text;
SqlCeConnection con = new SqlCeConnection(#"Data Source=" + Directory.GetCurrentDirectory() + #"\Database\condrokothadb.sdf;Password=000;");
//in combobox there are 2 option(language)
//if select one language(option) from combobox
if(mood=="bangla")
{
SqlCeDataAdapter b = new SqlCeDataAdapter("SELECT english,bangla FROM dic WHERE (bangla like '" + word + "%')", con);
DataTable tt = new DataTable();
b.Fill(tt);
dataGridView1.DataSource = tt;
}
else //by default english language is selected
{
using (con)
{
con.Open();
using (SqlCeDataAdapter b = new SqlCeDataAdapter("SELECT english,bangla FROM dic WHERE (english like '" + word + "%')", con))
{
DataTable tt = new DataTable();
b.Fill(tt);
dataGridView1.DataSource = tt;
}
}
}
}
private void comboBox1_SelectedIndexChanged(object sender, EventArgs e)
{
if (((ComboBox)sender).SelectedItem.ToString() == "Bangla")
{
mood = "bangla";
}
else if (((ComboBox)sender).SelectedItem.ToString() == "English")
{
mood = "english";
}
}
My problem is that when a user want to write something into the textbox it is getting so much slower to write. How can I overcome that?

This is an interesting question and here is how I would solve it.
I added a timer that starts to count as you type the first character into your textBox, and for every character you add the timer resets. The application wont execute the part where you search through the database untill the timer reaches a set number of ticks.
Make sure you add a timer and a backgroundWorker into the form. Create the events through the properties window and add this code:
int timerTicks;
int waitUntill = 10; //10 = 1 second. Change this to decide how long the application will wait.
string mood;
string word;
string langConnection;
DataTable tt;
SqlCeConnection con;
SqlCeDataAdapter b;
private void textBox1_TextChanged(object sender, EventArgs e)
{
if (!timer1.Enabled)
timer1.Start();
//Reset the timer when a character is entered in textBox1.
timerTicks = 0;
}
private void timer1_Tick(object sender, EventArgs e)
{
timerTicks++;
if (timerTicks > waitUntill && !backgroundWorker1.IsBusy && comboBox1.SelectedItem != null)
{
//Stop the timer and begin the search in a background thread.
timer1.Stop();
word = textBox1.Text;
mood = comboBox1.SelectedItem.ToString();
backgroundWorker1.RunWorkerAsync();
}
}
private void backgroundWorker1_DoWork(object sender, DoWorkEventArgs e)
{
tt = new DataTable();
con = new SqlCeConnection(#"Data Source=" + Directory.GetCurrentDirectory() + #"\Database\condrokothadb.sdf;Password=000;");
langConnection = String.Format("SELECT english,bangla FROM dic WHERE ({0} like '{1}%')", mood, word);
using (con)
{
con.Open();
b = new SqlCeDataAdapter(langConnection, con);
b.Fill(tt);
}
}
private void backgroundWorker1_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
dataGridView1.DataSource = tt;
}
Note that you don't need the comboBox1_SelectedIndexChanged event for this to work.
EDIT:
To make the actual search run faster you would have to open the connection at startup and keep it open throughout the entirety of the execution, like other answers suggest as well. You should be able to figure that out for yourself though.

Instead of getting data from server in every text changed you can get all data in grid at once and then filter them with dataview.
// to get data in grid
CustomList<wordlistDAO> WordList = null;
WordList = WordListBLL.GetAllWord();
GridWord.DataSource = WordList ;
// create word datatable for filtering
DataTable dtWord = null;
dtWord = new DataTable();
foreach (DataGridViewColumn colu in GridWord.Columns)
dtWord .Columns.Add(new DataColumn(colu.HeaderText));
foreach (DataGridViewRow row in GridWord.Rows)
{
DataRow dr = dtWord.NewRow();
foreach (DataGridViewCell cell in row.Cells)
dr[row.Cells.IndexOf(cell)] = cell.Value;
dtWord .Rows.Add(dr);
}
//create data view
DataView wordlistview = new DataView();
wordlistview = new DataView(dtWord);
// filter dataview and show in grid
if (cboLanguage.Text == "Bangla")
{
wordlistview.RowFilter = "bangla LIKE '" + txtSearchValue.Text.Trim().ToUpper() + "%'";
}
else
{
wordlistview.RowFilter = "english LIKE '" + txtSearchValue.Text.Trim().ToUpper() + "%'";
}
GridWord.DataSource = wordlistview;

The main problem is that, every time you press a key in the textbox, you are creating a database connection and querying the database. This is very inefficient! Also, because your Bangla code doesn't dispose of the connection, you may be keeping a lot of objects referenced that don't need to be, so you may find performance is degrading over time.
One basic suggestion would be to use a single connection instead of opening new connections for each keypress. This will reduce somewhat the time taken for the query. Realistically though, I suspect that you want to load the full content of the data at once, and run your query in-memory. This will give you much better speed.
Running your query on a background thread will help maintain the responsiveness of your UI, but will potentially end up with lots of queries running at once trying to catch up with the user's typing.
A better solution is to consider running an "idle-timer," and only starting the query when the user has stopped typing for a short amount of time. I'd recommend still using a background thread for this. You won't query the database for every keypress, and you won't affect the responsiveness of the UI.

Delay could be due to large amount of data in the database and you're calling database for every text changed event.
What I would suggest is to get all the data into DataView and keep filtering and binding the grid with results from view. That way you can minimise the number of times database is called.

Doing a search in the database on any keystroke is a bad practice. As you already experienced, it makes the UI very slow. A better option would be to do the search in a background thread, and also not for every keystroke. You can wait some time (0.5 seconds for example) before doing the search. If the user pressed another key in the meantime, expand the wait again to another 0.5 seconds.

problem with RX and web service collection loading wp7

I'm beginner with C# and wp7 platform and I have some problem with good idea to get request from web service.
I made webservice in PHP (nusoap - WSDL) and everything is working fine in "normal" using.
Now I have ObservableCollection saved in IsolatedStorage with I load when Page is open (List of watched stacks exchange). Then I want to refresh data for every item from web service.
I don't know whether this is a good idea.
Code:
private GPWWebservicePortTypeClient client = new GPWWebservicePortTypeClient();
private ObservableCollection<WebServiceClass.ItemGetValues> StoredStock =
new ObservableCollection<WebServiceClass.ItemGetValues>();
public const string _fileName = "listaObserwowanych.xml";
public Page()
{
InitializeComponent();
DataContext = App.ViewModel;
this.Loaded += new RoutedEventHandler(Page_Loaded);
client.GetLastValueCompleted +=
new EventHandler<GetLastValueCompletedEventArgs>(client_GetLastValueCompleted);
foreach (var itemGetValuese in App.ViewModel.Items)
{
client.GetLastValueAsync(itemGetValuese.name);
}
var o =
Observable.FromEvent<GetLastValueCompletedEventArgs(client,"GetLastValueCompleted")
.Subscribe(setList);
}
void client_GetLastValueCompleted(object sender, GetLastValueCompletedEventArgs e)
{
if (e.Error != null)
{
MessageBox.Show(Convert.ToString(e.Error));
}
else
{
ObservableCollection<WebServiceClass.ItemGetValues> ListValues =
(ObservableCollection<WebServiceClass.ItemGetValues>)
JsonConvert.DeserializeObject(e.Result,
typeof(ObservableCollection<WebServiceClass.ItemGetValues>));
StoredStock.Add(ListValues[0]);
}
}
private void setList(IEvent<GetLastValueCompletedEventArgs> ex)
{
List.ItemsSource = StoredStock;
}
void Page_Loaded(object sender, RoutedEventArgs e)
{
App.ViewModel.LoadData();
List.ItemsSource = App.ViewModel.Items;
}
Like u see I use RX to call method client_GetLastValueCompleted add store result to auxiliary variable (StoredStock). Then refresh List in setList method, but that method is client_GetLastValueCompleted what is not soo good idea, becouse I need to run that method only when all of runned GetLastValueAsync in foreach is completed.
Second problem: becouse of async web service method StoredStock sometime have different order than App.ViewModel.Items .
Any good idea how to do that in right way?
Best regards,
Lukas

You're really mixing up a number of ways to call web services and Rx. You really need to decide on a single way and stick to it.
If you're going to use Rx, then you'll have something like this:
public Page()
{
InitializeComponent();
DataContext = App.ViewModel;
this.Loaded += new RoutedEventHandler(Page_Loaded);
}
void Page_Loaded(object sender, RoutedEventArgs e)
{
App.ViewModel.LoadData();
var storedStock =
new ObservableCollection<WebServiceClass.ItemGetValues>();
List.ItemsSource = storedStock;
var values =
Observable.Using<WebServiceClass.ItemGetValues, GPWWebservicePortTypeClient>
(() => new GPWWebservicePortTypeClient(), ws =>
{
var clientGetLastValue = Observable
.FromAsyncPattern<string, GetLastValueResponse>
(ws.BeginGetLastValue, ws.EndGetLastValue);
Func<string, WebServiceClass.ItemGetValues> deserializeFirst = r =>
((List<WebServiceClass.ItemGetValues>)JsonConvert
.DeserializeObject(r,
typeof(List<WebServiceClass.ItemGetValues>)))
.First();
return
from item in App.ViewModel.Items
from e in clientGetLastValue(item)
select deserializeFirst(e.Result);
});
values.Subscribe(storedStock.Add);
}
You'll have to get the right method call names for your web service client, but the code should roughly be right. Let me know how you go.
I corrected the code above. Should have returned the query inside the Using call rather than assign it to values.
I corrected the call to FromAsyncPattern to use the correct method names and return type from the actual web service reference class sent via email.
It should look like this:
Observable.FromAsyncPattern<string, GetLastValueResponse>
(ws.BeginGetLastValue, ws.EndGetLastValue);

If you're a beginner with C#, try to avoid RX for the time being. It is a cool technology, but if you use it without clear understanding of what is going on, it will bring more problems than solve.
Use a simple event, and when each async item arrives, locate and update the correspondent one in the stored list.