The criteria is simple. I am developing a winform application using C# 4.0 and have a win form having a grid view. On the load event, I retrieve data from database and then assign the datasource to the gridview. I have used BackGroundWorker object for database retrieval. This is nice. My GUI is responsive. The records are more than about 10 lac. So i fill my datatable using background worker and then assign the datatable to gridview. The problem is, my system gets slower. When i minimize mdi form of my application and try to do other things like opening internet browser and stuff, my pc gets slower. I have core i3 2ith 2GB RAM. A grid having 10 lac records makes my system slow. How to manage memory in this case? Here is the code:
BackgroundWorker bWorker;
DataTable dt;
public Form1()
{
InitializeComponent();
bWorker = new BackgroundWorker();
bWorker.DoWork += new DoWorkEventHandler(m_oWorker_DoWork);
bWorker.ProgressChanged += new ProgressChangedEventHandler(m_oWorker_ProgressChanged);
bWorker.RunWorkerCompleted += new RunWorkerCompletedEventHandler(m_oWorker_RunWorkerCompleted);
bWorker.WorkerReportsProgress = true;
bWorker.WorkerSupportsCancellation = true;
}
void m_oWorker_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
dataGridView1.DataSource = dt;
pictureBox1.Visible = false;
//progressBar1.Style = ProgressBarStyle.Blocks;
//label1.Text = "Data Loaded Successfully!";
this.Hide();
this.Show();
}
void m_oWorker_DoWork(object sender, DoWorkEventArgs e)
{
dt = getDataTable();
//bWorker.ReportProgress(100);
}
void m_oWorker_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
//Here you play with the main UI thread
//progressBar1.Value = e.ProgressPercentage;
}
private void Form1_Load(object sender, EventArgs e)
{
bWorker.RunWorkerAsync();
}
private DataTable getDataTable()
{
string conStr = ConString;
SqlConnection con = new SqlConnection(conStr);
SqlCommand cmd = new SqlCommand("Select *from testtable",con);
SqlDataAdapter da = new SqlDataAdapter(cmd);
dt = new DataTable();
try
{
da.Fill(dt);
return dt;
}
catch (Exception ex)
{
MessageBox.Show(ex.ToString());
return null;
}
}
Please guide me how to manage memory so that system should not get slower. If i dispose datatable after assigning it to grid, would that help?
I am actually a beginner. And want a complete guidance about memory management. Application should be fast. Thanx
Most likely your problem is that your DataGridView creates a row for each row in the result set. You should read about what the virtual display mode is, the VirtualMode property and how to implement it.
You can't expect to load a million rows of data into a GUI control and expect everything to behave well, can you?
The problem is here:
SqlCommand cmd = new SqlCommand("Select *from testtable",con);
There's no way you should be SELECTing the entire table. You'll want to LIMIT your query to only those rows that the GUI can reasonably display. Shooting completely in the dark, I'd say somewhere on the order of a thousand rows at maximum.
My assumation is you try to load all those rocords one time in getDataTable. Why not load by demand? I.e., only load the first level load. If all those are first level, you can separate into multi pages whose size may be a screen width.
You can use the DataGrid control with setting all appropriate select, insert, update, delete statements, add paging rules and this will also to the trick. The grid will not load all the records in once, but instead, it will use the clever internal functionality to select from the db and display only the appropriate data.
Hope this helps also.
Related
I've been struggling to find efficient solution to get live data from MySQL database to datagridview. In my database values are changing every second or even faster. I need to get live feed directly to my windows app. The only solution I could found is to use async, but then app get laggy when trying to move it arround or do other tasks. My attempt:
public async void refreshOutput()
{
while (true)
{
DataTable dbResults = Connect_ToDB.executeQuery("SELECT * FROM mysql.test_table;");
dataGridView2.DataSource = dbResults;
await Task.Delay(200);
}
}
Also I tried:
private void button24_Click(object sender, EventArgs e)
{
DataTable data = Connect_ToDB.executeQuery("SELECT * FROM mysql.test_table;");
BindingSource bSource = new BindingSource();
bSource.DataSource = data;
dataGridView2.DataSource = bSource;
}
In any case i always need to push the button again to get new data. Is there a way to get it live without disturbing the application background processes? Thank you!
I would like to ask if it is possible to load a database into an array or list, and then run queries on it? I have the following code.
string cs = "Data Source=dataBase.sqlite;Version=3;";
SQLiteConnection con;
SQLiteDataAdapter adapt;
DataTable dt;
private void textBox1_TextChanged(object sender, EventArgs e)
{
con = new SQLiteConnection(cs);
con.Open();
adapt = new SQLiteDataAdapter("select * from Table1 where CnName1 like '" + textBox1.Text + "%'", con);
dt = new DataTable();
adapt.Fill(dt);
dataGridView1.DataSource = dt;
con.Close();
}
This works, but it creates a new dataTable whenever a query is run, the problem code is:
dt = new DataTable();
The program is meant to be constantly running, so this is inefficient, since it will eat up a lot of memory. How do I load the database into an object, and then run queries on that object? The table is only meant to have 1 column, and the queries run will only serve as a search function. I want to load the database only once, that is when the program is started, then the connection will be closed, and everything else will be done with the program, not with the database.
Edit: I would like to state for anyone else viewing this question to also view saab669's answer, as it provides useful information as well, however I can not choose two answers.
Assuming you have a form containing your text box. Declare a class level variable to store you datatable.
private DataTable _data;
Create a class to encapsulate your database connection and retrieval of data.
public class MyDataBaseConnection
{
public DataTable ReturnMyData(string valueFromTextBox)
{
var cs = "Data Source=dataBase.sqlite;Version=3;";
SQLiteConnection con;
SQLiteDataAdapter adapt;
DataTable dt;
try
{
con = new SQLiteConnection(cs);
con.Open();
adapt = new SQLiteDataAdapter("select * from Table1 where CnName1 like '" + textBox1.Text + "%'", con);
dt = new DataTable();
adapt.Fill(dt);
con.Close();
return dt;
}
catch (Exception ex)
{
//Log here.
throw;
}
finally
{
con = null;
adapt = null;
//Or Dispose. I dont have SQL lite so dont know if they implement IDispose
}
}
}
In your textbox change event, call the db code and assign to your class level var
private void textBox1_TextChanged(object sender, EventArgs e)
{
var myDBConnection = new MyDataBaseConnection();
_data = myDBConnection.ReturnMyData(textBox1.Text);
dataGridView1.DataSource = null;
dataGridView1.DataSource = _data;
}
When the text changed event fires again, the data will be changed in the grid.
Hope that helps.
My response is too long for a comment, but I want to reply to a few things you said:
but it creates a new dataTable whenever a query is run, the problem code is dt = new DataTable(); The program is meant to be constantly running, so this is inefficient, since it will eat up a lot of memory.
Sort of. You only ever have one DataTable object. It's just that every time that event fires, it's re-querying the database. So you're not consuming an excessive amount of memory (unless your DB is huge!), or at least not in the way you think you are.
How do I load the database into an object, and then run queries on that object?
As I mentioned in my comment, you're doing that already. Just in a less-than-ideal event handler. Depending on how you want the application to behave, you should move this to the form load event as defined here. Or perhaps it might make more sense to have a button with a click event. The form load could add a long delay to your application's start up and the user might think it froze, depending on how long it takes you to fetch all the records from the DB.
Also, from the code snippet you provide, am I correct in assuming you define DataTable dt; outside of the event handler, on the class level?
Anyways, per the MSDN article I linked in my comment, once you have a DataTable populated then you can simply do things like this:
DataRow[] foundRows;
foundRows = dataSet1.Tables["TableName"].Select("ColumnName LIKE 'your_search_value%'");
Lastly, and I cannot stress this enough about the code you provided: DO NOT concatenate strings for a query which will execute against a database. I don't care if it's for a homework assignment or a little tool only you will use, there's no excuse for not taking the extra time to learn how to do it the right way. There's no reason to build a bad habbit, as this is susceptible to SQL injection which is comically easy to protect against. You absolutely should spend the 30 minutes to learn about parameterized queries.
I have a DataGridView in which I load data from a SQL server database. When I load the data it takes quite long time.
I would like to give user information that the data is loading. May I ask you what is the best way connecting Progressbar when data is loading into the DataGridView?
I don't want anyone to make a fully working code for me. I just would like to know how it can be done.
I see someone awarded my question with bounty. I would like to say that at the moment Iam using this code which I would appriciate if it would fit.
DTGdataTable = new DataTable();
SqlDataAdapter SDA = new SqlDataAdapter
SDA.Fill(DTGdataTable);
dataGridView1.DataSource = DTGdataTable ;
Thank you everyone for your time.
If the problem is that it takes long to fetch the data from the database, i have a possible solution for you:
private void buttonLoad_Click(object sender, EventArgs e)
{
progressBar.Visible = true;
progressBar.Style = ProgressBarStyle.Marquee;
System.Threading.Thread thread =
new System.Threading.Thread(new System.Threading.ThreadStart(loadTable));
thread.Start();
}
private void loadTable()
{
// Load your Table...
DataTable table = new DataTable();
SqlDataAdapter SDA = new SqlDataAdapter();
SDA.Fill(table);
setDataSource(table);
}
internal delegate void SetDataSourceDelegate(DataTable table);
private void setDataSource(DataTable table)
{
// Invoke method if required:
if (this.InvokeRequired)
{
this.Invoke(new SetDataSourceDelegate(setDataSource), table);
}
else
{
dataGridView.DataSource = table;
progressBar.Visible = false;
}
}
Put the method loading the data into another thread and set the datasource when it's finished. There should be an invoke required. If you want to show percentage values in the progressbar, don't use the style 'Marquee' and add another function and delegate you can invoke for setting the value of the progress bar.
If binding the data to the grid is the problem, then you can not put the binding into another thread and you may show a progress-popup that runs in another thread.
I hope this helps.
Try this...This should be the quickest route...
1) Add a button control.
2) Add a datagridview control.
3) Add a single column into the datagridview control.
4) Add a progressbar control.
5) Leave them all with default names.
Under the button1_Click event, add this chunk of code...
int maxnumber = 1000;
progressBar1.Value = 0;
progressBar1.Maximum = maxnumber;
for (int x = 0; x <= maxnumber - 1; x++)
{
Application.DoEvents();
dataGridView1.Rows.Add(new string[] {Convert.ToString(x)});
progressBar1.Value += 1;
label1.Text = progressBar1.Value.ToString();
}
That's it. Edit to suit your needs. This isn't the complete exact code you want but it should get you started. I've taken the liberty to declare maxnumber to hold the limit of the progressbar. In your case, this should be the row count of your database and it's subtracted by 1 since index always starts with zero :)
The method above is single threaded, which means you still can't do multitasking while your loop still isn't done yet. In some cases, depending on the algorithm of the code, using the single threaded method may be faster than going for the multithreading route. Anyhow, the second method would be using a backgroundworker. This is most commonly preferred by people since the user can still do things while the list loads. Kind of like installers where you can cancel installation even if it's in the middle of doing something. The website below should get you started. The code is in VB.NET but can be easily converted to C# :)
http://social.msdn.microsoft.com/Forums/vstudio/en-US/efd7510c-43ed-47c4-86a3-1fa350eb0d30/fill-datagridview-using-backgroundworker-progressbar?forum=vbgeneral
The problem is all the data load together and return back from database.
You need to get the data loading progress. This can be done from database.
Get the count to data rows from database. (This query would return a single number, so it would be quicker).
Get the data in parts (Divide & conquer strategy), you can use OFFSET and FETCH database queries. (This is return part of data on each call)
OFFSET and FETCH is available on different database. In MSSQL, it was introduced in version 2012.
When we are getting the data in parts from server, we can calculate the progress.
Within your code, this line will take up most of the processing time:
SDA.Fill(DTGdataTable);
So I think the most important thing is to keep track of progress while this is executing. To do this, you first need to know how many rows you're expecting. The SQLDataAdapter can't provide this information, so you would need to run an extra COUNT(*) query first to get this number. You can then use this as the Maximum on your ProgressBar.
My actual loading code is based upon Michael Hofmeiers solution, but instead of using the ProgressBar in Marquee mode, I would feed it with real progress data from a Timer Control. So on your form, you add a Timer control (named progressTimer in my example), set its Interval to 100 msec and Enabled to false. The code then becomes:
private DataTable table = null;
private void buttonLoad_Click(object sender, EventArgs e)
{
// TODO: Select the row count here and assign it to progressBar.Maximum
progressBar.Visible = true;
System.Threading.Thread thread =
new System.Threading.Thread(new System.Threading.ThreadStart(loadTable));
thread.Start();
progressTimer.Enabled = true;
}
private void loadTable()
{
// Load your Table...
this.table = new DataTable();
SqlDataAdapter SDA = new SqlDataAdapter();
SDA.Fill(table);
setDataSource(table);
}
internal delegate void SetDataSourceDelegate(DataTable table);
private void setDataSource(DataTable table)
{
// Invoke method if required:
if (this.InvokeRequired)
{
this.Invoke(new SetDataSourceDelegate(setDataSource), table);
}
else
{
progressTimer.Enabled = false;
dataGridView.DataSource = table;
progressBar.Visible = false;
}
}
private void progressTimer_Tick(object sender, EventArgs e)
{
if (this.table != null)
progressBar.Value = this.table.Rows.Count;
}
When you run this, your application will stay responsive during the loading, and you'll see the ProgressBar changing to reflect the number of rows loaded. Only at the end (when you set the DataSource on the DataGridView) will the application seem to "hang", but I don't think there's a way to avoid that. You can only do this operation from the main thread, so it's unavoidable that the UI becomes unresponsive. But in my test, the DataGridView easily handles 300K+ rows in about a second, so that shouldn't be that much of a problem.
1.Download an ajax loader.gif from google.
2.replace that image with this code..
3.Set time accordingly..
<div id="spinner" style="display: none;">
<span id="ss" style="float: left; margin-left: 50% !Important; margin-top: 22% !Important;">
<img src="ajax-pink-loader.gif" alt="Loading..." />
</span>
</div>
$("#ssubmit").click(function () {
$("#spinner").show();
setInterval(function () {
$("#spinner").hide();
}, 20000);
});
I am making a dictionary using C# and Windows forms. In my dictionary I have a textBox where the user can search for a word to get the meaning. I also have some options in a comboBox where the user can choose a language to see the meaning for that language. Because I am making the dictionary for different languages.
My code looks like:
private void textBox1_TextChanged(object sender, EventArgs e)
{
string word = textBox1.Text;
SqlCeConnection con = new SqlCeConnection(#"Data Source=" + Directory.GetCurrentDirectory() + #"\Database\condrokothadb.sdf;Password=000;");
//in combobox there are 2 option(language)
//if select one language(option) from combobox
if(mood=="bangla")
{
SqlCeDataAdapter b = new SqlCeDataAdapter("SELECT english,bangla FROM dic WHERE (bangla like '" + word + "%')", con);
DataTable tt = new DataTable();
b.Fill(tt);
dataGridView1.DataSource = tt;
}
else //by default english language is selected
{
using (con)
{
con.Open();
using (SqlCeDataAdapter b = new SqlCeDataAdapter("SELECT english,bangla FROM dic WHERE (english like '" + word + "%')", con))
{
DataTable tt = new DataTable();
b.Fill(tt);
dataGridView1.DataSource = tt;
}
}
}
}
private void comboBox1_SelectedIndexChanged(object sender, EventArgs e)
{
if (((ComboBox)sender).SelectedItem.ToString() == "Bangla")
{
mood = "bangla";
}
else if (((ComboBox)sender).SelectedItem.ToString() == "English")
{
mood = "english";
}
}
My problem is that when a user want to write something into the textbox it is getting so much slower to write. How can I overcome that?
This is an interesting question and here is how I would solve it.
I added a timer that starts to count as you type the first character into your textBox, and for every character you add the timer resets. The application wont execute the part where you search through the database untill the timer reaches a set number of ticks.
Make sure you add a timer and a backgroundWorker into the form. Create the events through the properties window and add this code:
int timerTicks;
int waitUntill = 10; //10 = 1 second. Change this to decide how long the application will wait.
string mood;
string word;
string langConnection;
DataTable tt;
SqlCeConnection con;
SqlCeDataAdapter b;
private void textBox1_TextChanged(object sender, EventArgs e)
{
if (!timer1.Enabled)
timer1.Start();
//Reset the timer when a character is entered in textBox1.
timerTicks = 0;
}
private void timer1_Tick(object sender, EventArgs e)
{
timerTicks++;
if (timerTicks > waitUntill && !backgroundWorker1.IsBusy && comboBox1.SelectedItem != null)
{
//Stop the timer and begin the search in a background thread.
timer1.Stop();
word = textBox1.Text;
mood = comboBox1.SelectedItem.ToString();
backgroundWorker1.RunWorkerAsync();
}
}
private void backgroundWorker1_DoWork(object sender, DoWorkEventArgs e)
{
tt = new DataTable();
con = new SqlCeConnection(#"Data Source=" + Directory.GetCurrentDirectory() + #"\Database\condrokothadb.sdf;Password=000;");
langConnection = String.Format("SELECT english,bangla FROM dic WHERE ({0} like '{1}%')", mood, word);
using (con)
{
con.Open();
b = new SqlCeDataAdapter(langConnection, con);
b.Fill(tt);
}
}
private void backgroundWorker1_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
dataGridView1.DataSource = tt;
}
Note that you don't need the comboBox1_SelectedIndexChanged event for this to work.
EDIT:
To make the actual search run faster you would have to open the connection at startup and keep it open throughout the entirety of the execution, like other answers suggest as well. You should be able to figure that out for yourself though.
Instead of getting data from server in every text changed you can get all data in grid at once and then filter them with dataview.
// to get data in grid
CustomList<wordlistDAO> WordList = null;
WordList = WordListBLL.GetAllWord();
GridWord.DataSource = WordList ;
// create word datatable for filtering
DataTable dtWord = null;
dtWord = new DataTable();
foreach (DataGridViewColumn colu in GridWord.Columns)
dtWord .Columns.Add(new DataColumn(colu.HeaderText));
foreach (DataGridViewRow row in GridWord.Rows)
{
DataRow dr = dtWord.NewRow();
foreach (DataGridViewCell cell in row.Cells)
dr[row.Cells.IndexOf(cell)] = cell.Value;
dtWord .Rows.Add(dr);
}
//create data view
DataView wordlistview = new DataView();
wordlistview = new DataView(dtWord);
// filter dataview and show in grid
if (cboLanguage.Text == "Bangla")
{
wordlistview.RowFilter = "bangla LIKE '" + txtSearchValue.Text.Trim().ToUpper() + "%'";
}
else
{
wordlistview.RowFilter = "english LIKE '" + txtSearchValue.Text.Trim().ToUpper() + "%'";
}
GridWord.DataSource = wordlistview;
The main problem is that, every time you press a key in the textbox, you are creating a database connection and querying the database. This is very inefficient! Also, because your Bangla code doesn't dispose of the connection, you may be keeping a lot of objects referenced that don't need to be, so you may find performance is degrading over time.
One basic suggestion would be to use a single connection instead of opening new connections for each keypress. This will reduce somewhat the time taken for the query. Realistically though, I suspect that you want to load the full content of the data at once, and run your query in-memory. This will give you much better speed.
Running your query on a background thread will help maintain the responsiveness of your UI, but will potentially end up with lots of queries running at once trying to catch up with the user's typing.
A better solution is to consider running an "idle-timer," and only starting the query when the user has stopped typing for a short amount of time. I'd recommend still using a background thread for this. You won't query the database for every keypress, and you won't affect the responsiveness of the UI.
Delay could be due to large amount of data in the database and you're calling database for every text changed event.
What I would suggest is to get all the data into DataView and keep filtering and binding the grid with results from view. That way you can minimise the number of times database is called.
Doing a search in the database on any keystroke is a bad practice. As you already experienced, it makes the UI very slow. A better option would be to do the search in a background thread, and also not for every keystroke. You can wait some time (0.5 seconds for example) before doing the search. If the user pressed another key in the meantime, expand the wait again to another 0.5 seconds.
I'm trying to make a DataGridView that displays data from an SQL database (GSM.sdf) and saves changes made to the DataGridView back to the database when a save button is pressed. The data displays fine, but nothing happens when the save button is pressed. I've been following the top answer from this thread:
http://social.msdn.microsoft.com/Forums/en/csharpgeneral/thread/98bc0b4d-a2ea-4b74-81f0-473da624528a
But it isn't working out. Here is my code:
namespace WindowsFormsApplication5
{
public partial class Zeelot : Form
{
DataTable table = new DataTable();
SqlCeDataAdapter z;
DataSet gSMDataSet = new DataSet();
public Zeelot()
{
InitializeComponent();
}
private void Zeelot_Load(object sender, EventArgs e)
{
string b = #"Data Source =.\SQLEXPRESS;database=GSM;Integrated Security=FALSE;Connection Timeout=30;User Instance=FALSE";
SqlCeConnection conn = new SqlCeConnection(b);
conn.Open();
string cd = "SELECT * FROM PhoneNumbers";
z = new SqlCeDataAdapter(cd, conn);
z.Fill(gSMDataSet, "PhoneNumbers");
table = gSMDataSet.Tables[0];
conn.Close();
dataGridView1.DataSource = table;
}
private void SaveButton_Click(object sender, EventArgs e)
{
SqlCeCommandBuilder local_SqlCommandBuilder = new SqlCeCommandBuilder(z);
local_SqlCommandBuilder.ConflictOption = System.Data.ConflictOption.OverwriteChanges;
z.UpdateCommand = local_SqlCommandBuilder.GetUpdateCommand();
z.Update(((System.Data.DataTable)this.dataGridView1.DataSource));
((System.Data.DataTable)this.dataGridView1.DataSource).AcceptChanges();
}
}
}
I believe your problem lies where you are casting a DataSource to System.Data.DataTable
Try z.Update(gSMDataSet);
I also don't believe you need AcceptChanges()
you dont need to call .AcceptChanges() on the DataGridView at the end.
AcceptChanges() submits the changes from your DGV to the connected DataSource (your DataTable)
Try calling AcceptChanges() before you try to submit the changes to your Database.
I think your problem is with your connection. You manually create and open a connection and assign it to your DataAdapter. But within your _Load() Method, you close the connection. As the DataAdapter used inside the _Click EventHandler is the same as the one used in your _Load() Method (therefore uses the same connection). It can't submit any changes to your Database because you already closed the connection.
Don't you get any exceptions at runtime?
Try using breakpoints to examine the current state of your connection Object before you attempt to submit your changes to your Database.
Also, AFAIK MS advises against creating connections manually if you want to use them within a DataAdapter.
DataAdapter's Constructor can create a connection on its own