I am working on a serial monitor project based on the arduino serial monitor, but with a lot of more functionalities. Serial monitor UI with messages every 2 seconds
There are 2 basic functions - reading serial data and visualising it to the richtextbox, writing serial data and also visualising it to the richtextbox. There is a problem when the communication is very intensive (arduino sends a line as fast as it can) and the user input cuts some of the recieved lines in halves. I made an algorythm to separate the input strings from the output strings in the richtextbox by setting sending/receiving flags. However this made the program act strange. Sometimes the command sent to the arduino just doesn't visualise in the richtextbox, but the data from the arduino is visualised just fine. I tried setting breakpoints on the user input visualising method and they were rarely activated. What is more I also set breakpoints on the visualisation of the read data method and they were also rarely activated. However the form was not lagging or freezing and the commands were flying in the richtextbox, contrary to the observed behaviour of the breakpoints. The printing algorythm is heavy, because I implemented multi-color printing.
What I tried:
-tried the blockingcollection approach. Just putting the printing actions in a queue and executing the actions one by one using 2 additional threads to the main one.(Task.Factory.StartNew). The problem was the queue was filled with over 3000 actions in the matter of a minute and the whole thing was lagging behind with like 15 seconds.
-i tried starting a new Task.Factory for every receive/send methods for every printing of a new command and locking the thread until the command is printed in the richtextbox. This is also too slow.
Finally I came up with the idea of just setting flags and allowing or not the print event. Using this approach the user input is almost never printed in the richtextbox. :(
Printing algorythm:
void printToConsole(string print, Color txtColor, string part, Color partColor, bool isMsg, bool endNL)
{
while (print.Contains('\r'))
print = print.Replace("\r", "");
for (int i = 0; i < print.Length; i++)
{
if (newString)
{
newString = false;
printTimeAndPart(part, partColor);
}
else if (i == 0 && !prevStrHadNL && isMsg != prevWasMsg)
{
printNLTimeAndPart(part, partColor);
}
else if (i == 0 && prevStrHadNL)
{
printNLTimeAndPart(part, partColor);
}
else if (i == print.Length - 1)
{
if (print[i] == '\n')
{
if (endNL && isMsg != prevWasMsg)
AppendText("\n");
prevStrHadNL = true;
break;
}
else
{
if (endNL)
AppendText("\n");
prevStrHadNL = false;
}
}
if (print[i] == '\n')
{
printNLTimeAndPart(part, partColor);
}
else
AppendText(print[i].ToString(), txtColor);
}
prevWasMsg = isMsg;
}
private void printNLTimeAndPart(string part, Color partColor)
{
AppendText("\n");
printTimeAndPart(part, partColor);
}
private void printTimeAndPart(string part, Color partColor)
{
if (timestamp == 1)
AppendText(DateTime.Now.ToString("HH:mm:ss.fff"), timeColor);
AppendText(part, partColor);
}
private void AppendText(string txt, Color clr)
{
if (serialControl.ReadAllowed)
{
if (rtArea.InvokeRequired)
{
MethodInvoker mi = delegate ()
{ rtArea.AppendText(txt, clr); };
Invoke(mi);
}
else
rtArea.AppendText(txt, clr);
}
}
private void AppendText(string txt)
{
if (serialControl.ReadAllowed)
{
if (rtArea.InvokeRequired)
{
MethodInvoker mi = delegate ()
{ rtArea.AppendText(txt); };
Invoke(mi);
}
else
rtArea.AppendText(txt);
}
}
Serial read data print:
wait1 and wait2 - flags. When wait1 is false there is no current user input message being printed.
When wait2 is false user input is free to print. If the access to a print operation was denied it is performed after the operation which blocked it. (At least that is the idea)
private void Port_dataRecieved(string a)
{
if (!consoleUsed)
{
newString = true;
consoleUsed = true;
}
inputVal = a;
if (!wait1)
{
inputAccDenied = false;
wait2 = true;
PrintInput(inputVal);
wait2 = false;
}
else
inputAccDenied = true;
if (msgAccDenied)
{
PrintMsg(msgVal);
msgAccDenied = false;
}
}
private void PrintInput(string input)
{
printToConsole(input, inTxtColor, inPrefix, inPrefColor, false, false);
}
This is the dataReceived method from my serial controller class:
I tried several ways to read serial data and the uncommented one is the most stable one I tried.
private void port_DataReceived(object sender, SerialDataReceivedEventArgs e)
{
/* byte[] buffer = new byte[blockLimit];
Action kickoffRead = null;
kickoffRead = delegate
{
port.BaseStream.BeginRead(buffer, 0, buffer.Length, delegate (IAsyncResult ar)
{
try
{
int actualLength = port.BaseStream.EndRead(ar);
byte[] received = new byte[actualLength];
Buffer.BlockCopy(buffer, 0, received, 0, actualLength);
string rcv = Encoding.Default.GetString(received);
dataRecieved(rcv);
}
catch (IOException exc)
{
//handleAppSerialError(exc);
}
kickoffRead();
}, null);
};
kickoffRead();
*/
if (ReadAllowed)
{
string a = port.ReadExisting();
//port.DiscardInBuffer();
dataRecieved(a); //Raise event and pass the string to Form1 serial_dataReceived
}
}
So, I need some advice on how exactly to print the multi-color messages without cutting into each other, printing every time, fast, with low cpu load (now it is like 35% on full load (arduino intensive transmittion) on i5-4310m). I would be glad if you could provide some examples, too.
Related
private void serialPort_DataReceived(object sender, System.IO.Ports.SerialDataReceivedEventArgs e)
{
if (Clos_flag) return;
try
{
Listening = true;
if (serialPort.IsOpen)
{
this.txt_weight.Invoke(new MethodInvoker(delegate
{
//txt_weight.Text = serialPort.ReadLine();
serialPort.NewLine = "\r";
string weight = serialPort.ReadLine();
if (weight.IndexOf(")") > 0)
{
weight = weight.Substring(3, 8);
// txt_weight.Text = weight.Substring(0, weight.LastIndexOf("0") + 1);
}
}));
}
}
catch (Exception eg)
{
MessageBox.Show(eg.ToString());
}
finally
{
Listening = false;
}
}
What should i change to remove the first zero in the picture, above is the code i use. I tried to change the substring to (2,8) but still it is not working.
I get the value from a weighing machine, once the user click the "Open Serial Port" button
Change this line:
string weight = serialPort.ReadLine();
to this:
string weight = serialPort.ReadLine().TrimStart('0').Trim();
What that will do is readline, trim the first 0, then trim the outside (start amd end) blank spaces before loading the value into the declared variable
Seems like this could be caused by many factors.
Looking at the code, the leading zero should be stripped out, since you already substring out the first two char.
I see that your function serialPort_DataReceived reads the data from a device in your port, so I presume your screen shot is from the output device.
Try and hard code
weight = "12345";
If the problem still persists, it would be caused not by the code, but by your device, etc..
Hope this helps~
I'm trying to create an application which communicates with hardware via serial port and reports the results to the gui.
Currently moving through GUI is made by KeyEvents which trigger the drawing of the next "page" of GUI. However at one step (after the key is pressed) I need to draw new page and send few commands via serial port.
The command sending is done via :
port.Write(data, 0, data.Length);
I then wait for the answer by waiting for DataReceivedHandler to trigger - it just pins out that there is data awaiting and data is being processed in another method.
At first I just put sending & receiving command in the function drawing the page after the "draw parts" however it made it stuck - the data was being transfered, but the page wasn't drawn - it was frozen.
Then I made an async method :
private async void SendData()
{
await Task.Run(() => serialClass.SendAndReceive(command));
// process reply etc.
}
Which is used like that :
public void LoadPage()
{
image = Image.FromFile(path);
//do some stuff on image using Graphics, adding texts etc.
picturebox1.Image = image;
SendData();
}
It works fine, however I need to "reload" the page (to call again LoadPage). If I do it inside the async method like this :
private async void SendData()
{
await Task.Run(() => serialClass.SendAndReceive(command));
// process reply etc.
LoadPage();
}
Then obviously the image won't be refreshed, though the data will be send via serial port. Is it possible to somehow check if async function was finished and trigger an event where I could reload the page?
So far I've tried using the BackGroundWorker Work Complete and Property Change. The data was send again, but the image wasn't reloaded. Any idea how I can achieve that?
Thanks in advance for the help,
Best regards
You need to use a state machine and delegates to achieve what you are trying to do. See the code below, I recommend doing all this in a separate thread other then Main. You keep track of the state you're in, and when you get a response you parse it with the correct callback function and if it is what you are expecting you move onto the next send command state.
private delegate void CallbackFunction(String Response); //our generic Delegate
private CallbackFunction CallbackResponse; //instantiate our delegate
private StateMachine currentState = StateMachine.Waiting;
SerialPort sp; //our serial port
private enum StateMachine
{
Waiting,
SendCmd1,
Cmd1Response,
SendCmd2,
Cmd2Response,
Error
}
private void do_State_Machine()
{
switch (StateMachine)
{
case StateMachine.Waiting:
//do nothing
break;
case StateMachine.SendCmd1:
CallbackResponse = Cmd1Response; //set our delegate to the first response
sp.Write("Send first command1"); //send our command through the serial port
currentState = StateMachine.Cmd1Response; //change to cmd1 response state
break;
case StateMachine.Cmd1Response:
//waiting for a response....you can put a timeout here
break;
case StateMachine.SendCmd2:
CallbackResponse = Cmd2Response; //set our delegate to the second response
sp.Write("Send command2"); //send our command through the serial port
currentState = StateMachine.Cmd2Response; //change to cmd1 response state
break;
case StateMachine.Cmd2Response:
//waiting for a response....you can put a timeout here
break;
case StateMachine.Error:
//error occurred do something
break;
}
}
private void Cmd1Response(string s)
{
//Parse the string, make sure its what you expect
//if it is, then set the next state to run the next command
if(s.contains("expected"))
{
currentState = StateMachine.SendCmd2;
}
else
{
currentState = StateMachine.Error;
}
}
private void Cmd2Response(string s)
{
//Parse the string, make sure its what you expect
//if it is, then set the next state to run the next command
if(s.contains("expected"))
{
currentState = StateMachine.Waiting;
backgroundWorker1.CancelAsync();
}
else
{
currentState = StateMachine.Error;
}
}
//In my case, I build a string builder until I get a carriage return or a colon character. This tells me
//I got all the characters I want for the response. Now we call my delegate which calls the correct response
//function. The datareceived event can fire mid response, so you need someway to know when you have the whole
//message.
private void serialPort1_DataReceived(object sender, System.IO.Ports.SerialDataReceivedEventArgs e)
{
string CurrentLine = "";
string Data = serialPortSensor.ReadExisting();
Data.Replace("\n", "");
foreach (char c in Data)
{
if (c == '\r' || c == ':')
{
sb.Append(c);
CurrentLine = sb.ToString();
sb.Clear();
CallbackResponse(CurrentLine); //calls our correct response function depending on the current delegate assigned
}
else
{
sb.Append(c);
}
}
}
I would put this in a background worker, and when you press a button or something you can set the current state to SendCmd1.
Button press
private void buttonStart_Click(object sender, EventArgs e)
{
if(!backgroundWorker1.IsBusy)
{
currentState = StateMachine.SendCmd1;
backgroundWorker1.RunWorkerAsync();
}
}
Background worker do work event
private void backgroundWorker1_DoWork(object sender, DoWorkEventArgs e)
{
while (true)
{
if (backgroundWorker1.CancellationPending)
break;
do_State_Machine();
Thread.Sleep(100);
}
}
edit: you can use invoke to update the GUI from your background worker thread.
this.Invoke((MethodInvoker)delegate
{
image = Image.FromFile(path);
//do some stuff on image using Graphics, adding texts etc.
picturebox1.Image = image;
});
I am reading data values from a serial port and plotting the data in realtime on a zedgraph control using invoke and delegate. I would like to click a disconnect button that will stop the plotting and when a connect button is pressed, the plotting will resume. My problem is when I reconnect, the plotting will not be displayed at 1 point per second. There will be unexpected behavior that sometimes includes a delay that will not plot for 2 seconds and the next second it will plot 2 points at the same time. Sometimes it will not plot for 3 seconds and the next second, it will plot 3 points at the same time and sometimes it will work fine plotting 1 point per second. Sometimes it will plot twice or three times per second. This problem occurs only when I disconnect and then reconnect. My GUI response also begins to get worse with every time I disconnect and reconnect until finally not responding.
When I click disconnect, I clear the zedgraph but don't close the serial port. When I re-click connect again, the serial port will begin sending data again and continue to plot, calling control.invoke every time a new data point is plotted onto the zedgraph. Is the problem that I am calling invoke too many times and how would I get around this? I need to plot in realtime, so each time I receive a data point, plot it on the zedgraph right away.
Here is how I connect to the serial port:
private void btnConnect_Click_1(object sender, EventArgs e)
{
curveUSB = myPaneUSB.AddCurve("Load", listUSB, Color.Black, SymbolType.Circle);
isConnected = true;
DateTime startTime = DateTime.Now;
txtStartTime.Text = startTime.ToString();
sw.Start();
createCSVFile(startTime);
try
{
if (!serialPort1.IsOpen)
{
serialPort1.PortName = cmbPort.Items[cmbPort.SelectedIndex].ToString();
//Other Serial Port Property
serialPort1.Parity = Parity.None;
serialPort1.StopBits = StopBits.One;
serialPort1.DataBits = 8;
serialPort1.BaudRate = 9600;
//Open our serial port
serialPort1.Open();
}
}
catch(Exception ex)
{
MessageBox.Show(ex.Message);
}
Thread.Sleep(100); //Always sleep before reading
serialPort1.DataReceived += new SerialDataReceivedEventHandler(serialPort1_DataReceivedHandler);
// Save the beginning time for reference
tickStart = Environment.TickCount; //used the calculate the time
setGraphAxis(myPaneUSB, zgControlUSB);
//Disable Connect button
btnConnect.Enabled = false;
//Enable Disconnect button
btnDisconnect.Enabled = true;
}
Invoke and serialPort1_DataReceivedHandler:
private void serialPort1_DataReceivedHandler(object sender, SerialDataReceivedEventArgs e)
{
if (isConnected == true)
{
txtUSBLoad.Invoke(new EventHandler(delegate
{
strRawData = serialPort1.ReadLine();
txtUSBLoad.Text = strRawData + loadUnit;
Display_Data(strRawData, curveUSB, listUSB, zgControlUSB);
}));
}
}
Display_Data function to display the plot:
private void Display_Data(String data, LineItem curve, IPointListEdit list, ZedGraphControl zgControl)
{
ts = sw.Elapsed;
tsBT = swBT.Elapsed;
elapsedTime = String.Format("{0:00}:{1:00}:{2:00}",tsBT.Hours, tsBT.Minutes, tsBT.Seconds);
elapsedTimeBT = String.Format("{0:00}:{1:00}:{2:00}", tsBT.Hours, tsBT.Minutes, tsBT.Seconds);
txtElapsedTime.Text = elapsedTime;
txtBTElapsedTime.Text = elapsedTimeBT;
if (zgControl.GraphPane.CurveList.Count <= 0) //Make sure that the curvelist has at least one curve
return;
curve = zgControl.GraphPane.CurveList[0] as LineItem; //Get the first CurveItem in the graph
if (curve == null)
return;
list = curve.Points as IPointListEdit; //Get the PointPairList
if (list == null) //If this is null, it means the reference at curve.Points does not
return; //support IPointListEdit, so we won't be able to modify it
time = (Environment.TickCount - tickStart) / 1000.0; //Time is measured in seconds
//Get current time
now = DateTime.Now;
timestamp = now.ToOADate();
list.Add(timestamp, Convert.ToDouble(data));
if(fileWriter.BaseStream != null){ //If fileWriter is open, then write to file, else don't write
fileWriter.WriteLine(String.Format("{0:yyyy-MM-dd_HH-mm-ss}", now) + "," + data); //writes the timestamp and data to the CSV file
}
if (Convert.ToDouble(data) > tempLoad) //Checks if the current load is the maximum load
{
tempLoad = Convert.ToDouble(data);
txtMaxLoad.Text = tempLoad.ToString() + " " + loadUnit;
txtBTMaxLoad.Text = tempLoad.ToString() + " " + loadUnit;
}
//if ((Convert.ToDouble(data) > dblThreshold) && (emailAlerts == true)) {
if ((Convert.ToDouble(data) > 50) && (emailAlerts == true))
{
sendEmail(Convert.ToDouble(data));
}
XDate dateTime = curve[0].X;
zgControl.AxisChange(); //changes the x-axis
zgControl.Invalidate(); //Force a redraw
}
Disconnect button:
private void btnDisconnect_Click_1(object sender, EventArgs e)
{
zgControlUSB.GraphPane.CurveList.Clear();
if (fileWriter != null)
{
fileWriter.Close();
}
isConnected = false;
btnConnect.Enabled = true;
btnDisconnect.Enabled = false;
txtUSBLoad.Text = initText;
DateTime endTime = DateTime.Now;
txtEndTime.Text = endTime.ToString(); //displays the time stamp when the plotting stops
sw.Stop(); //stops the stop watch
}
When I disconnect, I do not close the serial port because the problem is worse when I do and I read that it is not good to always close and reopen a serial port. Instead, I used an isConnected flag to start and stop plotting.
Any help is appreciated! Thanks in advance!
Calling txtUSBLoad.Invoke() will switch processing onto the GUI thread (the same thread that is used to handle user input and draw the controls). By putting all the processing inside the call to Invoke() you are effectively swamping the GUI thread.
If your serial updates are frequent enough this will result in delayed interactions/re-drawing of the GUI.
You have a few options here, the first thing is to keep all the processing in the thread that receives the messages from the serial port. Look at all the visual controls that you are updating and the determine what data they need to be updated and process as much of that before calling Invoke(). Create a class to hold the data.
This might be too slow which will mean that you'll have a backlog of serial data building up which you can't process in time. If so, you can conflate the incoming data if you're ok with losing updates. You'll have to see if this applies to your situation.
If you want to try and parallelise the processing of the incoming data look into TPL Dataflow for an example of how to create a processing pipeline for the data.
I am interfacing with a device, and need a way to constantly check if any bytes are available to be read by the serialport, and if so I need to read them and perform different functions. I can't create a data_received event for a number of reasons due to the project I am working on. As of now I have an infinite for loop running in a new thread, but it is causing the computer to slow way down.
Does anyone have any other ideas? Any help is appreciated. Below is the for loop code I am currently using.
void bw_DoWork(object sender, DoWorkEventArgs e)
{
object[] parameters = (object[])e.Argument;
SerialPort port = (SerialPort)parameters[0];
agentRadio agentR = (agentRadio)parameters[1];
for (; ; )
{
if (port.IsOpen)
{
try
{
if (port.BytesToRead > 0)
{
byte value = (byte)port.ReadByte();
if (value == Global.OFF_CHAR)
{
port.Close();
Global.isConnected = false;
Form form = new Form();
form.deviceDisconnect(agentR);
MessageBox.Show("Your device has disconnected on its own, due to inactivity or because a key was switched off.", "Device Disconnect", MessageBoxButton.OK, MessageBoxImage.Information);
break;
}
else if (value == Global.WARNING_CHAR[0])
{
if (Global.activeClick)
{
port.BaseStream.Write(Global.POLL_CHAR, 0, Global.POLL_CHAR.Length);
}
}
}
}
catch { }
}
else
{
break;
}
}
}
Get rid of the horrible 'if (port.BytesToRead > 0)' polling loop! Why do you do that - you do nothing else if the check retuns false, so just let the read block in the normal way.
Use the DataReceived event. This event is called on a secondary thread, there's no need to create another, or event a BackgroundWorker.
I have the code below where it update the data within listview, I was trying to pass parameter into main code which update the listview table.
I have external code that can send burst data stream and it shown same value on the list.
Below is based on anonymous method which work but earlier data get overwritten.
The listview is too slow which slow down main program (not listed here) the burst data goes to this code and use separate thread to handle the display (about 20 set). The burst data is about 20 set of dataTX array, etc.
I'm open for suggestion how to fix this.
==========================================================================
public void LIN_Request_Add_Message(bool isCRCIncluded) // This Add new line for request based message.
{
byte[] dataTX = new byte[10];
dataTX = myLinTools.LinArrayTXArray();
DateTime d = DateTime.Now;
this.ReqAddMessageThread = new Thread(delegate() { ReqAddMessageThreadProc(isCRCIncluded, dataTX, d); }); //anonymous method
this.ReqAddMessageThread.Start();
}
#endregion
private void ReqAddMessageThreadProc(bool isCRCIncluded, byte[] dataTX, DateTime d)
{
if (this.OutputView.InvokeRequired)
{
test1Callback del = new test1Callback(ReqAddMessageThreadProc);
this.Invoke(del, new object[] { isCRCIncluded, dataTX,d });
return;
}
if (this.Visible == true)
{
SendMessage(this.Handle, WM_SETREDRAW, false, 0);
}
int length = myLinTools.LINDataLength;
int pCRC = 0;
elem = new ListViewItem(m_Item.ToString());
elem.SubItems.Add(d.Date.ToShortDateString());
elem.SubItems.Add(d.ToShortTimeString() + ":" + d.Second.ToString());
elem.SubItems.Add("");
for (int i = 0; i < length + 1; i++)
{
elem.SubItems.Add(dataTX[i].ToString("X2"));
pCRC = i;
}
for (int i = length; i < 8; i++)
{
elem.SubItems.Add(" "); // fill gaps
}
if (isCRCIncluded == true) // Does the message contains processed CRC data?
{
elem.SubItems.Add(dataTX[pCRC + 1].ToString("X2"));
}
else // No, then make one for display only!!
{
Byte CRC = myLinTools.CRC_Processor(false);
elem.SubItems.Add(CRC.ToString("X2"));
}
this.OutputView.Items.Add(elem);
this.OutputView.EnsureVisible(m_Item);
if (myLinTools.IsRequestResponse == true) // Request Message Only
{
if (this.Visible == true) // Is form open?
{
SendMessage(this.Handle, WM_SETREDRAW, true, 0);
this.Refresh();
}
}
m_Item++;
}
=================================================================================
Thanks KazR, I modified the code which worked fine, however it slow down the other high level program (call it main program), that making data transfer to this program that display data. One of the requirement that the main program stream the data without delay or pause cause by listview in this display program. That why I'm looking for way to use thread, so it release the control back to main program and thus operates faster, but however there is issue in keep data from being over-written by next thread, since listview is slow. Perhaps I should consider a buffer, which update only when there is no activity in main program.
I do not wish to use virtual, I'm open for alternative suggestion.
==================================================================================
delegate void ReqAddMessageTCallback(bool isCRCIncluded, byte[] dataTX, DateTime d);
#region//==================================================LIN_Request_Add_Message
public void LIN_Request_Add_Message(bool isCRCIncluded) // This Add new line for request based message.
{
byte[] dataTX = new byte[10];
dataTX = myLinTools.LinArrayTXArray();
DateTime d = DateTime.Now;
ReqAddMessageThreadProc(isCRCIncluded, dataTX, d);
}
#endregion
#region//==================================================ReqAddMessageThreadProc
private void ReqAddMessageThreadProc(bool isCRCIncluded, byte[] dataTX, DateTime d)
{
if (this.OutputView.InvokeRequired)
{
ReqAddMessageTCallback del = new ReqAddMessageTCallback(ReqAddMessageThreadProc);
this.BeginInvoke(del, new object[] { isCRCIncluded, dataTX, d });
return;
}
if (this.Visible == true)
{
SendMessage(this.Handle, WM_SETREDRAW, false, 0);
}
int length = myLinTools.LINDataLength;
int pCRC = 0;
elem = new ListViewItem(m_Item.ToString());
From your code example it appears that you're creating a new Thread object each time you receive data and all this thread is doing is calling the ReqAddMessageThreadProc method. Assuming that calls to LIN_Request_Add_Message are not being made in the main UI thread, you could try removing the Thread creation & start calls, replace them with a direct call the ReqAddMessageThreadProc and use BeginInvoke rather than Invoke.
e.g.
public void LIN_Request_Add_Message(bool isCRCIncluded) // This Add new line for request based message.
{
byte[] dataTX = new byte[10];
dataTX = myLinTools.LinArrayTXArray();
DateTime d = DateTime.Now;
ReqAddMessageThreadProc(isCRCIncluded, dataTX, d);
}
#endregion
private void ReqAddMessageThreadProc(bool isCRCIncluded, byte[] dataTX, DateTime d)
{
if (this.OutputView.InvokeRequired)
{
test1Callback del = new test1Callback(ReqAddMessageThreadProc);
this.BeginInvoke(del, new object[] { isCRCIncluded, dataTX,d });
return;
}
etc...
The BeginInvoke call is the async version of Invoke, this should negate the need to use separate new Thread objects each time you receive new data.
You should make your ListView virtual. Then you can operate at full speed with your threads and the listview will display only the items which are currently visible. If you do append things to the ListView and not force to make it visible every time something is added you can let the user decide when he does want to scroll down. He does see when things are appended because the scrollbar is becoming smaller and smaller while items are added.
This way you can display millions of entries without any issues and you will get rid of ivoking, redrawing, refreshing the listview via SendMessage. Here is a virtual list view sample how you can change it.
By doing so your code will become simpler and faster because you do not mix background data processing with UI stuff. E.g. sorting can be done at raw data array level without doing anything in the UI except to trigger it.