The requirement is like this:
I must issue a command requesting a screenshot. The twist? The command specifies a time - and that time is in the PAST. Not forever, mind you, just within a 30 second window.
My command indicates the actual second, and the corresponding screenshot is pulled from the rolling cache and saved in a permanent location for another process.
To accomplish this, I must take a screenshot every second, and I must preserve them for 30 seconds. After that the cache can purge itself.
My question is
What method for screenshots is least impactful against the desktop?
I can't imagine that one frame per second is going to be terrible for performance. Have you tried the simplest possible method to see if it impacts performance?
Take a look at http://www.dotnetjalps.com/2007/06/how-to-take-screenshot-in-c.html for information on how to get a screen shot.
I would suggest you use a timer:
var ssTimer = new System.Threading.Timer((s) =>
{
GetAndStoreScreenShot();
}, null, 1000, 1000);
You'll need some synchronization on your cache to prevent threading problems (i.e. trying to read the first item in the cache while it's being pushed out). Probably easiest to use a lock, since that's not going to be particularly performance sensitive.
As for what to use for a cache, I'd suggest a LinkedList. It's easy to append to the list (AddLast()) and remove the first item (RemoveFirst()), and since you'll only have 30 of them and requests for screen shots will be relatively infrequent, a sequential scan to get the nth item wouldn't be too time consuming.
Or you can implement a simple circular buffer using List or Array. Increment an index to be the insertion spot, wrapping around when you increment off the end. A lookup then becomes a simple matter of modulo arithmetic.
Along the lines of what Jim said, you could store your working set of screenshots in a ConcurrentDictionary<DateTime, Bitmap>, and have two timers: The first timer would add your screenshot to the dictionary, and the second would remove anything older than 30 seconds from the dictionary.
However, this is going to be terrible for performance pretty much any way you slice it.
Something along these lines:
var imageDictionary = new ConcurrentDictionary<DateTime, Bitmap>();
var screenshotTaker = new System.Timers.Timer(1000);
screenshotTaker.Elapsed += (sender, e) =>
{
Bitmap bmp = GetScreenshot();
imageDictionary.TryAdd(DateTime.Now, bmp);
};
var screenshotRemover = new System.Timers.Timer(1000);
screenshotRemover.Elapsed += (sender, e) =>
{
RemoveExpiredBitmaps();
};
screenshotTaker.Start();
screenshotRemover.Start();
Related
So, I have a resizable window which draws a graph given on live values it calculates. If I make the window bigger, it redraws and scales everything, also when the graph hits a new maxY and maxX, using those as reference points to determine the position of the other points. This works fine, but if I resize the window, and a lot of things are being scheduled for updating, it the window resizes fine, but the Graph mimics what it was supposed to do a couple (up to 4) seconds ago.
If I understand this correctly, every time I invoke my UI thread, which is blocked, and give him new coordinates to draw, it finishes the current one and then goes on to the next one. I see why you would want that, but since the graph is more or less constantly scaling, it is deleting itself every update anyway, so I would save quite some processing power and runtime, if I would just finish every current task and jump over all the accumulated task right to the newest one, since the stored ones are outdated anyway
Is there a way to do this?
I thought of 3 things which might work, of which the third one the only one is, I know will work or, rather, is possible, even if rather slow:
// (1) If there is a way to check if the UI thread is busy, I could just not send
// a new request to draw another line
if(!uithread.busy)
{
Application.Current.Dispatcher.BeginInvoke(DispatcherPriority.Background,
new Action(() => this.DrawGraph(new List<List<Coordinates>> { CoordinateListA, CoordinateListB })));
}
// (2) As soon as UI thread finishes the current task, it jumps to the newest one
// (3) In my opinion the ugliest solution, I check how long the last drawing took and
// don't send any draw-requests in that time. I will just wait that certain time and
// just then send him the newest results to draw and measure how long that took
If there is no better solution, I think I will go with (3), but since I hope that there is another way, I wanted to ask if maybe someone else here had a similar issue.
So I fixed it. My Architecture saves all the logs, so I do not need to save coordinates, if I need new values I can just, every time I need new Coordinates, calculate them from the Logs. Like this:
// beginning of the class
private bool _isDrawing = false;
if (!_graphIsDrawing)
{
_graphIsDrawing = true;
Application.Current.Dispatcher.BeginInvoke(DispatcherPriority.Background,
new Action(() => this.DrawGraph(new List<List<Coordinates>> { CoordinateListA, CoordinateListB }, scalingFactors, canvasSize)));
//I need to invoke, since I am working with multiple threads here. Else it
//would be enough to just call 'this.DrawGraph(...)'
}
///////////
public void DrawGraph(List<List<Coordinates>> listOfGraphs, float[] scalingFactor, int[] canvasSize)
{
lock (_graphDrawLock)
{
this._AlgorithmRuntimeViewA.DrawGraph(listOfGraphs[0], scalingFactor, canvasSize);
this._AlgorithmRuntimeViewB.DrawGraph(listOfGraphs[1], scalingFactor, canvasSize);
_graphIsDrawing = false;
}
}
Here I lock it again, so not both threads draw at the same time breaking everything. At the end I set _graphIsDrawing to false again, so I can call it again.
I am using ObservableCollection to store information on my CPU Usage and transferring this information to a line chart. the information is updated every second. It is working fine but I realize that this is going to jam up my memory overtime cos it just keeps adding information to the list.
What is the norm in this situation? Do you reset the list after every minute? I feel that would mess up how the chart looks every time it resets. Please advice how I could manage this memory issue swiftly. Thanks.
ObservableCollection<KeyValuePair<double, double>> chart1 = new ObservableCollection<KeyValuePair<double, double>>();
chart1.Add(new KeyValuePair<double, double>(DateTime.now, getCurrentCpuUsage()));
What exactly you should do depends on your requirements.
If you only need to retain a certain amount of data (e.g. 10 minutes or whatever), a bounded queue may be more appropriate than an ObservableCollection. That way, events that are "too old" automatically fall out of the data structure, allowing you to cap memory usage.
If you still want to be able to access older data in the future, you could write the data coming out of the end of the queue to a file or database instead of just dropping it.
For one implementation of a bounded queue see
Limit size of Queue<T> in .NET?
Since you may need an observable bounded queue, here are some notes on how to implement one (fairly straightforward)
Observable Stack and Queue
Bounded Queue Explained
A regular queue is like the line at a checkout stand. People line up (or as the British would say queue up) at the end of the line, and the cashier takes the person at the front of the line. First in, first out. FIFO.
A bounded queue sets a maximum length for the line. For a real-life line, new people would be prevented from joining the line if it is too long. Some bounded queues in software work that way too. The other option to keep the queue length from exceeding a limit is to remove from the front of the queue when the line is too long. In real life that's probably not very fair, but for software algorithms that's sometimes just what you need.
Store as much information as you need and drop (or save to file) the rest.
If you need to store data for very long time spans (more than i.e. 1M of data points) you may try compress data. But figure out your goals and measure performance time/memory usage first.
Short Question
I have a loop that runs 180,000 times. At the end of each iteration it is supposed to append the results to a TextBox, which is updated real-time.
Using MyTextBox.Text += someValue is causing the application to eat huge amounts of memory, and it runs out of available memory after a few thousand records.
Is there a more efficient way of appending text to a TextBox.Text 180,000 times?
Edit I really don't care about the result of this specific case, however I want to know why this seems to be a memory hog, and if there is a more efficient way to append text to a TextBox.
Long (Original) Question
I have a small app which reads a list of ID numbers in a CSV file and generates a PDF report for each one. After each pdf file is generated, the ResultsTextBox.Text gets appended with the ID Number of the report that got processed and that it was successfully processed. The process runs on a background thread, so the ResultsTextBox gets updated real-time as items get processed
I am currently running the app against 180,000 ID numbers, however the memory the application is taking up is growing exponentially as time goes by. It starts by around 90K, but by about 3000 records it is taking up roughly 250MB and by 4000 records the application is taking up about 500 MB of memory.
If I comment out the update to the Results TextBox, the memory stays relatively stationary at roughly 90K, so I can assume that writing ResultsText.Text += someValue is what is causing it to eat memory.
My question is, why is this? What is a better way of appending data to a TextBox.Text that doesn't eat memory?
My code looks like this:
try
{
report.SetParameterValue("Id", id);
report.ExportToDisk(ExportFormatType.PortableDocFormat,
string.Format(#"{0}\{1}.pdf", new object[] { outputLocation, id}));
// ResultsText.Text += string.Format("Exported {0}\r\n", id);
}
catch (Exception ex)
{
ErrorsText.Text += string.Format("Failed to export {0}: {1}\r\n",
new object[] { id, ex.Message });
}
It should also be worth mentioning that the app is a one-time thing and it doesn't matter that it is going to take a few hours (or days :)) to generate all the reports. My main concern is that if it hits the system memory limit, it will stop running.
I'm fine with leaving the line updating the Results TextBox commented out to run this thing, but I would like to know if there is a more memory efficient way of appending data to a TextBox.Text for future projects.
I suspect the reason the memory usage is so large is because textboxes maintain a stack so that the user can undo/redo text. That feature doesn't seem to be required in your case, so try setting IsUndoEnabled to false.
Use TextBox.AppendText(someValue) instead of TextBox.Text += someValue. It's easy to miss since it's on TextBox, not TextBox.Text. Like StringBuilder, this will avoid creating copies of the entire text each time you add something.
It would be interesting to see how this compares to the IsUndoEnabled flag from keyboardP's answer.
Don't append directly to the text property. Use a StringBuilder for the appending, then when done, set the .text to the finished string from the stringbuilder
Instead of using a text box I would do the following:
Open up a text file and stream the errors to a log file just in case.
Use a list box control to represent the errors to avoid copying potentially massive strings.
Personally, I always use string.Concat* . I remember reading a question here on Stack Overflow years ago that had profiling statistics comparing the commonly-used methods, and (seem) to recall that string.Concat won out.
Nonetheless, the best I can find is this reference question and this specific String.Format vs. StringBuilder question, which mentions that String.Format uses a StringBuilder internally. This makes me wonder if your memory hog lies elsewhere.
**based on James' comment, I should mention that I never do heavy string formatting, as I focus on web-based development.*
Maybe reconsider the TextBox? A ListBox holding string Items will probably perform better.
But the main problem seem to be the requirements, Showing 180,000 items cannot be aimed at a (human) user, neither is changing it in "Real Time".
The preferable way would be to show a sample of the data or a progress indicator.
When you do want to dump it at the poor User, batch string updates. No user could descern more than 2 or 3 changes per second. So if you produce 100/second, make groups of 50.
Some responses have alluded to it, but nobody has outright stated it which is surprising.
Strings are immutable which means a String cannot be modified after it is created. Therefore, every time you concatenate to an existing String, a new String Object needs to be created. The memory associated with that String Object also obviously needs to be created, which can get expensive as your Strings become larger and larger. In college, I once made the amateur mistake of concatenating Strings in a Java program that did Huffman coding compression. When you're concatenating extremely large amounts of text, String concatenation can really hurt you when you could have simply used StringBuilder, as some in here have mentioned.
Use the StringBuilder as suggested.
Try to estimate the final string size then use that number when instantiating the StringBuilder. StringBuilder sb = new StringBuilder(estSize);
When updating the TextBox just use assignment eg: textbox.text = sb.ToString();
Watch for cross-thread operations as above. However use BeginInvoke. No need to block
the background thread while the UI updates.
A) Intro: already mentioned, use StringBuilder
B) Point: don't update too frequently, i.e.
DateTime dtLastUpdate = DateTime.MinValue;
while (condition)
{
DoSomeWork();
if (DateTime.Now - dtLastUpdate > TimeSpan.FromSeconds(2))
{
_form.Invoke(() => {textBox.Text = myStringBuilder.ToString()});
dtLastUpdate = DateTime.Now;
}
}
C) If that's one-time job, use x64 architecture to stay within 2Gb limit.
StringBuilder in ViewModel will avoid string rebindings mess and bind it to MyTextBox.Text. This scenario will increase performance many times over and decrease memory usage.
Something that has not been mentioned is that even if you're performing the operation in the background thread, the update of the UI element itself HAS to happen on the main thread itself (in WinForms anyway).
When updating your textbox, do you have any code that looks like
if(textbox.dispatcher.checkAccess()){
textbox.text += "whatever";
}else{
textbox.dispatcher.invoke(...);
}
If so, then your background op is definitely being bottlenecked by the UI Update.
I would suggest that your background op use StringBuilder as noted above, but instead of updating the textbox every cycle, try updating it at regular intervals to see if it increases performance for you.
EDIT NOTE:have not used WPF.
You say memory grows exponentially. No, it is a quadratic growth, i.e. a polynomial growth, which is not as dramatic as an exponential growth.
You are creating strings holding the following number of items:
1 + 2 + 3 + 4 + 5 ... + n = (n^2 + n) /2.
With n = 180,000 you get total memory allocation for 16,200,090,000 items, i.e. 16.2 billion items! This memory will not be allocated at once, but it is a lot of cleanup work for the GC (garbage collector)!
Also, bear in mind, that the previous string (which is growing) must be copied into the new string 179,999 times. The total number of copied bytes goes with n^2 as well!
As others have suggested, use a ListBox instead. Here you can append new strings without creating a huge string. A StringBuild does not help, since you want to display the intermediate results as well.
here's the problem: I have a custom hardware device and I have to grab images from it in C#/WPF and display them in a window, all with 120+ FPS.
The problem is that there is no event to indicate the images are ready, but I have to constantly poll the device and check whether there are any new images and then download them.
There are apparently a handful of ways to do it, but I haven't been able to find the right one yet.
Here's what I tried:
A simple timer (or DispatcherTimer) - works great for slower frame rates but I can't get it past let's say, 60 FPS.
An single threaded infinite loop - quite fast but I have to put the DoEvents/it's WPF equivalent in the loop in order for window to be redrawn; this has some other unwanted (strange) consequences such as key press events from some controls not being fired etc..
Doing polling/downloading in another thread and displaying in UI thread, something like this:
new Thread(() =>
{
while (StillCapturing)
{
if (Camera.CheckForAndDownloadImage(CameraInstance))
{
this.Dispatcher.Invoke((Action)this.DisplayImage);
}
}
}).Start();
Well, this works relatively well, but puts quite a load on a CPU and of course completely kills the machine if it doesn't have more than one CPU/core, which is unacceptable. Also, I there is a large number of thread contentions this way.
The question is obvious - are there any better alternatives, or is one of these the way to go in this case?
Update:
I somehow forgot to mention that (well, forgot to think about it while writing this question), but of course I don't need all frames to be displayed, however I still need to capture all of them so they can be saved to a hard drive.
Update2:
I found out that the DispatcherTimer method is slow not because it can't process everything fast enough, but because the DispatcherTimer waits for the next vertical sync before firing the tick event; which is actually good in my case, because in the tick event I can save all pending images to a memory buffer (used for saving images to disk) and display just the last one.
As for the old computers being completely "killed" by capturing, it appears that WPF falls back to software rendering which is very slow. There's probably nothing I can do about.
Thanks for all the answers.
I think you're trying for too simplistic of an approach. Here's what I would do.
a) put a Thread.Sleep(5) in your polling loop, that should allow you to get close to 120fps while still keeping CPU times low.
b) Only update the display with every 5th frame or so. That will cut down on the amount of processing as I'm not sure that WPF is made to handle much more than 60fps.
c) Use ThreadPool to spawn a subtask for each frame that will then go and save it to the disk (in a seperate file per frame), that way you won't be as limited by disk performance. Extra frames will just pile up in memory.
Personally I would implement them in that order. Chances are a or b will fix your problems.
You could do the following (all psuedocode):
1. Have a worker thread running dealing with the capture process:
List<Image> _captures = new List<Image>();
new Thread(() =>
{
while (StillCapturing)
{
if (Camera.CheckForAndDownloadImage(CameraInstance))
{
lock(_locker){_captures.Add(DisplayImage);
}
}
}).Start();
Have the dispatcher timer thread take latest captured image (obviously it will have missed some captures since last tick) and display. Therefore, UI thread is throttled and doing as little as possible, it isn't doing all the "capturing", this is done by worker threads. sorry I can't get this bit to format (but you get the idea):
void OnTimerTick(can't remember params)
{
Image imageToDisplay;
lock(_locker){imageToDisplay = _captures[k.Count - 1];
DisplayFunction(imageToDisplay);
}
it might be that the list is a queue and another thread is used to bleed the queue and write to disk or whatever.
I have a C# application in which a LOT of information is being added to a Textbox for display to the user. Upon processing of the data, almost immediately, the application becomes very slow and unresponsive. This is how I am currently attempting to handle this:
var saLines = textBox1.Lines;
var saNewLines = saLines.Skip(50);
textBox1.Lines = saNewLines.ToArray();
This code is run from a timer every 100mS. Is there a better way to handle this? I am using Microsoft Visual C# 2008 Express Edition. Thanks.
The simple answer is TextBox.AppendText().
You get much better performance initially.
I tested writing a 500 char message every 20 ms for 2 mins (with BackgroundWorker) and the UI remained responsive and CPU minimal. At some point, of course, it will become unresponsive but it was good enough for my needs.
Try by having in memory a list with the content, and removing the first 50 elements by RemoveRange and then going with ToArray();
Like this :
lst.RemoveRange(0,50);
textBox1.Lines = lst.ToArray();
It should be a lot faster.
I'd say your main problem here is that you are using the TextBox as your primary storage for your text. Everytime you call TextBox.Lines, the string is split on Environment.NewLine.
Try turning it around:
Store the text in a new List<String>(maxLines)
In your AddLine method, check the length of your text buffer and use RemoveRange(0, excessCount)
Update your display TextBox by calling String.Join(Environment.NewLine, textBuffer.ToArray())
That last call is a bit expensive, but it should stop your slowdowns. To get it any faster you'd need to use a statically sized string array and move the references around yourself.
The most efficient way to trim an array is to create a new array of the desired size, then use Array.Copy to copy the desired portion of the old array.
I would recommend that you maintain a List<string> containing all of your lines.
You should use a StringBuilder to build a string containing the lines you're looking for, and set the textbox's Text proeprty to the StringBuilder's string. For added performance, set the StringBuilder's capacity to a reasonable guess of the final sie of the string. (Or to list.Skip(...).Take(...).Sum(s => s.Length))
If you're concerned about memory, you can trim the List<string> by calling RemoveRange.
As long as you don't put too much in the textbox at once, doing it this way should be extremely fast. All of the manipulation of the List<string> and the StringBuilder can be done in a background thread, and you can pass the completed string to the UI thread.
The TextBox.Lines property simply concatenates the array you give it using a StringBuilder, so there's no point in using it (and making a needless array).
Instead of splitting the text and then re-joining it, just get the sub-string from the 51st line:
int i = textBox1.GetFirstCharIndexFromLine(50);
if (i > 0) textBox1.Text = textBox1.Text.Substring(i);