Develop Reference

Full CPU usage for Parallel.For loops

I am writing a WPF application that processes an image data stream from an IR camera. The application uses a class library for processing steps such as rescaling or colorizing, which I am also writing myself. An image processing step looks something like this:
ProcessFrame(double[,] frame)
{
int width = frame.GetLength(1);
int height = frame.GetLength(0);
byte[,] result = new byte[height, width];
Parallel.For(0, height, row =>
{
for(var col = 0; col < width; ++col)
ManipulatePixel(frame[row, col]);
});
}
Frames are processed by a task that runs in the background. The issue is, that depending on how costly the specific processing algorithm is ( ManipulatePixel() ), the application can't keep up with the camera's frame rate any more. However, I have noticed that despite me using parallel for loops, the application simply won't use all of the CPU that is available - task manager performance tab shows about 60-80% CPU usage.
I have used the same processing algorithms in C++ before, using the concurrency::parallel_for loops from the parallel patterns library. The C++ code uses all of the CPU it can get, as I would expect, and I also tried PInvoking a C++ DLL from my C# code, doing the same algorithm that runs slowly in the C# library - it also uses all the CPU power available, CPU usage is right at 100% virtually the whole time and there is no trouble at all keeping up with the camera.
Outsourcing the code into a C++ DLL and then marshalling it back into C# is an extra hassle I'd of course rather avoid. How do I make my C# code actually make use of all the CPU potential? I have tried increasing process priority like this:
using (Process process = Process.GetCurrentProcess())
process.PriorityClass = ProcessPriorityClass.RealTime;
Which has an effect, but only a very small one. I also tried setting the degree of parallelism for the Parallel.For() loops like this:
ParallelOptions parallelOptions = new ParallelOptions();
parallelOptions.MaxDegreeOfParallelism = Environment.ProcessorCount;
and then passing that to the Parallel.For() loop, this had no effect at all but I suppose that's not surprising since the default settings should already be optimized. I also tried setting this in the application configuration:
<runtime>
<Thread_UseAllCpuGroups enabled="true"></Thread_UseAllCpuGroups>
<GCCpuGroup enabled="true"></GCCpuGroup>
<gcServer enabled="true"></gcServer>
</runtime>
but this actually makes it run even slower.
EDIT:
The ProcessFrame code block I quoted originally was actually not quite correct. What I was doing at the time was:
ProcessFrame(double[,] frame)
{
byte[,] result = new byte[frame.GetLength(0), frame.GetLength(1)];
Parallel.For(0, frame.GetLength(0), row =>
{
for(var col = 0; col < frame.GetLength(1); ++col)
ManipulatePixel(frame[row, col]);
});
}
Sorry for this, I was paraphrasing code at the time and I didn't realize that this is an actual pitfall, that produces different results. I have since changed the code to what I originally wrote (i.e. the width and height variables set at the beginning of the function, and the array's length properties only queried once each instead of in the for loop's conditional statements). Thank you #Seabizkit, your second comment inspired me to try this. The change in fact already makes the code run noticeably faster - I didn't realize this because C++ doesn't know 2D arrays so I had to pass the pixel dimensions as separate arguments anyway. Whether it is fast enough as it is I cannot say yet however.
Also thank you for the other answers, they contain a lot of things I don't know yet but it's great to know what I have to look for. I'll update once I reached a satisfactory result.

I would need to have all of your code and be able to run it locally in order to diagnose the problem because your posting is devoid of details (I would need to see inside your ManipulatePixel function, as well as the code that calls ProcessFrame). but here's some general tips that apply in your case.
2D arrays in .NET are significantly slower than 1D arrays and staggered arrays, even in .NET Core today - this is a longstanding bug.
See here:
https://github.com/dotnet/coreclr/issues/4059
Why are multi-dimensional arrays in .NET slower than normal arrays?
Multi-dimensional array vs. One-dimensional
So consider changing your code to use either a jagged array (which also helps with memory locality/proximity caching, as each thread would have its own private buffer) or a 1D array with your own code being responsible for bounds-checking.
Or better-yet: use stackalloc to manage the buffer's lifetime and pass that by-pointer (unsafe ahoy!) to your thread delegate.
Sharing memory buffers between threads makes it harder for the system to optimize safe memory accesses.
Avoid allocating a new buffer for each frame encountered - if a frame has a limited lifespan then consider using reusable buffers using a buffer-pool.
Consider using the SIMD and AVX features in .NET. While modern C/C++ compilers are smart enough to compile code to use those instructions, the .NET JIT isn't so hot - but you can make explicit calls into SMID/AVX instructions using the SIMD-enabled types (you'll need to use .NET Core 2.0 or later for the best accelerated functionality)
Also, avoid copying individual bytes or scalar values inside a for loop in C#, instead consider using Buffer.BlockCopy for bulk copy operations (as these can use hardware memory copy features).
Regarding your observation of "80% CPU usage" - if you have a loop in a program then that will cause 100% CPU usage within the time-slices provided by the operating-system - if you don't see 100% usage then your code then:
Your code is actually running faster than real-time (this is a good thing!) - (unless you're certain your program can't keep-up with the input?)
Your codes' thread (or threads) is blocked by something, such as a blocking IO call or a misplaced Thread.Sleep. Use tools like ETW to see what your process is doing when you think it should be CPU-bound.
Ensure you aren't using any lock (Monitor) calls or using other thread or memory synchronization primitives.

Efficiency matters ( it is not true-[PARALLEL], but may, yet need not, benefit from a "just"-[CONCURRENT] work
The BEST, yet a rather hard way, if ultimate performance is a MUST :
in-line an assembly, optimised as per cache-line sizes in the CPU hierarchy and keep indexing that follows the actual memory-layout of the 2D data { column-wise | row-wise }. Given there is no 2D-kernel-transformation mentioned, your process does not need to "touch" any topological-neighbours, the indexing can step in whatever order "across" both of the ranges of the 2D-domain and the ManipulatePixel() may get more efficient on transforming rather blocks-of pixels, instead of bearing all overheads for calling a process just for each isolated atomicised-1px ( ILP + cache-efficiency are on your side ).
Given your target production-platform CPU-family, best use (block-SIMD)-vectorised instructions available from AVX2, best AVX512 code. As you most probably know, may use C/C++ using AVX-intrinsics for performance optimisations with assembly-inspection and finally "copy" the best resulting assembly for your C# assembly-inlining. Nothing will run faster. Tricks with CPU-core affinity mapping and eviction/reservation are indeed a last resort, yet may help for indeed an almost hard-real-time production settings ( though, hard R/T systems are seldom to get developed in an ecosystem with non-deterministic behaviour )
A CHEAP, few-seconds step :
Test and benchmark the run-time per batch of frames of a reversed composition of moving the more-"expensive"-part, the Parallel.For(...{...}) inside the for(var col = 0; col < width; ++col){...} to see the change of the costs of instantiations of the Parallel.For() instrumentation.
Next, if going this cheap way, think about re-factoring the ManipulatePixel() to at least use a block of data, aligned with data-storage layout and being a multiple of cache-line length ( for cache-hits ~ 0.5 ~ 5 [ns] improved costs-of-memory accesses, being ~ 100 ~ 380 [ns] otherwise - here, a will to distribute a work (the worse per 1px) across all NUMA-CPU-cores will result in paying way more time, due to extended access-latencies for cross-NUMA-(non-local) memory addresses and besides never re-using an expensively cached block-of-fetched-data, you knowingly pay excessive costs from cross-NUMA-(non-local) memory fetches ( from which you "use" just 1px and "throw" away all the rest of the cached-block ( as those pixels will get re-fetched and manipulated in some other CPU-core in some other time ~ a triple-waste of time ~ sorry to have mentioned that explicitly, but when shaving each possible [ns] this cannot happen in production pipeline ) )
Anyway, let me wish you perseverance and good luck on your steps forwards to gain the needed efficiency back onto your side.

Here's what I ended up doing, mostly based on Dai's answer:
made sure to query image pixel dimensions once at the beginning of the processing functions, not within the for loop's conditional statement. With parallel loops, it would seem this creates competitive access of those properties from multriple threads which noticeably slows things down.
removed allocation of output buffers within the processing functions. They now return void and accept the output buffer as an argument. The caller creates one buffer for each image processing step (filtering, scaling, colorizing) only, which doesn't change in size but gets overwritten with each frame.
removed an extra data processing step where raw image data in the format ushort (what the camera originally spits out) was converted to double (actual temperature values). Instead, processing is applied to the raw data directly. Conversion to actual temperatures will be dealt with later, as necessary.
I also tried, without success, to use 1D arrays instead of 2D but there is actually no difference in performance. I don't know if it's because the bug Dai mentioned was fixed in the meantime, but I couldn't confirm 2D arrays to be any slower than 1D arrays.
Probably also worth mentioning, the ManipulatePixel() function in my original post was actually more of a placeholder rather than a real call to another function. Here's a more proper example of what I am doing to a frame, including the changes I made:
private static void Rescale(ushort[,] originalImg, byte[,] scaledImg, in (ushort, ushort) limits)
{
Debug.Assert(originalImg != null);
Debug.Assert(originalImg.Length != 0);
Debug.Assert(scaledImg != null);
Debug.Assert(scaledImg.Length == originalImg.Length);
ushort min = limits.Item1;
ushort max = limits.Item2;
int width = originalImg.GetLength(1);
int height = originalImg.GetLength(0);
Parallel.For(0, height, row =>
{
for (var col = 0; col < width; ++col)
{
ushort value = originalImg[row, col];
if (value < min)
scaledImg[row, col] = 0;
else if (value > max)
scaledImg[row, col] = 255;
else
scaledImg[row, col] = (byte)(255.0 * (value - min) / (max - min));
}
});
}
This is just one step and some others are much more complex but the approach would be similar.
Some of the things mentioned like SIMD/AVX or the answer of user3666197 unfortunately are well beyond my abilities right now so I couldn't test that out.
It's still relatively easy to put enough processing load into the stream to tank the frame rate but for my application the performance should be enough now. Thanks to everyone who provided input, I'll mark Dai's answer as accepted because I found it the most helpful.

Binary Search - How to load +5M records from a file into Range<int>[] array?

This question is a follow up to my previous question regarding binary search (Fast, in-memory range lookup against +5M record table).
I have sequential text file, with over 5M records/lines, in the format below. I need to load it into Range<int>[] array. How would one do that in a timely fashion?
File format:
start int64,end int64,result int
start int64,end int64,result int
start int64,end int64,result int
start int64,end int64,result int
...

I'm going to assume you have a good disk. Scan through the file once and count the number of entries. If you can guarantee your file has no blank lines, then you can just count the number of newlines in it -- don't actually parse each line.
Now you can allocate your array once with exactly that many entries. This avoids excessive re-allocations of the array:
var numEntries = File.ReadLines(filepath).Count();
var result = new Range<int>[numEntries];
Now read the file again and create your range objects with code something like:
var i = 0;
foreach (var line in File.ReadLines(filepath))
{
var parts = line.Split(',');
result[i++] = new Range<int>(long.Parse(parts[0]), long.Parse(parts[1]), int.Parse(parts[2]);
}
return result;
Sprinkle in some error handling as desired. This code is easy to understand. Try it out in your target environment. If it is too slow, then you can start optimizing it. I wouldn't optimize prematurely though because that will lead to much more complex code that might not be needed.

This is a typical (?) producer-consumer problem which can be solved using multiple threads. In your case the producer is reading data from disk and the consumer is parsing the lines and populating the array. I can see two different cases:
Producer is (much) faster than the consumer: in this case you should try using more consumer threads;
Consumer is (much) faster than the producer: you can't do very much to speed up things other than affecting your hardware configuration such as buying a faster hard disk or using a RAID 0. In this case I wouldn't even use a multithreading solution because it's not worth the added complexity.
This question might help you implementing that in C#.

C#: Reading Huge CSV File

I'm parsing a 40MB CSV file.
It works nicely right now, and it's rather easy to parse, the only problem I have is performance, which of course is rather slow.
I'd like to know if there is a way I can improve this, as I only need to find by key I find and then stop looping, so if the entry is at the beginning of the file it finishes quickly, but if it's at the end it takes a while.
I could balance this by giving it a random start line, but the algorithm would still be O(n)... So I'm not sure if it's really worth it.
Is there a way I can improve my sequential parsing algorithm?

First: "Reading Huge CSV File" and "So I'm parsing a 40MB CSV file.". Ihave space delimited files here of 10+ GIGAbyte - what would you call those?
Also: the size of the file is irrelevant, you process them normally anyway line by line.
the only problem I have is performance, which of course is rather slow
Define. What do you think is slow? Parsing them is quite fast when done properly.
I'd like to know if there is a way I can improve this, as I only need to find by key I find and
then stop looping, so if the entry is at the beggining of the file it
finishes quickly, but if it's at the end it takes a while.
Do NOT use a CSV file? More than 60 years ago people invented databases for this.
Is there a way I can improve my secuential parsing algorithm?
YOu mean except pulling the parsing into a separate thread, and using an efficient code (which you may not have - noone knows).
Theoretically you could:
Read on one thread, with a decent buffer (less IO = faster)
Move field split into thread 2 (optional)
Use tasks to parse the fields (one per field per line) so you use all processors).
I am currently processing some (around 10.000) files (with sizes in double digit gigabte sadly) and... I go this way (Have to process them in a specific order) to use my computer fully.
That should give you a lot - and seriously, a 40mb file should load in 0.x seconds (0.5 - 0.6).
STILL that is very inefficient. Any reason you do not load the file into a database like all people do? CSV is good as some transport format, it sucks as a database.

Why don't you convert your csv to a normal database. Even sqlexpress will be fine.

Of course.
Say you order it alphabetically.
Then, start in the middle.
Each iteration, move to the middle of the top or bottom; whichever has the appropriate key.
This algorithm has O(log n).
This is called a "binary search," and is what "Mike Christianson" suggests in his comment.

Will suggest you to break one 40Mb File into smaller size few files.
And using Parallel.ForEach you could improve file processing performace

You can load the CSV into DataTable and use available operations that could be faster than looping through
Loading it to database and perform your operation on that is another option

This, I believe, is the fastest way to read a CSV file sequentially. There may be other ways to extract data from CSV, but if you limited to this approach, then this solution might work for you.
const int BUFFER_SIZE = 0x8000; //represents 32768 bytes
public unsafe void parseCSV(string filePath)
{
byte[] buffer = new byte[BUFFER_SIZE];
int workingSize = 0; //store how many bytes left in buffer
int bufferSize = 0; //how many bytes were read by the file stream
StringBuilder builder = new StringBuilder();
char cByte; //character representation of byte
using (FileStream fs = new FileStream(filePath, FileMode.Open, FileAccess.Read))
{
do
{
bufferSize = fs.Read(buffer, 0, BUFFER_SIZE);
workingSize = bufferSize;
fixed (byte* bufferPtr = buffer)
{
byte* workingBufferPtr = bufferptr;
while (workingSize-- > 0)
{
switch (cByte = (char)*workingBufferPtr++)
{
case '\n':
break;
case '\r':
case ',':
builder.ToString();
builder.Clear();
break;
default:
builder.Append(cByte);
break;
}
}
}
} while (bufferSize != 0);
}
}
Explanation:
Reading the file into a byte buffer. This will be done using the basic Filestream class, which gives access to the always fast Read()
Unsafe code. While I generally recommend not using unsafe code, when traversing any kind of buffer, using pointers can bring a speedup.
StringBuilder since we will be concatenating bytes into workable strings to test againt the key. StringBuilder is by far the fastest way to append bytes together and get a workable string out them.
Note that this method fairly complaint with RFC 4180, but if you deal with quotes, you can easily modify the code I posted to handle trimming.

Parsing large data file from disk significantly slower than parsing in memory?

While writing a simple library to parse a game's data files, I noticed that reading an entire data file into memory and parsing from there was significantly faster (by up to 15x, 106s v 7s).
Parsing is usually sequential but seeks will be done every now and then to read some data stored elsewhere in a file, linked by an offset.
I realise that parsing from memory will definitely be faster, but something is wrong if the difference is so significant. I wrote some code to simulate this:
public static void Main(string[] args)
{
Stopwatch n = new Stopwatch();
n.Start();
byte[] b = File.ReadAllBytes(#"D:\Path\To\Large\File");
using (MemoryStream s = new MemoryStream(b, false))
RandomRead(s);
n.Stop();
Console.WriteLine("Memory read done in {0}.", n.Elapsed);
b = null;
n.Reset();
n.Start();
using (FileStream s = File.Open(#"D:\Path\To\Large\File", FileMode.Open))
RandomRead(s);
n.Stop();
Console.WriteLine("File read done in {0}.", n.Elapsed);
Console.ReadLine();
}
private static void RandomRead(Stream s)
{
// simulate a mostly sequential, but sometimes random, read
using (BinaryReader br = new BinaryReader(s)) {
long l = s.Length;
Random r = new Random();
int c = 0;
while (l > 0) {
l -= br.ReadBytes(r.Next(1, 5)).Length;
if (c++ <= r.Next(10, 15)) continue;
// simulate seeking
long o = s.Position;
s.Position = r.Next(0, (int)s.Length);
l -= br.ReadBytes(r.Next(1, 5)).Length;
s.Position = o;
c = 0;
}
}
}
I used one of the game's data files as input to this. That file was about 102 MB, and it produced this result (Memory read done in 00:00:03.3092618. File read done in 00:00:32.6495245.) which has memory reading about 11x faster than file.
The memory read was done before the file read to try and improve its speed via the file cache. It's still that much slower.
I've tried increasing or decreasing FileStream's buffer size; nothing produced significantly better results, and increasing or decreasing it too much just worsened the speed.
Is there something I'm doing wrong, or is this to be expected? Is there any way to at least make the slowdown less significant?
Why is reading the entire file at once and then parsing it so much faster than reading and parsing simultaneously?
I've actually compared to a similar library written in C++, which uses the Windows native CreateFileMapping and MapViewOfFile to read files, and it's very fast. Could it be the constant switching from managed to unmanaged and the involved marshaling that causes this?
I've also tried MemoryMappedFiles present in .NET 4; the speed gain was only about one second.

Is there something I'm doing wrong, or is this to be expected?
No, nothing wrong. This is entirely expected. That accessing the disk is an order of magnitude slower than accessing memory is more than reasonable.
Update:
That a single read of the file followed by processing is faster than processing while reading is also expected.
Doing a large IO operation and processing in memory would be faster than getting a bit from disk, processing it, calling the disk again (waiting for the IO to complete), processing that bit etc...

Is there something I'm doing wrong, or is this to be expected?
A harddisk has, compared to RAM, huge access times. Sequential reads are pretty speedy, but as soon as the heads have to move (because data is fragmented) it takes lots of milliseconds to get the next bit of data, during which your application is idling.
Is there any way to at least make the slowdown less significant?
Buy an SSD.
You also can take a look at Memory-Mapped Files for .NET:
MemoryMappedFile.CreateFromFile().
As for your edit: I'd go with #Oded and say that reading the file on beforehand adds a penalty. However, that should not cause the method that first reads the whole file to be seven times as slow as 'process-as-you-read'.

I decided to do some benchmarks comparing various ways of reading a file in C++ and C#. First I created a 256mb file. In the c++ benchmarks, buffered means I first copied the entire file to a buffer then read the data from the buffer. All the benchmarks read the file, directly or indirectly, byte by byte sequentially and calculate a checksum. All times are measured from the moment I open the file until I am completely done and the file is closed. All benchmarks were run multiple times to maintain consistent OS file caching.
C++
Unbuffered memory mapped file: 300ms
Buffered memory mapped file: 500ms
Unbuffered fread: 23,000ms
Buffered fread: 500ms
Unbuffered ifstream: 26,000ms
Buffered ifstream: 700ms
C#
MemoryMappedFile: 112,000ms
FileStream: 2,800ms
MemoryStream: 2,300ms
ReadAllBytes: 600ms
Interpret the data as you wish. C#'s memory mapped files are slower than even the worst case c++ code, whereas c++'s memory mapped files are the fastest things around. C#'s ReadAllBytes is decently fast, only twice as slow as c++'s memory mapped files. So if you want the best performance I recommend you use ReadAllBytes and then access the data directly from the array without using a stream.

C++ ">>" and "<<" IO in C#?

Is there a C# library that provides the functionality of ">>" and "<<" for IO in C++? It was really convenient for console apps. Granted not a lot of console apps are in C#, but some of us use it for them.
I know about Console.Read[Line]|Write[Line] and Streams|FileStream|StreamReader|StreamWriter thats not part of the question.
I dont think im specific enough
int a,b;
cin >> a >> b;
IS AMAZING!!
string input = Console.ReadLine();
string[] data = input.split( ' ' );
a = Convert.ToInt32( data[0] );
b = Convert.ToInt32( data[1] );
... long winded enough? Plus there are other reasons why the C# solution is worse. I must get the entire line or make my own buffer for it. If the line im working on is IDK say the 1000 line of Bells Triangle, I waste so much time reading everything at one time.
EDIT:
GAR!!!
OK THE PROBLEM!!!
Using IntX to do HUGE number like the .net 4.0 BigInteger to produce the bell triangle. If you know the bell triangle it gets freaking huge very very quickly. The whole point of this question is that I need to deal with each number individually. If you read an entire line, you could easily hit Gigs of data. This is kinda the same as digits of Pi. For Example 42pow1048576 is 1.6 MB! I don't have time nor memory to read all the numbers as one string then pick the one I want

No, and I wouldn't. C# != C++
You should try your best to stick with the language convention of whatever language you are working in.

I think I get what you are after: simple, default formatted input. I think the reason there is no TextReader.ReadXXX() is that this is parsing, and parsing is hard: for example: should ReadFloat():
ignore leading whitespace
require decimal point
require trailing whitespace (123abc)
handle exponentials (12.3a3 parses differently to 12.4e5?)
Not to mention what the heck does ReadString() do? From C++, you would expect "read to the next whitespace", but the name doesn't say that.
Now all of these have good sensible answers, and I agree C# (or rather, the BCL) should provide them, but I can certainly understand why they would choose to not provide fragile, nearly impossible to use correctly, functions right there on a central class.
EDIT:
For the buffering problem, an ugly solution is:
static class TextReaderEx {
static public string ReadWord(this TextReader reader) {
int c;
// Skip leading whitespace
while (-1 != (c = reader.Peek()) && char.IsWhiteSpace((char)c)) reader.Read();
// Read to next whitespace
var result = new StringBuilder();
while (-1 != (c = reader.Peek()) && !char.IsWhiteSpace((char)c)) {
reader.Read();
result.Append((char)c);
}
return result.ToString();
}
}
...
int.Parse(Console.In.ReadWord())

Nope. You're stuck with Console.WriteLine. You could create a wrapper that offered this functionality, though.

You can Use Console.WriteLine , Console.ReadLine ..For the purpose.Both are in System NameSpace.

You have System.IO.Stream(Reader|Writer)
And for console: Console.Write, Console.Read

Not that I know of. If you are interested of the chaining outputs you can use System.Text.StringBuilder.
http://msdn.microsoft.com/en-us/library/system.text.stringbuilder(VS.71).aspx
StringBuilder builder = new StringBuilder();
builder.Append("hello").Append(" world!");
Console.WriteLine(builder.ToString());
Perhaps not as pretty as C++, but as another poster states, C# != C++.

This is not even possible in C#, no matter how hard you try:
The left hand side and right hand side of operators is always passed by value; this rules out the possibility of cin.
The right hand side of << and >> must be an integer; this rules out cout.
The first point is to make sure operator overloading is a little less messy than in C++ (debatable, but it surely makes things a lot simpler), and the second point was specifically chosen to rule out C++'s cin and cout way of dealing with IO, IIRC.