my question is mainly about code optimization(at the moment)
I have created a network monitor that monitors different connections on the PC, what i had done is I'm sniffing packets at the 3rd level of the stack(the network level), after capturing the packet, i am supposed to create an object on the UI for each connection, what i am doing at the moment is looking at the overall consumed bandwidth and total data sent every second the program is run. here is that part of the code:
temp= packet_rtxt.TextLength;
tempdr = temp / 1024;
dr_txt.Text=tempdr.ToString();
totaldata = totaldata + temp;
totaldatadisp = totaldata;
packet_rtxt.Text = "";
//unit
if (totaldata < 10485760)
{
if (totaldata < 10240)
unit.Text = "bytes";
else
{
totaldatadisp = totaldatadisp / 1024;
unit.Text = "KBs";
}
}
else
{
totaldata = totaldatadisp / 1048576;
unit.Text = "MBs";
}
test.Text = totaldatadisp.ToString();
tds.Enabled = true;
}
so what im doing so far is writing out the captured packets into a rich text box, taking the length of that rtxt and adding it to a counter for the total data, taking the length and using it as the data rate, then clearing the rtxt for the next bits of data.
the total data recieved part is working fine, however the BPs section works fine for low amounts of data, then it goes crazy if the data rate is over 10kbps(on my pc)
should i try optimizing the whole code, or is there some other method(keep in mind i need to monitor every single connection), or do i need to use different UI controls?
should I focus on optimization or using new ways?
thanks in advance
The standard controls are not made for such a load. You need to separate the logging of data from the display of data.
I'd only show the last say 10kb of text once per second. You can still keep all of the log records in some data structure. But you don't have to push all of them to the UI.
Alternatively you can write your own text-display control but that is going to be a lot more work.
Related
Context
I have been creating a system where an raspberry PI is sending images to a remote client in real-time.
The raspberry PI captures the images using a raspberry PI camera. A captured image is available as a 3-dimensional array of all the pixels (rows, colums and rgb). By sending and displaying the images really fast it will appear as a video to the user.
My goal is to send these images in real-time with the image resolution being as high as possible. An acceptable frame rate is around 30 fps. I selected the protocol UDP and not TCP. I did this because data can be transferred much faster in UDP due to less overhead. Re-transmissions of individual packets is not necessary because losing some pixels is acceptable in my case. The raspberry PI and the client are located in the same network so not many packets will be dropped anyway.
Taking into account that the maximum transmission unit (MTU) on the ethernet layer is 1500 bytes, and the UDP packets should not be fragmented or dropped, I selected a maximum payload length of 1450 bytes, of which 1447 bytes are data, and 3 bytes are application layer overhead. The remaining 50 bytes are reserved
for overhead that is automatically added by the TCP/IP and transport layers.
I mentioned that captured images are available as an array. Assuming the size of this array is, for example, 1.036.800 bytes (e.g. width=720 * height=480 * numberOfColors=3), then 717 (1.036.800 / 1447) UDP packets are needed to send the entire array. The c++ application on the raspberry PI does this by fragmenting the array into fragments of 1447 bytes, and adding an fragment index number, which is between 1-717, as overhead to the packet. We also add an image number, to distinguish from a previously sent image/array. The packet looks like this:
udp packet
Problem
On the client side, I developed a C# application that receives all the packets and reassembles the array using the included index numbers. Using the EgmuCV library, the received array is converted to an image and drawn in a GUI. However, some of the received images are drawn with black lines/chunks. When debugging, I discovered that this problem is not caused by drawing the image, but the black chunks are actually missing array fragments that did never arrive. Because the byte values in an array are initialized as 0 by default, the missing fragments are shown as black chunks
Debugging
Using Wireshark on the client's side, I searched for the index of such a missing fragment, and was surprised to find it, intact. This would mean that the data is received correctly on the transport layer (and observed by wireshark), but never read on the application layer.
This image shows that a chunk of a received array is missing, at index 174.000. Because there are 1447 data bytes in a packet, the index of this missing data corresponds to an UDP packet with the fragment index 121 (174.000/1447). The hexadecimal equivalent for 121 is 79. The following image shows the packet corrosponding UDP packet in wireshark, proving the data was still intact on the transport layer. image
What have I tried soo far
When I lower the frame rate, there will be less black chunks, and they are often smaller. With a framerate of 3FPS there is no black at all. However, this frame rate is not desired. That is a speed of around (3fps * 720x480x3) 3.110.400 bits per second (379kb/s). A normal computer should be capable to read more bits per seconds than this. And as I explained, the packets DID arrive in wireshark, they are only not read in the application layer.
I have also tried changing the UDP payload length from 1447 to 500. This only makes it worse, see image.
I implemented multi threading so that data is read and processed in different threads.
I tried a TCP implementation. The images were received intact, but it was not fast enough to transfer the images in real-time.
It is notable that a 'black chunk' does not represent a single missing fragment of 1447 bytes, but many consecutive fragments. So at some point when reading for data, a number of packets is not read. Also not every image has this problem, some are arrived intact.
I am wondering what is wrong with my implementation that results in this unwanted effect. So I will be posting some of my code below.
Please note that the exception 'SocketException' is never really thrown and the Console.Writeline for 'invalid overhead' is also never printed. The _client.Receive always receives 1450 bytes, expect for the last fragment of an array, which is smaller.
Also
Besides solving this bug, if anyone has alternative suggestions for transmitting these arrays in a more efficient way (requiring less bandwidth but without quality loss), I would gladly hear it. As long as the solution has the array as input/output on both endpoints.
Most importantly: NOTE that the missing packets were never returned by the UdpClient.Receive() method.
I did not post code for c++ application running on the raspberry PI, because the data did arrive (in wireshark) as I have already proved. So the transmission is working fine, but receiving is not.
private const int ClientPort = 50000;
private UdpClient _client;
private Thread _receiveThread;
private Thread _processThread;
private volatile bool _started;
private ConcurrentQueue<byte[]> _receivedPackets = new ConcurrentQueue<byte[]>();
private IPEndPoint _remoteEP = new IPEndPoint(IPAddress.Parse("192.168.4.1"), 2371);
public void Start()
{
if (_started)
{
throw new InvalidCastException("Already started");
}
_started = true;
_client = new UdpClient(_clientPort);
_receiveThread = new Thread(new ThreadStart(ReceiveThread));
_processThread = new Thread(new ThreadStart(ProcessThread));
_receiveThread.Start();
_processThread.Start();
}
public void Stop()
{
if (!_started)
{
return;
}
_started = false;
_receiveThread.Join();
_receiveThread = null;
_processThread.Join();
_processThread = null;
_client.Close();
}
public void ReceiveThread()
{
_client.Client.ReceiveTimeout = 100;
while (_started)
{
try
{
byte[] data = _client.Receive(ref _remoteEP);
_receivedPackets.Enqueue(data);
}
catch(SocketException ex)
{
Console.Writeline(ex.Message);
continue;
}
}
}
private void ProcessThread()
{
while (_started)
{
byte[] data;
bool dequeued = _receivedPackets.TryDequeue(out data);
if (!dequeued)
{
continue;
}
int imgNr = data[0];
int fragmentIndex = (data[1] << 8) | data[2];
if (imgNr <= 0 || imgNr > 255 || fragmentIndex <= 0)
{
Console.WriteLine("Received data with invalid overhead");
return;
}
// i omitted the code for this method because is does not interfere with the
// socket and therefore not really relevant to the issue that i described
ProccessReceivedData(imgNr, fragmentIndex , data);
}
}
We are working on Windows Desktop application in which we need to capture current internet Bandwidth.
We are downloading a ZIP file multiple times sequentially but our results are not matching with Speed Test.
We are capturing bytes received on ACTIVE network card, but sequential download doesn't provide expected result. We even tried parallel downloading of different files multiple times but failed.
We got success only when we downloaded different files in parallel and performed test using Speed Test simultaneously.
Now here are my few questions:
Does bandwidth between TCP HOPS affects our bandwidth?
Does traffic between TCP HOPS affects our bandwidth?
How to effectively consume entire bandwidth using HTTP / TCP downloads and C# .NET?
Does ISP throttles bandwidth per TCP Socket connection?
Does ISP provides bandwidth to http://www.speedtest.net? (Could be possible as it can always show expected result but other sites cannot)
for (int downloadCount = 0; downloadCount < iterations; downloadCount++)
{
try
{
string downloadUrl = GetUniqueDownloadUrl();
bool isValidUrl = Uri.IsWellFormedUriString(downloadUrl, UriKind.Absolute);
if (true != isValidUrl)
{
return result;
}
// Download file and register total time to download file.
Stopwatch stopwatch = new Stopwatch();
stopwatch.Start();
byte[] fileContent = webclient.DownloadData(new Uri(downloadUrl, UriKind.Absolute));
stopwatch.Stop();
double downloadTime = stopwatch.ElapsedMilliseconds / 1000; // Milliseconds to Seconds
// Convert bytes to Mbits.
fileSizeInMbits = fileContent.Length / 125000; // bytes to Megabits
double speed = fileSizeInMbits / downloadTime; // speed in Mbps
// Store speeds for average calculation.
speeds.Add(speed);
}
catch (Exception e)
{
result.Error = e;
break;
}
}
}
// Calculate average bandwidth for total successful downloads.
double totalAvgSpeed = speeds.Average();
result.FileSizeInMB = fileSizeInMbits / 8;
result.Speed = Math.Round(totalAvgSpeed, 2, MidpointRounding.AwayFromZero);
return result;
}
There's no such thing as internet "speed" there's only speed between 2 hosts, if you have 1 computer on gigabit ethernet and a server also on gigabit ethernet, even if just 1 node on the way is saturated speed will go down, when you use speedtest.net it has a lot of close servers (including likely one at your isp) so you're going to get a very positive estimate.
And if your isp throttled you'd see it on speedtest just the same.
The only thing to remember is that downloading a file from a server will only give you an estimate of the speed TO/FROM that server, and not an "internet speed" which is a concept that doesn't really exist to begin with.
So I'm trying to read real time data from the serial port object in C# / Gtk#. I have a product which talks over RF to the computer and every time it gets a command it sends back an ACK. If I use AccessPort and auto send a command every 500ms, I get my ACK. I've ran AccessPort for hours and been able to match every single command to an ACK, so I know this is physically working.
I wrote a small program in C# / Gtk# that send data out the serial port at X ms and has a delegated thread which reads the serial port for any bytes that come back. My problem is that no matter how I write the method for the serial reading, it never actually captures all the bytes that I know are there.
So far this is the "lightest" code I have:
private void readSerial(){
byte readByte = 0x00;
Gtk.Application.Invoke (delegate {
try {
readByte = (byte)serialPort.ReadByte();
Console.WriteLine(readByte.ToString("X2"));
} catch (System.ArgumentException sae) {
Console.WriteLine(sae.Message);
}
});
}
I have assigned that method to a thread in the main function:
writeThread = new Thread (writeSerial);
readThread = new Thread (readSerial);
And I start it when a connect button is pressed, along with the writeThread. The writeThread is working fine as I can see the product execute the correct instruction every X ms ( currently I'm testing at 500ms). The ACK should arrive at the computer every X ms + 35 ms * module ID, so if my end product has a module id of 2 the response would be delayed by 70ms and hence the computer should see it at 570ms or X + 70ms.
Is there a better way to do this? I'm I doing something boneheadedly wrong?
Some other code I've played with was reading 0x0E bytes from the serial port and storing the bytes into a buffer, this also missed a lot of the bytes I know are coming back.
Can anyone offer some help? I do know the readSerial method is actually starting as I do see a 0x00 pop out on the console, which is correct as 0x00 are dispersed among the data I'm looking for.
Figured it out!
I'm not sure what the exact issue was but when I removed the delegation and just used a while(true) look inside that method it worked fine.
Okay from my knowledge UDP works like this:
You have data you want to send, you say to the UDP client, hey send this data.
The UDP client then says, sure why not, and sends the data to the selected IP and Port.
If it get´s through or in the right order is another story, it have sent the data, you didn´t ask for anything else.
Now from this perspective, it´s pretty much impossible to send data and assemble it.
for example, i have a 1mb image, and i send it.
So i send divide it in 60kb files (or something to fit the packages), and send them one by one from first to last.
So in theory, if all get´s added, the image should be exactly the same.
But, that theory breaks as there is no law that tells the packages if it can arrive faster or slower than another, so it may only be possible if you make some kind of wait timer, and hope for the best that the arrive in the order they are sent.
Anyway, what i want to understand is, why does this work:
void Sending(object sender, NAudio.Wave.WaveInEventArgs e)
{
if (connect == true && MuteMic.Checked == false)
{
udpSend.Send(e.Buffer, e.BytesRecorded, otherPartyIP.Address.ToString(), 1500);
}
}
Recieving:
while (connect == true)
{
byte[] byteData = udpReceive.Receive(ref remoteEP);
waveProvider.AddSamples(byteData, 0, byteData.Length);
}
So this is basically, it sends the audio buffer through udp.
The receiving par just adds the udp data received in a buffer and plays it.
Now, this works.
And i wonder.. why?
How can this work, how come the data is sent in the right order and added so it appears as a constant audio stream?
Cause if i would to this with an image, i would probably get all the data.
But they would be in a random order probably, and i can only solve that by marking packages and stuff like that. And then there is simply no reason for it, and TCP takes over.
So if someone can please explain this, i just don´t get it.
Here is a code example that is when sending an image, and well, it works. But it seems to work better when the entire byte array isn´t sent, meanign some part of the image is corrupted (not sure why, probably something to do with how the size of the byte array are).
Send:
using (var udpcap = new UdpClient(0))
{
udpcap.Client.SendBufferSize *= 16;
bsize = ms.Length;
var buff = new byte[7000];
int c = 0;
int size = 7000;
for (int i = 0; i < ms.Length; i += size)
{
c = Math.Min(size, (int)ms.Length - i);
Array.Copy(ms.GetBuffer(), i, buff, 0, c);
udpcap.Send(buff, c, adress.Address.ToString(), 1700);
}
Receive:
using (var udpcap = new UdpClient(1700))
{
udpcap.Client.SendBufferSize *= 16;
var databyte = new byte[1619200];
int i = 0;
for (int q = 0; q < 11; ++q)
{
byte[] data = udpcap.Receive(ref adress);
Array.Copy(data, 0, databyte, i, data.Length);
i += data.Length;
}
var newImage = Image.FromStream(new MemoryStream(databyte));
gmp.DrawImage(newImage,0,0);
}
You should be using TCP. You write: it´s pretty much impossible to send data and assemble it. for example, i have a 1mb image, and i send it. So i send divide it in 60kb files (or something to fit the packages), and send them one by one from first to last. ... But, that theory breaks as there is no law that tells the packages if it can arrive faster or slower than another, so it may only be possible if you make some kind of wait timer, and hope for the best that the arrive in the order they are sent. That's exactly what TCP does: ensure that all the pieces of a stream of data are received in the order they were sent, with no omissions, duplications, or modifications. If you really want to re-implement that yourself, you should be reading RFC 793 - it talks at length about how to build a reliable data stream atop an unreliable packet service.
But really, just use TCP.
You're missing a lot of helpful details from your question, but based on the level of understanding presented I'll attempt to answer at a similar level:
You're absolutely right, in general the UDP protocol doesn't guarantee order of delivery or even delivery at all. Your local host is going to send the packets (i.e. parts of your request message) in the order it receives them from the sending application, and from there its up to network components to decide how your request gets delivered. In local networks however (within a handful of hops of the original requester) there aren't really a lot of directions for the packets to go. As such they will likely just flow in line and never see a hiccup.
On the greater internet however, there is likely a wide variety of routing choices available to each router between your requesting host and the destination. Every router along the way can make a choice on which direction parts of your message take. Assuming all paths are equal (which they aren't) and guaranteed reliability of every network segment between the 2 hosts its likely to see similar results as within network (with added latency). Unfortunately neither of the posed conditions can be considered reliable (different paths on the internet perform differently depending on client and server, and no single path on the internet should ever be considered to be reliable (that's why it's a "net").
These are of course based on general observations from my own experience in network support and admin roles. Members of other StackExchange sites may be able to provide better feedback.
I've been trying to get what I believe to be the simplest possible form of threading to work in my application but I just can't do it.
What I want to do: I have a main form with a status strip and a progress bar on it. I have to read something between 3 and 99 files and add their hashes to a string[] which I want to add to a list of all files with their respective hashes. Afterwards I have to compare the items on that list to a database (which comes in text files).
Once all that is done, I have to update a textbox in the main form and the progressbar to 33%; mostly I just don't want the main form to freeze during processing.
The files I'm working with always sum up to 1.2GB (+/- a few MB), meaning I should be able to read them into byte[]s and process them from there (I have to calculate CRC32, MD5 and SHA1 of each of those files so that should be faster than reading all of them from a HDD 3 times).
Also I should note that some files may be 1MB while another one may be 1GB. I initially wanted to create 99 threads for 99 files but that seems not wise, I suppose it would be best to reuse threads of small files while bigger file threads are still running. But that sounds pretty complicated to me so I'm not sure if that's wise either.
So far I've tried workerThreads and backgroundWorkers but neither seem to work too well for me; at least the backgroundWorkers worked SOME of the time, but I can't even figure out why they won't the other times... either way the main form still froze.
Now I've read about the Task Parallel Library in .NET 4.0 but I thought I should better ask someone who knows what he's doing before wasting more time on this.
What I want to do looks something like this (without threading):
List<string[]> fileSpecifics = new List<string[]>();
int fileMaxNumber = 42; // something between 3 and 99, depending on file set
for (int i = 1; i <= fileMaxNumber; i++)
{
string fileName = "C:\\path\\to\\file" + i.ToString("D2") + ".ext"; // file01.ext - file99.ext
string fileSize = new FileInfo(fileName).Length.ToString();
byte[] file = File.ReadAllBytes(fileName);
// hash calculations (using SHA1CryptoServiceProvider() etc., no problems with that so I'll spare you that, return strings)
file = null; // I didn't yet check if this made any actual difference but I figured it couldn't hurt
fileSpecifics.Add(new string[] { fileName, fileSize, fileCRC, fileMD5, fileSHA1 });
}
// look for files in text database mentioned above, i.e. first check for "file bundles" with the same amount of files I have here; then compare file sizes, then hashes
// again, no problems with that so I'll spare you that; the database text files are pretty small so parsing them doesn't need to be done in an extra thread.
Would anybody be kind enough to point me in the right direction? I'm looking for the easiest way to read and hash those files quickly (I believe the hashing takes some time in which other files could already be read) and save the output to a string[], without the main form freezing, nothing more, nothing less.
I'm thankful for any input.
EDIT to clarify: by "backgroundWorkers working some of the time" I meant that (for the very same set of files), maybe the first and fourth execution of my code produces the correct output and the UI unfreezes within 5 seconds, for the second, third and fifth execution it freezes the form (and after 60 seconds I get an error message saying some thread didn't respond within that time frame) and I have to stop execution via VS.
Thanks for all your suggestions and pointers, as you all have correctly guessed I'm completely new to threading and will have to read up on the great links you guys posted.
Then I'll give those methods a try and flag the answer that helped me the most. Thanks again!
With .NET Framework 4.X
Use Directory.EnumerateFiles Method for efficient/lazy files enumeration
Use Parallel.For() to delegate parallelism work to PLINQ framework or use TPL to delegate single Task per pipeline Stage
Use Pipelines pattern to pipeline following stages: calculating hashcodes, compare with pattern, update UI
To avoid UI freeze use appropriate techniques: for WPF use Dispatcher.BeginInvoke(), for WinForms use Invoke(), see this SO answer
Considering that all this stuff has UI it might be useful adding some cancellation feature to abandon long running operation if needed, take a look at the CreateLinkedTokenSource class which allows triggering CancellationToken from the "external scope"
I can try adding an example but it's worth do it yourself so you would learn all this stuff rather than simply copy/paste - > got it working -> forgot about it.
PS: Must read - Pipelines paper at MSDN
TPL specific pipeline implementation
Pipeline pattern implementation: three stages: calculate hash, match, update UI
Three tasks, one per stage
Two Blocking Queues
//
// 1) CalculateHashesImpl() should store all calculated hashes here
// 2) CompareMatchesImpl() should read input hashes from this queue
// Tuple.Item1 - hash, Typle.Item2 - file path
var calculatedHashes = new BlockingCollection<Tuple<string, string>>();
// 1) CompareMatchesImpl() should store all pattern matching results here
// 2) SyncUiImpl() method should read from this collection and update
// UI with available results
var comparedMatches = new BlockingCollection<string>();
var factory = new TaskFactory(TaskCreationOptions.LongRunning,
TaskContinuationOptions.None);
var calculateHashesWorker = factory.StartNew(() => CalculateHashesImpl(...));
var comparedMatchesWorker = factory.StartNew(() => CompareMatchesImpl(...));
var syncUiWorker= factory.StartNew(() => SyncUiImpl(...));
Task.WaitAll(calculateHashesWorker, comparedMatchesWorker, syncUiWorker);
CalculateHashesImpl():
private void CalculateHashesImpl(string directoryPath)
{
foreach (var file in Directory.EnumerateFiles(directoryPath))
{
var hash = CalculateHashTODO(file);
calculatedHashes.Add(new Tuple<string, string>(hash, file.Path));
}
}
CompareMatchesImpl():
private void CompareMatchesImpl()
{
foreach (var hashEntry in calculatedHashes.GetConsumingEnumerable())
{
// TODO: obviously return type is up to you
string matchResult = GetMathResultTODO(hashEntry.Item1, hashEntry.Item2);
comparedMatches.Add(matchResult);
}
}
SyncUiImpl():
private void UpdateUiImpl()
{
foreach (var matchResult in comparedMatches.GetConsumingEnumerable())
{
// TODO: track progress in UI using UI framework specific features
// to do not freeze it
}
}
TODO: Consider using CancellationToken as a parameter for all GetConsumingEnumerable() calls so you easily can stop a pipeline execution when needed.
First off, you should be using a higher level of abstraction to solve this problem. You have a bunch of tasks to complete, so use the "task" abstraction. You should be using the Task Parallel Library to do this sort of thing. Let the TPL deal with the question of how many worker threads to create -- the answer could be as low as one if the work is gated on I/O.
If you do want to do your own threading, some good advice:
Do not ever block on the UI thread. That's is what is freezing your application. Come up with a protocol by which working threads can communicate with your UI thread, which then does nothing except for responding to UI events. Remember that methods of user interface controls like task completion bars must never be called by any other thread other than the UI thread.
Do not create 99 threads to read 99 files. That's like getting 99 pieces of mail and hiring 99 assistants to write responses: an extraordinarily expensive solution to a simple problem. If your work is CPU intensive then there is no point in "hiring" more threads than you have CPUs to service them. (That's like hiring 99 assistants in an office that only has four desks. The assistants spend most of their time waiting for a desk to sit at instead of reading your mail.) If your work is disk-intensive then most of those threads are going to be idle most of the time waiting for the disk, which is an even bigger waste of resources.
First, I hope you are using a built-in library for calculating hashes. It's possible to write your own, but it's far safer to use something that has been around for a while.
You may need only create as many threads as CPUs if your process is CPU intensive. If it is bound by I/O, you might be able to get away with more threads.
I do not recommend loading the entire file into memory. Your hashing library should support updating a chunk at a time. Read a chunk into memory, use it to update the hashes of each algorighm, read the next chunk, and repeat until end of file. The chunked approach will help lower your program's memory demands.
As others have suggested, look into the Task Parallel Library, particularly Data Parallelism. It might be as easy as this:
Parallel.ForEach(fileSpecifics, item => CalculateHashes(item));
Check out TPL Dataflow. You can use a throttled ActionBlock which will manage the hard part for you.
If my understanding that you are looking to perform some tasks in the background and not block your UI, then the UI BackgroundWorker would be an appropriate choice. You mentioned that you got it working some of the time, so my recommendation would be to take what you had in a semi-working state, and improve upon it by tracking down the failures. If my hunch is correct, your worker was throwing an exception, which it does not appear you are handling in your code. Unhandled exceptions that bubble out of their containing threads make bad things happen.
This code hashing one file (stream) using two tasks - one for reading, second for hashing, for more robust way you should read more chunks forward.
Because bandwidth of processor is much higher than of disk, unless you use some high speed Flash drive you gain nothing from hashing more files concurrently.
public void TransformStream(Stream a_stream, long a_length = -1)
{
Debug.Assert((a_length == -1 || a_length > 0));
if (a_stream.CanSeek)
{
if (a_length > -1)
{
if (a_stream.Position + a_length > a_stream.Length)
throw new IndexOutOfRangeException();
}
if (a_stream.Position >= a_stream.Length)
return;
}
System.Collections.Concurrent.ConcurrentQueue<byte[]> queue =
new System.Collections.Concurrent.ConcurrentQueue<byte[]>();
System.Threading.AutoResetEvent data_ready = new System.Threading.AutoResetEvent(false);
System.Threading.AutoResetEvent prepare_data = new System.Threading.AutoResetEvent(false);
Task reader = Task.Factory.StartNew(() =>
{
long total = 0;
for (; ; )
{
byte[] data = new byte[BUFFER_SIZE];
int readed = a_stream.Read(data, 0, data.Length);
if ((a_length == -1) && (readed != BUFFER_SIZE))
data = data.SubArray(0, readed);
else if ((a_length != -1) && (total + readed >= a_length))
data = data.SubArray(0, (int)(a_length - total));
total += data.Length;
queue.Enqueue(data);
data_ready.Set();
if (a_length == -1)
{
if (readed != BUFFER_SIZE)
break;
}
else if (a_length == total)
break;
else if (readed != BUFFER_SIZE)
throw new EndOfStreamException();
prepare_data.WaitOne();
}
});
Task hasher = Task.Factory.StartNew((obj) =>
{
IHash h = (IHash)obj;
long total = 0;
for (; ; )
{
data_ready.WaitOne();
byte[] data;
queue.TryDequeue(out data);
prepare_data.Set();
total += data.Length;
if ((a_length == -1) || (total < a_length))
{
h.TransformBytes(data, 0, data.Length);
}
else
{
int readed = data.Length;
readed = readed - (int)(total - a_length);
h.TransformBytes(data, 0, data.Length);
}
if (a_length == -1)
{
if (data.Length != BUFFER_SIZE)
break;
}
else if (a_length == total)
break;
else if (data.Length != BUFFER_SIZE)
throw new EndOfStreamException();
}
}, this);
reader.Wait();
hasher.Wait();
}
Rest of code here: http://hashlib.codeplex.com/SourceControl/changeset/view/71730#514336