Develop Reference

BoundedCapacity of linked ActionBlock is not respected

I have a sequential pipeline that consists of two steps.
(simplified example)
The first step simply adds 1000 to the input number.
The second step simply displays the number.
var transformBlock = new TransformBlock<int, long>(StepOne, new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = 1,
BoundedCapacity = DataflowBlockOptions.Unbounded,
});
var actionBlock = new ActionBlock<long>(StepTwo, new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = 1,
BoundedCapacity = 2,
});
transformBlock.LinkTo(actionBlock, new DataflowLinkOptions
{
PropagateCompletion = true
});
for (int i = 0; i < 100; i++)
{
transformBlock.Post(i);
}
static async Task<long> StepOne(int item)
{
await Task.Delay(500);
Console.WriteLine("transforming: " + item);
return (long)item + 1000;
}
static async Task StepTwo(long item)
{
await Task.Delay(1000);
Console.WriteLine("final product: " + item);
}
Since step 2 is taking longer than step 1, I would expect step 1 to throttle after some time since it cannot send the result to the bounded buffer of step 2.
Expected output:
Transforming: 0
Transforming: 1
Final product: 1000
Transforming: 2
Final product: 1001
Transforming: 3
Final product: 1002
Transforming: 4
Final product: 1003
...
Actual output:
Transforming: 0
Transforming: 1
Final product: 1000
Transforming: 2
Transforming: 3
Final product: 1001
Transforming: 4
Transforming: 5
Final product: 1002
Transforming: 6
Transforming: 7
Final product: 1003
...

A TransformBlock maintains two queues internally, an input queue and an output queue. The size of these two queues can be monitored at any moment through the InputCount and OutputCount properties. The accumulated size of these two queues is configured by the BoundedCapacity option, so the sum InputCount+OutputCount is always less than or equal to the BoundedCapacity value. In your case the BoundedCapacity of the block is Unbounded, so there is no limiting factor at how large these two queues can become (other than some hard limits like the Int32.MaxValue probably). The fact that the linked ActionBlock has a limited bounded capacity is mostly irrelevant, and has no consequence other than delaying the transfer of the transformed values from the output queue of the TransformBlock to the input queue of the ActionBlock. This consequence is only observable if you monitor the OutputCount property of the source block, and the InputCount property of the target block. It wouldn't even matter if the TransformBlock was not linked to any target block. It would happily continue crunching numbers by itself, until some hard limit was hit, or the memory of the machine was exhausted.

How to partition large buffer to get start, end and length in c#?

How to partition large buffer to get start, end and length in c#?
like if i have a buffer size of 1048576, how can i partition in to 5 parts?
this example is greatly reduce in size
length: 14
start: 0 || end: 2 || length: 3
start: 3 || end: 5 || length: 3
start: 6 || end: 8 || length: 3
start: 9 || end: 11 || length: 3
start: 12 || end: 13 || length: 2
update: i just need to get the start, end and length, so i can write data on one large buffer, it's kinda like a buffer pool, but with no predefine size, so have i have to partition it into parts.

Is it possible to reduce the length of DateTime.Now.Ticks.ToString("X") and still maintain uniqueness?

I have a limitation with some hardware with which I am working wherein I can only broadcast ( wirelessly ) 26 characters.
To overcome this limitation, the first broadcast transmits a timestamp converted into hexadecimal ( DateTime.Now.Ticks.ToString( "X" ) ), along with the length of the message being transmitted ( also as a hexadecimal string ).
The receiving software tests for header messages, and when it confirms that it receives one, stores the time stamp ( reconverted into a long ) in a dictionary :
/*************************************************************************
* _pendingMessages.Add( DateTime.Now.Ticks, Tuple.Create( MessageLength, string.Empty ) );
* T.Item1 = Message Length
* T.Item2 = Message ( when Message.Length == Length, Pop Message )
*************************************************************************/
private static Dictionary<long, Tuple<long, string>> _pendingMessages;
Unfortunately, the time stamp has to be passed each time, and it's... over half the allotted character length ( at 15 characters right now ).
So I was thinking that, rather than pass the entire time stamp, that I might be able to reduce it by summing the value of the characters of the hex string :
For Example :
DateTime.Now.Ticks.ToSTring("X").Sum( C => C ).ToString("X");
Unfortunately, a quick test blew that idea away rather unceremoniously
( duplicate keys rather quickly ) :
Dictionary<string, long> _dctTest = new Dictionary<string, long>( );
while ( true ){
long dtNow = DateTime.Now.Ticks;
string strKey = dtNow.ToString("X").Sum( C => C ).ToStrings("X");
_dctTest.Add( strKey, dtNow ); //<=====Explodes after less than a second.
}
So my question is - is there any way for me to reliably reduce the length of my "Key" while still ( reasonably ) guaranteeing uniqueness?

Here's something to kick-start some answers. I'm not claiming this is an optimal solution but I can get you millisecond precision with only 11 characters 8 characters 7 characters of encoded data.
Assuming millisecond accuracy is good enough, we can reduce the precision of our algorithm from the get-go. A tick represents 100 nanoseconds. There are 10,000 ticks in a millisecond. Here's the algorithm:
Start with a known, large number of ticks that occurred in the past. This example uses the beginning of the century.
long centuryBegin = new DateTime(2001, 1, 1).Ticks;
// 631139040000000000
Now take a snapshot of the current timestamp:
long currentDate = DateTime.Now.Ticks;
// 636083231371736598
Take the difference, and reduce the precision to millisecond-level:
long shortTicks = (currentDate - centuryBegin) / 10000L;
// 494419137173
Now we just base64-encode the string:
string base64Ticks = Convert.ToBase64String(BitConverter.GetBytes(shortTicks));
// lVKtHXMAAAA=
However, without going into too much detail of why, the trailing "AAAA=" will be present on any encoded number of this number of bytes, so we can remove it!
base64Ticks = base64Ticks.Substring(0, 7);
// lVKtHXM
You now have a 7-character string lVKtHXM for transmission. On the other side:
// Decode the base64-encoded string back into bytes
// Note we need to add on the "AAAA=" that we stripped off
byte[] data = new byte[8];
Convert.FromBase64String(base64Ticks + "AAAA=").CopyTo(data, 0);
// From the bytes, convert back to a long, multiply by 10,000, and then
// add on the known century begin number of ticks
long originalTicks = (BitConverter.ToInt64(data, 0) * 10000L) + centuryBegin;
// 636083231371730000
Let's check the difference between the two:
636083231371736598 (original ticks)
-636083231371730000 (decoded ticks)
===================
6598 (difference)
And you can see this gets you to within 6,598 ticks, or 0.6598 milliseconds, of the original timestamp. The difference will always be <= 1 ms.
In terms of uniqueness, I tried this on 100,000 fake transmissions, sleeping for 1 millisecond in between each attempt, and there were no collisions.
To round out this post, here are some helper methods you might use:
public static string EncodeTransmissionTimestamp(DateTime date)
{
long shortTicks = (date.Ticks - 631139040000000000L) / 10000L;
return Convert.ToBase64String(BitConverter.GetBytes(shortTicks)).Substring(0, 7);
}
public static DateTime DecodeTransmissionTimestamp(string encodedTimestamp)
{
byte[] data = new byte[8];
Convert.FromBase64String(encodedTimestamp + "AAAA=").CopyTo(data, 0);
return new DateTime((BitConverter.ToInt64(data, 0) * 10000L) + 631139040000000000L);
}
Some of this work was inspired by this post: Compress large Integers into smallest possible string

BitArray stores bits in reverse order when initialized with an Int32 array

Best way to describe my miss understanding is with the code itself:
var emptyByteArray = new byte[2];
var specificByteArray = new byte[] {150, 105}; //0x96 = 150, 0x69 = 105
var bitArray1 = new BitArray(specificByteArray);
bitArray1.CopyTo(emptyByteArray, 0); //[0]: 150, [1]:105
var hexString = "9669";
var intValueForHex = Convert.ToInt32(hexString, 16); //16 indicates to convert from hex
var bitArray2 = new BitArray(new[] {intValueForHex}) {Length = 16}; //Length=16 truncates the BitArray
bitArray2.CopyTo(emptyByteArray, 0); //[0]:105, [1]:150 (inversed, why??)
I've been reading that the bitarray iterates from the LSB to the MSB, what's the best way for me to initialize the bitarray starting from a hex string then?

I think you are thinking about it wrong. Why are you even using a BitArray? Endianness is a byte-related convention, BitArray is just an array of bits. Since it is least-significant bit first, the correct way to store a 32-bit number in a bit array is with bit 0 at index 0 and bit 31 at index 31. This isn't just just my personal bias towards little-endianness (bit 0 should be in byte 0 not byte 3 for goodness sake), it's because BitArray stores bit 0 of a byte at index 0 in the array. It also stores bit 0 of a 32-bit integer in bit 0 of the array, no matter the endianness of the platform you are on.
For example, instead of your integer 9669, let's look at 1234. No matter what platform you are on, that 16-bit number has the following bit representation, because we write a hex number with the most significant hex digit 1 to the left and the least significant hex digit 4 to the right, bit 0 is on the right (a human convention):
1 2 3 4
0001 0010 0011 0100
No matter how an architecture orders the bytes, bit 0 of a 16-bit number always means the least-significant bit (the right-most here) and bit 15 means the most-significant bit (the left-most here). Due to this, your bit array will always be like this, with bit 0 on the left because that's the way I read an array (with index 0 being bit 0 and index 15 being bit 15):
---4--- ---3--- ---2--- ---1---
0 0 1 0 1 1 0 0 0 1 0 0 1 0 0 0
What you are doing is trying to impose the byte order you want onto an array of bits where it doesn't belong. If you want to reverse the bytes, then you'll get this in the bit array which makes a lot less sense, and means you'll have to reverse the bytes again when you get the integer back out:
---2--- ---1--- ---4--- ---3---
0 1 0 0 1 0 0 0 0 0 1 0 1 1 0 0
I don't think this makes any kind of sense for storing an integer. If you want to store the big-endian representation of a 32-bit number in the BitArray then what you are really storing is a byte array that just happens to be the big-endian representation of a 32-bit number and you should convert to a byte array first make it big-endian if necessary before putting it in the BitArray:
int number = 0x1234;
byte[] bytes = BitConverter.GetBytes(number);
if (BitConverter.IsLittleEndian)
{
bytes = bytes.Reverse().ToArray();
}
BitArray ba = new BitArray(bytes);

How does this C# code work out the answer?

I have solved the project Euler problem 16 but discovered this rather novel approach but I cannot get my head around the technique employed (from http://www.mathblog.dk/project-euler-16/):
int result = 0;
BigInteger number = BigInteger.Pow(2, 1000);
while (number > 0) {
result += (int) (number % 10);
number /= 10;
}
My version seems more conventional but I think the above approach is cooler.
var result = BigInteger
.Pow(2, 1000)
.ToString()
.Aggregate(0, (total, next) => total + (int) Char.GetNumericValue(next));
How does the mathematics work on the first approach, it is cool, but I need some explanation to help me understand, so if someone would be so kind to explain to me I would really appreciate it.
NOTE: If I have posted in the wrong section please let me know the better place to ask.

value % 10 will return the last digit (remainder after dividing by 10). Dividing the integer by 10 will remove this number.
Think of the number as a list, and you're just dequeuing the list and summing the values.

The modulus operator provides the remainder from the division. So, mod 10 is going to be the one's place in the number. The integer division by 10 will then shift everything so it can be repeated.
Example for the number 12345:
12345 % 10 = 5
12345 / 10 = 1234
1234 % 10 = 4
1234 / 10 = 123
123 % 10 = 3
123 / 10 = 12
12 % 10 = 2
12 / 10 = 1
1 % 10 = 1
1 / 10 = 0 (loop ends)
The addition is performed on the result of each modulus so you'd get 5+4+3+2+1

number % 10 extracts the least significant decimal digit. e.g. 12345 => 5
number / 10 removes the least significant decimal digit. This works because integer division in C# throws away the remainder. e.g. 12345 => 1234
Thus the above code extracts each digit, adds it to the sum, and then removes it. It repeats the process until all digits have been removed, and the number is 0.

It's quite simple:
Imagine this:
number = 54
It uses modulo to get the remainder of this divded by 10
e.g. 54 / 10 = 5 remainder 4
It then adds this digit (4) to the result, then divides the number by 10 (storing into int which discards decimal places)
so then number = 5
same again, 5 / 10 = 0 remainder 5
Adds them togther, result is now 9
and so on until the number is 0 :)
(in this case 9 is the answer)

They found the number 2^1000.
Modulo 10 gets the least significant digit.
E.G. 12034 % 10 = 4
Dividing by 10 strips the least significant digit.
E.G. 12034 / 10 = 1203
They sum up these least significant digits.