I'm playing with a TRNG usb device, and successfully turning the random bytes into various usable data in C#.
I'd like to create a .NET decimal value between 0..1 (ex: 0.37327) directly from the bytes using binary reader (or other direct bytes -> decimal) method.
// assume: byte[] random_data of appropriate length, patterned for a 0..1 range
decimal value = new BinaryReader(new MemoryStream(random_data)).ReadDecimal();
I was looking for the byte format of decimal, but it looks like it may not be a standard?
How do I convert byte values into decimals?
.NET decimal cross-platform standard
This is for hobby work, so it's acceptable to me to use something that might change in future.
I've looked at the bytes generated for sample input decimal values - I see negative and precision flags (1Ch max in last int32?), but the min/max data values dumping compiler constants are stumping me a bit, and I'm generate above zero values or invalid values:
fractional value nearest 1 (.99...9): FFFFFF0F 6102253E 5ECE4F20 00001C00
fractional value nearest 0 (.00...1): 01000000 00000000 00000000 00001C00
Can you help me onto the correct path for generating a full fractional 0..1 range?
Edit with "final" code, thanks to everyone!
Here's the C# code I ended up with to create an unbiased random decimal of range [0..1] from a suitable random byte stream (like a TRNG device, www.Random.org, or CSPRNG algorithm). Generated values look good to the eyeball, boundary tests pass, and as long as I avoided embarassing typos and copy/paste bugs, this should be usable as-is.
Thanks for the help and interesting discussions!
private decimal RandomDecimalRange01()
{
// 96 bits of random data; we'll use 94 bits to directly map decimal's max precision 0..1 range
byte[] data = new byte[12];
decimal value = 0;
// loop until valid value is generated, discarding invalids values. Mostly controlled by top 2 bits: 11 is always invalid, 00 or 01, is always valid, 10 has valid and invalid ranges. Odds make loop generally find value in one or a few iterations.
while (true)
{
// Acquire random bytes from random source (like TRNG device or CSPRNG api)
if (!trng.GetBytes(data))
{
throw new Exception("Failed to aquire random bytes from source");
}
else
{
// Read 94 random bits (pull 96 bits, discard 2)
BinaryReader reader = new BinaryReader(new MemoryStream(data));
int low = reader.ReadInt32();
int mid = reader.ReadInt32();
int high = reader.ReadInt32() & 0x3FFFFFFF; // don't consume upper 2 random bits - out of range
// Discard invalid values and reloop (interpret special invalid value as 1)
if (high > 542101086)
{
continue;
}
else if (high == 542101086)
{
if (mid > 1042612833)
{
continue;
}
else if (mid == 1042612833)
{
if (low > 268435455)
{
// Special override to generate 1 value for inclusive [0..1] range - interpret the smallest invalid value as 1. Remove code for exclusive range [0..1)
if (low == 268435456)
{
value = 1m; // return 1.0
break;
}
continue;
}
}
}
// return random decimal created from parts - positive, maximum precision 28 (1C) scale
value = new decimal(low, mid, high, false, 28);
break;
}
}
return value;
}
Sample generated values running TrueRNGPro TRNG device bytes through algorithm
0.8086691474438979082567747041
0.4268035919422123276460607186
0.7758625805098585303332549015
0.0701321080502462116399370731
0.3127190777525873850928167447
0.6022236739048965325585049764
0.1244605652187291191393036867
Tests around interesting boundary values
// test databyte values for max & min ranges
new byte[] { 0x01, 0x00, 0x00, 0x10, 0x61, 0x02, 0x25, 0x3E, 0x5E, 0xCE, 0x4F, 0x20 }; // boundary: 1 too large for algorithm, will be discarded
new byte[] { 0x00, 0x00, 0x00, 0x10, 0x61, 0x02, 0x25, 0x3E, 0x5E, 0xCE, 0x4F, 0x20 }; // boundary: special 1 more than largest valid .99999..., interpret as 1 value
new byte[] { 0xFF, 0xFF, 0xFF, 0x0F, 0x61, 0x02, 0x25, 0x3E, 0x5E, 0xCE, 0x4F, 0x20 }; // boundary: largest valid value .9999...
new byte[] { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 }; // boundary: smallest valid value, should be 0
new byte[] { 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 }; // boundary: 1 more than smallest valid value, should be .000...1
From https://msdn.microsoft.com/en-us/library/system.decimal.getbits(v=vs.110).aspx
The binary representation of a Decimal number consists of a 1-bit
sign, a 96-bit integer number, and a scaling factor used to divide the
integer number and specify what portion of it is a decimal fraction.
The scaling factor is implicitly the number 10, raised to an exponent
ranging from 0 to 28.
The return value is a four-element array of 32-bit signed integers.
The first, second, and third elements of the returned array contain
the low, middle, and high 32 bits of the 96-bit integer number.
The fourth element of the returned array contains the scale factor and
sign. It consists of the following parts:
Bits 0 to 15, the lower word, are unused and must be zero.
Bits 16 to 23 must contain an exponent between 0 and 28, which
indicates the power of 10 to divide the integer number.
Bits 24 to 30 are unused and must be zero.
Bit 31 contains the sign: 0 mean positive, and 1 means negative.
You are looking for 0 <= 96bitinteger / 10exponent <= 1
Multiplying through, this is the same as 0 <= 96bitinteger <= 10^exponent
This would enumerate every possibile pair of 96bitinteger and exponent that would produce a decimal value between 0 and 1 (assuming the sign bit is set to 0).
for (int exponent=0; exponent<=28; exponent++) {
BigInteger max = BigInteger.Pow(10, exponent);
for (int i = 0; i <= max; i++) {
var fmt = "96bitinteger: {0}, exponent: {1}";
Console.WriteLine(String.Format(fmt, i, exponent));
}
}
Using exponent=28
1028 in hex is 204fce5e 3e250261 10000000. So once you place the 32-bit numbers according to the docs, and then create fhe final 32-bits accounting for the fact for some reason when they say bit 0 they mean the highest order bit, it isn't too hard to construct the decimal number 1 from bytes.
int[] data = new int[] { 0x10000000, 0x3e250261, 0x204fce5e, 0x1C0000 };
var random_data = data.SelectMany(BitConverter.GetBytes).ToArray();
decimal value = new BinaryReader(new MemoryStream(random_data)).ReadDecimal();
Console.WriteLine(value);
Consider the more general new int[] { a, b, c, 0x1C0000 }. The constraints on a b and c, for creating a decimal number 0 <= d <= 1 are
if c < 0x204fce5e:
a can be anything
b can be anything
elif c = 0x204fce5e:
if b < 0x3e250261:
a can be anything
elif b = 0x3e250261
constrain a <= 10000000
b can not be greater than 0x3e250261
c can not be greater than 0x204fce5e.
Do you want uniform distribution between 0 and 1 (including both ends)?
Do the original data is also uniform at the byte level?
How much precision do you really need?
Do you need decimal or float/double would do?
If you use decimal, I don't think you would get uniform distribution as you cannot use whole range of all bytes.
With floating points, I think it might be easier to do bits manipulations if you create a number between 1 and 2 and then subtract 1. However, you would never get exactly 1.0.
Related
In C/C++ you could do this O(n) with a pointer to a buffer and casting to 32bit datatype, but in C# given byte[] or IEnumerable<byte> and an int32 how might you efficiently find the first position of the DWORD in the byte buffer, ideally using built-in library methods?
I don't want to venture into unsafe code so I can of course search for a 4-element byte sub-buffer. But is there a neater way to take advantage of my search value being 32bits long?
Trivial workup (pseudo-ish):
int Find(IEnumerable<byte> buf, int val)
{
byte d = val & 0xff, c = (val >>8) & 0xff, b = (val >>16) & 0xff, a = (val>>24) & 0xff;
for(int i=0;i<buf.Length - 3;++i)
{
if(buf[i] == d && buf[i+1] == c && buf[i+2] == b && buf[i+3] == a)
return i
}
return -1;
}
Basically wondering if I can combine all 4 byte-wise checks with a single 32-bit check.
wondering if I can combine all 4 byte-wise checks with a single 32-bit check
Not without violating alignment. Some options you do have are:
a masked 32-bit comparison against the first part of the value for each of the 4 possible alignments (this will be testing 32, 24, 16, or 8 bits). After you get a success, you'll still have to test against the remaining bits
A SIMD comparison looking for the first byte (or most unique -- I certainly wouldn't want to search for 0x00 since there will be far more than 0.5% false positives) of the needle at 16 or so possible adjacent locations simultaneously. Once you get a match you need to test the three following bytes also.
I am trying to copy the value of a uint into a byte array in C#. I have managed to accomplish this using code in an unsafe context but ideally, I would like to do this in a safe context
The code I am currently using is this
var bytes = new byte[] {0x68, 0x00, 0x00, 0x00, 0x00}
fixed (byte* bytesPointer = bytes )
{
*(ulong*)(bytesPointer + 1) = value;
}
The equivalent of what I am trying to accomplish in C# can be done like this in C++
unsigned char bytes[] = {0x68, 0x00, 0x00, 0x00, 0x00}
memcpy(((unsigned long)bytes + 1), value, 4);
How could I do this in a safe context in C#?
You could use these
Array.Copy(Array, Int32, Array, Int32, Int32)
Copies a range of elements from an Array starting at the specified
source index and pastes them to another Array starting at the
specified destination index. The length and the indexes are specified
as 32-bit integers.
Buffer.BlockCopy(Array, Int32, Array, Int32, Int32) Method
Copies a specified number of bytes from a source array starting at a
particular offset to a destination array starting at a particular
offset.
My goal is to get a 64bit value hence a byte array of size 8. However my problem is that I want to set the first 20 bits myself and then have the rest to be 0s. Can this be done with the shorthand byte array initialisation?
E.g. if I wanted all 0s I would say:
byte[] test = new byte[] {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
What I've thought about/tried:
So each hexadecimal digit corresponds to 4 binary digits. Hence, if I want to specify the first 20bits, then I specify the first 5 hexadecimal digits? But I'm not sure of how to do this:
byte[] test = new byte[] {0xAF, 0x17, 0x10, 0x00, 0x00, 0x00, 0x00, 0x00};
That would mean that I've specified the first 24 bits right? And not 20.
I could use a BitArray and do it that way but I'm just wondering whether it can be done in the above way.
How about:
byte byte1 = 0xFF;
byte byte2 = 0xFF;
byte byte3 = 0xFF;
// 8bits 8bits 4bits : total = 20 bits
// 11111111 11111111 11110000
byte[] test = new byte[] { byte1, byte2, (byte)(byte3 & 0xF0), 0x00, 0x00, 0x00, 0x00, 0x00 };
You can write your bytes backward, and use BitConverter.GetBytes(long):
var bytes = BitConverter.GetBytes(0x117AF);
Demo.
Since each hex digit corresponds to a single four-bit nibble, you can initialize data in "increments" of four bits. However, the data written in reverse will be almost certainly less clear to human readers of your code.
As I understand it, Random.Next() uses the system time for getting the seed, but when iterating through a loop very fast the system time hasn't changed or has hardly changed, giving me the same "random" number. I'm attempting to use random to select starting positions to begin writing bytes for about 2 seconds of static here and there to music files, 30 different positions are selected, but they're almost the same. I'm getting almost continuous static from the beginning, broken about 3 times for just a few seconds before it resumes playing the music normally at around 30 seconds in; which isn't what I want, I need it spread out throughout the entire clip. "int pos" is the problem, its not random, each starting position is nearly identical to all the others, so I have a prolonged amount of static, not static randomly spread throughout the music. My randoms are also static.
FileStream stream = new FileStream(file, FileMode.Open, FileAccess.ReadWrite);
for (int i = 0; i < 20; i++)
{
int pos = rand2.Next(75000, 4000000 /*I've been too lazy to get the file length, so I'm using 4000000*/ );
for (int x = 0; x < 500000/*500000 is a little over 2 seconds of static*/; x++)
{
byte number = array[rand.Next(array.Length)];
stream.Position = pos;
pos++;
stream.WriteByte(number);
}
}
I assumed that it would take 5 seconds or so to make each write (on my slow CPU), which would be enough time for the next random to give me a position that isn't identical to or extremely close to the previous one. As it stands, each time I seem to be getting an initial position of about ~90000 (first few seconds of music); and all of the next ones are within 20 seconds of that. So my question is, what do I need to modify/do differently in order to achieve my desired result? I'd like a couple seconds of static scattered throughout the entire clip, not clustered together.
I have a byte array which stores my hex digits, which are randomly selected, that appears to work fine, its just the randomness of the writing positions isn't random at all, they're all in extreme proximity.
byte[] array = { 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, 0x0B, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1A, 0x1B, 0x1C, 0x1D, 0x1E, 0x1F };
Thanks.
P.S. I know I should have using blocks for the FileStream, I'll add it when I get around to it.
If 500000 bytes is about 2 seconds of audio, and you're starting somewhere between 75000 and 4000000, you're starting between 0.15s and 8s into the song. That explains why you're not hearing any static after about 10s into the song. Try using the actual file size (minus 500000) as the upper bound of the rand.Next call used to populate pos.
When you write :
int pos = rand2.Next(75000, 4000000)
If your file is stereo 16 bits 48KHz, then 4000000 / (2*2*48000) = 20 seconds
of sound. Use the file size, it's easy. You shouldn't be lazy :-) .
But if you 'Add' noise you shouldn't replace the file value, but rather add to
it a random value.
And why have a noise amplitude of only 16 / Byte ? and fixed ? using number = rand.Next(Amplitude)
would seem fine to compute the added noise.
Note also that you're creating two white noises, one on each channel.
First, Random.Next does not use system clock to generate a random number. Random number is generated on the basis of seed provided and the default constructor(parameterless) uses the system clock as seed but it only set the system clock as seed.
Correct me if I am wrong but it looks u r getting random numbers but these numbers are close enough to produce almost same sound and u want your numbers to be scattered with large margin(difference between consecutive random numbers). To achieve this I suggest you to use a minimum threshold. If difference between consecutive number is less than threshold either try to generate next one or adjust the value to satisfy the threshold. A code can better explain what I want to say.
int RandomNumber(Random rand,int? lastRandomNumber, int min, int max, int threshold)
{
var number = rand.Next(min, max);
if (lastRandomNumber.HasValue && Math.Abs(lastRandomNumber.Value - number) < threshold)
{
Console.WriteLine("Actual:{0}", number);
if (lastRandomNumber.Value < number)
{
number += threshold;
if (number > max)
number = max;
}
else
{
number -= threshold;
if (number < min)
number = min;
}
}
return number;
}
I had the same problem when generating random numbers . It was generating the same values. The following simple code solved my problem.
private static readonly Random random = new Random();
private static readonly object syncLock = new object();
public int RandomNumber(int min, int max)
{
lock (syncLock)
{ // synchronize
return random.Next(min, max);
}
}
You will use is like below:
int rn = RandomNumber(0, 10); //for example
I'm building arrays of bytes to be communicated over Bluetooth. These bytes are partly built from enumerated types, such as the following :
public enum Motor
{
A = 0x00,
B = 0x01,
C = 0x02,
AB = 0x03,
AC = 0x04,
BC = 0x05,
}
Later in my code I create a variable called MyMotor of type MyMotor.B. I then pass this variable to a method in which I build my byte array.
My issue is that the software I'm communicating with via Bluetooth expects the hex value of the enumerated value as a string, ie MyMotor.B = byte 0x01 = dec 1 = hex 31. However casting MyMotor directly to a char would result in it evaluating to it's enumerated value ie MyMotor = B = hex 42.
For various reasons I can't change my enurated list, so I've settled on what feels like a very hacked together two line section of code :
String motorchar = Convert.ToString(Convert.ToInt32(MyMotor)); // convert to temp var
command[5] = (byte)(motorchar[0]); // store hex value of var
It works as I'd like ie command[5] = hex31
I wonder if there's a better way. All the articles I've found talk about dealing with entire byte arrays rather than individual bytes and chars.
chars[0] = (char)('0' + ((byte)myMotor & 0x0F));
chars[1] = (char)('0' + (((byte)myMotor & 0xF0) >> 4));
This needs a little more tweaking for hexadecimal, though.
If your other app expects a string then provide one.
Make an array of strings to hold the values (which you know) and use the int value of the enum as an index into that array.
Unless I am missing something your two line code is equivalent to just calling;
BitConverter.ToString(MyMotor);
No?
If you know that your program's values and the values the API expects always differ by some fixed amount (for example, AC = 0x04, but the API wants "4", then you can write a simple conversion:
char c = (char)((int)Motor + '0');
That gets kind of ugly when there are more than 10 values, though. You can special case it for hexadecimal digits, but after that it's pretty bad.
You're better off creating a lookup table, the most general being a dictionary:
Dictionary<Motor, string> MotorLookup = new Dictionary<Motor, string>() {
{ Motor.A, "0" },
{ Motor.B, "1" },
// etc, etc.
};
That's going to be the most flexible and most maintainable.
Why not use:
//If you want the ASCII representation.
// e.g. myMotor == B, then the Byte Decimal value will be 49 or 0x31.
command[5] = (byte)((byte)myMotor).ToString()[0];
or
//If you want the numeric representation:
// e.g. myMotor == B, then the Byte Decimal value will be 1 or 0x01.
command[5] = (byte)myMotor;
I see you using values like "0x01" and "0x05". The "0x" prefix means it's a hexadecimal number, but you never go past 5, so it's exactly the same as using integer values "1" and "5".
I don't see how you're even getting Decimal 1 == Hex 31 or Hex 42 that you mention in your post. The ASCII equivalent of Char '1' is Decimal 49 or Hex 31.