Develop Reference

Math.NET CryptoRandomSource Next is Biased

I am developing a gaming platform that is subject to heavy regulatory scrutiny. I chose Math.NET because it seemed like a good fit. However I have just received this comment back from our auditors.
Comments please if this is accurate and how it can be resolved?
In RandomSource(), Next(int, int) is defined as follows:
public override sealed int Next(int minValue, int maxValue)
{
if (minValue > maxValue)
{
throw new ArgumentException(Resources.ArgumentMinValueGreaterThanMaxValue);
}
if (_threadSafe)
{
lock (_lock)
{
return (int)(DoSample()*(maxValue - minValue)) + minValue;
}
}
return (int)(DoSample()*(maxValue - minValue)) + minValue;
}
This creates a bias in the same way as before. Using an un-scaled value from the RNG and multiplying it by the range without previously eliminating the bias (unless the range is a power of 2, there will be a bias ).

Update: The implementation of Next(minInclusive, maxExclusive) has been changed in Math.NET Numerics v3.13 following this discussion. Since v3.13 it is no longer involving floating point numbers, but instead samples integers with as many bits as needed to support the requested range (power of two) and rejects those outside of the actual range. This way it avoids adding any bias on top of the byte sampling itself (as provided e.g. by the crypto RNG)
Assumption: DoSample() returns a uniformly distributed sample in the range [0,1) (double precision floating point number).
Multiplying it with the range R=max-min will result in a uniformly distributed sample in the range [0,R). Casting this to an integer, which is essentially a floor, will result in a uniformly distributed discrete sample of one of 0,1,2,...,R-1. I don't see where the fact that R is even, odd, or a power of two may affect bias in this step.
A few runs to compute 100'000'000 samples also do not indicate obvious bias, but of course this is no proof:
var r = new CryptoRandomSource();
long[] h = new long[8];
for (int i = 0; i < 100000000; i++)
{
h[r.Next(2,7)]++;
}
0
0
19996313
20001286
19998092
19998328
20005981
0
0
0
20000288
20002035
20006269
19994927
19996481
0
0
0
19998296
19997777
20001463
20002759
19999705
0

I've come up with this solution for a value between 0 and max inclusive. I'm no maths expert so comments welcome.
It seems to satisfy the regulatory spec I have which says
2b) If a particular random number selected is outside the range of equal distribution of re-scaling values, it is permissible to discard that random number and select the next in sequence for the purpose of re-scaling."
private readonly CryptoRandomSource _random = new CryptoRandomSource();
private int GetRandomNumber(int max)
{
int number;
var nextPowerOfTwo = (int)Math.Pow(2, Math.Ceiling(Math.Log(max) / Math.Log(2)));
do
{
// Note: 2nd param of Next is an *exclusive* value. Add 1 to satisfy this
number = _random.Next(0, nextPowerOfTwo + 1);
} while (number > max);
return number;
}

BitArray change bit within range

How can I ensure that when changing a bit from a BitArray, the BitArray value remains in a range.
Example:
Given the range [-5.12, 5.12] and
a = 0100000000000000011000100100110111010010111100011010100111111100 ( = 2.048)
By changing a bit at a random position, I need to ensure that the new value remains in the given range.

I'm not 100% sure what you are doing and this answer assumes you are storing a as a 64-bit value (long) currently. The following code may help point you in the right direction.
const double minValue = -5.12;
const double maxValue = 5.12;
var initialValue = Convert.ToInt64("100000000000000011000100100110111010010111100011010100111111100", 2);
var changedValue = ChangeRandomBit(initialValue); // However you're doing this
var changedValueAsDouble = BitConverter.Int64BitsToDouble(initialValue);
if ((changedValueAsDouble < minValue) || (changedValueAsDouble > maxValue))
{
// Do something
}

It looks like double (64 bits and result has decimal point).
As you may know it has sign bit, exponent and fraction, so you can not change random bit and still have value in the range, with some exceptions:
sign bit can be changed without problem if your range is [-x;+x] (same x);
changing exponent or fraction will require to check new value range but:
changing exponent of fraction bit from 1 to 0 will make |a| less.
I don't know what you are trying to achieve, care to share? Perhaps you are trying to validate or correct something, then you may have a look at this.

Here's an extension method that undoes the set bit if the new value of the float is outside the given range (this is an example only, it relies on the BitArray holding a float with no checks, which is pretty horrible so just hack a solution out of this, incl changing to double):
static class Extension
{
public static void SetFloat(this BitArray array, int index, bool value, float min, float max)
{
bool old = array.Get(index);
array.Set(index, value);
byte[] bytes = new byte[4];
array.CopyTo(bytes, 0);
float f = BitConverter.ToSingle(bytes, 0);
if (f < min || f > max)
array.Set(index, old);
}
}
Example use:
static void Main(string[] args)
{
float f = 2.1f;
byte[] bytes = System.BitConverter.GetBytes(f);
BitArray array = new BitArray(bytes);
array.Set(20, true, -5.12f, 5.12f);
}

If you can actually limit your precision, then this would be a lot easier. For example given the range:
[-5.12, 5.12]
If I multiply 5.12 by 100, I get
[-512, 512]
And the integer 512 in binary is, of course:
1000000000
So now you know you can set any of the first 9 bits and you'll be < 512 if the 10th bit is 0. If you set the 10th bit, you will have to set all the other bits to 0. With a little extra effort, this can be extended to deal with 2's complement negative values too (although, I might be inclined just to convert them to positive values)
Now if you actually need to accommodate the 3 d.p. of 2.048, then you'll need to multiply all you values by 1000 instead and it will be a little more difficult because 5120 in binary is 1010000000000
You know you can do anything you want with everything except the most significant bit (MSB) if the MSB is 0. In this case, if the MSB is 1, but the next 2 bits are 0, you can do anything you want with the remaining bits.
The logic involved with dealing directly with the number in IEEE-754 floating point format is probably going to be torturous.
Or you could just go with the "mutate the value and then test it" approach, if it's out-of-range, go back and try again. Which might be suitable (in practice), but won't be guaranteed to exit.
A final thought, depending on exactly what you are doing, you might want to also look at Gray Codes. The idea of a Gray Code is to make it such that each value is only 1 bit flip apart. With naturally encoded binary, a flip of the MSB has orders of magnitude more impact on the final value than a flip of the LSB.

Random.Next returns always the same values [duplicate]

This question already has answers here:
Random number generator only generating one random number
(15 answers)
Closed 9 years ago.
I use the method to generate unique number but I always get the same number -2147483648. Even if I stop the program, recompile it and run again I still see the same number.
public static int GetRandomInt(int length)
{
var min = Math.Pow(10, length - 1);
var max = Math.Pow(10, length) - 1;
var random = new Random();
return random.Next((int)min, (int)max);
}

Try externalizing the random instance:
private readonly Random _random = new Random();
public static int GetRandomInt(int length)
{
var min = Math.Pow(10, length - 1);
var max = Math.Pow(10, length) - 1;
return _random.Next((int)min, (int)max);
}

This is not an issue of not reusing Random instance, the results he gets should be random on multiple starts, not always being -(2^32)
This is the issue with length being too big, and casting powers of length to int. If you break the code into following lines:
var min = Math.Pow(10, length - 1);
var max = Math.Pow(10, length) - 1;
var random = new Random();
var a = (int)min;
var b = (int)max;
return random.Next(a, b);
You'll see that a and b are -2147483648, making that the only possible result of Next(min, max) (the doc specifies if min==max, return min).
The largest length you can safely use with this method is 9. For a length of 10 you'll get System.ArgumentOutOfRangeException, for length > 10 you'll get the -2147483648 result.

You have three problems with your code.
You should externalize your random variable.
You have a problem with truncation error.
The range between min and max is way to large.
The first problem is because you may not have enough time to advance the seed when reinitializing your random variable. The second error comes from truncating your (what would b very large) numbers down to ints. Finally, your biggest problem is your range between your min and your max. Consider finding the range between min and max (as defined in your code) with inputs 1->20:
length max-min
1 8
2 89
3 899
4 8999
5 89999
6 899999
7 8999999
8 89999999
9 899999999
10 8,999,999,999
11 89999999999
12 899999999999
13 8999999999999
14 89999999999999
15 899999999999999
16 9E+15
17 9E+16
18 9E+17
19 9E+18
And keep in mind that the maximum integer is 2,147,483,647, which is passed on any number greater than 9.

You should keep an instance of Random and not new() it up all the time, that should give you better results.
Also check for what length actually is. It may be giving you funny results as to the limits.

I think the problem is the calculation of min and max. They will be greater than Int32.MaxValue pretty fast...

In your class, have one instance of Random, e.g.:
public class MyClass
{
private readonly Random random = new Random();
public static int GetRandomInt(int length)
{
var min = Math.Pow(10, length - 1);
var max = Math.Pow(10, length) - 1;
return random.Next((int)min, (int)max);
}
}
The fact that random always returns the same values only exists for testing purposes.

Random classes usually use a seed to initialize themselves, and will usually return the same sequence provided the seed is the same one :
Always reuse the same Random() instance instead of recreating one over and over again
if you want unpredictable results, use a time-dependent seed rather than an hard-coded one
It's very difficult to code a truly random number generator. Most methods use external entropy generators (such as mouse movement, cpu temperature, or even complex physical mechanisms such as helium balloons colliding one another...).

The Random instance should be created only once and then reused. The reason for this is that the RNG is by default seeded with the current system time. If you rapidly create new Random instances (and pull one value from it) then many of them will be seeded with the same timestamp, because the loop probably executes faster than the system clock advances.
Remember, a RNG initialized by seed A will always return sequence B. So if you create three Random instances all seeded with for example 123, these three instances will always return the same number on the same iteration.

Need a way to pick a common bit in two bitmasks at random

Imagine two bitmasks, I'll just use 8 bits for simplicity:
01101010
10111011
The 2nd, 4th, and 6th bits are both 1. I want to pick one of those common "on" bits at random. But I want to do this in O(1).
The only way I've found to do this so far is pick a random "on" bit in one, then check the other to see if it's also on, then repeat until I find a match. This is still O(n), and in my case the majority of the bits are off in both masks. I do of course & them together to initially check if there's any common bits at all.
Is there a way to do this? If so, I can increase the speed of my function by around 6%. I'm using C# if that matters. Thanks!
Mike

If you are willing to have an O(lg n) solution, at the cost of a possibly nonuniform probability, recursively half split, i.e. and with the top half of the bits set and the bottom half set. If both are nonzero then chose one randomly, else choose the nonzero one. Then half split what remains, etc. This will take 10 comparisons for a 32 bit number, maybe not as few as you would like, but better than 32.
You can save a few ands by choosing to and with the high half or low half at random, and if there are no hits taking the other half, and if there are hits taking the half tested.
The random number only needs to be generated once, as you are only using one bit at each test, just shift the used bit out when you are done with it.
If you have lots of bits, this will be more efficient. I do not see how you can get this down to O(1) though.
For example, if you have a 32 bit number first and the anded combination with either 0xffff0000 or 0x0000ffff if the result is nonzero (say you anded with 0xffff0000) conitinue on with 0xff000000 of 0x00ff0000, and so on till you get to one bit. This ends up being a lot of tedious code. 32 bits takes 5 layers of code.

Do you want a uniform random distribution? If so, I don't see any good way around counting the bits and then selecting one at random, or selecting random bits until you hit one that is set.
If you don't care about uniform, you can select a set bit out of a word randomly with:
unsigned int pick_random(unsigned int w, int size) {
int bitpos = rng() % size;
unsigned int mask = ~((1U << bitpos) - 1);
if (mask & w)
w &= mask;
return w - (w & (w-1));
}
where rng() is your random number generator, w is the word you want to pick from, and size is the relevant size of the word in bits (which may be the machine wordsize, or may be less as long as you don't set the upper bits of the word. Then, for your example, you use pick_random(0x6a & 0xbb, 8) or whatever values you like.

This function uniformly randomly selects one bit which is high in both masks. If there are
no possible bits to pick, zero is returned instead. The running time is O(n), where n is the number of high bits in the anded masks. So if you have a low number of high bits in your masks, this function could be faster even though the worst case is O(n) which happens when all the bits are high. The implementation in C is as follows:
unsigned int randomMasksBit(unsigned a, unsigned b){
unsigned int i = a & b; // Calculate the bits which are high in both masks.
unsigned int count = 0
unsigned int randomBit = 0;
while (i){ // Loop through all high bits.
count++;
// Randomly pick one bit from the bit stream uniformly, by selecting
// a random floating point number between 0 and 1 and checking if it
// is less then the probability needed for random selection.
if ((rand() / (double)RAND_MAX) < (1 / (double)count)) randomBit = i & -i;
i &= i - 1; // Move on to the next high bit.
}
return randomBit;
}

O(1) with uniform distribution (or as uniform as random generator offers) can be done, depending on whether you count certain mathematical operation as O(1). As a rule we would, though in the case of bit-tweaking one might make a case that they are not.
The trick is that while it's easy enough to get the lowest set bit and to get the highest set bit, in order to have uniform distribution we need to randomly pick a partitioning point, and then randomly pick whether we'll go for the highest bit below it or the lowest bit above (trying the other approach if that returns zero).
I've broken this down a bit more than might be usual to allow the steps to be more easily followed. The only question on constant timing I can see is whether Math.Pow and Math.Log should be considered O(1).
Hence:
public static uint FindRandomSharedBit(uint x, uint y)
{//and two nums together, to find shared bits.
return FindRandomBit(x & y);
}
public static uint FindRandomBit(uint val)
{//if there's none, we can escape out quickly.
if(val == 0)
return 0;
Random rnd = new Random();
//pick a partition point. Note that Random.Next(1, 32) is in range 1 to 31
int maskPoint = rnd.Next(1, 32);
//pick which to try first.
bool tryLowFirst = rnd.Next(0, 2) == 1;
// will turn off all bits above our partition point.
uint lowerMask = Convert.ToUInt32(Math.Pow(2, maskPoint) - 1);
//will turn off all bits below our partition point
uint higherMask = ~lowerMask;
if(tryLowFirst)
{
uint lowRes = FindLowestBit(val & higherMask);
return lowRes != 0 ? lowRes : FindHighestBit(val & lowerMask);
}
uint hiRes = FindHighestBit(val & lowerMask);
return hiRes != 0 ? hiRes : FindLowestBit(val & higherMask);
}
public static uint FindLowestBit(uint masked)
{ //e.g 00100100
uint minusOne = masked - 1; //e.g. 00100011
uint xord = masked ^ minusOne; //e.g. 00000111
uint plusOne = xord + 1; //e.g. 00001000
return plusOne >> 1; //e.g. 00000100
}
public static uint FindHighestBit(uint masked)
{
double db = masked;
return (uint)Math.Pow(2, Math.Floor(Math.Log(masked, 2)));
}

I believe that, if you want uniform, then the answer will have to be Theta(n) in terms of the number of bits, if it has to work for all possible combinations.
The following C++ snippet (stolen) should be able to check if any given num is a power of 2.
if (!var || (var & (var - 1))) {
printf("%u is not power of 2\n", var);
}
else {
printf("%u is power of 2\n", var);
}

If you have few enough bits to worry about, you can get O(1) using a lookup table:
var lookup8bits = new int[256][] = {
new [] {},
new [] {0},
new [] {1},
new [] {0, 1},
...
new [] {0, 1, 2, 3, 4, 5, 6, 7}
};
Failing that, you can find the least significant bit of a number x with (x & -x), assuming 2s complement. For example, if x = 46 = 101110b, then -x = 111...111010010b, hence x & -x = 10.
You can use this technique to enumerate the set bits of x in O(n) time, where n is the number of set bits in x.
Note that computing a pseudo random number is going to take you a lot longer than enumerating the set bits in x!

This can't be done in O(1), and any solution for a fixed number of N bits (unless it's totally really ridiculously stupid) will have a constant upper bound, for that N.

Hardware RNG - How to use?

Ok, I've just picked up a hardware RNG and it contains some simple functions as below,
GetRandomBytes(UInt Length,out object Array)
GetRandomDoubles(UInt Length,out object Array)
The functions seem to explain themselves pretty well, how would one use these functions effectivly to generate a number between a certain range?
More info from some docs we have found,
GetRandomByte
Return a single byte containing 8 random bits.
GetRandomWord
Return an unsigned integer containing 32 random bits.
GetRandomDouble
Returns a double-precision floating point value uniformly
distributed between 0 (inclusive) and 1 (exclusive).
GetRandomBytes
GetRandomWords
GetRandomDoubles
Fill in an array with random values. These methods all take
two arguments, an integer specifying the number of values
to return (as an unsigned long integer), and the array to
return the values in (as a COM Variant).

To get a random int within a given range, you can use the GetRandomDouble function that is provided by the hardware, and scale that value to fit the desired range. The maximum value is exclusive, since the underlying double range [0,1) is half-open.
int GetRandomInt(int min, int max) {
double d = randHardware.GetRandomDouble();
return ((max-min)*d)+min;
}

If I had those functions without absolutely any other help or indication the first try I would do is this (just looking at the signature):
uint length = 20;
object array;
GetRandomBytes(length, out array);
Then I will try to debug this and see what the actual type of array is after calling the function. Looking at the name of the function I would assume byte[], so I would cast:
byte[] result = (byte[])array;
As far as a range is concerned those function signatures are far from self-explaining. Maybe the length parameter?
Also note that in C# there's no such thing as UInt. There's System.UInt32 and uint which is a shortcut.

NOTE: This uses inclusive ranges. You might want exclusive max, as is typical. Obviously this should be modified to your needs.
Let's say you get a double that's random
public int getIntInRangeFromDouble(int min, int max, double rand) {
int range = max-min+1;
int offset = (int)(range*rand);
return min + offset - 1;
}
You can apply this by taking your random doubles and doing
int[] getIntsFromRandomDoubles(int min, int max, double[] rands) {
int[] result = new int[rands.length];
for(int i = 0; i < rands.length; i++) result[i] = getIntInRangeFromDouble(min,max,rands[i]);
return result;
}