Develop Reference

Math.NET CryptoRandomSource Next is Biased

I am developing a gaming platform that is subject to heavy regulatory scrutiny. I chose Math.NET because it seemed like a good fit. However I have just received this comment back from our auditors.
Comments please if this is accurate and how it can be resolved?
In RandomSource(), Next(int, int) is defined as follows:
public override sealed int Next(int minValue, int maxValue)
{
if (minValue > maxValue)
{
throw new ArgumentException(Resources.ArgumentMinValueGreaterThanMaxValue);
}
if (_threadSafe)
{
lock (_lock)
{
return (int)(DoSample()*(maxValue - minValue)) + minValue;
}
}
return (int)(DoSample()*(maxValue - minValue)) + minValue;
}
This creates a bias in the same way as before. Using an un-scaled value from the RNG and multiplying it by the range without previously eliminating the bias (unless the range is a power of 2, there will be a bias ).

Update: The implementation of Next(minInclusive, maxExclusive) has been changed in Math.NET Numerics v3.13 following this discussion. Since v3.13 it is no longer involving floating point numbers, but instead samples integers with as many bits as needed to support the requested range (power of two) and rejects those outside of the actual range. This way it avoids adding any bias on top of the byte sampling itself (as provided e.g. by the crypto RNG)
Assumption: DoSample() returns a uniformly distributed sample in the range [0,1) (double precision floating point number).
Multiplying it with the range R=max-min will result in a uniformly distributed sample in the range [0,R). Casting this to an integer, which is essentially a floor, will result in a uniformly distributed discrete sample of one of 0,1,2,...,R-1. I don't see where the fact that R is even, odd, or a power of two may affect bias in this step.
A few runs to compute 100'000'000 samples also do not indicate obvious bias, but of course this is no proof:
var r = new CryptoRandomSource();
long[] h = new long[8];
for (int i = 0; i < 100000000; i++)
{
h[r.Next(2,7)]++;
}
0
0
19996313
20001286
19998092
19998328
20005981
0
0
0
20000288
20002035
20006269
19994927
19996481
0
0
0
19998296
19997777
20001463
20002759
19999705
0

I've come up with this solution for a value between 0 and max inclusive. I'm no maths expert so comments welcome.
It seems to satisfy the regulatory spec I have which says
2b) If a particular random number selected is outside the range of equal distribution of re-scaling values, it is permissible to discard that random number and select the next in sequence for the purpose of re-scaling."
private readonly CryptoRandomSource _random = new CryptoRandomSource();
private int GetRandomNumber(int max)
{
int number;
var nextPowerOfTwo = (int)Math.Pow(2, Math.Ceiling(Math.Log(max) / Math.Log(2)));
do
{
// Note: 2nd param of Next is an *exclusive* value. Add 1 to satisfy this
number = _random.Next(0, nextPowerOfTwo + 1);
} while (number > max);
return number;
}

BitArray change bit within range

How can I ensure that when changing a bit from a BitArray, the BitArray value remains in a range.
Example:
Given the range [-5.12, 5.12] and
a = 0100000000000000011000100100110111010010111100011010100111111100 ( = 2.048)
By changing a bit at a random position, I need to ensure that the new value remains in the given range.

I'm not 100% sure what you are doing and this answer assumes you are storing a as a 64-bit value (long) currently. The following code may help point you in the right direction.
const double minValue = -5.12;
const double maxValue = 5.12;
var initialValue = Convert.ToInt64("100000000000000011000100100110111010010111100011010100111111100", 2);
var changedValue = ChangeRandomBit(initialValue); // However you're doing this
var changedValueAsDouble = BitConverter.Int64BitsToDouble(initialValue);
if ((changedValueAsDouble < minValue) || (changedValueAsDouble > maxValue))
{
// Do something
}

It looks like double (64 bits and result has decimal point).
As you may know it has sign bit, exponent and fraction, so you can not change random bit and still have value in the range, with some exceptions:
sign bit can be changed without problem if your range is [-x;+x] (same x);
changing exponent or fraction will require to check new value range but:
changing exponent of fraction bit from 1 to 0 will make |a| less.
I don't know what you are trying to achieve, care to share? Perhaps you are trying to validate or correct something, then you may have a look at this.

Here's an extension method that undoes the set bit if the new value of the float is outside the given range (this is an example only, it relies on the BitArray holding a float with no checks, which is pretty horrible so just hack a solution out of this, incl changing to double):
static class Extension
{
public static void SetFloat(this BitArray array, int index, bool value, float min, float max)
{
bool old = array.Get(index);
array.Set(index, value);
byte[] bytes = new byte[4];
array.CopyTo(bytes, 0);
float f = BitConverter.ToSingle(bytes, 0);
if (f < min || f > max)
array.Set(index, old);
}
}
Example use:
static void Main(string[] args)
{
float f = 2.1f;
byte[] bytes = System.BitConverter.GetBytes(f);
BitArray array = new BitArray(bytes);
array.Set(20, true, -5.12f, 5.12f);
}

If you can actually limit your precision, then this would be a lot easier. For example given the range:
[-5.12, 5.12]
If I multiply 5.12 by 100, I get
[-512, 512]
And the integer 512 in binary is, of course:
1000000000
So now you know you can set any of the first 9 bits and you'll be < 512 if the 10th bit is 0. If you set the 10th bit, you will have to set all the other bits to 0. With a little extra effort, this can be extended to deal with 2's complement negative values too (although, I might be inclined just to convert them to positive values)
Now if you actually need to accommodate the 3 d.p. of 2.048, then you'll need to multiply all you values by 1000 instead and it will be a little more difficult because 5120 in binary is 1010000000000
You know you can do anything you want with everything except the most significant bit (MSB) if the MSB is 0. In this case, if the MSB is 1, but the next 2 bits are 0, you can do anything you want with the remaining bits.
The logic involved with dealing directly with the number in IEEE-754 floating point format is probably going to be torturous.
Or you could just go with the "mutate the value and then test it" approach, if it's out-of-range, go back and try again. Which might be suitable (in practice), but won't be guaranteed to exit.
A final thought, depending on exactly what you are doing, you might want to also look at Gray Codes. The idea of a Gray Code is to make it such that each value is only 1 bit flip apart. With naturally encoded binary, a flip of the MSB has orders of magnitude more impact on the final value than a flip of the LSB.

Generate a Random Int32 for the full range of possible numbers

If I wanted to generate a random number for all possible numbers an Int32 could contain would the following code be a reasonable way of doing so? Is there any reason why it may not be a good idea? (ie. a uniform distribution at least as good as Random.Next() itself anyway)
public static int NextInt(Random Rnd) //-2,147,483,648 to 2,147,483,647
{
int AnInt;
AnInt = Rnd.Next(System.Int32.MinValue, System.Int32.MaxValue);
AnInt += Rnd.Next(2);
return AnInt;
}

You could use Random.NextBytes to obtain 4 bytes, then use BitConverter.ToInt32 to convert those to an int.
Something like:
byte[] buf = new byte[4];
Rnd.NextBytes(buf);
int i = BitConverter.ToInt32(buf,0);

Your proposed solution will slightly skew the distribution. The minValue and maxValue will occur less frequently than the interior values. As an example, assume that int has a MinValue of -2 and a MaxValue of 1. Here are the possible initial values, with each followed by the resulting values after the Random(2):
-2: -2 -1
-1: -1 0
0: 0 1
half of the negative -2 values will get modified up to -1, and only half of 0 will get modified up to 1. So the values -2 and 1 will occur less frequently than -1 and 0.
Damien's solution is good. Another choice would be:
if (Random(2) == 0) {
return Random(int.MinValue, 0);
} else {
return 1 + Random(-1, int.MaxValue);
}
another solution, similar to Damiens approach, and faster than the previous one would be
int i = r.Next(ushort.MinValue, ushort.MaxValue + 1) << 16;
i |= r.Next(ushort.MinValue, ushort.MaxValue + 1);

A uniform distribution does not mean you get each number exactly once. For that you need a permutation
Now, if you need a random permutation of all 4-billion numbers you're a bit stuck. .NET does not allow objects to be larger than 2GBs. You can work around that, but I assume that's not really what you need.
If you less numbers (say, 100, or 5 million, less than a few billions) without repetitions, you should do this:
Maintain a set of integers, starting empty. Choose a random number. If it's already in the set, choose another random number. If it's not in the set, add it and return it.
That way you guarantee each number will be returned only once.

I have a class where I get random bytes into a 8KB buffer and distribute numbers from by converting them from the random bytes. This gives you the full int distribution. The 8KB buffer is used to you do not need to call NextBytes for every new random byte[].
// Get 4 bytes from the random buffer and cast to int (all numbers equally this way
public int GetRandomInt()
{
CheckBuf(sizeof(int));
return BitConverter.ToInt32(_buf, _idx);
}
// Get bytes for your buffer. Both random class and cryptoAPI support this
protected override void GetNewBuf(byte[] buf)
{
_rnd.NextBytes(buf);
}
// cyrptoAPI does better random numbers but is slower
public StrongRandomNumberGenerator()
{
_rnd = new RNGCryptoServiceProvider();
}

Arbitrarily large integers in C#

How can I implement this python code in c#?
Python code:
print(str(int(str("e60f553e42aa44aebf1d6723b0be7541"), 16)))
Result:
305802052421002911840647389720929531201
But in c# I have problems with big digits.
Can you help me?
I've got different results in python and c#. Where can be mistake?

Primitive types (such as Int32, Int64) have a finite length that it's not enough for such big number. For example:
Data type Maximum positive value
Int32 2,147,483,647
UInt32 4,294,967,295
Int64 9,223,372,036,854,775,808
UInt64 18,446,744,073,709,551,615
Your number 305,802,052,421,002,911,840,647,389,720,929,531,201
In this case to represent that number you would need 128 bits. With .NET Framework 4.0 there is a new data type for arbitrarily sized integer numbers System.Numerics.BigInteger. You do not need to specify any size because it'll be inferred by the number itself (it means that you may even get an OutOfMemoryException when you perform, for example, a multiplication of two very big numbers).
To come back to your question, first parse your hexadecimal number:
string bigNumberAsText = "e60f553e42aa44aebf1d6723b0be7541";
BigInteger bigNumber = BigInteger.Parse(bigNumberAsText,
NumberStyles.AllowHexSpecifier);
Then simply print it to console:
Console.WriteLine(bigNumber.ToString());
You may be interested to calculate how many bits you need to represent an arbitrary number, use this function (if I remember well original implementation comes from C Numerical Recipes):
public static uint GetNeededBitsToRepresentInteger(BigInteger value)
{
uint neededBits = 0;
while (value != 0)
{
value >>= 1;
++neededBits;
}
return neededBits;
}
Then to calculate the required size of a number wrote as string:
public static uint GetNeededBitsToRepresentInteger(string value,
NumberStyles numberStyle = NumberStyles.None)
{
return GetNeededBitsToRepresentInteger(
BigInteger.Parse(value, numberStyle));
}

If you just want to be able to use larger numbers there is BigInteger which has a lot of digits.

To find the number of bits you need to store a BigInteger N, you can use:
BigInteger N = ...;
int nBits = Mathf.CeilToInt((float)BigInteger.Log(N, 2.0));

Generation of (pseudo) random constrained values of (U)Int64 and Decimal

Note: For brevity's sake, the following will not discern between randomness and pseudo-randomness. Also, in this context, constrained means between given min and max values)
The System.Random class provides random generation of integers, doubles and byte arrays.
Using Random.Next, one can easily generate random constrained values of type Boolean, Char, (S)Byte, (U)Int16, (U)Int32. Using Random.NextDouble(), one can similarly generate constrained values of types Double and Single (as far as my understanding of this type goes). Random string generation (of a given length and alphabet) has also been tackled before.
Consider the remaining primitive data types (excluding Object): Decimal and (U)Int64. Their random generation has been tackled as well (Decimal, (U)Int64 using Random.NextBytes()), but not when constrained. Rejection sampling (i.e. looping until the generated value is the desired range) could theoretically be used, but it is obviously not a practical solution. Normalizing NextDouble() won't work because it doesn't have enough significant digits.
In short, I am asking for the proper implementation of the following functions:
long NextLong(long min, long max)
long NextDecimal(decimal min, decimal max)
Note that, since System.DateTime is based on a ulong, the first function would allow for random constrained generation of such structs as well (similar to here, only in ticks instead of minutes).

This should do it. For decimal I utilized Jon Skeet's initial approach to generating random decimals (no constraints). For long I provided a method to produced random non-negative longs which is then used to create the a value in the random range.
Note that for decimal the resulting distribution is not a uniform distribution on [minValue, maxValue]. It merely is uniform on all the bit representations of decimals that fall in the range [minValue, maxValue]. I do not see an easy way around this without using rejection sampling.
For long the resulting distribution is uniform on [minValue, maxValue).
static class RandomExtensions {
static int NextInt32(this Random rg) {
unchecked {
int firstBits = rg.Next(0, 1 << 4) << 28;
int lastBits = rg.Next(0, 1 << 28);
return firstBits | lastBits;
}
}
public static decimal NextDecimal(this Random rg) {
bool sign = rg.Next(2) == 1;
return rg.NextDecimal(sign);
}
static decimal NextDecimal(this Random rg, bool sign) {
byte scale = (byte)rg.Next(29);
return new decimal(rg.NextInt32(),
rg.NextInt32(),
rg.NextInt32(),
sign,
scale);
}
static decimal NextNonNegativeDecimal(this Random rg) {
return rg.NextDecimal(false);
}
public static decimal NextDecimal(this Random rg, decimal maxValue) {
return (rg.NextNonNegativeDecimal() / Decimal.MaxValue) * maxValue; ;
}
public static decimal NextDecimal(this Random rg, decimal minValue, decimal maxValue) {
if (minValue >= maxValue) {
throw new InvalidOperationException();
}
decimal range = maxValue - minValue;
return rg.NextDecimal(range) + minValue;
}
static long NextNonNegativeLong(this Random rg) {
byte[] bytes = new byte[sizeof(long)];
rg.NextBytes(bytes);
// strip out the sign bit
bytes[7] = (byte)(bytes[7] & 0x7f);
return BitConverter.ToInt64(bytes, 0);
}
public static long NextLong(this Random rg, long maxValue) {
return (long)((rg.NextNonNegativeLong() / (double)Int64.MaxValue) * maxValue);
}
public static long NextLong(this Random rg, long minValue, long maxValue) {
if (minValue >= maxValue) {
throw new InvalidOperationException();
}
long range = maxValue - minValue;
return rg.NextLong(range) + minValue;
}
}

Let's assume you know how to generate N random bits. This is pretty easily done either using NextBytes or repeated calls to Random.Next with appropriate limits.
To generate a long/ulong in the right range, work out how large the range is, and how many bits it takes to represent it. You can then use rejection sampling which will at worst reject half the generated values (e.g. if you want a value in the range [0, 128], which means you'll generate [0, 255] multiple times). If you want a non-zero based range, just work out the size of the range, generate a random value in [0, size) and then add the base.
Generating a random decimal is signficantly harder, I believe - aside from anything else, you'd have to specify the distribution you wanted.

I came here looking for a way to generate 64 bit values within an arbitrary range. The other answers failed to produce a random number when given certain ranges (e.g. long.MinValue to long.MaxValue). Here's my version that seems to solve the problem:
public static long NextInt64(this Random random, long minValue, long maxValue)
{
Contract.Requires(random != null);
Contract.Requires(minValue <= maxValue);
Contract.Ensures(Contract.Result<long>() >= minValue &&
Contract.Result<long>() < maxValue);
return (long)(minValue + (random.NextUInt64() % ((decimal)maxValue - minValue)));
}
It uses the following extension Methods:
public static ulong NextUInt64(this Random random)
{
Contract.Requires(random != null);
return BitConverter.ToUInt64(random.NextBytes(8), 0);
}
public static byte[] NextBytes(this Random random, int byteCount)
{
Contract.Requires(random != null);
Contract.Requires(byteCount > 0);
Contract.Ensures(Contract.Result<byte[]>() != null &&
Contract.Result<byte[]>().Length == byteCount);
var buffer = new byte[byteCount];
random.NextBytes(buffer);
return buffer;
}
The distribution is not perfectly even when the size of the requested range is not a clean divisor of 2^64, but it at least provides a random number within the request range for any given range.

Based upon Jon Skeet's method, here's my stab at it:
public static long NextLong(this Random rnd, long min, long max)
{
if (max <= min)
{
throw new Exception("Min must be less than max.");
}
long dif = max - min;
var bytes = new byte[8];
rnd.NextBytes(bytes);
bytes[7] &= 0x7f; //strip sign bit
long posNum = BitConverter.ToInt64(bytes, 0);
while (posNum > dif)
{
posNum >>= 1;
}
return min + posNum;
}
Let me know if you see any errors.

long posNum = BitConverter.ToInt64(Guid.NewGuid().ToByteArray(), 0);
use this instead of NextBytes