Develop Reference

Does multiplying a floating point number by 0 always yield 0?

Floating point numbers are tricky, since many natural arithmetic properties don't hold.
I SUPPOSE that this particular property holds nevertheless, but I prefer to ask rather than be hit by hard to detect errors.
Assume that d is an arbitrary variable of type double. May I assume that after the following operation:
d *= 0;
The following check will always return true?
d == 0
I suppose that this will not hold if d is positive / negative infinity or NaN. However, are there any other problems I need to be aware of?
I hear that in floating point there are actually two zeroes, namely +0 and -0. If d is negative in the beginning, will d *= 0 return -0 instead? If so, will it be equal to 0?
I hear that floating point operations are subjected to inaccuracy. Thus, is it possible that multiplying by 0 will instead return something like 0.0000001 or -0.000001 which will not be equal to 0? My assumption is that this will likely be impossible, but I don't have enough knowledge to back this assumption up so I prefer to ask.
Any other problems I didn't foresee?

The short answer is yes, given your specific example, d will be 0. This is because 0 has an exact representation in every FP model, so when you multiply a double by the value '0' (or '0.0'), the result is not subject to rounding/truncation errors.
The issues you mentioned come into play during FP arithmetic as a result of the inherent approximations that occur when you have a finite resolution.
For example, the most accurate 64-bit representation of the value 0.002 in the IEEE-754 floating point standard is 0x3F60624D_D2F1A9FC which amounts to 2.00000000000000004163336342344E-3 (source: http://www.binaryconvert.com/convert_double.html)
The value 0 does not suffer from this loss of precision. However, obviously if you were to do something like this, you would run into problems:
double d = 0.009;
d -= (0.001 * 9);
if (d == 0)
{
//This check will fail
}
This is why it is almost always advised against to use exact equality when comparing floating point values.

C# and the mischief of floats

In testing as to why my program is not working as intended, I tried typing the calculations that seem to be failing into the immediate window.
Math.Floor(1.0f)
1.0 - correct
However:
200f * 0.005f
1.0
Math.Floor(200f * 0.005f)
0.0 - incorrect
Furthermore:
(float)(200f * 0.005f)
1.0
Math.Floor((float)(200f * 0.005f))
0.0 - incorrect
Probably some float loss is occuring, 0.99963 ≠ 1.00127 for example.
I wouldn't mind storing less pricise values, but in a non lossy way, for example if there were a numeric type that stored values as integers do, but to only three decimal places, if it could be made performant.
I think probably there is a better way of calculating (n * 0.005f) in regards to such errors.
edit:
TY, a solution:
Math.Floor(200m * 0.005m)
Also, as I understand it, this would work if I didn't mind changing the 1/200 into 1/256:
Math.Floor(200f * 0.00390625f)
The solution I'm using. It's the closest I can get in my program and seems to work ok:
float x = ...;
UInt16 n = 200;
decimal d = 1m / n;
... = Math.Floor((decimal)x * d)

Floats represent numbers as fractions with powers of two in the denominator. That is, you can exactly represent 1/2, or 3/4, or 19/256. Since .005 is 1/200, and 200 is not a power of two, instead what you get for 0.005f is the closest fraction that has a power of two on the bottom that can fit into a 32 bit float.
Decimals represent numbers as fractions with powers of ten in the denominator. Like floats, they introduce errors when you try to represent numbers that do not fit that pattern. 1m/333m for example, will give you the closest number to 1/333 that has a power of ten as the denominator and 29 or fewer significant digits. Since 0.005 is 5/1000, and that is a power of ten, 0.005m will give you an exact representation. The price you pay is that decimals are much larger and slower than floats.
You should always always always use decimals for financial calculations, never floats.

The problem is that 0.005f is actually 0.004999999888241291046142578125... so less than 0.005. That's the closest float value to 0.005. When you multiply that by 200, you end up with something less than 1.
If you use decimal instead - all the time, not converting from float - you should be fine in this particular scenario. So:
decimal x = 0.005m;
decimal y = 200m;
decimal z = x * y;
Console.WriteLine(z == 1m); // True
However, don't assume that this means decimal has "infinite precision". It's still a floating point type with limited precision - it's just a floating decimal point type, so 0.005 is exactly representable.

If you cannot tolerate any floating point precision issues, use decimal.
http://msdn.microsoft.com/en-us/library/364x0z75.aspx
Ultimately even decimal has precision issues (it allows for 28-29 significant digits). If you are working in it's supported range ((-7.9 x 10^28 to 7.9 x 10^28) / (100^28)), you are quite unlikely to be impacted by them.

Fourier transform rounding error

I'm messing around with Fourier transformations. Now I've created a class that does an implementation of the DFT (not doing anything like FFT atm). This is the implementation I've used:
public static Complex[] Dft(double[] data)
{
int length = data.Length;
Complex[] result = new Complex[length];
for (int k = 1; k <= length; k++)
{
Complex c = Complex.Zero;
for (int n = 1; n <= length; n++)
{
c += Complex.FromPolarCoordinates(data[n-1], (-2 * Math.PI * n * k) / length);
}
result[k-1] = 1 / Math.Sqrt(length) * c;
}
return result;
}
And these are the results I get from Dft({2,3,4})
Well it seems pretty okay, since those are the values I expect. There is only one thing I find confusing. And it all has to do with the rounding of doubles.
First of all, why are the first two numbers not exactly the same (0,8660..443 8 ) vs (0,8660..443). And why can't it calculate a zero, where you'd expect it. I know 2.8E-15 is pretty close to zero, but well it's not.
Anyone know how these, marginal, errors occur and if I can and want to do something about it.
It might seem that there's not a real problem, because it's just small errors. However, how do you deal with these rounding errors if you're for example comparing 2 values.
5,2 + 0i != 5,1961524 + i2.828107*10^-15
Cheers

I think you've already explained it to yourself - limited precision means limited precision. End of story.
If you want to clean up the results, you can do some rounding of your own to a more reasonable number of siginificant digits - then your zeros will show up where you want them.
To answer the question raised by your comment, don't try to compare floating point numbers directly - use a range:
if (Math.Abs(float1 - float2) < 0.001) {
// they're the same!
}
The comp.lang.c FAQ has a lot of questions & answers about floating point, which you might be interested in reading.

From http://support.microsoft.com/kb/125056
Emphasis mine.
There are many situations in which precision, rounding, and accuracy in floating-point calculations can work to generate results that are surprising to the programmer. There are four general rules that should be followed:
In a calculation involving both single and double precision, the result will not usually be any more accurate than single precision. If double precision is required, be certain all terms in the calculation, including constants, are specified in double precision.
Never assume that a simple numeric value is accurately represented in the computer. Most floating-point values can't be precisely represented as a finite binary value. For example .1 is .0001100110011... in binary (it repeats forever), so it can't be represented with complete accuracy on a computer using binary arithmetic, which includes all PCs.
Never assume that the result is accurate to the last decimal place. There are always small differences between the "true" answer and what can be calculated with the finite precision of any floating point processing unit.
Never compare two floating-point values to see if they are equal or not- equal. This is a corollary to rule 3. There are almost always going to be small differences between numbers that "should" be equal. Instead, always check to see if the numbers are nearly equal. In other words, check to see if the difference between them is very small or insignificant.
Note that although I referenced a microsoft document, this is not a windows problem. It's a problem with using binary and is in the CPU itself.
And, as a second side note, I tend to use the Decimal datatype instead of double: See this related SO question: decimal vs double! - Which one should I use and when?

In C# you'll want to use the 'decimal' type, not double for accuracy with decimal points.
As to the 'why'... repsensenting fractions in different base systems gives different answers. For example 1/3 in a base 10 system is 0.33333 recurring, but in a base 3 system is 0.1.
The double is a binary value, at base 2. When converting to base 10 decimal you can expect to have these rounding errors.

C# float infinite loop

The following code in C# (.Net 3.5 SP1) is an infinite loop on my machine:
for (float i = 0; i < float.MaxValue; i++) ;
It reached the number 16777216.0 and 16777216.0 + 1 is evaluates to 16777216.0. Yet at this point: i + 1 != i.
This is some craziness.
I realize there is some inaccuracy in how floating point numbers are stored. And I've read that whole numbers greater 2^24 than cannot be properly stored as a float.
Still the code above, should be valid in C# even if the number cannot be properly represented.
Why does it not work?
You can get the same to happen for double but it takes a very long time. 9007199254740992.0 is the limit for double.

Right, so the issue is that in order to add one to the float, it would have to become
16777217.0
It just so happens that this is at a boundary for the radix and cannot be represented exactly as a float. (The next highest value available is 16777218.0)
So, it rounds to the nearest representable float
16777216.0
Let me put it this way:
Since you have a floating amount of precision, you have to increment up by a higher-and-higher number.
EDIT:
Ok, this is a little bit difficult to explain, but try this:
float f = float.MaxValue;
f -= 1.0f;
Debug.Assert(f == float.MaxValue);
This will run just fine, because at that value, in order to represent a difference of 1.0f, you would need over 128 bits of precision. A float has only 32 bits.
EDIT2
By my calculations, at least 128 binary digits unsigned would be necessary.
log(3.40282347E+38) * log(10) / log(2) = 128
As a solution to your problem, you could loop through two 128 bit numbers. However, this will take at least a decade to complete.

Imagine for example that a floating point number is represented by up to 2 significant decimal digits, plus an exponent: in that case, you could count from 0 to 99 exactly. The next would be 100, but because you can only have 2 significant digits that would be stored as "1.0 times 10 to the power of 2". Adding one to that would be ... what?
At best, it would be 101 as an intermediate result, which would actually be stored (via a rounding error which discards the insignificant 3rd digit) as "1.0 times 10 to the power of 2" again.

To understand what's going wrong you're going to have to read the IEEE standard on floating point
Let's examine the structure of a floating point number for a second:
A floating point number is broken into two parts (ok 3, but ignore the sign bit for a second).
You have a exponent and a mantissa. Like so:
smmmmmmmmeeeeeee
Note: that is not acurate to the number of bits, but it gives you a general idea of what's happening.
To figure out what number you have we do the following calculation:
mmmmmm * 2^(eeeeee) * (-1)^s
So what is float.MaxValue going to be? Well you're going to have the largest possible mantissa and the largest possible exponent. Let's pretend this looks something like:
01111111111111111
in actuality we define NAN and +-INF and a couple other conventions, but ignore them for a second because they're not relevant to your question.
So, what happens when you have 9.9999*2^99 + 1? Well, you do not have enough significant figures to add 1. As a result it gets rounded down to the same number. In the case of single floating point precision the point at which +1 starts to get rounded down happens to be 16777216.0

It has nothing to do with overflow, or being near the max value. The float value for 16777216.0 has a binary representation of 16777216. You then increment it by 1, so it should be 16777217.0, except that the binary representation of 16777217.0 is 16777216!!! So it doesn't actually get incremented or at least the increment doesn't do what you expect.
Here is a class written by Jon Skeet that illustrates this:
DoubleConverter.cs
Try this code with it:
double d1 = 16777217.0;
Console.WriteLine(DoubleConverter.ToExactString(d1));
float f1 = 16777216.0f;
Console.WriteLine(DoubleConverter.ToExactString(f1));
float f2 = 16777217.0f;
Console.WriteLine(DoubleConverter.ToExactString(f2));
Notice how the internal representation of 16777216.0 is the same 16777217.0!!

The iteration when i approaches float.MaxValue has i just below this value. The next iteration adds to i, but it can't hold a number bigger than float.MaxValue. Thus it holds a value much smaller, and begins the loop again.

Is it safe to check floating point values for equality to 0?

I know you can't rely on equality between double or decimal type values normally, but I'm wondering if 0 is a special case.
While I can understand imprecisions between 0.00000000000001 and 0.00000000000002, 0 itself seems pretty hard to mess up since it's just nothing. If you're imprecise on nothing, it's not nothing anymore.
But I don't know much about this topic so it's not for me to say.
double x = 0.0;
return (x == 0.0) ? true : false;
Will that always return true?

It is safe to expect that the comparison will return true if and only if the double variable has a value of exactly 0.0 (which in your original code snippet is, of course, the case). This is consistent with the semantics of the == operator. a == b means "a is equal to b".
It is not safe (because it is not correct) to expect that the result of some calculation will be zero in double (or more generally, floating point) arithmetics whenever the result of the same calculation in pure Mathematics is zero. This is because when calculations come into the ground, floating point precision error appears - a concept which does not exist in Real number arithmetics in Mathematics.

If you need to do a lot of "equality" comparisons it might be a good idea to write a little helper function or extension method in .NET 3.5 for comparing:
public static bool AlmostEquals(this double double1, double double2, double precision)
{
return (Math.Abs(double1 - double2) <= precision);
}
This could be used the following way:
double d1 = 10.0 * .1;
bool equals = d1.AlmostEquals(0.0, 0.0000001);

For your simple sample, that test is okay. But what about this:
bool b = ( 10.0 * .1 - 1.0 == 0.0 );
Remember that .1 is a repeating decimal in binary and can't be represented exactly, the same as trying to write 1/3 as a base 10 decimal. Now compare that to this code:
double d1 = 10.0 * .1; // make sure the compiler hasn't optimized the .1 issue away
bool b = ( d1 - 1.0 == 0.0 );
I'll leave you to run a test to see the actual results: you're more likely to remember it that way.

From the MSDN entry for Double.Equals:
Precision in Comparisons
The Equals method should be used with
caution, because two apparently
equivalent values can be unequal due
to the differing precision of the two
values. The following example reports
that the Double value .3333 and the
Double returned by dividing 1 by 3 are
unequal.
...
Rather than comparing for equality,
one recommended technique involves
defining an acceptable margin of
difference between two values (such as
.01% of one of the values). If the
absolute value of the difference
between the two values is less than or
equal to that margin, the difference
is likely to be due to differences in
precision and, therefore, the values
are likely to be equal. The following
example uses this technique to compare
.33333 and 1/3, the two Double values
that the previous code example found
to be unequal.
Also, see Double.Epsilon.

The problem comes when you are comparing different types of floating point value implementation e.g. comparing float with double. But with same type, it shouldn't be a problem.
float f = 0.1F;
bool b1 = (f == 0.1); //returns false
bool b2 = (f == 0.1F); //returns true
The problem is, programmer sometimes forgets that implicit type cast (double to float) is happening for the comparison and the it results into a bug.

If the number was directly assigned to the float or double then it is safe to test against zero or any whole number that can be represented in 53 bits for a double or 24 bits for a float.
Or to put it another way you can always assign and integer value to a double and then compare the double back to the same integer and be guaranteed it will be equal.
You can also start out by assigning a whole number and have simple comparisons continue to work by sticking to adding, subtracting or multiplying by whole numbers (assuming the result is less than 24 bits for a float abd 53 bits for a double). So you can treat floats and doubles as integers under certain controlled conditions.

No, it is not OK. So-called denormalized values (subnormal), when compared equal to 0.0, would compare as false (non-zero), but when used in an equation would be normalized (become 0.0). Thus, using this as a mechanism to avoid a divide-by-zero is not safe. Instead, add 1.0 and compare to 1.0. This will ensure that all subnormals are treated as zero.

Try this, and you will find that == is not reliable for double/float.
double d = 0.1 + 0.2;
bool b = d == 0.3;
Here is the answer from Quora.

Actually, I think it is better to use the following codes to compare a double value against to 0.0:
double x = 0.0;
return (Math.Abs(x) < double.Epsilon) ? true : false;
Same for float:
float x = 0.0f;
return (Math.Abs(x) < float.Epsilon) ? true : false;