I'm messing around with Fourier transformations. Now I've created a class that does an implementation of the DFT (not doing anything like FFT atm). This is the implementation I've used:
public static Complex[] Dft(double[] data)
{
int length = data.Length;
Complex[] result = new Complex[length];
for (int k = 1; k <= length; k++)
{
Complex c = Complex.Zero;
for (int n = 1; n <= length; n++)
{
c += Complex.FromPolarCoordinates(data[n-1], (-2 * Math.PI * n * k) / length);
}
result[k-1] = 1 / Math.Sqrt(length) * c;
}
return result;
}
And these are the results I get from Dft({2,3,4})
Well it seems pretty okay, since those are the values I expect. There is only one thing I find confusing. And it all has to do with the rounding of doubles.
First of all, why are the first two numbers not exactly the same (0,8660..443 8 ) vs (0,8660..443). And why can't it calculate a zero, where you'd expect it. I know 2.8E-15 is pretty close to zero, but well it's not.
Anyone know how these, marginal, errors occur and if I can and want to do something about it.
It might seem that there's not a real problem, because it's just small errors. However, how do you deal with these rounding errors if you're for example comparing 2 values.
5,2 + 0i != 5,1961524 + i2.828107*10^-15
Cheers
I think you've already explained it to yourself - limited precision means limited precision. End of story.
If you want to clean up the results, you can do some rounding of your own to a more reasonable number of siginificant digits - then your zeros will show up where you want them.
To answer the question raised by your comment, don't try to compare floating point numbers directly - use a range:
if (Math.Abs(float1 - float2) < 0.001) {
// they're the same!
}
The comp.lang.c FAQ has a lot of questions & answers about floating point, which you might be interested in reading.
From http://support.microsoft.com/kb/125056
Emphasis mine.
There are many situations in which precision, rounding, and accuracy in floating-point calculations can work to generate results that are surprising to the programmer. There are four general rules that should be followed:
In a calculation involving both single and double precision, the result will not usually be any more accurate than single precision. If double precision is required, be certain all terms in the calculation, including constants, are specified in double precision.
Never assume that a simple numeric value is accurately represented in the computer. Most floating-point values can't be precisely represented as a finite binary value. For example .1 is .0001100110011... in binary (it repeats forever), so it can't be represented with complete accuracy on a computer using binary arithmetic, which includes all PCs.
Never assume that the result is accurate to the last decimal place. There are always small differences between the "true" answer and what can be calculated with the finite precision of any floating point processing unit.
Never compare two floating-point values to see if they are equal or not- equal. This is a corollary to rule 3. There are almost always going to be small differences between numbers that "should" be equal. Instead, always check to see if the numbers are nearly equal. In other words, check to see if the difference between them is very small or insignificant.
Note that although I referenced a microsoft document, this is not a windows problem. It's a problem with using binary and is in the CPU itself.
And, as a second side note, I tend to use the Decimal datatype instead of double: See this related SO question: decimal vs double! - Which one should I use and when?
In C# you'll want to use the 'decimal' type, not double for accuracy with decimal points.
As to the 'why'... repsensenting fractions in different base systems gives different answers. For example 1/3 in a base 10 system is 0.33333 recurring, but in a base 3 system is 0.1.
The double is a binary value, at base 2. When converting to base 10 decimal you can expect to have these rounding errors.
Related
A colleague has written some code along these lines:
var roundedNumber = (float) Math.Round(someFloat, 2);
Console.WriteLine(roundedNumber);
I have an uncertainty about this code - is the number that gets written here even guaranteed to have 2 decimal places any more? It seems plausible to me that truncation of the double Math.Round(someFloat, 2) to float might result in a number whose string representation has more than 2 digits. Can anybody either provide an example of this (demonstrating that such a cast is unsafe) or else demonstrate somehow that it is safe to perform such a cast?
Assuming single and double precision IEEE754 representation and rules, I have checked for the first 2^24 integers i that
float(double( i/100 )) = float(i/100)
in other words, converting a decimal value with 2 decimal places twice (first to the nearest double, then to the nearest single precision float) is the same as converting the decimal directly to single precision, as long as the integer part of the decimal is not too large.
I have no guarantee for larger values.
The double approximation and the single approximation are different, but that's not really the question.
Converting twice is innocuous up to at least 167772.16, it's the same as if Math.Round would have done it directly in single precision.
Here is the testing code in Squeak/Pharo Smalltalk with ArbitraryPrecisionFloat package (sorry to not exhibit it in c# but the language does not really matter, only IEEE rules do).
(1 to: 1<<24)
detect: [:i |
(i/100.0 asArbitraryPrecisionFloatNumBits: 24) ~= (i/100 asArbitraryPrecisionFloatNumBits: 24) ]
ifNone: [nil].
EDIT
Above test was superfluous because, thanks to excellent reference provided by Mark Dickinson (Innocuous double rounding of basic arithmetic operations) , we know that doing float(double(x) / double(y)) produces a correctly-rounded value for x / y, as long as x and y are both representable as floats, which is the case for any 0 <= x <= 2^24 and for y=100.
EDIT
I have checked with numerators up to 2^30 (decimal value > 10 millions), and converting twice is still identical to converting once. Going further with an interpreted language is not good wrt global warming...
When I run the following code, I get 0 printed on both lines:
Double a = 9.88131291682493E-324;
Double b = a*0.1D;
Console.WriteLine(b);
Console.WriteLine(BitConverter.DoubleToInt64Bits(b));
I would expect to get Double.NaN if an operation result gets out of range. Instead I get 0. It looks that to be able to detect when this happens I have to check:
Before the operation check if any of the operands is zero
After the operation, if neither of operands were zero, check if the result is zero. If not let it run. If it is zero, assign Double.NaN to it instead to indicate that it's not really a zero, it's just a result that can't be represented within this variable.
That's rather unwieldy. Is there a better way? What Double.NaN is designed for? I'm assuming some operations must have return it, surely designers did not put it there just in case? Is it possible that this is a bug in BCL? (I know unlikely, but, that's why I'd like to understand how that Double.NaN is supposed to work)
Update
By the way, this problem is not specific for double. decimal exposes it all the same:
Decimal a = 0.0000000000000000000000000001m;
Decimal b = a* 0.1m;
Console.WriteLine(b);
That also gives zero.
In my case I need double, because I need the range they provide (I'm working on probabilistic calculations) and I'm not that worried about precision.
What I need though is to be able to detect when my results stop mean anything, that is when calculations drop the value so low, that it can no longer be presented by double.
Is there a practical way of detecting this?
Double works exactly according to the floating point numbers specification, IEEE 754. So no, it's not an error in BCL - it's just the way IEEE 754 floating points work.
The reason, of course, is that it's not what floats are designed for at all. Instead, you might want to use decimal, which is a precise decimal number, unlike float/double.
There's a few special values in floating point numbers, with different meanings:
Infinity - e.g. 1f / 0f.
-Infinity - e.g. -1f / 0f.
NaN - e.g. 0f / 0f or Math.Sqrt(-1)
However, as the commenters below noted, while decimal does in fact check for overflows, coming too close to zero is not considered an overflow, just like with floating point numbers. So if you really need to check for this, you will have to make your own * and / methods. With decimal numbers, you shouldn't really care, though.
If you need this kind of precision for multiplication and division (that is, you want your divisions to be reversible by multiplication), you should probably use rational numbers instead - two integers (big integers if necessary). And use a checked context - that will produce an exception on overflow.
IEEE 754 in fact does handle underflow. There's two problems:
The return value is 0 (or -1 for negative undreflow). The exception flag for underflow is set, but there's no way to get that in .NET.
This only occurs for the loss of precision when you get too close to zero. But you lost most of your precision way long before that. Whatever "precise" number you had is long gone - the operations are not reversible, and they are not precise.
So if you really do care about reversibility etc., stick to rational numbers. Neither decimal nor double will work, C# or not. If you're not that precise, you shouldn't care about underflows anyway - just pick the lowest reasonable number, and declare anything under that as "invalid"; may sure you're far away from the actual maximum precision - double.Epsilon will not help, obviously.
All you need is epsilon.
This is a "small number" which is small enough so you're no longer interested in.
You could use:
double epsilon = 1E-50;
and whenever one of your factors gets smaller than epislon you take action (for example treat it like 0.0)
The following test will fail in C#
Assert.AreEqual<double>(10.0d, 16.1d - 6.1d);
The problem appears to be a floating point error.
16.1d - 6.1d == 10.000000000000002
This is causing me headaches in writing unit tests for code that uses double. Is there a way to fix this?
There is no exact conversion between the decimal system and the binary representation of a double (see excellent comment by #PatriciaShanahan below on why).
In this case the .1 part of the numbers is the problem, it cannot be finitely represented in a double (like 1/3 can't be finitely represented exactly as a decimal number).
A code snippet to explain what happends:
double larger = 16.1d; //Assign closest double representation of 16.1.
double smaller = 6.1; //Assign closest double representation of 6.1.
double diff = larger - smaller; //Assign closest diff between larger and
//smaller, but since a smaller value has a
//larger precision the result will have better
//precision than larger but worse than smaller.
//The difference shows up as the ...000002.
Always use the Assert.Equal overload which takes a delta parameter when comparing doubles.
Alternatively if you really need exact decimal conversion, use the decimal data type, that has another binary representation and would return exactly 10 in your example.
Floatingoint numbers are an estimate of the actual value based on an exponent so the test fails correctly. If you require exact equivalence in two decimal numbers you may need to check out the decimal data type.
If you are using NUnit please use the Within option. Here can you find additional information: http://www.nunit.org/index.php?p=equalConstraint&r=2.6.2.
I agree with anders abel. There won't be a way to do this using a float number representation. In direct result of IEE 1985-754 only the numbers that can be represented by
can be stored and calculated with precisly (as long as the chosen bit number allows this).
For Example : 1024 * 1.75 * 183.375 / 1040.0675 <-- will be stored precisly
10 / 1.1 <-- wont be stored precisly
If you are hardly interested in exact representation of rational numbers you could write your own number-implementation using fractions.
This could be done by saving numerator, denominator and sign. Then operations like multiply, subtract, etc. need to be implemented (very hard to ensure good performance). A toString()-method could look like this (I assume cachedRepresentation, cachedDotIndex and cachedNumerator to be member-variables)
public String getString(int digits) {
if(this.cachedRepresentation == ""){
this.cachedRepresentation += this.positiveSign ? "" : "-";
this.cachedRepresentation += this.numerator/this.denominator;
this.cachedNumerator = 10 * (this.numerator % this.denominator);
this.cachedDotIndex = this.cachedRepresentation.Length;
this.cachedRepresentation += ".";
}
if ((this.cachedDotIndex + digits) < this.cachedRepresentation.Length)
return this.cachedRepresentation.Substring(0, this.cachedDotIndex + digits + 1);
while((this.cachedDotIndex + digits) >= this.cachedRepresentation.Length){
this.cachedRepresentation += this.cachedNumerator / this.denominator;
this.cachedNumerator = 10 * (this.cachedNumerator % denominator);
}
return cachedRepresentation;
}
This worked for me. At the operations itself with long numbers I got some problems with too small datatypes (usually I don't use c#). I think for an experienced c#-developer it should be no problem to implement this without problems of to small datatypes.
If you want to implement this you should do minifications of the fraction at initializing and before operations using euclids greatest-common-divider.
Non rational numbers can (in every case I know) be specified by a algorithm that comes as close to the exact representation as you want (and computer allows).
Is it safe to cast from double to decimal in the following manner in C#:
int downtimeMinutes = 90;
TimeSpan duration = TimeSpan.FromHours(2d);
decimal calculatedDowntimePercent = duration.TotalMinutes > 0?
(downtimeMinutes / (decimal)duration.TotalMinutes) * 100.0m : 0.0m;
If the answer is yes, then no fuss, I'll just mark as accepted.
In general, double -> decimal conversions aren't safe, because decimal has a smaller range.
However, as long as TotalMinutes is less than the maximum decimal value* it will be fine. This is true, because TimeSpan.MaxValue.TotalMinutes < (double)decimal.MaxValue (I believe TimeSpan uses a long internally.)
So: yes.
*: (79,228,162,514,264,337,593,543,950,335 minutes is 1.1×10^13 times the age of the universe)
No, in general, casting from double to decimal is not always safe:
[TestCase(double.MinValue)]
[TestCase(double.MaxValue)]
[TestCase(double.NaN)]
[TestCase(double.NegativeInfinity)]
[TestCase(double.PositiveInfinity)]
public void WillFail(double input)
{
decimal result = (decimal)input; // Throws OverflowException!
}
As OP clarified in a comment to the question, "safe" being "doesn't cause run time exceptions", the above shows that exceptions can occur when casting a double to a decimal.
The above is the generic answer many Googlers might've come here for. However, to also answer the specific question by OP, here's a strong indication that the code will not throw exceptions, even on edge cases:
[Test]
public void SpecificCodeFromOP_WillNotFail_NotEvenOnEdgeCases()
{
int downtimeMinutes = 90;
foreach (TimeSpan duration in new[] {
TimeSpan.FromHours(2d), // From OP
TimeSpan.MinValue,
TimeSpan.Zero,
TimeSpan.MaxValue })
{
decimal calculatedDowntimePercent = duration.TotalMinutes > 0 ?
(downtimeMinutes / (decimal)duration.TotalMinutes) * 100.0m : 0.0m;
}
}
Yes it is safe, because decimal has greater precision
http://msdn.microsoft.com/en-us/library/364x0z75(VS.80).aspx
The compiler will put in casts around the other non decimal numbers, but they'll all fit into decimal * (see caveat).
--
Caveat
Decimal is not a floating point type. Its mandate is to always uphold precision. Whereas a floating point number such as double (which I mostly use) makes a tradeoff on precision to accommodate very large numbers). Very large or very small numbers will not fit into decimal. So Lisa needs to ask herself if the magnitude of the operation is likely to be less than 28 significant digital digits. 28 significant digits are adequate for most scenarios.
Floating point is good for astronomically large or infintessimally small numbers... or operations inbetween that yield enough accuracy. I should look this up, but double is okay for plus or minus a few billion with accuracy of up to several decimal points (up to 7 or 8?).
in the sciences there's no point measuring beyond the accuracy of your equipment. In finance, often the logical choice is double because a double is computationally more efficient for most situations (sometimes they want a bit more accuracy, but the efficiency is not worth throwing away for something like decimal). In the end we all have to get pragmatic and map business needs to a digital domain. There are tools out there that have a dynamic number representation. Probably there are libraries in .net for the same. However, is it worth it? Sometimes it is. Often it's overkill.
Every time I use Math.Round/Floor/Ceiling I always cast to int (or perhaps long if necessary). Why exactly do they return double if it's always returning an integer.
The result might not fit into an int (or a long). The range of a double is much greater.
Approximate range of double: ±5.0 × 10−324 to ±1.7 × 10308
(Source)
I agree with Mark's answer that the result might not fit in a long, but you might wonder: what if C# had a much longer long type? Well, here's what happens in Python with it's arbitary-length integers:
>>> round(1.23e45)
1229999999999999973814869011019624571608236032
Most of the digits are "noise" from the floating-point rounding error. Perhaps part of the motivation for Round/Floor/Ceiling returning double in C# was to avoid the illusion of false precision.
An alternative explanation is that the .NET Math module uses code written in C, in which floor and ceil return floating-point types.
Range arguments aside, none of these answers addresses what, to me, is a fundamental problem with returning a floating point number when you really want an exact integer. It seems to me that the calculated floating point number could be less than or greater than the desired integer by a small round off error, so the cast operation could create an off by one error. I would think that, instead of casting, you need to apply an integer (not double) round-nearest function to the double result of floor(). Or else write your own code. The C library versions of floor() and ceil() are very slow anyway.
Is this true, or am I missing something? There is something about an exact representation of integers in an IEEE floating point standard, but I am not sure whether or not this makes the cast safe.
I would rather have range checking in the function (if it is needed to avoid overflow) and return a long. For my own private code, I can skip the range checking. I have been doing this:
long int_floor(double x)
{
double remainder;
long truncate;
truncate = (long) x; // rounds down if + x, up if negative x
remainder = x - truncate; // normally + for + x, - for - x
//....Adjust down (toward -infinity) for negative x, negative remainder
if (remainder < 0 && x < 0)
return truncate - 1;
else
return truncate;
}
Counterparts exist for ceil() and round() with different considerations for negative and positive numbers.
There is no reason given on the docs that I could find. My best guess is that if you are working with doubles, chances are you would want any operations on doubles to return a double. Rounding it to cast to an int was deemed by the language designer less common then rounding and keeping as a double.
You could write your own method that cast it to an int for you in about 2 lines of code, and much less work than posting a question on stack overflow...