[PD] floating-point question

[Mathieu Bouchard]

Le 2012-02-19 à 21:52:00, Hans-Christoph Steiner a écrit :
> On Feb 17, 2012, at 4:27 PM, Mirko Petrovich wrote:
>> I'm having a problem with a patch doing some very simple math. The problem is that float 0.1 represents as 0.0999985 in some cases.
> That's a problem with floating point calculations on computers, unfortuantely.  Hard to work around that.

0.1 is usually represented as exactly
   13421773/134217728 = 0.100000001490116...
   where 134217728 = pow(2,27)

For various reasons, the numerator can be a few units up or down :
   13421772/134217728 = 0.0999999940395355...
   13421769/134217728 = 0.0999999716877937...
   13421776/134217728 = 0.1000000238418580...

As you can see, 0.0999985 is way off. You get that kind of value with a 
numerator around 13421571. That's 202 notches off !

If you sum together a lot of 0.001 values to make a 0.1, the error 
accumulates. It's more accurate to count using whole numbers, and divide 
the whole number each time.

It's sometimes important to cut down on accuracy for the benefit of 
efficiency, but doing audio on today's laptops, you will probably not 
encounter them. However, tablets and phones often have slow float 
calculators.

  ______________________________________________________________________
| Mathieu BOUCHARD ----- téléphone : +1.514.383.3801 ----- Montréal, QC