[PD] integer (mis)calculator

[Mathieu Bouchard]

Le 2011-12-08 à 00:13:00, Joe Newlin a écrit :

> Can someone explain what's going on in the attached example? I'm getting 
> 7.1 minus 7 equals .099. I need to split two-digit integers, so this 
> result is giving me problems.

in fractional powers of two,

1/10 = 1/16 + 1/32 + 1/256 + 1/512 + 1/4096 + 1/8192 + 1/65536 + ...

in fractional binary digits, it's written like 0.000110011001100110011... 
it's periodic. However, a float is never periodic, so, the digits are 
chopped.

  ______________________________________________________________________
| Mathieu BOUCHARD ----- téléphone : +1.514.383.3801 ----- Montréal, QC