[PD] "wrong" floating point values in pyext

Mathieu Bouchard matju at artengine.ca
Mon Feb 20 14:19:15 CET 2006


On Mon, 20 Feb 2006, Tim Blechmann wrote:

> IEEE 754 floating point numbers aren't able to encode 0.2 ... pd's print
> is rounding to 6 digits, python doesn't round, though ...

Python rounds too, but computes with 64-bit floats, just like Ruby. You 
see that Python only printed 11 significant digits, but with 64-bit floats 
you can have as many as 15, so there's rounding going on here.

 _ _ __ ___ _____ ________ _____________ _____________________ ...
| Mathieu Bouchard - tél:+1.514.383.3801 - http://artengine.ca/matju
| Freelance Digital Arts Engineer, Montréal QC Canada




More information about the Pd-list mailing list