[PD] "wrong" floating point values in pyext
Mathieu Bouchard
matju at artengine.ca
Mon Feb 20 14:19:15 CET 2006
On Mon, 20 Feb 2006, Tim Blechmann wrote:
> IEEE 754 floating point numbers aren't able to encode 0.2 ... pd's print
> is rounding to 6 digits, python doesn't round, though ...
Python rounds too, but computes with 64-bit floats, just like Ruby. You
see that Python only printed 11 significant digits, but with 64-bit floats
you can have as many as 15, so there's rounding going on here.
_ _ __ ___ _____ ________ _____________ _____________________ ...
| Mathieu Bouchard - tél:+1.514.383.3801 - http://artengine.ca/matju
| Freelance Digital Arts Engineer, Montréal QC Canada
More information about the Pd-list
mailing list