[PD] integer (mis)calculator

[Roman Haefeli]

Hi Joe


Pd uses 32 bit floating point number format which has (as any other
number format) some limitations. Not every number can be exactly
expressed by this format. '7.1' seems to be such a number. 

In case you work with only one digit after the decimal point, I suggest
to multiply your number inputs by 10 so that they are all integers. Then
perform your calculations in this scale. In the end you can downscale
your results again with [/ 10]. The reason is that all integers up to
2^23 (or was it 2^24?) can be represented _exactly_ by 32 bit float. 

BTW: This topic has extensively been discussed several times on this
list. You may find more information in the Pd-list archives.


Roman


On Thu, 2011-12-08 at 00:13 -0800, Joe Newlin wrote:
> Can someone explain what's going on in the attached example? I'm
> getting 7.1 minus 7 equals .099. I need to split two-digit integers,
> so this result is giving me problems.
> 
> Thanks,
> 
> JN
> 
> -- 
> www.joenewlin.net
> www.twitter.com/joe_newlin
> 
> 
> _______________________________________________
> Pd-list at iem.at mailing list
> UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list