[PD] finite precision in basic math?

[IOhannes m zmoelnig]

On 07.01.19 13:24, Peter P. wrote:
> Hi,
> 
> when I send the numer 0.7 to both inlets of a [* ] object (yes, in
> correct order) the result is printed as 0.49
> If I now subtract [- 0.49] the result is -2.98023e-08
> While this is no major concern for me right now, I am still wondering
> which step in this simple calculation is triggering this. If I remember
> my CS lecture correctly it is the subtraction, right?

no. it's the representation of any floating point number.
neither "0.7" nor "0.49"  nor "x*x" (with x="0.7") can be represented
*exactly* in a base-2 floating point representation. (honestly, i'm just
making that up now without checking the representability of these exact
numbers; however, i'm pretty sure it *is* valid though).

because both numbers are inaccurately represented, they happen to be
different.

fgmasdr
IOhannes

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.puredata.info/pipermail/pd-list/attachments/20190107/0f2cc8a4/attachment.sig>