[PD] symbol2list float error

[IOhannes m zmoelnig]

hi,

please read http://puredata.info/community/lists/Netiquette#Threads on
how to start new threads.


On 2010-06-02 13:47, Julian Villegas wrote:
> Hi everyone,
> 
> I'm trying to read data from a GPS receiver connected to a serial port in a MacBook (Mac OS 10.6.3) running Pd-extended (v. 0.41.4). The  GPS receiver sends floats with three decimal places, but I'm losing the last decimal place in the conversion process. I found that the truncation is being done at a [symbol2list] object inside a [convert.ascii2pd LF] abstraction (v 3.1), so if I print before and after this is what I get:
> 
> before: symbol 3731.428
> after: 3731.43
> 
> before: symbol 13956.320
> after: 13956.3
> 
> it seems to me that [symbol2list] is doing some rounding, how can I avoid it? the rest of the patch works like a charm but that last decimal place is crucial for this project. Could you guys help me?
>

[symbol2list] internally converts cstrings to doubles, which have higher
precision than Pd usually works with.
it then converts these doubles to Pd's native numbers (single floats).

so i doubt, that there is a rounding problem on this side.

another thing to notice is, that Pd does some rounding when it
_displays_ numbers.
so you might only thing that there is a (rounding) problem, even though
there is none.
try putting a [wrap] object after the symbol2float conversion to only
see the part after the decimal point.

fgmadr
IOhannes

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3636 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.puredata.info/pipermail/pd-list/attachments/20100602/02841a37/attachment.bin>