[PD] strange behaviour for look up table operation

[Matt Barber]

Hi,

My guess is that the problem is a difference in the exact floating point
representation of each 0.8f which gets propagated through the
multiplication, which will unfortunately give you index trouble with these
very large indices.

The closest binary representation of 0.8 is:
0.800000012 = 13421773/2^24

Based on the content of your files, the [hsv2rgb] seems to be outputting
something like this, which gets printed as 0.8:
0.80000025 = 13421777/2^24

I'm not sure there's anything you can do about this, unfortunately.

On Wed, Nov 11, 2015 at 8:43 PM, mick mengucci <misturapura at gmail.com>
wrote:

> Hi lists,
> we are making a new abstraction in Pd to convert from color space RGB to
> CIE-Lch using a look up table. We map all the RGB triplets to an index of
> the LUT that points to the correct conversion to CIE-Lch.
>
> The problem is that when using [hsv2rgb] we get the same index number as
> inputting directly the rgb triplet but different result from the look up
> table.
>
> How is that possible???
>
> Here's a simplification of the patch to illustrate the problem (warning:
> there are three tables with 1.65814e+07 values)
>
> https://drive.google.com/file/d/0B6MlQyl-quuaYnEzQmtDOFFyckU
>
> I hope somebody will have the patience to explain this very strange
> behaviour of an apparently simple message to expression and look up table
> operation. Maybe the size of the table and of the index is the reason for
> it? We run Xubuntu and Pd 64bits.
>
> Cheers
>
> --
> mickmengucci.bandcamp.com
> facebook.com/mickmengucci3
> facebook.com/Labio.pt
> <http://WWW.MISTURAPURA.NET>MISTURAPURA.NET <http://WWW.MISTURAPURA.NET>
> pure mixture vibration since 1995
>
> _______________________________________________
> Pd-list at lists.iem.at mailing list
> UNSUBSCRIBE and account-management ->
> http://lists.puredata.info/listinfo/pd-list
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puredata.info/pipermail/pd-list/attachments/20151111/37557859/attachment-0002.html>