[PD] integer (mis)calculator

[katja]

On Thu, Dec 8, 2011 at 1:06 PM, Eduardo Flores Abad
<mail at eduardoflores.de> wrote:
> Hi Katja,
>
> tries to round this way:
>
> 100 * (7.1 - 7)   / 100  =  0.1



Did you try that, Eduardo? In Pd, you get:

100 * (7.1 - 7) / 100 = 0.0999999

But:

((100 * 7.1) - (100 * 7)) / 100 = 0.1

Or:

((10 * 7.1) - (10 * 7)) / 100 = 0.1

That is:

(71 - 70) / 10 = 0.1


So this is what Roman already suggested: do subtraction (or addition)
with integers, to be sure they're exact (up till 2^24), then scale if
required.

The original quest was to split two digits, which can be done like so:

truncate(71/10) = 7
71 - (truncate(71/10))*10 = 1

Where truncate() is done with the [i ] object. This works for all
two-digit ints, for  more digits there would be similar solutions.

The main point is: integer subtraction and addition is exact, and
floats can represent integers with exactness up to a certain point. A
single precision float can represent integers up till 2^24 with
exactness, that is 16,777,216, more than 16 million. A 64 bit float
('double') can represent integers up till 2^53, that is
9,007,199,254,740,992 (more than you'll need in dsp).

By the way, many fractions can be represented with exactness in both
decimal and binary floating point format. This includes values in the
form 1/2^n (1/2, 1/4, 1/8 etc.).



Katja