[PD-dev] d_soundfile.c MAXBUFSIZE

[Mathieu Bouchard]

On Tue, 28 Feb 2006, David Plans Casal wrote:

> #define MAXBUFSIZE 16777216 /* arbitrary; just don't want to hang malloc
> */
> I'm not sure about the 'hanging malloc' thing...

Maybe a better word than hanging would be flooding, as in taking too much
RAM. But there is another reason not stated, so it's not that arbitrary.
 
16777216, which is 2^24, is the last contiguous integer representable
in the float32 format. 16777217 (and following odd integers) get rounded 
to the closest even integer. starting at 2^25 you only get multiples of 4 
(which is 2^2) and so on. note that 2^25/2^2 = 2^23 which is the degree of 
precision of float32.

This means that [tabread~] doesn't do what you want starting at 16777216; 
it also means that [tabread4~] does exactly the same as [tabread~] 
starting at 2^23=8388608, which is not what you want, and that appears 
gradually, because between 2^22 and 2^23 you only have steps of 0.5, 
between 2^21 and 2^22 you only have steps of 0.25, etc.

There are still some objects that can handle big arrays, so the limit
should be configurable at run time... but it's not.

> does it mean that enlarging that buffer size could screw with memory
> allocation because it we might 'run out'?

You're allowed to malloc something close to 2^30 bytes = 2^28 float32's in
one chunk if you have enough RAM. In 64-bit mode you would be able to
malloc more, but pd arrays don't work in 64-bit mode.

If Pd used float64 you wouldn't have problems with tabread~ because 
float64 contains all integers up to 2^53. However Pd doesn't support 
float64 yet, and once it does, you can't use float32 anymore, so all your 
arrays will take double the RAM. To support multiple number formats 
in signals would be quite a overhaul of Pd. To support them in arrays 
only, would be easier but still quite a bit of work. (But then you would 
be able to store your values in int16 format, which is half of the RAM of 
float32!)

> AFAICT, 16777216 is about 6.3 minutes. If I need more like 26460000 (10m)
> allocation, could I just alter MAXBUFSIZE and recompile?

yes.

> Or is there a saner way of doing it, so that I don't end up with a
> 'custom' pd?

no... well, do you have an hex editor? ;-)

 _ _ __ ___ _____ ________ _____________ _____________________ ...
| Mathieu Bouchard - tél:+1.514.383.3801 - http://artengine.ca/matju
| Freelance Digital Arts Engineer, Montréal QC Canada