[PD] Large File Support on Linux

[Miller Puckette]

To answer Ico's question, the trouble I forsee is musician A gives
musician B a patch, containing compiled externs - and then musician B
runs it and gets a crash instead of music.  Sub-optimal in my opinion :)

I now think that it should be OK to use open64() only in the file d_soundfile.c
and only for linux - so putting at the head of the file,

#ifdef __linux__
#define _LARGEFILE64_SOURCE
#endif

... then, because opening soundfiles currently uses open_via_path, simply
closing the file and re-opening it via open64().  There's a similar hack
to open "binbufs" via paths in the function binbuf_read_via_canvas().

There are two other festering problems in open_via_path() that all should
probably be fixed in one go by defining an updated function call:

1.  externs further down the path are chosen in front of abstractions that
should be taken instead;

2.  open_via_path isn't thread safe - d_soundfile.c could call it at the same
moment someone in the "Pd" thread is changing the path.  This should almost
never hapen but should be fixed.

This is a big enough change that I think it should wait for 0.45, whereas the
hack described above could go in right now as a local bug-fix.

BTW I've got a couple of other bug-fixes underway; wil push to git now.

cheers
Miller

Oe Mon, Mar 18, 2013 at 12:47:47PM -0400, Ivica Ico Bukvic wrote:
> > the problem with that is, that s_path implements an API available for
> > externals.
> > if open_via_path() returns a filehandle for an LFS file, and the
> > external has been compiled without LFS, the filehande will be
> > incompatible. see [1] for a discussion.
> > 
> > i guess the only clean way to solve that, is to provide a complete
> > wrapper around the file API, so an external has functions guaranteed
> > to work with the filehandle returned by Pd.
> > currently there are sys_open()/sys_close() and its f*-variants.
> > but we would need at least sys_(f)seek and sys_(f)stat, but
> > preferrably a complete wrapper around LFS-compatible file functions [2].
> > 
> > all this functionality (including the handilng of UTF filenames on
> > W32) is not really Pd-related, and could well be factored out into a
> > separate library.
> 
> Thanks for the clear explanation of the matter, IOhannes.
> 
> Why not simply recompile externals after fixing s_path? Pd-extended already
> comes with most externals recompiled for every new release and adding legacy
> stuff just creates more confusion in maintaining the source down the road.
> In other words, FWIW and IMHO I think there is way too much effort the
> community is trying to pour into binary compatibility for a system that
> clearly begs for further enhancements in the core API.
> 
> Best wishes,
> 
> Ico
> 
> 
> _______________________________________________
> Pd-list at iem.at mailing list
> UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list