[PD] Large File Support on Linux

[Ivica Ico Bukvic]

I completely agree with your example, Miller, as far as pd vanilla is
concerned. OTOH pd-extended ships with all its externals compiled with each
version, so this would be for the most part a non-issue unless:

1) users A or B are using different versions of pd (which could be
elaborated upon in the documentation when sharing patches which is rather
common in other software environments), or

2) if A is using custom externals that are not found in pd-extended (in
which case B would not want to use those anyways unless both A and B are
using the same platform, which is the only scenario where it would make
sense to keep this as a workaround)

Either way FWIW I still feel this is too much of a workaround for little
gain and potentially a lot of headache down the road.

Best wishes,

Ico

> -----Original Message-----
> From: Miller Puckette [mailto:msp at ucsd.edu]
> Sent: Monday, March 18, 2013 2:47 PM
> To: Ivica Ico Bukvic
> Cc: 'IOhannes m zmoelnig'; pd-list at iem.at
> Subject: Re: [PD] Large File Support on Linux
> 
> To answer Ico's question, the trouble I forsee is musician A gives
> musician B a patch, containing compiled externs - and then musician B
> runs it and gets a crash instead of music.  Sub-optimal in my opinion :)
> 
> I now think that it should be OK to use open64() only in the file
d_soundfile.c
> and only for linux - so putting at the head of the file,
> 
> #ifdef __linux__
> #define _LARGEFILE64_SOURCE
> #endif
> 
> ... then, because opening soundfiles currently uses open_via_path, simply
> closing the file and re-opening it via open64().  There's a similar hack
> to open "binbufs" via paths in the function binbuf_read_via_canvas().
> 
> There are two other festering problems in open_via_path() that all should
> probably be fixed in one go by defining an updated function call:
> 
> 1.  externs further down the path are chosen in front of abstractions that
> should be taken instead;
> 
> 2.  open_via_path isn't thread safe - d_soundfile.c could call it at the
same
> moment someone in the "Pd" thread is changing the path.  This should
> almost
> never hapen but should be fixed.
> 
> This is a big enough change that I think it should wait for 0.45, whereas
the
> hack described above could go in right now as a local bug-fix.
> 
> BTW I've got a couple of other bug-fixes underway; wil push to git now.
> 
> cheers
> Miller
> 
> Oe Mon, Mar 18, 2013 at 12:47:47PM -0400, Ivica Ico Bukvic wrote:
> > > the problem with that is, that s_path implements an API available for
> > > externals.
> > > if open_via_path() returns a filehandle for an LFS file, and the
> > > external has been compiled without LFS, the filehande will be
> > > incompatible. see [1] for a discussion.
> > >
> > > i guess the only clean way to solve that, is to provide a complete
> > > wrapper around the file API, so an external has functions guaranteed
> > > to work with the filehandle returned by Pd.
> > > currently there are sys_open()/sys_close() and its f*-variants.
> > > but we would need at least sys_(f)seek and sys_(f)stat, but
> > > preferrably a complete wrapper around LFS-compatible file functions
[2].
> > >
> > > all this functionality (including the handilng of UTF filenames on
> > > W32) is not really Pd-related, and could well be factored out into a
> > > separate library.
> >
> > Thanks for the clear explanation of the matter, IOhannes.
> >
> > Why not simply recompile externals after fixing s_path? Pd-extended
> already
> > comes with most externals recompiled for every new release and adding
> legacy
> > stuff just creates more confusion in maintaining the source down the
road.
> > In other words, FWIW and IMHO I think there is way too much effort the
> > community is trying to pour into binary compatibility for a system that
> > clearly begs for further enhancements in the core API.
> >
> > Best wishes,
> >
> > Ico
> >
> >
> > _______________________________________________
> > Pd-list at iem.at mailing list
> > UNSUBSCRIBE and account-management ->
> http://lists.puredata.info/listinfo/pd-list