[PD] Large File Support on Linux

[IOhannes m zmoelnig]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 2013-03-18 13:10, IOhannes m zmoelnig wrote:
> On 2013-03-18 11:18, Charles Goyard wrote:
>> Hi,
> 
>> thanks for taking that up.
> 
>> On my machine (archlinux, Linux 3.7.10-1-ARCH i686 GNU/Linux, 
>> glibc 2.17), changing the LARGEFILE define to __linux__ (or 
>> __gnu_linux__ yields:
> 
> 
> nonono. you MUST have "_LARGEFILE64_SOURCE" defined before
> including unistd.h, else you won't have LFS support. there are
> configure checks that test whether doing defining 
> _LARGEFILE64_SOURCE makes sense on your system.
> 
> having said that, i just noticed that these checks seem to have
> gone from configure(.ac) during the 0.43 rewrite of the
> buildsystem. so most likely this is the source of the bug.
> 

so i rechecked and things are a bit more complicated than i originally
thought when it comes to large file support (LFS).

it is (well, should be) easy enough to open/read/write large (>2GB)
files within Pd.
simply using "_LARGEFILE64_SOURCE" or "_FILE_OFFSET_BITS=64" is the
recommended way.
those defines have to be set whenever unistd is included for
open/close/lseek/...

now Pd uses some magic in d_soundfile.c (e.g. redefining "open()" to
"open64()"), but that doesn't really help at all, since the files are
not opened in d_soundfile.c but instead are opened by open_via_path()
in s_path.c. open_via_path() currently does not handle LFS.

the easy way would obviously be to move LFS into s_path.c.

the problem with that is, that s_path implements an API available for
externals.
if open_via_path() returns a filehandle for an LFS file, and the
external has been compiled without LFS, the filehande will be
incompatible. see [1] for a discussion.

i guess the only clean way to solve that, is to provide a complete
wrapper around the file API, so an external has functions guaranteed
to work with the filehandle returned by Pd.
currently there are sys_open()/sys_close() and its f*-variants.
but we would need at least sys_(f)seek and sys_(f)stat, but
preferrably a complete wrapper around LFS-compatible file functions [2].

all this functionality (including the handilng of UTF filenames on
W32) is not really Pd-related, and could well be factored out into a
separate library.


fgasd,
IOhannes


[1] http://ohse.de/uwe/articles/lfs.html
[2] http://opengroup.org/platform/lfs.html
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAlFHDpoACgkQkX2Xpv6ydvQ+wACeMLeUFh9Bgfor7SV4YvcZokwL
MhwAn23vz/Zl6xpp+/EWQPvWrW8gyEm8
=AE4m
-----END PGP SIGNATURE-----