[PD-dev] how to load shared code for libraries

[IOhannes m zmölnig]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09/26/2011 06:52 PM, Hans-Christoph Steiner wrote:
> 
> IOhannes and I were discussing how to load shared code for a libdir on
> IRC.  The sticking point is that Mac OS X seems to require dylibs to
> have a hard-coded path.  IOhannes posted a test lib to try to work out
> how to do it using DYLD_LIBRARY_PATH:
> 
> http://iem.at/~zmoelnig/OSX/test.tgz
> 
> So here are my results from Mac OS X 10.5.8:
> 
> creating a [test] object:
> -------------------------
> 
> /private/tmp/test/tmp/test.pd_darwin:
> dlopen(/private/tmp/test/tmp/test.pd_darwin, 10): no suitable image
> found.  Did find:
>    /private/tmp/test/tmp/test.pd_darwin: unknown required load command
> 0x80000022
> test
> ... couldn't create
> 
> 
> hans at dhcp-10 tmp $ ~/code/pd-extended.git/src/pd -noprefs -lib test
> -------------------------------------------------------------------
> 
> ./test.pd_darwin: dlopen(./test.pd_darwin, 10): no suitable image
> found.  Did find:
>     ./test.pd_darwin: unknown required load command 0x80000022
> test: can't load library
> 


just to clarify a bit:
the binaries posted under the link above were compiled under OSX-10.6,
without any precautions for backward compatibility.
they don't work on 10.5, nor 10.4

i also put the same set of binaries online, but this time compiled under
10.4, and they work and function as expected under 10.4 and 10.6 (and i
expect them to work on 10.5 as well)



on w32, the search paths for dll's should be modifiable via the %PATH%
environment, whereas on linux (and most i think BSD and hurd) the envvar
in question is LD_LIBRARY_PATH




after the technical details, let's step back and have a look at what
this is all about:
the main motivation is to be able to bundle shared code and 3rd party
libraries with externals.

the dynamic linkers of the various OSs, will automatically search for
dylinked libraries your external depends on, in various places (e.g.
/usr/lib).

this is great if you have an easy way to install the needed library into
this place (e.g. a dependy-informed installer system like debian's "apt").
it's not so great, if you don't have such a system, and the user is
expected to find and download the given dependencies and put them into
your system's search path.
even if an external ships with all dependencies, they still have to be
installed in the "correct" places, which makes installation more
complicated than just dragging a folder onto your desktop.
why?

because your OS's dynamic linker doesn't look for 3rd party libraries in
weird places like a folder on your desktop.

now the idea, hans and me came up with, was to make Pd tell the OS to
look into those places.
e.g. if library "foo.pd_darwin" gets installed into ~/Library/Pd/foo/,
then it would be nice, if the dynamic linker would search for any
depending libraries in the ~/Library/Pd/foo/ directory.
likewise "bar.pd_linux" (installed in ~/my-externals/pub/) might have
it's dependencies searched for in ~/my-externals/pub/.

if both "foo" and "bar" depend on "libdudelsack", and both provide a
"libdudelsack.dll" in their homedirectories, then of course we want both
libraries to choose "their" correct one.

so the idea is, to modify the automatic search path of the dynamic
linker whenever a library is loaded.

example:
the user installs the "frotzel" library, by downloading it, and
unzipping it into /usr/local/pd-externals/

now, if the user requests a new object [frotzel], Pd will search for a
matching binary and eventually find frotzel.l_ia64 in
/usr/local/pd-externals/frotzel/
before dlopen()ing this binary, it will read the LD_LIBRARY_PATH
variable, push it to some stack, and then add
/usr/local/pd-externals/frotzel/ at the beginning of the LD_LIBRARY_PATH.
it will then dlopen() the frotzel.l_ia64 binary, and the linker
discovers, that it has to load the libknurbel.so library, which it will
(magically) find in /usr/local/pd-externals/frotzel/
because all dependencies are fullfilled, [frotzel] will successfully be
loaded and will work.
after fortzel.l_ia64 has been successfully loaded, we no longer need to
search for libraries in /usr/local/pd-externals/frotzel/, and therefore
the original content of LD_LIBRARY_PATH is restored (popped from the stack).


the exact name of "LD_LIBRARY_PATH" is system dependent, but it seems
that LD_LIBRARY_PATH, DYLD_LIBRARY_PATH and PATH cover most systems.


fasdmrt
IOhannes








-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk6Ax4MACgkQkX2Xpv6ydvTbrQCePMZCWLbqBNAGe18J5Lbwj2a4
aLIAn25HcmduV4mJgMUG+dH/XOJHWjCL
=07As
-----END PGP SIGNATURE-----