[PD-dev] libraries that crash in a double-precision runtime

[IOhannes m zmoelnig]

On 03.03.20 14:55, Thomas Grill wrote:
> Hi all,
> thanks to IOhannes for debugging.
> I have pushed fixes to https://github.com/grrrr/flext which should make flext (and flext-based objects) fully compatible with 64-bit DSP, that is, flext-based project should compile and run (or at least, not crash).
> 
> My question is now how to separate 32-bit DSP from 64-bit DSP binaries. There doesn't seem a separate file extension, right?

right.
there's an ongoing discussion at [900]

the gist is:

## current situation
pd-double uses "class_new64()" instead of "class_new()" (which is
reserved for registering sijngle-precision externals).
if a (single-precision) external calls "class_new()" in a
double-precision runtime, it will spit out a warning and return a
NULL-pointer.
vice versa, if a (double-precision) external calls "class_new64()" in a
single-precision runtime, it will spit out a warning and return a
NULL-pointer.
this is to protect externals from being loaded into a runtime of the
wrong floatsize.
to allow simple compilation of double-precision externals, "class_new"
is redefined (in m_pd.h) to "class_new64" if PD_FLOATSIZE==64.

i personally am convinced that this is a *required* protection of the
runtime.


## TODO
however, people are not entirely happy yet ([900]).
there's basically two classes of problems:
- distribution of packages that contain binaries for both float-sizes
- backward compatibility

### packages with both float-sizes
the current implementation doesn't solve the problem on how to
distribute binaries (e.g. via deken) either side-by-side (multiple
binary files) or in a single phat binary (that containes code for both
precisions).

the former would require a way to distinguish the filenames based on the
precision (that is: a filename-extension schema).
people have proposed something like "<library>.l_amd64-DP", which i
personally find rather ugly. a somewhat nicer schema might be
"<library>.float64.l_amd64" (which i would prefer, as it keeps the final
extension intact)

the latter (phat binaries that are compiled for both double- and
single-precision) is theoretically possible, but people have to jump
through a lot of hoops to make it work.
this should, and probably could be made simpler. see [1] for a
suggestion to mangle the entry-point (aka setup() function) of an external.
this could make compilation of phat externals much simpler...iff the
only public functions are the setup-functions.


### backward compatibility
the other problem is, that binaries compiled for double-precision use a
new symbol ("class_new64") which is not present in older versions of Pd
(Pd<<0.51).
This is of course not a problem for double-precision only binaries
(which you don't want to load anyhow in the olde single-precision runtime).
but if the binary contains code for *both* single- abd double-precision,
then you might expect to be able to load the single-precision variant in
legacy Pd's.

i think this can be solved by weak linking (aka: "optional symbols" that
are set to NULL if not available).
i have an experimental branch at [2] where i toyed with this idea today,
and so far it seems to work nicely.
the biggest caveat is, that not all OSs support weak linking, and of
course MicroSoft Windows does not.
so my branch uses a little extra magic to implement a replacement using
GetProcAddress(), but that requires to include <windows.h>, which
somewhat bloats m_pd.h


> How are two variants handled in deken?

deken (the cmdline tool) currently inspects the binary to see whether it
imports the "class_new" symbol (in which case it is a single-precision
binary) resp the "class_new64" symbol (in which case it is a
double-precision binary). if both symbols are imported, then the binary
is multi-precision.
this information is encoded into the filename, e.g. a deken package
"foobar[v1.2.3](Linux-ppc-64)(Linux-ppc-32).dek" contains PowerPC
binaries for both single and double-precision.

the deken-plugin (the downloader/installer) just uses the the filename
to determine whether a given search-result matches it's own
architecture/precision.


gmasdr
IOhannes

[900 ]https://github.com/pure-data/pure-data/issues/900
[1] https://lists.puredata.info/pipermail/pd-dev/2019-12/022214.html
[2] https://git.iem.at/pd/pure-data/-/tree/tests/double-API

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.puredata.info/pipermail/pd-dev/attachments/20200303/6b79c25c/attachment.sig>