[PD-dev] loading classes: search by directory rather than extension

Jonathan Wilkes jancsika at yahoo.com
Wed Sep 23 23:09:45 CEST 2015


And a question to make sure I understand what you're proposing:
Inside "foo.pd", I have a single object [bar]
"foo.pd" lives in directory /home/blah
The only other file in /home/blah is "bar.pd"
Under your revision of the loading mechanism, when I load "foo.pd" is [bar] guaranteed to be loaded from /home/blah/bar.pd ?
-Jonathan

 


     On Wednesday, September 23, 2015 5:05 PM, Jonathan Wilkes <jancsika at yahoo.com> wrote:
   

 Oh man, I was absolutely dreading addressing all that loading mess, so I'm excited to see you're tackling it.
Question:what does it mean to "tag" an objectname with "abstraction"?
Another question:How were you planning to implement canvas-local loading?  (Or did you have a plan in mind?)
-Jonathan


 


     On Wednesday, September 23, 2015 4:20 PM, IOhannes m zmölnig <zmoelnig at iem.at> wrote:
   

 hi miller et all,

the current implementation of Pd's loading mechanism works as follows:

- each registered loader is asked whether they can handle a given
objectclass
- each loader in turn searches the entire (canvas-enhanced) path for
files matching their pattern (eg.g. "*.pd_linux", or "*.pd_lua")
- if all fails, Pd searches the entire (canvas-enhanced) path for
abstractions.

e.g.
 /A/x.pd_linux
 /B/x.pd_linux
 /C/x.pd_linux
 /A/x.pd_lua
 /B/x.pd_lua
 /C/x.pd_lua
 /A/x.pd
 /B/x.pd
 /C/x.pd


this has a number of disadvantages, mainly:

- the associated loader has a higher priority then the path.
this means that it is nigh¹ impossible to override a binary external
(e.g. "*.pd_darwin") installed system-wide (e.g. in extra/) with a
user-provided alternative (e.g. a pd_lua implementation in the current
patch path).

- abstractions (arguably the most used external classes) have an extra
load time penalty, since each time a new (abstraction-provided) object
is created, Pd will first search the entire path for binaries,
lua-scripts and what not.


iirc, this has been discussed at length, and the proper solution for
this is to change the loading mechanism, so that paths have a higher
that loaders:
Pd shall iterate over each search-path and ask the loaders whether there
is something for them in there.

this way an abstraction in a "-path" enhanced searchpath is always found
early in the search, AND it can shadow an external in a later search path.

e.g.
 /A/x.pd_linux
 /A/x.pd_lua
 /A/x.pd
 /B/x.pd_linux
 /B/x.pd_lua
 /B/x.pd
 /C/x.pd_linux
 /C/x.pd_lua
 /C/x.pd



so i have taken the liberty and implement that (i'll submit the patches
in a second).

here's a few observations:

compatibility with legacy loaders
===
the new implementation needs an API that tells the loaders where they
should look for.
rather than create a new API, i used the existing one and enhanced the
callback function with a new argument <path>, so we now have:
  int loader_callback(t_canvas*c, char*name, char*path);

the good thing about this is that it is actually backward compatible (at
least with sane calling conventions, as on my tested linux system).
legacy loaders will simply ignore the <path> argument, and continue to
search their files in the entire path.
this will result in an overall performance degredation (while loading
objects), as the legacy loaders will call the entire search path for
each element in the search path :-(
the good news is that:
- the legacy loaders will continue to work!
- the performance degredation will only happen if legacy loaders are
involved.
- the performance degredation will be at *load time*, so should not be
too important
- there are only very few (known) loaders, so they should be fixed in no
time


abstraction loading
===
one big change is that abstractions are now loaded via a proper "loader"
(rather than via some special handling as fallback)
this implies that whenever the abstraction-loader wins (that is, a given
objectname can first be resolved via an abstraction), this objectname is
tagged as "abstraction".
when another object of the same name is created, the loader competition
will not take place any more, and instead the object will be loaded
immediately as "abstraction" (the current implementation then still
searches the entire path for the .pd file)
this has two side effects:
- until now it was possible to replace "future instances" of an
abstraction-object with externals.
e.g. imagine that [foo] is resolved via ~/pd-externals/foo.pd. if
someone installs a foo.pd_linux into ~/pd-externals/ during the Pd
session, then the next time someone creates [foo] it will be the external!
whether you like that behaviour or not, my implementation prevents it
(however it is still possible to change the actual abstraction; e.g. if
[foo] is resolved via /usr/lib/pd/extra/foo.pd and somebody saves a
foo.pd into ~/pd-externals/, then the next time someone creates [foo] it
will be the abstraction in ~/pd-externals/!)
- since this means that we don't need to search the entire path for
externals (that have not been there when we looked a second ago)
whenever we create another instance of our abstraction, patches
generally load faster (esp. when an abstraction is used often).
the speedup is not tremendous, about 3x; but still...


canvas path iterator
===
to ease the implementation (and to avoid code duplication), i introduced
an iterator, that will call a callback function for each searchpath of a
canvas.
this is now also used in canvas_open(), but might be useful elsewhere.


interactive use of declare
===
as a side effect (and not strictly related to the loading problem), i
fixed [declare] so that creating a [declare -path foo] would allow you
to *immediately* use the foo/ path.
i always found it *very* annoying that you had to close/open the
abstraction for [declare] to have an effect.
(this functionality is disabled while the patch is currently /loading/)


not done
===
what i wanted to do as well, but haven't gotten around was proper
canvas-local library loading, so you could do [declare -lib
foo/bla]+[bla] in one abstraction and [declare -lib moo/bla]+[bla] in
another abstraction and the two [bla] objectclasses would refer to
different externals.



hmm, that mail is already quite long. and i have forgotten what else i
wanted to say.

in any case, i would like to hear your opinion on all this.

in the meantime i'll submit the 4 patches of this implementation to the
sf tracker.

gfamds
IOhannes







¹ ah well, for certain use-cases one could disable the standard
searchpaths altogether; not.

_______________________________________________
Pd-dev mailing list
Pd-dev at lists.iem.at
http://lists.puredata.info/listinfo/pd-dev


   

  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puredata.info/pipermail/pd-dev/attachments/20150923/395a9fdd/attachment.html>


More information about the Pd-dev mailing list