<html><body><div style="color:#000; background-color:#fff; font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px"><div id="yui_3_16_0_1_1443041410138_3480">Oh man, I was absolutely dreading addressing all that loading mess, so I'm excited to see you're tackling it.</div><div id="yui_3_16_0_1_1443041410138_3450"><br></div><div id="yui_3_16_0_1_1443041410138_3416">Question:</div><div dir="ltr" id="yui_3_16_0_1_1443041410138_3415">what does it mean to "tag" an objectname with "abstraction"?</div><div id="yui_3_16_0_1_1443041410138_3449" dir="ltr"><br></div><div id="yui_3_16_0_1_1443041410138_3481" dir="ltr">Another question:</div><div id="yui_3_16_0_1_1443041410138_3519" dir="ltr">How were you planning to implement canvas-local loading? (Or did you have a plan in mind?)</div><div id="yui_3_16_0_1_1443041410138_3606" dir="ltr"><br></div><div id="yui_3_16_0_1_1443041410138_3607" dir="ltr">-Jonathan<br></div><div id="yui_3_16_0_1_1443041410138_3518" dir="ltr"><br></div><div id="yui_3_16_0_1_1443041410138_3517" dir="ltr"><br></div><div id="yui_3_16_0_1_1443041410138_3345"><span></span></div> <br><div class="qtdSeparateBR"><br><br></div><div style="display: block;" class="yahoo_quoted"> <div style="font-family: HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif; font-size: 16px;"> <div style="font-family: HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif; font-size: 16px;"> <div dir="ltr"> <font face="Arial" size="2"> On Wednesday, September 23, 2015 4:20 PM, IOhannes m zmölnig <zmoelnig@iem.at> wrote:<br> </font> </div> <br><br> <div class="y_msg_container">hi miller et all,<br><br>the current implementation of Pd's loading mechanism works as follows:<br><br>- each registered loader is asked whether they can handle a given<br>objectclass<br>- each loader in turn searches the entire (canvas-enhanced) path for<br>files matching their pattern (eg.g. "*.pd_linux", or "*.pd_lua")<br>- if all fails, Pd searches the entire (canvas-enhanced) path for<br>abstractions.<br><br>e.g.<br> /A/x.pd_linux<br> /B/x.pd_linux<br> /C/x.pd_linux<br> /A/x.pd_lua<br> /B/x.pd_lua<br> /C/x.pd_lua<br> /A/x.pd<br> /B/x.pd<br> /C/x.pd<br><br><br>this has a number of disadvantages, mainly:<br><br>- the associated loader has a higher priority then the path.<br>this means that it is nigh¹ impossible to override a binary external<br>(e.g. "*.pd_darwin") installed system-wide (e.g. in extra/) with a<br>user-provided alternative (e.g. a pd_lua implementation in the current<br>patch path).<br><br>- abstractions (arguably the most used external classes) have an extra<br>load time penalty, since each time a new (abstraction-provided) object<br>is created, Pd will first search the entire path for binaries,<br>lua-scripts and what not.<br><br><br>iirc, this has been discussed at length, and the proper solution for<br>this is to change the loading mechanism, so that paths have a higher<br>that loaders:<br>Pd shall iterate over each search-path and ask the loaders whether there<br>is something for them in there.<br><br>this way an abstraction in a "-path" enhanced searchpath is always found<br>early in the search, AND it can shadow an external in a later search path.<br><br>e.g.<br> /A/x.pd_linux<br> /A/x.pd_lua<br> /A/x.pd<br> /B/x.pd_linux<br> /B/x.pd_lua<br> /B/x.pd<br> /C/x.pd_linux<br> /C/x.pd_lua<br> /C/x.pd<br><br><br><br>so i have taken the liberty and implement that (i'll submit the patches<br>in a second).<br><br>here's a few observations:<br><br>compatibility with legacy loaders<br>===<br>the new implementation needs an API that tells the loaders where they<br>should look for.<br>rather than create a new API, i used the existing one and enhanced the<br>callback function with a new argument <path>, so we now have:<br> int loader_callback(t_canvas*c, char*name, char*path);<br><br>the good thing about this is that it is actually backward compatible (at<br>least with sane calling conventions, as on my tested linux system).<br>legacy loaders will simply ignore the <path> argument, and continue to<br>search their files in the entire path.<br>this will result in an overall performance degredation (while loading<br>objects), as the legacy loaders will call the entire search path for<br>each element in the search path :-(<br>the good news is that:<br>- the legacy loaders will continue to work!<br>- the performance degredation will only happen if legacy loaders are<br>involved.<br>- the performance degredation will be at *load time*, so should not be<br>too important<br>- there are only very few (known) loaders, so they should be fixed in no<br>time<br><br><br>abstraction loading<br>===<br>one big change is that abstractions are now loaded via a proper "loader"<br>(rather than via some special handling as fallback)<br>this implies that whenever the abstraction-loader wins (that is, a given<br>objectname can first be resolved via an abstraction), this objectname is<br>tagged as "abstraction".<br>when another object of the same name is created, the loader competition<br>will not take place any more, and instead the object will be loaded<br>immediately as "abstraction" (the current implementation then still<br>searches the entire path for the .pd file)<br>this has two side effects:<br>- until now it was possible to replace "future instances" of an<br>abstraction-object with externals.<br>e.g. imagine that [foo] is resolved via ~/pd-externals/foo.pd. if<br>someone installs a foo.pd_linux into ~/pd-externals/ during the Pd<br>session, then the next time someone creates [foo] it will be the external!<br>whether you like that behaviour or not, my implementation prevents it<br>(however it is still possible to change the actual abstraction; e.g. if<br>[foo] is resolved via /usr/lib/pd/extra/foo.pd and somebody saves a<br>foo.pd into ~/pd-externals/, then the next time someone creates [foo] it<br>will be the abstraction in ~/pd-externals/!)<br>- since this means that we don't need to search the entire path for<br>externals (that have not been there when we looked a second ago)<br>whenever we create another instance of our abstraction, patches<br>generally load faster (esp. when an abstraction is used often).<br>the speedup is not tremendous, about 3x; but still...<br><br><br>canvas path iterator<br>===<br>to ease the implementation (and to avoid code duplication), i introduced<br>an iterator, that will call a callback function for each searchpath of a<br>canvas.<br>this is now also used in canvas_open(), but might be useful elsewhere.<br><br><br>interactive use of declare<br>===<br>as a side effect (and not strictly related to the loading problem), i<br>fixed [declare] so that creating a [declare -path foo] would allow you<br>to *immediately* use the foo/ path.<br>i always found it *very* annoying that you had to close/open the<br>abstraction for [declare] to have an effect.<br>(this functionality is disabled while the patch is currently /loading/)<br><br><br>not done<br>===<br>what i wanted to do as well, but haven't gotten around was proper<br>canvas-local library loading, so you could do [declare -lib<br>foo/bla]+[bla] in one abstraction and [declare -lib moo/bla]+[bla] in<br>another abstraction and the two [bla] objectclasses would refer to<br>different externals.<br><br><br><br>hmm, that mail is already quite long. and i have forgotten what else i<br>wanted to say.<br><br>in any case, i would like to hear your opinion on all this.<br><br>in the meantime i'll submit the 4 patches of this implementation to the<br>sf tracker.<br><br>gfamds<br>IOhannes<br><br><br><br><br><br><br><br>¹ ah well, for certain use-cases one could disable the standard<br>searchpaths altogether; not.<br><br>_______________________________________________<br>Pd-dev mailing list<br><a ymailto="mailto:Pd-dev@lists.iem.at" href="mailto:Pd-dev@lists.iem.at">Pd-dev@lists.iem.at</a><br><a href="http://lists.puredata.info/listinfo/pd-dev" target="_blank">http://lists.puredata.info/listinfo/pd-dev</a><br><br><br></div> </div> </div> </div></div></body></html>