[PD-dev] new loader replacement progress report
Hans-Christoph Steiner
hans at at.or.at
Wed Aug 25 17:45:21 CEST 2010
There is a lot to grapple with here, I've started to try to make sense
of the work you've done so far, and in the process I'm throwing things
in the wiki:
http://puredata.info/dev/PdSearchPath
http://puredata.info/dev/PdNamespaces
So far its making sense, here's one perhaps naive idea of how to
structure things: There are multiple forms for storing objects on the
filesystem, including abstractions, single- and multi-object library
files, etc. Then there is the representation of what objects are
assigned to which symbols. These don't need to be the same thing. Each
patch has its own table of which symbols are attached to which
objects, and this can be separate from which objects are actually
loaded into memory.
One idea for handling this is to have separate tables of symbols for
which object files are loaded, versus which symbols are tied to which
objects for a given patch. It seems for this to be workable, there
would need to be a "loaded" table, then each patch would generate its
table from the loaded table. Then if an object isn't in the loaded
table, the loaded table handler would go and try to load it.
.hc
On Jul 18, 2010, at 3:41 AM, Claude Heiland-Allen wrote:
> Hi everyone,
>
> I've been working on some algorithms to eventually replace the way
> Pd finds and loads and creates objects. Sorry for the length of
> this mail, but you can skip some sections if you don't care about
> implementation details and/or user interface details :)
>
>
> Table of Contents
> * Specifications
> * Example Loaders
> * Clearer Error Messages
> * Test Inputs
> * Test Outputs
>
>
> Specifications
> --------------
>
> As agreed by everyone so far, the new load order will be roughly:
>
> for path in paths do -- the core does this bit
> for loader in loaders do
> loader(path, libray, object)
>
> instead of the old, problematic:
>
> for loader in loaders do
> for path in paths do -- the loader does this bit
> loader(path, library, object)
>
> I restructured the loader plugin mechanism to be (pseudocode):
>
> typedef err libloadfun(symbol *dir, symbol *lib);
> typedef err objloadfun(symbol *dir, symbol *lib, symbol *obj);
> struct loader {
> symbol *name;
> float priority;
> libloadfun *libfun;
> objloadfun *objfun;
> }
>
>
> Example Loaders
> ---------------
>
> -- binary loader is super simple
> Loader.new("bin", 70,
> function(dir, lib) return loadbin(dir, lib) end,
> function(dir, lib, obj) return loadbin(dir, obj) end)
>
> -- directory loader is needed if there is no main library file
> -- the core code checks the results to see if it succeeded
> Loader.new("dir", 10,
> function(dir, lib)
> if dir:sub(-1 - #lib, -1) == ("/" .. lib) then
> return function() end
> else
> return nil, "dir doesn't match lib"
> end
> end,
> nil -- directory loader can't load objects
> )
>
> -- the abstraction loader is quite complicated
> -- if the file exists it creates a callback that
> -- registers the class, with a constructor that
> -- runs the abstraction itself
> -- a Context here is essentially a 'canvas'
> Loader.new("txt", 30,
> nil, -- abstractions cannot register new classes
> function(dir, lib, obj)
> local src = dir .. "/" .. obj .. ".txt.lua"
> local f, err = fileexists(src)
> if f then
> local ldir, lobj = dir, obj
> return function()
> register(lobj, src, function(o, arguments)
> Context.push(Context.new({ ldir }))
> runtxt(ldir, lobj)
> Context.pop()
> end, src)
> end
> else
> return nil, err
> end
> end)
>
> The current implementation is entirely in Lua, in an unpublished git
> repository. I'll upload it somewhere if there is any interest.
>
>
> Clearer Error Messages
> ----------------------
>
> The main upshot so far is clearer error messages instead of random
> possibly-working-but-perhaps-not implementation defined behaviour
> when more than one library defines the same name:
>
>
> OBJECT [obj4]
> obj4 ... couldn't create (ambiguous)
> (found in: 'lib4')
> (found in: 'lib3')
> (found in: 'lib1')
> (found in: 'lib5')
> (found in: 'lib2')
> disambiguate by [import lib2/obj4]
> replacing lib2 by the library you want to import from
>
> OBJECT [obj2]
> obj2 ... couldn't create (ambiguous qualified import)
> (imported: 'lib2/obj2')
> (imported: 'lib1/obj2')
> disambiguate by removing extra qualified imports
>
>
> The error messages for objects that are not found are better too:
>
>
> OBJECT [obj8]
> obj8 ... couldn't create (no such object)
> (looked in: 'lib4')
> (looked in: 'vanilla')
> (looked in: 'lib3')
> (looked in: 'lib1')
> (looked in: 'lib5')
> (looked in: 'lib2')
>
> OBJECT [lib1/obj8]
> lib1/obj8 ... couldn't create (no such object in library)
>
> OBJECT [lib6/obj1]
> lib6/obj1 ... couldn't create (no such library)
>
>
> Note for that last [libc6/obj1] example: if libc6 actually did exist
> in the path but was not imported, the error would currently be the
> same. I want to make it so that [somelibrary/someobject] would work
> even when [somelibrary] is not imported (but such "implicit imports"
> would be deprecated, and warnings emitted).
>
>
> Test Inputs
> -----------
>
> The test input directory structure (.bin.lua approximates .pd_linux
> etc, .txt.lua approximates .pd patches/abstractions):
> ----8<----
> .:
> global prj1
>
> ./global:
> lib1 lib2 lib3 lib3.bin.lua lib4 lib5.bin.lua vanilla
>
> ./global/lib1:
> obj1.txt.lua obj3.txt.lua obj4.txt.lua obj5.txt.lua obj7.txt.lua
> obj2.txt.lua obj4.bin.lua obj5.bin.lua obj6.bin.lua
>
> ./global/lib2:
> lib2.bin.lua obj2.txt.lua obj4.bin.lua obj5.bin.lua obj6.bin.lua
> obj1.txt.lua obj3.txt.lua obj4.txt.lua obj5.txt.lua obj7.txt.lua
>
> ./global/lib3:
> obj1.txt.lua obj3.txt.lua obj4.txt.lua obj5.txt.lua obj7.txt.lua
> obj2.txt.lua obj4.bin.lua obj5.bin.lua obj6.bin.lua
>
> ./global/lib4:
> obj1.txt.lua obj3.txt.lua obj4.txt.lua obj5.txt.lua obj7.txt.lua
> obj2.txt.lua obj4.bin.lua obj5.bin.lua obj6.bin.lua
>
> ./global/vanilla:
> debug.bin.lua
>
> ./prj1:
> prj1.txt.lua
> ----8<----
>
> The test input file (prj1.txt.lua) imports stuff and creates objects:
> ----8<----
> import({
> "lib1", "lib1/obj1", "lib1/obj2",
> "lib2", "lib2/obj2", "lib2/obj3",
> "lib3",
> "lib4",
> "lib5"
> })
> for o = 1,8 do
> object("obj" .. o)
> end
> for l = 1,6 do for o = 1,8 do
> object("lib" .. l .. "/" .. "obj" .. o)
> end end
> ----8<----
>
>
> Test Outputs
> ------------
>
> The full raw output:
> http://claudiusmaximus.goto10.org/t/pd/pd-new-loader-test-4-
> verbose.txt
>
> The output with the STAT lines filtered out:
> http://claudiusmaximus.goto10.org/t/pd/pd-new-loader-test-4-quiet.txt
>
> The output with the STAT lines sorted and duplicates counted:
> http://claudiusmaximus.goto10.org/t/pd/pd-new-loader-test-4-statistics.txt
> (The counts are low because the results are cached, this will need
> to be changed for live-coding abstractions etc...)
>
>
> Hopefully I'm heading in the right direction with this, if not then
> yell loudly!
>
> Thanks,
>
>
> Claude
> --
> http://claudiusmaximus.goto10.org
>
> _______________________________________________
> Pd-dev mailing list
> Pd-dev at iem.at
> http://lists.puredata.info/listinfo/pd-dev
----------------------------------------------------------------------------
As we enjoy great advantages from inventions of others, we should be
glad of an opportunity to serve others by any invention of ours; and
this we should do freely and generously. - Benjamin Franklin
More information about the Pd-dev
mailing list