[PD-dev] urlloader, a web-based abstraction loader

Hans-Christoph Steiner hans at eds.org
Thu Jul 10 19:15:16 CEST 2008


Seems like an interesting idea.  Personally, I think an apt-get style  
of package management would work better for Pd.  Sounds like you have  
all the right infrastructure, but instead of auto-downloading  
patches, it could be used for package management.

FYI: if you want to be added as a developer, you need to introduce  
yourself to pd-dev and request to be added.  You can search the  
archives for examples.

.hc

On Jul 9, 2008, at 8:40 PM, Stephen Sinclair wrote:

> Hello,
>
> I have just submitted patch #, which implements a new loader for Pd.
> This allows one to specify in a patch a "base URL" in which missing
> abstractions are expected to be found.  The loader downloads an index
> at each listed URL, and if it knows about an abstraction that hasn't
> yet been loaded, it will download the patch file to a cache and open
> it from there.  This is an idea I've been toying with for a while, and
> I've given it some consideration on several levels.  I'm not even
> convinced it's a great idea, but I think it's something some people
> might be interested in.  Lately I haven't had time to work on it, but
> I've got it working well enough that perhaps I should publish it
> before it starts to just stagnate in my home folder.
>
> I thought I should just send a follow-up email explaining how to use
> it and expanding on some of the points listed in the patch comments:
>
> First, to test the patch you'll need to do a few things.
>
> 1) Get it compiled and loaded.
>
> The code is in externals/loaders/urlloader.  Put it in your startup
> path and make sure it gets loaded.  (pd should post some messages when
> it is loaded.)  You'll need the libcurl-dev package installed.
>
> 2) Make the files accessible through http.
>
> There is a directory under /urlloader called /remote.  These files
> must go to a web server.  For testing, I have a local Apache running
> on my laptop, and the files are hosted at http://localhost/pd.  The
> /pd folder is just a symlink from /var/www/pd to the /remote folder.
> Note that the test patch (test.pd) points to this location, so if you
> decide to use a different location, you'll have to modify it.
>
> 3) Sign the patches in the remote folder.
>
> The urlloader requires GPG to work (more on this below).  So you'll
> need to have gpg working to test this code.  If you have a gpg key,
> use it, otherwise make one.  (Not hard.)
> You want files ending in .asc, so issue these two commands:
>
> gpg -b --armor -s unknown.pd
> gpg -b --armor -s another.pd
>
> 4) Open test.pd
>
> It should open the test patch, and then the [another] and [unknown]
> objects should be instantiated.  Open them and you'll see that the
> actual location of the files is in a cache directory in your /home
> folder.
>
>
> Some points:
>
> - Requires GPG to verify downloaded patches before loading them.
>
> Since this patch essentially allows code downloaded from a website to
> run locally on your machine, it can obviously be a screaming security
> risk.  Therefore I did not want to publish this code before
> implementing this feature.  However, it's got a few ugly user
> interface implications.  Of course not everyone knows how to use GPG
> (I learned how just for this actually).  But also, what if you don't
> have the key to verify a signature?  Currently it just 'posts' this to
> the pd window, and you have to go download it.  The person publishing
> the abstractions must provide their public key for this purpose.  You
> then have to manually import it into your keyring with 'gpg --import'.
>
> - Also verifies the files MD5 sums
>
> I think this is probably redundant with the GPG stuff mentioned above.
>  Originally I was just going to use the MD5 sum of a file to verify
> its integrity.  However, I realized that trust issues go much deeper
> than that, since a loaded abstraction can easily specify another URL
> and then download patches from the new location.  You wouldn't want
> your computer automatically following and downloading anything it
> sees, so I saw it as a necessity to support some kind of trust
> infrastructure.  But I left the MD5 code in the patch for now anyways.
>  The pd.index file that sits in /remote is just the output of the
> 'md5sum' command.
>
> On the TODO list:
>
> - Port to non-Linux operating systems.
>
> Currently I developed this on Ubuntu.  It should be portable to other
> systems, provided libcurl and gpg work there.
>
> - Determine proper caching rules.
>
> Currently patches are cached for 100 seconds, then downloaded again
> after the expire.  This is just for testing purposes.  In real-world a
> more intelligent caching system would be necessary.  (That might be a
> good use of the MD5 sum in the index file, now that I think about it.)
>
> - Aliases for URLs which could be prepended to object names.
>   (e.g., urlA:myobject)
>
> If more than one URL has the same object listed, the first one will be
> downloaded, which may as well be considered undeterministic to the
> user.  It would be nice to support a way for the user to specify what
> URL an object belongs to.  Using some kind of alias prepended to the
> object name might be a way of doing it, with the alias specified in
> the [baseurl] object.  In fact, these should be related only to the
> canvas the [baseurl] objects reside in, whereas right now everything
> is pretty much global.  (It would probably require modification of the
> g_list data structure, if I understand right.)
>
> - Deal sanely with versioning.  (support svn URLs?)
>
> This brings up a problem with versions.  What if a patch is designed
> that depends on an abstraction at some server, but then the owner of
> that server makes an incompatible change which breaks the person's
> patch.  It would be nice to support some kind of versioning, so you
> could say not only, 'this patch depends on xyz', but 'this patch
> depends on version 2.5 of xyz'.  I was thinking that one way to
> support this might be to include support for subversion URLs, or at
> least websvn, so that specific versions of files would be downloaded,
> but still easily updated to a newer version.
>
> - Determine whether this is even a good idea.
>
> I'm not completely convinced that downloading patches and running them
> directly is the best way to deal with abstractions hosted in the
> 'cloud'.  It might be better to have a coherent and trusted repository
> system like APT.
>
>
> Please try it and give some feedback.  If you'd like me to commit this
> work to the svn, please feel free to grant me access, though I'll
> likely continue developing it on my own either way.
>
>
> Steve
>
> _______________________________________________
> Pd-dev mailing list
> Pd-dev at iem.at
> http://lists.puredata.info/listinfo/pd-dev



------------------------------------------------------------------------ 
----

Terrorism is not an enemy.  It cannot be defeated.  It's a tactic.   
It's about as sensible to say we declare war on night attacks and  
expect we're going to win that war.  We're not going to win the war  
on terrorism.        - retired U.S. Army general, William Odom






More information about the Pd-dev mailing list