[PD-dev] urlloader, a web-based abstraction loader
Hans-Christoph Steiner
hans at eds.org
Thu Jul 10 19:15:16 CEST 2008
Seems like an interesting idea. Personally, I think an apt-get style
of package management would work better for Pd. Sounds like you have
all the right infrastructure, but instead of auto-downloading
patches, it could be used for package management.
FYI: if you want to be added as a developer, you need to introduce
yourself to pd-dev and request to be added. You can search the
archives for examples.
.hc
On Jul 9, 2008, at 8:40 PM, Stephen Sinclair wrote:
> Hello,
>
> I have just submitted patch #, which implements a new loader for Pd.
> This allows one to specify in a patch a "base URL" in which missing
> abstractions are expected to be found. The loader downloads an index
> at each listed URL, and if it knows about an abstraction that hasn't
> yet been loaded, it will download the patch file to a cache and open
> it from there. This is an idea I've been toying with for a while, and
> I've given it some consideration on several levels. I'm not even
> convinced it's a great idea, but I think it's something some people
> might be interested in. Lately I haven't had time to work on it, but
> I've got it working well enough that perhaps I should publish it
> before it starts to just stagnate in my home folder.
>
> I thought I should just send a follow-up email explaining how to use
> it and expanding on some of the points listed in the patch comments:
>
> First, to test the patch you'll need to do a few things.
>
> 1) Get it compiled and loaded.
>
> The code is in externals/loaders/urlloader. Put it in your startup
> path and make sure it gets loaded. (pd should post some messages when
> it is loaded.) You'll need the libcurl-dev package installed.
>
> 2) Make the files accessible through http.
>
> There is a directory under /urlloader called /remote. These files
> must go to a web server. For testing, I have a local Apache running
> on my laptop, and the files are hosted at http://localhost/pd. The
> /pd folder is just a symlink from /var/www/pd to the /remote folder.
> Note that the test patch (test.pd) points to this location, so if you
> decide to use a different location, you'll have to modify it.
>
> 3) Sign the patches in the remote folder.
>
> The urlloader requires GPG to work (more on this below). So you'll
> need to have gpg working to test this code. If you have a gpg key,
> use it, otherwise make one. (Not hard.)
> You want files ending in .asc, so issue these two commands:
>
> gpg -b --armor -s unknown.pd
> gpg -b --armor -s another.pd
>
> 4) Open test.pd
>
> It should open the test patch, and then the [another] and [unknown]
> objects should be instantiated. Open them and you'll see that the
> actual location of the files is in a cache directory in your /home
> folder.
>
>
> Some points:
>
> - Requires GPG to verify downloaded patches before loading them.
>
> Since this patch essentially allows code downloaded from a website to
> run locally on your machine, it can obviously be a screaming security
> risk. Therefore I did not want to publish this code before
> implementing this feature. However, it's got a few ugly user
> interface implications. Of course not everyone knows how to use GPG
> (I learned how just for this actually). But also, what if you don't
> have the key to verify a signature? Currently it just 'posts' this to
> the pd window, and you have to go download it. The person publishing
> the abstractions must provide their public key for this purpose. You
> then have to manually import it into your keyring with 'gpg --import'.
>
> - Also verifies the files MD5 sums
>
> I think this is probably redundant with the GPG stuff mentioned above.
> Originally I was just going to use the MD5 sum of a file to verify
> its integrity. However, I realized that trust issues go much deeper
> than that, since a loaded abstraction can easily specify another URL
> and then download patches from the new location. You wouldn't want
> your computer automatically following and downloading anything it
> sees, so I saw it as a necessity to support some kind of trust
> infrastructure. But I left the MD5 code in the patch for now anyways.
> The pd.index file that sits in /remote is just the output of the
> 'md5sum' command.
>
> On the TODO list:
>
> - Port to non-Linux operating systems.
>
> Currently I developed this on Ubuntu. It should be portable to other
> systems, provided libcurl and gpg work there.
>
> - Determine proper caching rules.
>
> Currently patches are cached for 100 seconds, then downloaded again
> after the expire. This is just for testing purposes. In real-world a
> more intelligent caching system would be necessary. (That might be a
> good use of the MD5 sum in the index file, now that I think about it.)
>
> - Aliases for URLs which could be prepended to object names.
> (e.g., urlA:myobject)
>
> If more than one URL has the same object listed, the first one will be
> downloaded, which may as well be considered undeterministic to the
> user. It would be nice to support a way for the user to specify what
> URL an object belongs to. Using some kind of alias prepended to the
> object name might be a way of doing it, with the alias specified in
> the [baseurl] object. In fact, these should be related only to the
> canvas the [baseurl] objects reside in, whereas right now everything
> is pretty much global. (It would probably require modification of the
> g_list data structure, if I understand right.)
>
> - Deal sanely with versioning. (support svn URLs?)
>
> This brings up a problem with versions. What if a patch is designed
> that depends on an abstraction at some server, but then the owner of
> that server makes an incompatible change which breaks the person's
> patch. It would be nice to support some kind of versioning, so you
> could say not only, 'this patch depends on xyz', but 'this patch
> depends on version 2.5 of xyz'. I was thinking that one way to
> support this might be to include support for subversion URLs, or at
> least websvn, so that specific versions of files would be downloaded,
> but still easily updated to a newer version.
>
> - Determine whether this is even a good idea.
>
> I'm not completely convinced that downloading patches and running them
> directly is the best way to deal with abstractions hosted in the
> 'cloud'. It might be better to have a coherent and trusted repository
> system like APT.
>
>
> Please try it and give some feedback. If you'd like me to commit this
> work to the svn, please feel free to grant me access, though I'll
> likely continue developing it on my own either way.
>
>
> Steve
>
> _______________________________________________
> Pd-dev mailing list
> Pd-dev at iem.at
> http://lists.puredata.info/listinfo/pd-dev
------------------------------------------------------------------------
----
Terrorism is not an enemy. It cannot be defeated. It's a tactic.
It's about as sensible to say we declare war on night attacks and
expect we're going to win that war. We're not going to win the war
on terrorism. - retired U.S. Army general, William Odom
More information about the Pd-dev
mailing list