[PD] standard paths for externals

Thu Jun 14 13:33:35 CEST 2018

> On Jun 14, 2018, at 10:21 AM, pd-list-request at lists.iem.at wrote:
> 
> From: Roman Haefeli <reduzent at gmail.com <mailto:reduzent at gmail.com>>
> To: Pd-List <pd-list at lists.iem.at <mailto:pd-list at lists.iem.at>>
> Subject: Re: [PD] standard paths for externals
> Message-ID: <0b50bb13c24e850d65000e980526f8234a138f61.camel at gmail.com <mailto:0b50bb13c24e850d65000e980526f8234a138f61.camel at gmail.com>>
> Content-Type: text/plain; charset="utf-8"
> 
> Hey Dan
> 
> Thanks for trying to explain.
> 
> On Thu, 2018-06-14 at 01:18 +0200, Dan Wilcox wrote:
>> There seems to be a bit of FUD in this "discussion."
> 
> Maybe things aren't that bad as they were presented. For my part, I
> lost track of who favors what kind of design.
> 
>> Some things from my perspective:
>> 
>> 1. Pd's library/path mechanism is largely oriented toward *local*
>> libs first and *global* ones later, ie. download externals to
>> individual project (sub)folders.
> 
> Pd already used to work like that for quite a while, no?
> 
>> I believe the centralized loading was added later on in Pd-extended. 
> 
> I don't understand what you mean by that. Pd-extended already pre-
> loaded a subset of the installed externals (do you mean this?), which -
> imho - was a bad design choice. It meant there was one person deciding
> what comes pre-loaded and what not.

Yes and continuing that mechanism by installing things into more "standard places" is perhaps less ideal, even if it is easy, ie. ~/Library/Pd.

>> If we follow the "programming language model", this might be similar
>> to installing a Python egg locally or globally. I don't see how
>> supporting both modes of working is a problem.
> 
> Me neither.
> 
>> 2. IOhannes convinced me, over time, that the "stdpath" loading
>> mechanism used by the "extra" folder is problematic as it adds too
>> many (sub) search paths by default and it becomes harder to
>> tell/control what's being loaded.
> 
> Forgive my ignorance, but I don't understand. Does Pd now look into
> extra's subfolders? Currently I still have to actively load stuff with
> [declare -stdlib] or [declare -stdpath]. 

It always has, that's one of the "standard paths" aka those that [decalre's -stdpath] uses. These are different from the "user search paths" which are those  specified in the Path dialog and/or via [declare -path]. Similarly, [declare -stdlib] search for lib names *only* in the "standard paths" while [decalre -lib] looks only in supplied "user search paths". This is where great confusion comes and arguments over what is a path and how things should/can be searched.

Our utltimate solution is to move to dis-favor the separation and simply have [declare -path] and [dlecare -lib] search both the "standard paths" and the "user search paths", first locally, then the "suer paths", and finally the more global "standard paths".

>> The "better way" is for people to specify those externals paths
>> directly, either with the user search paths and/or declare.
>> There is much less ambiguity as to what's going on at the loss of
>> "just put things in this place" ease.
> 
> So paths added by the users will be searched by deken? So  instead of
> specifying paths to the library root - as we currently do it - adding a
> path in the future means adding a searchpath that is container for many
> libraries? And deken will find libraries inside the newly added
> searchpath root? 

By deken, do you mean [declare]?

Deken is unrelated to path searching, all it does is download things to a place, either the ~/Documents/Pd/externals directory or some other places specified by you.

> 
>> 3. As suggested by Alexandre, I wrote a PR to help address how
>> declare search when using -path & -lib by having them search along
>> user search paths as well: https://github.com/pure-data/pure- <https://github.com/pure-data/pure->
>> data/pull/205. I feel this is a much closer method of using declare
>> to how other programming languages handle such things, ie. Lua's
>> require.
> 
> OK, I hopefully understand now. Am I right in thinking that declare's
> -stdlib, -stdpath flags are considered superfluous because -lib and    
> -path will scan all searchpaths in a local -> global order, including
> the parent patch's folder?

That's the general idea. As I mention before, it's hard to tell which to use since people tend to put things in different places and, as you mention in a previous email, where is the "right" place to put things. I think the "right place" should be wherever you want as long as [declare] can find it *without* requiring the distinction between -path and -stdpath.

> 
>> 4. The Pd Documents path (aka ~/Documents/Pd) is simply a helper
>> location for beginners. It is only a regular user search path
>> added by default (when desired) and does not search subfolders. 
> 
> Now you challenge my understanding of the term searchpath again. I
> thought the new idea is that you will be able to load libraries with
> [declare] from user-added searchpaths? What's the point of creating
> that folder then? 

The folder is created to give you a place where externals can be downloaded. Previously, deken either installed externals inside the app bundle's /extra folder on macOS (hidden "standard path") or asked you *each time* where to put things. This simply automates where things can go for beginners, removing a pain point and follows the Arduio/Processing model.

The next step is the downloaded external needs to be added to the user search paths for things to work. This is why deken asks you if you want to add the path after you download the external and it's sort of a kludge until something like the [declare -path] fix can be integrated. It makes things "work" for now for beginners (and people missing extended) with two clicks. I know IOhannes is not a fan but, again, it's a stopgap.

>> Nobody has to use it and it can be disabled at any time.
> 
> My beef with it is that - the way it is presented to me -  it suggests
> to do things that create confusion later. 
> 
> As a new user:
> 
> "Do you want to create ~/Documents/Pd"
> 
> Yes
> 
> "Do you want to add it to the searchpaths?"
> 
> Yes
> 
> When downloading stuff with deken: "Do you want to install 'mylib' to
> ~/Documents/Pd?"
> 
> Yes
> 
> Now, I'm going to load mylib with [declare -{std}path mylib] in my
> patch which fails.
> 
> What am I missing here?

The fact that this won't work with [declare] yet... unless you install to a "standard path", ie /extra etc. Again, we're trying to move away form this model.

> 
>> 5. Yes, the macOS hiding the ~/Library folder is problematic for many
>> users as they don't know how to show it and are afraid to modify
>> things within it. I've recently taught classes to beginners that are
>> just learn gin the concept of a "file system" as they grew up mainly
>> using "mobile phones"
> 
> I don't see how deken requires users to have a notion about a
> filesystem. Ideally, I'd use deken to tell _what_ I want to install.
> Ideally, I'd tell my patch through [declare] _what_ to load. All the
> _where_ stuff shouldn't be a question neither at installing nor loading
> time. The _where_ stuff should only matter when the user tweaks their
> environment: "I'd rather put my 4.5 TB of Pd externals on my external
> drive ". 

Yeah, and that's the whole point. You should be free to put things wherever and you are.

The issue is that [declare -lib] currently doesn't look beyond local paths.

The whole file system thing is about creating a basic place for beginners who can later on learn more about the path mechanisms and be bale to handle it themselves. We're trying to find a way for people who "just want to make stuff" to do so without running into the (from what I can tell) immense pain of moving from Pd-extended to suddenly having to manage externals themselves. Please don't discount this as supporting beginners should be the health part of *any* open source project, otherwise it's just a bunch of "grey beards" complaining about fewer and fewer "newbs."

--------
Dan Wilcox
@danomatika <http://twitter.com/danomatika>
danomatika.com <http://danomatika.com/>
robotcowboy.com <http://robotcowboy.com/>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puredata.info/pipermail/pd-list/attachments/20180614/12d3e9b3/attachment-0001.html>