[PD] threadsafe multi-instance Pd? (Miller Puckette)

[Miller Puckette]

I think I should have used what appears to be the standard term,
"Thread-local storage" (https://en.wikipedia.org/wiki/Thread-local_storage).

The idea is that, if you invoke pd_this from two threads simultaneously,
they'll actually refer to two different memory locations and therefore
won't interfere with each other.  This will be true as long as everything that
is thread-unsafe is referenced through pd_this.

There are many, many static local variable running around in Pd, some of
which may have to be referenced through pd_this (while for others it may
suffice to make them thread local themselves, which would mean they would
be shared beween Pd instances that happened to use the same thread, and
their values will change when a pd instance is migrated from one thread to
another.  I think it will be necessary to look at each case individually.

cheers
Miller

On Mon, Jan 02, 2017 at 02:36:36PM +0100, Pierre Guillot wrote:
> That's a great news !
> I don't understand what you mean by " If this were (pd_this) were made
> per-thread". Perhaps that's the point I didn't understand when we were
> speaking about it at the Pd convention. How can we ensure that an object, a
> patch or whatever uses the right pd_instance ?
> 
> Here a concrete example of my problem, if you load a new patch (canvas) in
> a specific instance, to ensure that the canvas is added to the canvas list
> of this instance, you need:
> 1 - Lock a global lock
> 2 - Set the right instance
> 3 - Load the patch
> 4 - Unlock the global lock
> 
> I don't see how we can remove this global lock without changing the
> functions' prototypes that use pd_this (by adding a instance's pointer as
> argument and removing pd_this).
> 
> 
> > Date: Sun, 1 Jan 2017 15:31:51 -0800
> > From: Miller Puckette <msp at ucsd.edu>
> > To: pd-list at iem.at
> > Subject: [PD] threadsafe multi-instance Pd?
> > Message-ID: <20170101233151.GE21861 at fuzz.localdomain>
> > Content-Type: text/plain; charset=us-ascii
> >
> > To Pd List,
> >
> > Here's my promised followup mail on Pd thread-ability (the hoped for
> > ability to call Pd instances, via pdlib, from separate threads
> > concurrently).
> >
> > Peter Brinkmann made a suggestion during the Pd convention round-table
> > discussion that I'll paraphrase here.  There is a "pd_this" variable in
> > m_pd.c, pointing to the current Pd instance.  If this were made per-thread,
> > then it should be possible to run different instances on different threads
> > simultaneously; the only protection needed would be that each individual
> > instance should be protected by pdlib with its own lock.
> >
> > (There would also have to be a global lock to protect pd_init(), which need
> > only be called as setup time).
> >
> > But there's a snag, because the symbol table is global.  It wouldn't help
> > to
> > make this per-thread, since calls to pd instances might migrate from
> > thread to
> > thread.  Instead, we could do 1 of these 2 things:
> >
> > 1 (Peter's idea) : make gensym(), pd_bind(), and pd_unbind() threadsafe
> > using
> > a lock.  Access via gensym() could be nonexclusive, as could locking out
> > pd_bind() and pd_unbind() during accesses to s->s_thing (we'd have to hunt
> > down everywhere in the code this is done).
> >
> > I think there's a complication: what if you pass a message to an object via
> > a symbol's s_thing that's in a different instance from the current one
> > (pd_this) in the thread of the caller?  I don't know an easy way to
> > determine
> > what pd instance an arbitrary object belongs to.
> >
> > OR:
> >
> > 2. (another possibility): Go back and make the symbol table be
> > per-instance. I
> > tried this earlier and got stumped because classes, which are global to
> > all Pd
> > instances, contain a list of selectors (symbols) and their associated
> > messages.
> > If the symbol address changes because we switch to a new Pd instance,
> > messages
> > (like #N print) no longer are associated with the method (print_new()), so
> > nothing works.
> >
> > I think it's impractical to make classes per-instance, but what about
> > making
> > each class maintain a separate list of messages for each Pd instance.  When
> > a new Pd instance is created, we'd go find all the classes, and add a new
> > message list to each of them for the new instance.
> >
> > Then whenever one passes a message (assuming it's not one of s_bang,
> > s_float,
> > s_symbol, s_list, or s_anything, which are usually handled by a faster
> > mechanism), the message passing functions pd_typedmess(), getfn(), etc.,
> > would
> > have to look up the message list associated with the current pd instance,
> > and
> > then look down the list of selectors/methods as before.
> >
> > Anyone see anything fatally wrong with this, and/or can option (1) be made
> > workable and is it better?
> >
> > cheers
> > Miller
> >