[PD] threadsafe multi-instance Pd? (Miller Puckette)

[Pierre Guillot]

Thank, I understand much better ! It seems to me that can be good
workaround.

1 (Peter's idea) : make gensym(), pd_bind(), and pd_unbind() threadsafe
> using
> a lock.  Access via gensym() could be nonexclusive, as could locking out
> pd_bind() and pd_unbind() during accesses to s->s_thing (we'd have to hunt
> down everywhere in the code this is done).


One of the problems with this approach is the compatibility with external
libraries. You can check and lock everywhere in the "vanilla" code when you
access s_thing but you can't ensure that an external object does it.
Ignoring external libraries can be a compromise but I think it's pretty
restrictive. Or the drastic solution: make opaque the t_symbol and add new
function "sendsym" that could lock the symbol and call "pd_typemess" if
s_thing !=  NULL. This way, the future externals will be necessarily
compatible but we won't be able to compile the new ones except if we update
and correct the code...


> I think there's a complication: what if you pass a message to an object via
> a symbol's s_thing that's in a different instance from the current one
> (pd_this) in the thread of the caller?  I don't know an easy way to
> determine
> what pd instance an arbitrary object belongs to.


If we assume that each instance has its own thread. Perhaps, when we bind
an object to a symbol, the bindlist can "save" the current intance in the
element (bindelem). Then when one of the method of the bindlist is called,
it can check if the instance of the current thread is the same than the
ones of the element.

Anyone see anything fatally wrong with this, and/or can option (1) be made
> workable and is it better?


At first sight, the option 2 seems good.

There are many, many static local variable running around in Pd, some of
> which may have to be referenced through pd_this


Yes, indeed. In Camomile plugin, I encountered issues with sys_soundin,
sys_soundout, sys_inchannels, sys_outchannels and sys_dacsr. I think it
would be better if these variables were included to the pd instance (or
thread local).

Cheers


2017-01-02 17:40 GMT+01:00 Miller Puckette <msp at ucsd.edu>:

> I think I should have used what appears to be the standard term,
> "Thread-local storage" (https://en.wikipedia.org/wiki/Thread-local_storage
> ).
>
> The idea is that, if you invoke pd_this from two threads simultaneously,
> they'll actually refer to two different memory locations and therefore
> won't interfere with each other.  This will be true as long as everything
> that
> is thread-unsafe is referenced through pd_this.
>
> There are many, many static local variable running around in Pd, some of
> which may have to be referenced through pd_this (while for others it may
> suffice to make them thread local themselves, which would mean they would
> be shared beween Pd instances that happened to use the same thread, and
> their values will change when a pd instance is migrated from one thread to
> another.  I think it will be necessary to look at each case individually.
>
> cheers
> Miller
>
> On Mon, Jan 02, 2017 at 02:36:36PM +0100, Pierre Guillot wrote:
> > That's a great news !
> > I don't understand what you mean by " If this were (pd_this) were made
> > per-thread". Perhaps that's the point I didn't understand when we were
> > speaking about it at the Pd convention. How can we ensure that an
> object, a
> > patch or whatever uses the right pd_instance ?
> >
> > Here a concrete example of my problem, if you load a new patch (canvas)
> in
> > a specific instance, to ensure that the canvas is added to the canvas
> list
> > of this instance, you need:
> > 1 - Lock a global lock
> > 2 - Set the right instance
> > 3 - Load the patch
> > 4 - Unlock the global lock
> >
> > I don't see how we can remove this global lock without changing the
> > functions' prototypes that use pd_this (by adding a instance's pointer as
> > argument and removing pd_this).
> >
> >
> > > Date: Sun, 1 Jan 2017 15:31:51 -0800
> > > From: Miller Puckette <msp at ucsd.edu>
> > > To: pd-list at iem.at
> > > Subject: [PD] threadsafe multi-instance Pd?
> > > Message-ID: <20170101233151.GE21861 at fuzz.localdomain>
> > > Content-Type: text/plain; charset=us-ascii
> > >
> > > To Pd List,
> > >
> > > Here's my promised followup mail on Pd thread-ability (the hoped for
> > > ability to call Pd instances, via pdlib, from separate threads
> > > concurrently).
> > >
> > > Peter Brinkmann made a suggestion during the Pd convention round-table
> > > discussion that I'll paraphrase here.  There is a "pd_this" variable in
> > > m_pd.c, pointing to the current Pd instance.  If this were made
> per-thread,
> > > then it should be possible to run different instances on different
> threads
> > > simultaneously; the only protection needed would be that each
> individual
> > > instance should be protected by pdlib with its own lock.
> > >
> > > (There would also have to be a global lock to protect pd_init(), which
> need
> > > only be called as setup time).
> > >
> > > But there's a snag, because the symbol table is global.  It wouldn't
> help
> > > to
> > > make this per-thread, since calls to pd instances might migrate from
> > > thread to
> > > thread.  Instead, we could do 1 of these 2 things:
> > >
> > > 1 (Peter's idea) : make gensym(), pd_bind(), and pd_unbind() threadsafe
> > > using
> > > a lock.  Access via gensym() could be nonexclusive, as could locking
> out
> > > pd_bind() and pd_unbind() during accesses to s->s_thing (we'd have to
> hunt
> > > down everywhere in the code this is done).
> > >
> > > I think there's a complication: what if you pass a message to an
> object via
> > > a symbol's s_thing that's in a different instance from the current one
> > > (pd_this) in the thread of the caller?  I don't know an easy way to
> > > determine
> > > what pd instance an arbitrary object belongs to.
> > >
> > > OR:
> > >
> > > 2. (another possibility): Go back and make the symbol table be
> > > per-instance. I
> > > tried this earlier and got stumped because classes, which are global to
> > > all Pd
> > > instances, contain a list of selectors (symbols) and their associated
> > > messages.
> > > If the symbol address changes because we switch to a new Pd instance,
> > > messages
> > > (like #N print) no longer are associated with the method
> (print_new()), so
> > > nothing works.
> > >
> > > I think it's impractical to make classes per-instance, but what about
> > > making
> > > each class maintain a separate list of messages for each Pd instance.
> When
> > > a new Pd instance is created, we'd go find all the classes, and add a
> new
> > > message list to each of them for the new instance.
> > >
> > > Then whenever one passes a message (assuming it's not one of s_bang,
> > > s_float,
> > > s_symbol, s_list, or s_anything, which are usually handled by a faster
> > > mechanism), the message passing functions pd_typedmess(), getfn(),
> etc.,
> > > would
> > > have to look up the message list associated with the current pd
> instance,
> > > and
> > > then look down the list of selectors/methods as before.
> > >
> > > Anyone see anything fatally wrong with this, and/or can option (1) be
> made
> > > workable and is it better?
> > >
> > > cheers
> > > Miller
> > >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puredata.info/pipermail/pd-list/attachments/20170102/2c2a7db7/attachment.html>