[PD-dev] per-thread storage in Pd in support of pdlib - discussion?

[Miller Puckette]

Yikes... sounds like back to the drawing board.

The odd thing is, do make thread-local storage, the good C compiler
folks (and linker folks, etc) had to do all the work you'd need to make
a switch-all-my-static-storage-when-I-ask-you-to feature which would open
up all sorts of other ways to do things.  But I don't see any way to adapt
it to our needs, given the concerns you raised below.

cheers
Miller

On Sat, Jan 14, 2012 at 10:16:17PM -0500, Peter Brinkmann wrote:
> Hi Miller,
> Thanks for your message!
> 
> I'm afraid thread-local instances would be problematic from the point of
> view of libpd:
> 
> * The most common structure of a libpd-based application involves two
> threads, a GUI thread and an audio thread, where the GUI controls the Pd
> engine by invoking libpd functions from its thread.  This approach would
> break if Pd instances were thread local.
> 
> * In many cases, the audio thread is beyond the control of the programmer.
> For instance, if you want one Pd instance per JACK client, or one Pd
> instance per audio unit in iOS, then you just register a callback, and you
> have no real idea which thread will ultimately invoke your callback.
> 
> * If you use libpd via its Python bindings, then your threading options are
> limited due to the Global Interpreter Lock.
> 
> * libpd may also end up doing batch-processing, either by design or as a
> client running under JACK's freewheel mode, which has implications for
> threading.
> 
> I have a few more concerns, but these are the most important ones.  The
> upshot is that libpd may run in such a wide range of settings that it's
> hard to make assumptions about what kind of approach to threading is
> appropriate or even available.
> Cheers,
>      Peter
> 
> 
> On Sat, Jan 14, 2012 at 4:04 PM, Miller Puckette <msp at ucsd.edu> wrote:
> 
> > To Pd dev -
> >
> > For some time the good folks who brought us pdlib have been asking how
> > one could make it possible to run several instances of Pd in a single
> > address space.
> >
> > The question I'd like to rais is this: would it suffice to make Pd
> > instances
> > be per-thread?  This could be done by going through all the source and
> > modifyin global and static variables with the nonstandard __thread
> > leyword (gcc) or some declspec horror in Miscoroft C.  Alternatively,
> > one could use the C++11 standard thread_local keyword, although I believe
> > that's not widely implemented yet.
> >
> > To do this I'd replace all globals like
> > static t_symbol *symhash[HASHSIZE];
> > with
> > PDSTATIC  t_symbol *symhash[HASHSIZE];
> >
> > and define PDSTATIC to static (also PDGLOBAL to the empty string and
> > PDEXTERN to extern).  Then anyone wanting to come along and recompile pd
> > to work per-thread only has to redefine the three appropriately to the
> > compiler.
> >
> > Here are the gotchas I can foresee:
> >
> > 1.  external objects making explicit references to global storage
> > (such as canvas_dspstate and cos-table in m_pd.h and much stuff in the more
> > 'private' header files) would have to be recompiled to run per-thread.
> >  They'd
> > still work fine with vanilla Pd.
> >
> > 2. existing externs that create threads would break at the source level if
> > they use any Pd-supplied functions at all (outlet_bang(), clock_set(),
> > gensym(), etc) in 'other" threads.  Again they'd still work in Pd vanilla,
> > just not with versions of Pd recompiled to run per-thread.
> >
> > 3. lots of lines of code would be touched and this might make a number of
> > existing patches fail to apply cleanly.
> >
> > 4. supposing you use this to make a VST plug-in: what would happen if some
> > stupid host app called all its VST plug-ins from the same thread?  (I
> > think it's normal for hosts always to make a new thread for every VST
> > plug-in instance but I don't know if this is universal).
> >
> > 5.  If you wanted to run two instances of Pd in the same thread this
> > wouldn't
> > help.  You'd have to spawn new threads and pass control to them to get into
> > the alternate Pds.
> >
> > Comments anyone?
> >
> > thanks
> > Miller
> >
> > _______________________________________________
> > Pd-dev mailing list
> > Pd-dev at iem.at
> > http://lists.puredata.info/listinfo/pd-dev
> >