[PD-dev] per-thread storage in Pd in support of pdlib - discussion?

Peter Brinkmann peter.brinkmann at googlemail.com
Wed Jan 25 05:04:27 CET 2012


Oops, I just realized that I sent my reply IOhannes and didn't cc pd-dev...


On Thu, Jan 19, 2012 at 4:11 AM, IOhannes m zmoelnig <zmoelnig at iem.at>wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>


> >> right now, only #1 is possible at all and it takes some effort on the
>
> >> "thread host" (the external) to not fuck Pd's heap.
> >> i think Pd should be more helpful in this respect: a way to make Pd
> >> thread-safe is to eliminate global variables and if they can't (or
> >> shan't) be eliminate them properly protect them against parallel access.
> >>
> >
> > As far as libpd is concerned, I would prefer not to have any
> > synchronization inside Pd itself --- libpd can be used in a wide variety
> of
> > settings, with lots of different approaches to concurrency, and so it's
> > impossible to make any assumptions about threading at this level.
>
> what are the actual drawbacks if e.g. clock_delay() could be used from
> any thread (in an external)?
>

There's nothing wrong with accessing a Pd instance from multiple threads.
All I'm saying here is that the hypothetical replacement for libpd that
would form the core of this refactoring is not necessarily the place to do
synchronization.

By the way, once we have multiple instances of Pd, calls like clock_delay
will also need to take a context parameter because each instance of Pd will
have its own sys_time.


> which assumptions are made that might not hold true in all your use cases?
>

As I interpret it, the assumption is that synchronization at the audio
library level is desirable and sufficient.  There are use cases that might
not need synchronization at all (non-real-time batch processing, Python
applications, applications that use one Pd instance per thread, etc.),
which would render locks in Pd redundant as far as libpd is concerned
(support for threaded externals is a different story).  In other cases, an
invocation of a Pd function might be part of a larger context that also
requires synchronization.  The Java bindings of libpd, for example,
maintain a certain amount of state on the Java side.  In such a situation,
additional synchronization outside of Pd is needed.

In order to find the best solution for each case, each application would
have to be in charge of thread synchronization, but that would lead to a
lot of repetitive and error-prone code.  In libpd, I decided to handle
synchronization in the language bindings (no locks in Python, class-level
locks in Java and Objective-C), which seems like a reasonable compromise.
The upcoming C++ bindings that Dan Wilcox and Rich Eakin are working on
seem to be a different case due to portability issues, but the general
strategy of handling concurrency in language bindings seems to be sound.

In either case, locks in Pd would be redundant from the point of view of
libpd, although they'd also be benign because they'd always be uncontested
and hence cheap.



> my main reasoning is, that thread synchronisation is a re-curring
> problem that imho should not be re-implemented whenever it is needed.
> also a earlier attempts to fix this, using the great BIG kernel lock
> (aka sys_lock()), proved (at least for me) to be inadequate, as they
> slow down the entire processing significantly.
>

When I started working on libpd, I decided to discard sys_lock altogether.
The trade-off was between dealing with the complexity of sys_lock and
losing some threaded externals.  I haven't yet received any complaints
about broken externals, so I think the decision was okay.


>
> however, maybe i'm asking too much and what i really want is to have a
> standardized possibility to send messages to a Pd-instances without
> having to worry about threading (i use all the clock-stuff i keep
> mentioning mainly for doing exactly this: implementing a thread-safe
> message queue to send data from a worker thread back to Pd)
>

I see several ways of handling this.  One approach might be to have a core
library that remains blissfully unaware of concurrency, along with a
thread-safe wrapper that externals would operate on.  Another possibility
would be to think really big and rethink the way Pd processes audio (why
not do a topological sort of the signal processing graph and then
parallelize computations on the fly, using something like OpenMP?).  That
might involve a general refactoring into components that might be made
thread-safe with fine-grained locks or other concurrency techniques.


>
>
> > Of course, as you point out, Pd itself requires some synchronization in
> its
> > interaction with externals, so there's a bit of a conflict there.  My
> > favorite solution would be to refactor Pd so that it has an audio library
> > much like libpd at its core.  Then Pd would be able to do all the
> > synchronization it needs without affecting other applications that use
> the
> > same library.
>
> i have to admit i'm a bit unsure where you would draw the separation.
> Pd uses common infrastructures for a lot of things on different system
> levels: e.g. the message system is used to communicate between objects,
> to let the gui talk to the pd-core, to open a patch (instantiate it's
> objects),...
> while i think this is one of the strengths of Pd, i also think that this
> is probably the biggest problem when attempting a refactor as you
> describe it (but again: i might totally miss the point here)
>

I think we're already half-way there.  Just look at Chris McCormick's
PdDroidParty --- he's basically rebuilding a good chunk of the GUI
functionality of Pd on top of libpd.  What's missing is the ability to edit
patches, but I believe that could be done with a minor extension of the
messaging API of libpd.
Cheers,
     Peter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puredata.info/pipermail/pd-dev/attachments/20120124/e8d4632f/attachment.htm>


More information about the Pd-dev mailing list