[PD] readsf~ and writesf~, locking, blocking and real-time priority

[Matt Barber]

>
> 4) Has anyone ever “broken” these objects or experienced glitching?


​Once in 2005 we were having awful trouble streaming through Pd but we were
never sure whether it was [readsf~] per se, a very slow disk, or xruns in
ALSA/JACK, and we had only one performance laptop available. My best guess
is that it was an ALSA/JACK problem, since other software had a few issues
with glitching just on realtime audio processing.

pthread_mutex_lock() ... This might be a good time for a PSA for interested
newcomers to Pd, though, if any happen to be following this thread (ahem).
Having taught Pd for some 10 years now, one bad habit I've seen nearly
every student fall into is failing to preload the file before playing,
trying to do the initial read and the playing at the same logical time.
Usually there isn't a problem, but once in a while a taxed system that is
already streaming several files can glitch hard on a new stream. I've
attached a generic [readsf~] idiom that has been useful for first-year
students when they want to jump in and get Pd to play some sound files with
a GUI after they've fooled around with the control examples and
oscillators. This is before we get into event triggering, so the clunky
multiple play/stop buttons is edited out later on; the main thing is how to
keep the file open at all times. This turns out to be even more important
for rehearsal than for performance, when you need to be able to jump around
at will. ... pthread_mutex_unlock()



On Tue, Oct 6, 2015 at 12:15 AM, Miller Puckette <msp at ucsd.edu> wrote:

> I think you can get away with sharing a lock between two high-prioroty
> processes as long as neither one holds the lock for more than a small
> amount of time and if the OS can be counted on to give control to a
> real-time
> process quickly once it becomes runnable (i.e., if it's blocked on a lock,
> once that lock is released).
>
> The situation I don't know about is this:  if Pd's main thread failed to
> get
> the lock, so that control (presumably) passed back to the other thread that
> had the lock, how much time can pass before the other thread blocks on
> something so that control (again presumably) gets passed back to the main
> thread?
>
> But anyway, since neither thread holds onto the lock for more than a few
> lines of C code (with no system calls) it's probably blue-moon rare that
> the
> scheduler interrupts one thread right in the middle of a critical section
> and
> passes control to the other one that then blocks.  So this is essentially
> untested.
>
> Threads can never be used confidently in a real-time situation.  But I
> don't
> see any reasonable way without them to implement readsf~/writesf~, so there
> we are...
>
> cheers
> Miller
>
> P.S. one can issue non-blocking reads/writes, but there's also "open" which
> is much more likely to hiccup than "read", and I don't know of any async
> open
> call in any OS.
>
>
> On Tue, Oct 06, 2015 at 03:10:31AM +0000, Jonathan Wilkes via Pd-list
> wrote:
> > 1) One thing I noticed is that the article you cited seems to focus
> > on tasks not critical to the computation/delivery of audio samples.For
> example, if your program were blocking or locking in order to do a
> GUIupdate.  But here, the data must arrive in time to compute the next
> block.  If ittakes too long to read the next portion of the sound file,
> then you're going to geta glitch.
> > But I'm not sure I really grasp how locking works, nor really the whole
> file i/oprocess in general.
> >
> > Here's a naive question: why can't you just tell the OS to treat the
> file asif it were a non-blocking socket, add the fd to Pd's event loop with
> > sys_addpollfn, and then receive the incoming data to the relevant
> function?(Warning: some or all of the above may technically be gibberish...)
> >
> > -Jonathan
> >
> >
> >
> >
> >
> >
> >
> >
> >      On Monday, October 5, 2015 10:01 PM, Robert Esler <
> robert at urbanstew.org> wrote:
> >
> >
> >  I’m trying to understand why readsf~ and writesf~ work so well.
> >
> > I’m particularly referencing Ross Bencina’s article:
> http://www.rossbencina.com/code/real-time-audio-programming-101-time-waits-for-nothing
> and his subsequent paper,
> http://www.rossbencina.com/static/writings/File_IO_ACMC2014_Bencina.pdf
> >
> > If you are not into asynchronous message passing and lock-free queueing
> then I’ll summarize the articles briefly:
> >
> > When engaging in file I/O (e.g reading from or writing to an audio file)
> do not use locks or blocking. He goes on to say that this can lead to
> priority inversion, unbound execution time and “scheduler paranoia”.
> >
> > This is all absolutely true in my experience in the audio jungle.
> >
> > Pd’s async file I/O objects (readsf~ and writesf~) use both locks and
> blocking via a mutex and the pthread_cond_signal and pthread_cond_init
> functions.  Look at the source code file d_soundfile.c for more details.
> The gist of it is that these objects have two threads.  One parent thread
> that sends the data to the dsp scheduler, and a child thread that grabs the
> data from the file, and subsequently the child signals the parent when it
> has more data.
> >
> > Based on Bencina’s paper, readsf~ and writesf~ could (should?) glitch
> and may not be real-time safe.
> >
> > My questions are:
> >
> > 1) Have I completely misunderstood d_soundfile.c and it is actually
> entirely safe. If so, why is it safe?
> >
> > 2) Why doesn’t Pd glitch more often when using these objects?
> >
> > 3) Does Pd need lock-free message queueing for such inter-thread
> communication?
> >
> > 4) Has anyone ever “broken” these objects or experienced glitching?
> >
> > Thanks for the extra brain power.
> > -R
> >
> >
> >
> > _______________________________________________
> > Pd-list at lists.iem.at mailing list
> > UNSUBSCRIBE and account-management ->
> http://lists.puredata.info/listinfo/pd-list
> >
> >
> >
>
> > _______________________________________________
> > Pd-list at lists.iem.at mailing list
> > UNSUBSCRIBE and account-management ->
> http://lists.puredata.info/listinfo/pd-list
>
>
> _______________________________________________
> Pd-list at lists.iem.at mailing list
> UNSUBSCRIBE and account-management ->
> http://lists.puredata.info/listinfo/pd-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puredata.info/pipermail/pd-list/attachments/20151006/0fa2bca4/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Generic_readsf~_idiom.pd
Type: application/octet-stream
Size: 1433 bytes
Desc: not available
URL: <http://lists.puredata.info/pipermail/pd-list/attachments/20151006/0fa2bca4/attachment.obj>