<div dir="ltr">Ok cool. Thanks Miller, I will definitely look at the pipe code. <div><br></div><div>Is there an external that has become the defacto version of a transport or anything like that? I would like to support common use cases in the most helpful way possible.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Oct 25, 2020 at 8:32 AM Christof Ressi <<a href="mailto:info@christofressi.com">info@christofressi.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>Hi, Pd doesn't have a notion of global transport time, it just
knows about logical time. People can (and have) build their own
transport abstractions on top of that.</p>
<p>For further inspiration you could look into Supercollider's
TempoClock class. The idea is that each TempoClock can have its
own logical time and tempo. Changing the time or tempo of a
TempoClock affects all Routines that are scheduled on this clock.
You can, of course, have more than one TempoClock at the same
time.</p>
<p>Christof<br>
</p>
<div>On 25.10.2020 16:16, Iain Duncan wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Thanks Christof, that is helpful again, and also
encouraging as it describes pretty well what I've done so far in
the Max version. :-)
<div><br>
</div>
<div>I've enabled both one shot clocks (storing them in a
hashtable owned by the external) and periodic timers. The one
shot clocks exist in both a transport aware and
transport ignorant format for working with and quantizing off
the Max global transport, and there are periodic timers for
both too. (The transport time stops when the transport is
stopped, the other is just a general X ms timer). I have also
ensured the callback handle gets passed back so that any timer
or clock can be cancelled from Scheme user-space. (is there
such a thing as a global transport in PD?)</div>
<div><br>
</div>
<div>I was actually planning on something like you described in
Scheme space: a user defined scheduler running off the timers.
I will look into priority queues. I had one thought, which I
have not seen much, and was part of the reason I was asking on
here about schedulers. I would like to ensure the user can run
multiple transports at once, and hop around in time without
glitches. I was thinking that instead of using just a priority
queue, I would do something like a two stage structure,
perhaps with a hashmap or some other fast-read-anywhere
structure with entries representing a time period, and holding
priority queues for each period. This would be to enable the
system to seek instantly to the bar (say) and iterate through
the queue/list for that bar. Wondering if anyone has used or
seen this type of pattern or has suggestions? Basically I want
to make sure random access in time will work ok even if the
number of events in the schedule is very high, thus allowing
us to blur the lines between a scheduler and a full blown
sequencer engine. Thoughts, suggestions, and warnings are all
welcome.</div>
<div><br>
</div>
<div>iain</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Sun, Oct 25, 2020 at 4:21
AM Christof Ressi <<a href="mailto:info@christofressi.com" target="_blank">info@christofressi.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>Actually, there is no need to use a clock for every
scheduled LISP function. You can also maintain a seperate
scheduler, which is just a priority queue for callback
functions. In C++, you could use a std::map<double,
callback_type>. "double" is the desired (future) system
time, which you can get with "clock_getsystimeafter". </p>
<p>Then you create a *single* clock in the setup function *)
with a tick method that reschedules itself periodically
(e.g. clock_delay(x, 1) ). In the tick method, you get the
current logical time with "clock_getlogicaltime", walk
over the priority queue and dispatch + remove all items
which have a time equal or lower. You have to be careful
about possible recursion, though, because calling a
scheduled LISP function might itself schedule another
function. In the case of std::map, however, it is safe,
because insertion doesn't invalidate iterators.<br>
</p>
<p>Some more ideas:</p>
<p>Personally, I like to have both one-shot functions and
repeated functions, being able to change the time/interval
and also cancel them. For this, it is useful that the API
returns some kind of identifier for each callback (e.g. an
integer ID). This is what Javascript does with
"setTimeout"/"clearTimeout" and
"setInterval"/"clearInterval". I use a very similar system
for the Lua scripting API of my 2D game engine, but I also
have "resetTimeout" and "resetInterval" functions.</p>
<p>On the other hand, you could also have a look at the
scheduling API of the Supercollider, which is a bit
different: if a routine yields a number N, it means that
the routine will be scheduled again after N seconds.</p>
<p>Generally, having periodic timers is very convenient in a
musical environment :-)<br>
</p>
<p>Christof</p>
<p>*) Don't just store the clock in a global variable,
because Pd can have several instances. Instead, put the
clock in a struct which you allocate in the setup
function. The clock gets this struct as the owner.</p>
<p>typedef struct _myscheduler { t_clock *clock; }
t_myscheduler; // this would also be a good place to store
the priority queue<br>
</p>
<p>t_scheduler *x = getbytes(sizeof(t_myscheduler));</p>
<p>t_clock *clock = clock_new(x,
(t_method)myscheduler_tick);</p>
<p>x->clock = clock;<br>
</p>
<div>On 25.10.2020 02:02, Iain Duncan wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Thanks Christof, that's very helpful.
<div><br>
</div>
<div>iain</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Sat, Oct 24, 2020
at 5:53 PM Christof Ressi <<a href="mailto:info@christofressi.com" target="_blank">info@christofressi.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>But if you're still worried, creating a pool of
objects of the same size is actually quite easy,
just use a <a href="https://en.wikipedia.org/wiki/Free_list" target="_blank">https://en.wikipedia.org/wiki/Free_list</a>.</p>
<p>Christof<br>
</p>
<div>On 25.10.2020 02:45, Christof Ressi wrote:<br>
</div>
<blockquote type="cite">
<p> </p>
<blockquote type="cite">
<div>A) Am I right, both about being bad, and
about clock pre-allocation and pooling being a
decent solution?</div>
<div>B) Does anyone have tips on how one should
implement and use said clock pool?</div>
</blockquote>
ad A), basically yes, but in Pd you can get away
with it. Pd's scheduler doesn't run in the actual
audio callback (unless you run Pd in "callback"
mode) and is more tolerant towards operations that
are not exactly realtime friendly (e.g. memory
allocation, file IO, firing lots of messages,
etc.). The audio callback and scheduler thread
exchange audio samples via a lockfree ringbuffer.
The "delay" parameter actually sets the size of
this ringbuffer, and a larger size allows for
larger CPU spikes.
<p>In practice, allocating a small struct is
pretty fast even with the standard memory
allocator, so in the case of Pd it's nothing to
worry about. In Pd land, external authors don't
really care too much about realtime safety,
simply because Pd itself doesn't either.<br>
</p>
<p>---<br>
</p>
<p>Now, in SuperCollider things are different.
Scsynth and Supernova are quite strict regarding
realtime safety because DSP runs in the audio
callback. In fact, they use a special realtime
allocator in case a plugin needs to allocate
memory in the audio thread. Supercollider also
has a seperate non-realtime thread where you
would execute asynchronous commands, like
loading a soundfile into a buffer.</p>
<p>Finally, all sequencing and scheduling runs in
a different program (sclang). Sclang sends OSC
bundles to scsynth, with timestamps in the near
future. Conceptually, this is a bit similar to
Pd's ringbuffer scheduler, with the difference
that DSP itself never blocks. If Sclang blocks,
OSC messages will simply arrive late at the
Server.<br>
</p>
<p>Christof<br>
</p>
<div>On 25.10.2020 02:10, Iain Duncan wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi folks, I'm working on an
external for Max and PD embedding the S7
scheme interpreter. It's mostly intended to do
things at event level, (algo comp, etc) so I
have been somewhat lazy around real time
issues so far. But I'd like to make sure it's
as robust as it can be, and can be used for as
much as possible. Right now, I'm pretty sure
I'm being a bad real-time-coder. When the user
wants to delay a function call, ie (delay 100
foo-fun), I'm doing the following:
<div><br>
</div>
<div>- callable foo-fun gets registered in a
scheme hashtable with a gensym unique handle</div>
<div>- C function gets called with the handle</div>
<div>- C code makes a clock, storing it in a
hashtable (in C) by the handle, and passing
it a struct (I call it the "clock callback
info struct") with the references it needs
for it's callback</div>
<div>- when the clock callback fires, it gets
passed a void pointer to the
clock-callback-info-struct, uses it to get
the cb handle and the ref to the external
(because the callback only gets one arg),
calls back into Scheme with said handle</div>
<div>- Scheme gets the callback out of it's
registry and executes the stashed function</div>
<div><br>
</div>
<div>This is working well, but.... I am both
allocating and deallocating memory in those
functions: for the clock, and for the info
struct I use to pass around the reference to
the external and the handle. Given that I
want to be treating this code as high
priority, and having it execute as
timing-accurate as possible, I assume I
should not be allocating and freeing in
those functions, because I could get blocked
on the memory calls, correct? I think I
should probably have a pre-allocated pool of
clocks and their associated info structs so
that when a delay call comes in, we get one
from the pool, and only do memory management
if the pool is empty. (and allow the user to
set some reasonable config value of the
clock pool). I'm thinking RAM is cheap,
clocks are small, people aren't likely to
have more than 1000 delay functions running
concurrently or something at once, and they
can be allocated from the init routine.</div>
<div><br>
</div>
<div>My questions:</div>
<div>A) Am I right, both about being bad, and
about clock pre-allocation and pooling being
a decent solution?</div>
<div>B) Does anyone have tips on how one
should implement and use said clock pool?</div>
<div><br>
</div>
<div>I suppose I should probably also be
ensuring the Scheme hash-table doesn't do
any unplanned allocation too, but I can bug
folks on the S7 mailing list for that one...</div>
<div><br>
</div>
<div>Thanks!</div>
<div>iain</div>
</div>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
Pd-dev mailing list
<a href="mailto:Pd-dev@lists.iem.at" target="_blank">Pd-dev@lists.iem.at</a>
<a href="https://lists.puredata.info/listinfo/pd-dev" target="_blank">https://lists.puredata.info/listinfo/pd-dev</a>
</pre>
</blockquote>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
Pd-dev mailing list
<a href="mailto:Pd-dev@lists.iem.at" target="_blank">Pd-dev@lists.iem.at</a>
<a href="https://lists.puredata.info/listinfo/pd-dev" target="_blank">https://lists.puredata.info/listinfo/pd-dev</a>
</pre>
</blockquote>
</div>
_______________________________________________<br>
Pd-dev mailing list<br>
<a href="mailto:Pd-dev@lists.iem.at" target="_blank">Pd-dev@lists.iem.at</a><br>
<a href="https://lists.puredata.info/listinfo/pd-dev" rel="noreferrer" target="_blank">https://lists.puredata.info/listinfo/pd-dev</a><br>
</blockquote>
</div>
</blockquote>
</div>
_______________________________________________<br>
Pd-dev mailing list<br>
<a href="mailto:Pd-dev@lists.iem.at" target="_blank">Pd-dev@lists.iem.at</a><br>
<a href="https://lists.puredata.info/listinfo/pd-dev" rel="noreferrer" target="_blank">https://lists.puredata.info/listinfo/pd-dev</a><br>
</blockquote>
</div>
</blockquote>
</div>
_______________________________________________<br>
Pd-dev mailing list<br>
<a href="mailto:Pd-dev@lists.iem.at" target="_blank">Pd-dev@lists.iem.at</a><br>
<a href="https://lists.puredata.info/listinfo/pd-dev" rel="noreferrer" target="_blank">https://lists.puredata.info/listinfo/pd-dev</a><br>
</blockquote></div>