[PD] PdDroidParty latency, syncing etc.

[Chris McCormick]

Hi Gavin!

On 09/12/14 19:27, Gavin wrote:
> Is it ok to email you about this or would you prefer to keep it to the
> forums?

I hope you don't mind me CC'ing this to the Pd-list.

I'm happy you emailed about this because it is exactly what I have been
working on lately when I find spare time.

> I'm a bit confused as to the network options with pddroidparty. 
> There is netro which seems to be based on netsend and x-netreceive but
> also droidnetreceive and also netreceive!
> Which is the recommended or latest one?

The native netsend and netreceive now work in the version of Pd that
ships with PdDroidParty. The other libraries are deprecated. Sorry for
the confusion - there was a long standing bug with netreceive not
working in libpd that is now resolved. x-netreceive was just my way of
commenting out netreceive - you can change it back to netreceive and
remove the top level droidnetreceive hack object and netro should just
work again.

Netro is an abstraction for doing timing synchronisation of Pd patches
that runs on top of netreceive. I haven't updated it for a long time
because I've been working on something that does the job better:

http://github.com/chr15m/SyncJams/

SyncJams uses OSC underneath and so it can interoperate more widely with
other software. There is also a Python implementation and I am hoping
for implementations in other languages (especially Supercollider) down
the track. Patches welcome!

SyncJams provides a way to not only sync metronomes during live
performance but also a sort of distributed hash table ("set-state")
where everybody involved in a jam can share state information like
controller position, song key, etc. There is also a generic conduit for
in-time events ("message") to be propagated in order such as drum beat
hits from a live drummer.

I've been testing on Android, and latency aside things are working
pretty smoothly now.

> Also is there anyway to debug/print to a console on a phone?

If you use Pd's print object you can then use `adb logcat` to see what
comes out - you'll need to have your phone in debugging mode and
connected via USB or over TCP to adb.

> I'm hoping to guesstimate latency by calculating the time it takes to
> send data from one device to another and back and sync phasors on
> separate devices.
> 
> Have you tried this?

So latency is the exact problem I am tackling at the moment.

After speaking to Peter Brinkmann, the resident Android audio expert,
the outlook is as abysmal as ever. The way audio is implemented on
Android means you can't ever really know with 100% certainty how long it
takes between a software event and the speaker actually emitting the
triggered sound.

My approach, which is hopefully "good enough" is similar to what you
suggest, except that a single device can send a signal to its own
microphone to measure the round trip time. Hopefully half of the round
trip time will give the approximate output latency. This is what the
android-latency-tester patch did and I will probably use that same
method (with the bonk~ object).

<http://puredata.info/docs/AndroidLatency/>

I tested a FFT based cross correlation approach to detect the
same-device latency, but it wasn't really robust enough. That is in the
SyncJams repository if you want to take a look. Maybe I did something
wrong though as my DSP math is not very strong.

There is also Raph Levien's audio buffer size tool:

https://play.google.com/store/apps/details?id=com.levien.audiobuffersize

Which uploads its results for different devices:
http://audiobuffersize.appspot.com/

Code:
https://code.google.com/p/high-performance-audio/source/browse/audio-buffer-size/src/com/levien/audiobuffersize/AudioBufferSize.java

In the end I think the approach on Android is going to be:

 * Alert the user the first time they run the patch that latency needs
to be tested.
 * "Please turn up your speakers while we run the test."
 * Collect the timing data using bonk~ or FFT cross correlation or
whatever and write it to disk.
 * Everybody who makes an Android Pd app can use the same filename,
location, and format for the data (/sdcard/Pd/latency-test.txt) so
hopefully it only has to be done once.
 * Use the values in that file to compensate for latency.
 * Provide a menu option for the user to re-run the compensation if they
feel latency is getting out of hand.

> PS One more thing - where can I get x-netreceive so I can install it on
> my laptop. Is it the same as xnetreceive?

x-netreceive is not a real object - it's what I used to comment out the
existing netreceive object in netro when that wasn't working under
PdDroidParty. What you can do is replace it with netreceive instead, and
remove droidnetreceive entirely. Things should work after that.

Let us know how you go!

Cheers,

Chris.

-- 
http://mccormick.cx/