[PD-dev] signal object test template for automated unit testing

[Hans-Christoph Steiner]

On Monday, October 24, 2011 10:11 PM, "katja" <katjavetter at gmail.com>
wrote:
> Hans, thanks for your comments
> 
> On Mon, Oct 24, 2011 at 7:35 PM, Hans-Christoph Steiner <hans at at.or.at>
> wrote:
> >
> > That looks really good on many levels. :)  I like the layout, I hadn't
> > thought of standard deviation, that makes sense as long as we can specify
> > "exact" as a possibility.  My guess is that some of this stuff should
> > produce the same bit sequence every time, but I could be wrong there.
> 
> If you specify tolerance zero, that's exact. But we found that
> tolerance is needed for different reasons:
> 
> - floats are stored as text by Pd, and differences between computed
> and stored values occur because of the truncated stored floats
>
> - there's small differences between single precision and double precision
> Pd
>
> - objects with a 'memory' (like IIR filters) produce slightly
> different results if you test them repeatedly with the same input
> sequence

Ah, course, makes sense.  The third item there, the IIR filters, it
should be not too hard to reproduce the exact same operation with them
too.  With the tests, each one is run in a new Pd instance, so they're
always starting from scratch.  Pd is then quit, and restarted for the
next test.


 
> > Perhaps the subpatches should be abstractions as part of a 'test' library.
> >  If there was a bug or a new feature needed in any of those subpatches, it
> > would be rough to have to modify all of the tests.
> 
> Of course, thanks. I'll redesign it in that sense.
> 
> > About the sample rate, it should be possible to have each patch set the
> > sample rate it needs.  You can see hcs/get-audio-dialog-help.pd or the
> > mediasettings library for ways to do that.  I think we'll want to test at
> > different sample rates some day, but if its easier for now, we can stick to
> > 44100.
> 
> The idea was to only use vanilla objects around the object under test,
> so you don't depend on external libs for testing. Do you know if it's
> possible to send standard messages like 'audio-dialog 1 0 0 0 2 0 0 0
> 0 0 0 0 2 0 0 0 44100 20 0' or 'audio-dialog 1 0 0 0 2 0 0 0 0 0 0 0 2
> 0 0 0 48000 20 0' to pd on all systems, without requesting data first?
> I don't think so. If the device has different number of channels
> (instead of 2), Pd can not sync. And if the device doesn't allow the
> alternative samplerate, it won't work either. Samplerate 44100 is
> safest.

[pd audio-dialog 1 0 0 0 2 0 0 0 0 0 0 0 2 0 0 0 48000 20 0( is the
message that the Audio Settings preference panel sends when you click
OK. So it'll work anywhere.
 


> By the way if Pd does not sync with an audio device for whatever
> reason, you get weird test results anyhow. Even if you do not want to
> actually hear the sound, the audio device must work well, for these
> signal object tests. That was one of the first things we observed in
> practice.

Good to know.  We should be able to get a stable audio device setup with
the build farm, so that shouldn't be an issue.

.hc