[PD-dev] signal object test template for automated unit testing

[katja]

Hello,

Here is a .zip with an improved method for unit-testing of signal
objects. Instead of a template, there is now an abstraction
'unit-test-frame~.pd' handling the tests. The input of that
abstraction consists of:

- name of the reference file (32 bit float .wav file, like Roman suggested)
- signal (or system) under test
- optional delay between test-trigger and test-snapshot (for filters etc.)

The reference file can simply be created from within the unit test
patch (running in a 'good' Pd), and must be kept with the test and the
abstraction in a directory. The abstraction compares signal under test
with reference, computes standard deviation, and sends a tagged result
to Pd window and stdout:

STDDEV-ZERO 0 reference: <reference> (when stddev = 0)
STDDEV-SMALL <stddev> reference: <reference> (when 0 < stddev < 1e+05)
STDDEV-LARGE <stddev> reference: <reference> (when stddev >= 1e+05)

The tags facilitate grepping different categories of 'errors' from a
test log. Note that a small stddev may sometimes indicate improved
accuracy in an object! Like when Pd-double is tested against current
Pd.

By the way I did not find a way to set samplerate for a test. I
definitely do not want the test patch to mess up other audio settings,
like number of channels. Therefore an audio-dialog message to pd is
not an option. However there's an inlet reserved on
unit-test-frame~.pd for possible future solution of this issue. From a
remark by Roman Haefeli in another thread,
http://lists.puredata.info/pipermail/pd-dev/2011-10/017694.html, I
learnt that -noaudio may be a useful option for automated signal
object testing.


Please report if you have comments on the proposed test stuff. I
should mention a hidden agenda behind the efforts: the intention is to
produce dozends or even hundreds of unit-tests in short time, and use
these for testing Pd-double. Then we could finally make some
systematic progress in that field. Apart from that, the tests could be
useful for rewrites/refactorings in general, and for tracking down
optimizer-troubles.

Katja




.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: unit-tests.zip
Type: application/zip
Size: 9762 bytes
Desc: not available
URL: <http://lists.puredata.info/pipermail/pd-dev/attachments/20111026/94b4e736/attachment.zip>