[PD-dev] managing unit tests in SVN

[katja]

On Thu, Nov 3, 2011 at 3:28 PM, Hans-Christoph Steiner <hans at at.or.at> wrote:

> - each library would have a 'unittests' folder for the tests that are specific to that
> library.  Ideally each object would have a test patch in the 'unittests' subfolder in
> the library, but we're a ways off of that.


If I understand you well, one would get a subdir 'unittests' for every
lib like so: externals/creb/unittest/,
externals/miXed/cyclone/unittest/, externals/unauthorized/unittest/
etc. A copy of the test abstractions should be in every lib's
unittest/ dir, no? (In case a lib is copied individually from SVN).


A few remarks on unit test abstractions. There's two flavours now:

- [unit-test-frame~] for signal objects
- [unit-test-frame] for control objects with numeric output (floats
and/or lists of floats)

Both flavours store a 512 points reference file as 32 bit float .wav,
the most precise storage method in single precision vanilla Pd. See
http://www.katjaas.nl/pdunittests/pdunittests.html for screenshots and
.zip file with abstractions and some examples.

Of course, when testing an object, all other objects in the test patch
are part of the system under test as well. This is not only a
theoretic concern, as I soon found when testing double-precision-built
externals against their Pd-extended 0.42 reference. Signal classes in
Pd are designed for speed, not for accuracy. It is sometimes
inevitable to accumulate inaccuracies, for example when using
[phasor~] as test input signal for a filter object. Obviously, it is
impossible to test the exact accuracy of double precision objects
against a reference in single precision. Large deviations tell real
troubles, that is where the tests are most useful. For example, when a
double is read as type float due to erroneous pointer aliasing, output
is totally ridiculous. Or aggressive optimization may sometimes induce
unintended order of operations, mostly leading to crap output.

For all creb and smlib classes, unit tests are now written in this
sense, over 70 tests together. A fraction of the total to be done for
Pd-extended. Test development still takes more time than the actual
rewriting of bits in the code, even while the test abstractions save
loads of work. But for Pd-double in particular, it is indispensable.

Katja