[PD-dev] managing unit tests in SVN

[Hans-Christoph Steiner]

On Nov 3, 2011, at 3:56 PM, katja wrote:

> On Thu, Nov 3, 2011 at 3:28 PM, Hans-Christoph Steiner <hans at at.or.at> wrote:
> 
>> - each library would have a 'unittests' folder for the tests that are specific to that
>> library.  Ideally each object would have a test patch in the 'unittests' subfolder in
>> the library, but we're a ways off of that.
> 
> 
> If I understand you well, one would get a subdir 'unittests' for every
> lib like so: externals/creb/unittest/,
> externals/miXed/cyclone/unittest/, externals/unauthorized/unittest/
> etc.

Right, that's what I'm thinking, except having the folder called 'unittests' for grammatical consistency.

> A copy of the test abstractions should be in every lib's
> unittest/ dir, no? (In case a lib is copied individually from SVN).

I think the test abstractions should probably be distributed as a regular library, and included in Pd-extended.

> A few remarks on unit test abstractions. There's two flavours now:
> 
> - [unit-test-frame~] for signal objects
> - [unit-test-frame] for control objects with numeric output (floats
> and/or lists of floats)
> 
> Both flavours store a 512 points reference file as 32 bit float .wav,
> the most precise storage method in single precision vanilla Pd. See
> http://www.katjaas.nl/pdunittests/pdunittests.html for screenshots and
> .zip file with abstractions and some examples.
> 
> Of course, when testing an object, all other objects in the test patch
> are part of the system under test as well. This is not only a
> theoretic concern, as I soon found when testing double-precision-built
> externals against their Pd-extended 0.42 reference. Signal classes in
> Pd are designed for speed, not for accuracy. It is sometimes
> inevitable to accumulate inaccuracies, for example when using
> [phasor~] as test input signal for a filter object. Obviously, it is
> impossible to test the exact accuracy of double precision objects
> against a reference in single precision. Large deviations tell real
> troubles, that is where the tests are most useful. For example, when a
> double is read as type float due to erroneous pointer aliasing, output
> is totally ridiculous. Or aggressive optimization may sometimes induce
> unintended order of operations, mostly leading to crap output.
> 
> For all creb and smlib classes, unit tests are now written in this
> sense, over 70 tests together. A fraction of the total to be done for
> Pd-extended. Test development still takes more time than the actual
> rewriting of bits in the code, even while the test abstractions save
> loads of work. But for Pd-double in particular, it is indispensable.

The standard deviation stuff is definitely very valuable, its great you got that working.

.hc



----------------------------------------------------------------------------

I have always wished for my computer to be as easy to use as my telephone; my wish has come true because I can no longer figure out how to use my telephone."  --Bjarne Stroustrup (creator of C++)