[PD] regression testing WAS Re: [psql] object hand-holding

[Hans-Christoph Steiner]

On Dec 21, 2007, at 10:30 AM, IOhannes m zmölnig wrote:

> Mathieu Bouchard wrote:
>> On Mon, 10 Dec 2007, Hans-Christoph Steiner wrote:
>>> What we really need is regression testing, I think that would  
>>> make it
>>> easier to convince Miller to accept patches, and would be a great
>>> tool for finding bugs.  Matju started that project, any word on that
>>> Matju?
>> No, I haven't done much progress. The problem with that thing is  
>> that no-one wants to do it, not even me. It's hard to consider it  
>> a priority as much as it should be.
>
>
> at least i am using "kind-of" regression tests in zexy and iemlib.
> the "suite" is rather simplistic, but works well for my purposes.
>
> see zexy/tests for how it works

Any interest in making this a general purpose testing framework?  I  
would like to come up with a suite that is run every night  
automatically, like the nightly builds.  It sounds very similar to  
what I was thinking.

If we just come up with a standard set of inlets, outlets, and a  
standard output value, then it will be easy to write a script to run  
thru them all and report on them.  Sounds like you might already have  
that working to some degree.

I would like to make a set that tests for the strict aliasing bugs so  
I can safely use gcc optimizations.  This means testing tilde objects.

I think it might make sense to start a section in CVS for this called  
'tests', then we can start gathering tests there into subfolders  
organized by library.  Another option would be a standard folder in  
each library called 'tests' like you have.

That's my two bits, but we really should be doing a lot more of this.

.hc


------------------------------------------------------------------------ 
----

As we enjoy great advantages from inventions of others, we should be  
glad of an opportunity to serve others by any invention of ours; and  
this we should do freely and generously.         - Benjamin Franklin