Re: Updating the test suite

[Hallvard B Furuseth <h.b.furuseth@usit.uio.no>]

Pierangelo Masarati writes:
>Hallvard B Furuseth writes:
>> * Factor a lot of code out to shell functions in defines.sh.  (...)
>>   Shell functions are fairly portable now, and in any case OpenLDAP
>>   uses build/shtool which uses shell functions.  (...)
>
> You seem to be fairly optimistic about this.  I have no real arguments
> against; I hope you're correct.

I wasn't optimistic until I looked around a bit - and then noticed the
shell functions in shtool:-)  Info node (autoconf)Shellology says

     While most (at least most System V's) do have a Bourne shell that
     accepts shell functions most vendor `/bin/sh' programs are not the
     POSIX shell.

     So while most modern systems do have a shell _somewhere_ that
     meets the POSIX standard, the challenge is to find it.

Mascheck says this section is outdated and only Ultrix is still
relevant.  Since there are no relevant ITSes about shtool either,
that seems to be right at least for OpenLDAP.

Still, we chould allow the user to set a variable=<a shell which
supports functions> (both for shtool and tests), and if a user arrives
with such a need, see if we can include a configure test which finds
that shell.  I can ask on the autoconf list about that.

>> * Let ./run and scripts/all look for backend-specific as well as
>>   general tests.  Remove scripts/sql-all.
>>
>>   "./run -b foo all" (used by "make test" for backend foo) would
>>   first run scripts/foo-test*.  Then for bdb, ldbm, hdb or if a
>>   "-general" (-g) flag is given, also run scripts/test*.
>>
>>   scripts/all would first do ". $SRCDIR/scripts/foo-setup" or
>>   something, so we have some place to put the message in
>>   scripts/sql-all.
>>
>>   Maybe this will encourage someone to write tests for more
>>   backends:-)
>
> I like the idea in principle; one objection is that, for example, proxy
> testing may use different backends for storage.  Currently, they get
> tested against all configured storage types, while according to the
> suggested partitioning they would be tested by default only against one
> type (the "default"?  selected by the user?  what if all have to be
> tested?).

No, this is about what happens once you have chosen a backend, not
about which backend to choose.

There is no change in the default behaviour with the existing tests:
"make test" would still walk through "./run -b <backend> all" for all
backends supported by the test suite.  There are currently no
backend-specific tests for BDB, LDBM and HDB, while 'make test' for
SQL curently only runs the sql-specific tests.

-- 
Hallvard