[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: Updating the test suite (Was: commit: ldap/tests run.in)



Hallvard B Furuseth wrote:
Pierangelo Masarati writes:
In any case this is mainly a hack to allow importing the project
under AEGIS; I don't either like having to source defines.sh twice
only to get few directories. Maybe directories should simply be
set up in run and passed to defines.sh thru the environment?

I suppose ./run could run the scripts by sourcing them instead: test -x "$SCRIPT" && (. "$SCRIPT") Then the script will see variables from ./run, not just environment variables. It gives me a slightly uncomforable feel but should work fine. At least on Unix, don't know about other systems. E.g. skipping the 'test -x' would probably be a bad idea, so it should not be ported to a system where one cannot do that.


I'd like to update the test suite in various ways anyway. Comments welcome.

(Note that I only know Unix, not Cygwin or Mingw.  Or AEGIS,
whatever that is.)

* Factor a lot of code out to shell functions in defines.sh.  Then
  we can also easily move code between ./run and the test scripts.
  With functions to handle the details we'll also make fewer errors
  like sometimes forgetting to kill the daemons before exit.

The libtool folks have been debating this as well. I think consensus there is that it's safe to rely on shell functions today, sounds fine to me.

* Run each LDAP client and daemon as something like
$LDAP_TESTER <program> <args>...
where the user can set the (normally unset) $LDAP_TESTER
environment variable to e.g. valgrind or "xterm -e gdb --args".
Actually, we could use 4 variables:
$LDAP_RETCODE_TESTER for clients whose return code (other than
success/failure?) is important, defaulting to:
$LDAP_FG_TESTER for foreground processes, defaulting to:
$LDAP_TESTER for any process, and
$LDAP_BG_TESTER for background processes (default $LDAP_TESTER).

I've gotten pretty used to manually invoking valgrind when I want it. Dunno...
* Give ./run an "-ignore" (-i) argument, to ignore some errors and
keep running the tests. scripts/all would not abort if a test
fails, and specific checks in each script could be marked as
"soft" (keep going under ./run -i) or "hard" (always abort).

Perhaps.
* The tests should wait for daemons to exit after killing them, and
fail if they failed. Then we can also omit the sleeps after each
test in scripts/all, and a few others.

I was actually just about to commit a change for this. But now that we're discussing this restructuring, I'll hold off because there is probably a better way to organize things than my current patch.


We should also refuse the -k (KILLSERVERS) option if more than one test is being run.
* While waiting for slapd to start,
- sleep briefly after first failure and longer in each iteration.
- If ldapsearch fails, abort the loop if slapd is not running.
- Do not sleep 5 seconds before exiting the loop and failing.

Yes. The excessive sleeps here were getting annoying. I don't know about waiting for up to 30 or more seconds; if the server doesn't start up right away it probably isn't going to start up at all. Even on the slowest dev system I used (an OS/390 beast) it usually started within 10 seconds.
* "./run all" should only run tests matching "scripts/*[a-zA-Z0-9]"
  or something, so Emacs backup files "testxyz-foo.~19~" are not run.

  What are the backup file names to avoid from other editors?

Does Cygwin support filename expansion like *[a-zA-Z0-9], or must
this be handled with a case statement in scripts/all?

Filename expansion is implemented by the shell, not the operating system. "Does Cygwin support it" is a meaningless question. Does Cygwin support Unix-style command shells - of course, that's one of its reasons for existence.
* Allow "./run scripts/scriptname" and not just "./run scriptname",
so one can use filename expansion when typing a test command:
"./run scripts/<start-of-filename><tab>"
This seems unnecessary. Since run already knows to look for arg*, and you have to supply a unique prefix anyway, this is more work to invoke than the existing functionality.

i.e. "./run test033" is less to type than "./run scripts/test033<tab>" so what's the point?

# Kill_named <pid> || exit $?
# Kills and waits for one process.
# If the process returns failure or had died, runs Kill_all as well.
# Returns 0 iff the process was running and returned success when killed.
Kill_named () {
eval "fV_name=\${fV_PIDNAME$1-child} fV_PID$1="
echo "Stopping and waiting for $fV_name."
kill -HUP $1 && wait $1 && return 0
echo "$fV_name (pid $1) failed ($?)!" >&2
Kill_all
return 99
}

This function must unset the particular PID from fv_PIDVARS, otherwise Kill_all will complain.


--
 -- Howard Chu
 Chief Architect, Symas Corp.  http://www.symas.com
 Director, Highland Sun        http://highlandsun.com/hyc
 OpenLDAP Core Team            http://www.openldap.org/project/