Re: (ITS#8875) [Patch] Performance problems in back-mdb with large DITs and many aliases

[hbohnenkamp@united-internet.de]

Some months have passed but I finally found the time to set up a test
rig for regression testing of my patch under LIVE conditions.  I'll
describe the setup below.

The good news is, today I found the first bug in my patch (handling of
scope=one searches is incorrect). I say "good" news because it shows
that the testing setup is doing something right. 

In related news, it turns out that apparently the implementation of
alias-dereferencing in the hdb backend seems to be faulty, too... not
wrt performance, but pureley from a functional point of view. We found
a search request which should return around 6700 entries, but only
deliveres around 6150. My patched mdb-implementation returns more
closely to 6700 entries, but for obvious reasons also that might be
wrong still. I had not the time yet to find out what goes wrong in the
hdb backend. I'll open a bug  if I manage to reproduce it somehow
under lab conditions.

My test setup is like this:

I use the productive LDAP content of my company of about 2.4 million
Entries, slapcated from the LDAP master, to test against. All user
passwd hashes are replaced with a default hash so that the test tool
can bind successfully. Slapd configuration is essentially the same as
for our productive slaves (esp. the ACLs).

As test cases I use slapd logfiles of our productive clusters (approx
24 million log lines (stats) per server per 24h). Mostly binds and
searches. Those are directed against the hdb-configured slapd and a
patched-mdb-configured slapd (v 2.4.47). Results are compared: whether
binds succeed or fail, and the returned search results. I will extend
the tester to also include other operations, but this is waht is
relevant to test the patch, I guess.  

In the taxonomy of test-methods I would classify this as "advanced
monkey testing". 

While writing the test scripts I was wondering if somebody else did
ever do something similar. There are of course tests in the test
directory of the Openldap sources, but are there any (regression)
tests made (e.g. for Openldap release candidates) which try to
simulate large  productive setups?

-- 
Henrik Bohnenkamp