Large database/directory testing...

[Dave Lindner <david.lindner@compaq.com>]

Built several large test directories with the latest version of
openldap. Built with the 3.1 version of sleepycat. I thought folks
might be interested so...

Hardware: Two rather old AlphaStation 255 (EV4 255 MHz) 192 Meg of memory
          Running Tru64 Unix V5.1. For comparison, today's alphas
	  are using EV6 chips so these boxes are two generations behind.

Built a 1.5 million entry directory. Took approximately 12 hours
to build with the following indexes:

# Indices to maintain
index cn,sn,uid pres,eq
index   objectClass     eq

With dbcache set to 20meg, performance was quite good.

Built a 5 million entry directory. Took approximately 43 hours to build
with the same indexes. Again performance looks quite good. Forgive me
for not giving any real numbers for performance, but I tend to be
paranoid about that stuff in case I screwed something up. 

Total size of 1.5 million db with indexes was around 800 meg. Total
size of 5 million db with indexes was around 4.3 gig. Each entry
was a unix record:

uid=u345234, o=unix
objectClass=top
objectClass=posixAccount
uid=u345234
uidNumber=345234
gidNumber=15
userPassword=u345234
cn=Test User345234
homeDirectory=/home/u345234
loginShell=/bin/ksh

Some (rather obvious) observations:

DB build speed is more dependent on disk io speed, than processor
speed. Netscape directory server (iPlanet) seems to have comparable
build times. This shouldn't be too surprising as I believe they use
virtually the same database. Some performance improvements can be 
achieved if the indexes will fit into memory. At 5 million
my indexes were around 380 meg (your mileage WILL vary). Striping 
(via RAID) the index files also seemed to help performance, although 
not as much as I would have expected. That may help write performance 
as well, although I didn't have a chance to really test that.

Dave