index files grow with repeated add and delete (ITS#186)

[mharris@intermec.com]

Full_Name: Mike Harris
Version: current stable version
OS: HP-UX
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (204.57.247.20)


I'm running slapd 3.3 on hpux using ndbm as the backend database.  I built
it from source downloaded from www.umich.edu.  I have a simple test script
that repeatedly adds and deletes a single entry from LDAP.  I have observed
that the following backend database files grow over time when running the
test script:

	dn.ndbm.pag
	objectclass.ndbm.pag

The growth of the other files seems to  be limited only by the space available 
on the disk.  

In the umich source, I see calls to dn2id_delete() and id2entry_delete() in 
ldbm_back_delete() (back-ldbm/delete.c) that presumably update the 
dn2id.ndbm.pag and id2entry.ndbm.pag files when an entry is deleted.

In openldap source, I see that a call to id2children_remove() has been added 
to ldbm_back_delete().  I presume that this call updates the 
id2children.ndbm.pag file when an entry is deleted.  

In neither source do I find where the index files dn.ndbm.pag and 
objectclass.ndbm.pag are updated when an entry is deleted.  Are the index
files updated elsewhere in openldap?   Or is this a bug in both LDAP
implementations?  Is there a work-around other than using ldbmcat and ldif2ldbm
to dump and recreate the LDAP database?

To create the database:

ldif2ldbm -f slapd.conf -i domain.add

To invoke the test script:

	$ adddelete.ksh person.add person.del

adddelete.ksh:

while [ true ] 
do
    ldapadd -D "dc=MyDomain" -w secret -f $1
    ldapdelete -D "dc=MyDomain" -w secret -f $2
done

person.add:

dn: cn=Nat, dc=MyDomain
objectclass: person
sn: Cole 
cn: Nat

person.del:

cn=Nat, dc=MyDomain

domain.add:

dn: dc=MyDomain
dc: MyDomain
objectclass: domain

slapd.conf:

include         slapd.at.conf
include         slapd.oc.conf
schemacheck     on

#######################################################################
# ldbm database definitions
#######################################################################

database        ldbm
suffix          "dc=MyDomain"
directory       .
rootdn          "dc=MyDomain"
rootpw          secret
lastmod off

Sizes before running test script:

-rw-------   1 mharris    intermec      1024 May 28 10:57 dn.ndbm.pag
-rw-------   1 mharris    intermec      1024 May 28 10:57 dn2id.ndbm.pag
-rw-------   1 mharris    intermec      1024 May 28 10:57 id2children.ndbm.pag
-rw-------   1 mharris    intermec      1024 May 28 10:57 id2entry.ndbm.pag
-rw-------   1 mharris    intermec      1024 May 28 10:57 objectclass.ndbm.pag

Sizes after running test script for about 2 minutes:

-rw-------   1 mharris    intermec     86016 May 28 10:59 dn.ndbm.pag
-rw-------   1 mharris    intermec      1024 May 28 10:59 dn2id.ndbm.pag
-rw-------   1 mharris    intermec      1024 May 28 10:59 id2children.ndbm.pag
-rw-------   1 mharris    intermec      1024 May 28 10:59 id2entry.ndbm.pag
-rw-------   1 mharris    intermec      2048 May 28 10:59 objectclass.ndbm.pag