Issue 186 - index files grow with repeated add and delete
Summary: index files grow with repeated add and delete
Status: VERIFIED FIXED
Alias: None
Product: OpenLDAP
Classification: Unclassified
Component: slapd (show other issues)
Version: unspecified
Hardware: All All
: --- normal
Target Milestone: ---
Assignee: OpenLDAP project
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 1999-05-28 20:14 UTC by mharris@intermec.com
Modified: 2014-08-01 21:06 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description mharris@intermec.com 1999-05-28 20:14:15 UTC
Full_Name: Mike Harris
Version: current stable version
OS: HP-UX
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (204.57.247.20)


I'm running slapd 3.3 on hpux using ndbm as the backend database.  I built
it from source downloaded from www.umich.edu.  I have a simple test script
that repeatedly adds and deletes a single entry from LDAP.  I have observed
that the following backend database files grow over time when running the
test script:

	dn.ndbm.pag
	objectclass.ndbm.pag

The growth of the other files seems to  be limited only by the space available 
on the disk.  

In the umich source, I see calls to dn2id_delete() and id2entry_delete() in 
ldbm_back_delete() (back-ldbm/delete.c) that presumably update the 
dn2id.ndbm.pag and id2entry.ndbm.pag files when an entry is deleted.

In openldap source, I see that a call to id2children_remove() has been added 
to ldbm_back_delete().  I presume that this call updates the 
id2children.ndbm.pag file when an entry is deleted.  

In neither source do I find where the index files dn.ndbm.pag and 
objectclass.ndbm.pag are updated when an entry is deleted.  Are the index
files updated elsewhere in openldap?   Or is this a bug in both LDAP
implementations?  Is there a work-around other than using ldbmcat and ldif2ldbm
to dump and recreate the LDAP database?

To create the database:

ldif2ldbm -f slapd.conf -i domain.add

To invoke the test script:

	$ adddelete.ksh person.add person.del

adddelete.ksh:

while [ true ] 
do
    ldapadd -D "dc=MyDomain" -w secret -f $1
    ldapdelete -D "dc=MyDomain" -w secret -f $2
done

person.add:

dn: cn=Nat, dc=MyDomain
objectclass: person
sn: Cole 
cn: Nat

person.del:

cn=Nat, dc=MyDomain

domain.add:

dn: dc=MyDomain
dc: MyDomain
objectclass: domain

slapd.conf:

include         slapd.at.conf
include         slapd.oc.conf
schemacheck     on

#######################################################################
# ldbm database definitions
#######################################################################

database        ldbm
suffix          "dc=MyDomain"
directory       .
rootdn          "dc=MyDomain"
rootpw          secret
lastmod off

Sizes before running test script:

-rw-------   1 mharris    intermec      1024 May 28 10:57 dn.ndbm.pag
-rw-------   1 mharris    intermec      1024 May 28 10:57 dn2id.ndbm.pag
-rw-------   1 mharris    intermec      1024 May 28 10:57 id2children.ndbm.pag
-rw-------   1 mharris    intermec      1024 May 28 10:57 id2entry.ndbm.pag
-rw-------   1 mharris    intermec      1024 May 28 10:57 objectclass.ndbm.pag

Sizes after running test script for about 2 minutes:

-rw-------   1 mharris    intermec     86016 May 28 10:59 dn.ndbm.pag
-rw-------   1 mharris    intermec      1024 May 28 10:59 dn2id.ndbm.pag
-rw-------   1 mharris    intermec      1024 May 28 10:59 id2children.ndbm.pag
-rw-------   1 mharris    intermec      1024 May 28 10:59 id2entry.ndbm.pag
-rw-------   1 mharris    intermec      2048 May 28 10:59 objectclass.ndbm.pag

Comment 1 Kurt Zeilenga 1999-06-01 22:10:31 UTC
changed notes
changed state Open to Closed
Comment 2 Kurt Zeilenga 1999-06-01 22:11:29 UTC
Both OpenLDAP 1.x does not attempt to delete indexes for
a variety of reasons (see -devel archives).  This behavior
will like change for 2.0, if not sooner.  It's not a bug,
per say, but a feature of the implementation.

Kurt

At 08:14 PM 5/28/99 GMT, mharris@intermec.com wrote:
>Full_Name: Mike Harris
>Version: current stable version
>OS: HP-UX
>URL: ftp://ftp.openldap.org/incoming/
>Submission from: (NULL) (204.57.247.20)
>
>
>I'm running slapd 3.3 on hpux using ndbm as the backend database.  I built
>it from source downloaded from www.umich.edu.  I have a simple test script
>that repeatedly adds and deletes a single entry from LDAP.  I have observed
>that the following backend database files grow over time when running the
>test script:
>
>	dn.ndbm.pag
>	objectclass.ndbm.pag
>
>The growth of the other files seems to  be limited only by the space available 
>on the disk.  
>
>In the umich source, I see calls to dn2id_delete() and id2entry_delete() in 
>ldbm_back_delete() (back-ldbm/delete.c) that presumably update the 
>dn2id.ndbm.pag and id2entry.ndbm.pag files when an entry is deleted.
>
>In openldap source, I see that a call to id2children_remove() has been added 
>to ldbm_back_delete().  I presume that this call updates the 
>id2children.ndbm.pag file when an entry is deleted.  
>
>In neither source do I find where the index files dn.ndbm.pag and 
>objectclass.ndbm.pag are updated when an entry is deleted.  Are the index
>files updated elsewhere in openldap?   Or is this a bug in both LDAP
>implementations?  Is there a work-around other than using ldbmcat and ldif2ldbm
>to dump and recreate the LDAP database?
>
>To create the database:
>
>ldif2ldbm -f slapd.conf -i domain.add
>
>To invoke the test script:
>
>	$ adddelete.ksh person.add person.del
>
>adddelete.ksh:
>
>while [ true ] 
>do
>    ldapadd -D "dc=MyDomain" -w secret -f $1
>    ldapdelete -D "dc=MyDomain" -w secret -f $2
>done
>
>person.add:
>
>dn: cn=Nat, dc=MyDomain
>objectclass: person
>sn: Cole 
>cn: Nat
>
>person.del:
>
>cn=Nat, dc=MyDomain
>
>domain.add:
>
>dn: dc=MyDomain
>dc: MyDomain
>objectclass: domain
>
>slapd.conf:
>
>include         slapd.at.conf
>include         slapd.oc.conf
>schemacheck     on
>
>#######################################################################
># ldbm database definitions
>#######################################################################
>
>database        ldbm
>suffix          "dc=MyDomain"
>directory       .
>rootdn          "dc=MyDomain"
>rootpw          secret
>lastmod off
>
>Sizes before running test script:
>
>-rw-------   1 mharris    intermec      1024 May 28 10:57 dn.ndbm.pag
>-rw-------   1 mharris    intermec      1024 May 28 10:57 dn2id.ndbm.pag
>-rw-------   1 mharris    intermec      1024 May 28 10:57 id2children.ndbm.pag
>-rw-------   1 mharris    intermec      1024 May 28 10:57 id2entry.ndbm.pag
>-rw-------   1 mharris    intermec      1024 May 28 10:57 objectclass.ndbm.pag
>
>Sizes after running test script for about 2 minutes:
>
>-rw-------   1 mharris    intermec     86016 May 28 10:59 dn.ndbm.pag
>-rw-------   1 mharris    intermec      1024 May 28 10:59 dn2id.ndbm.pag
>-rw-------   1 mharris    intermec      1024 May 28 10:59 id2children.ndbm.pag
>-rw-------   1 mharris    intermec      1024 May 28 10:59 id2entry.ndbm.pag
>-rw-------   1 mharris    intermec      2048 May 28 10:59 objectclass.ndbm.pag
>
>
>
>
Comment 3 OpenLDAP project 2014-08-01 21:06:08 UTC
Provided story.