[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
RE: why is adding records slow?
I have analyzed add performance for a while and have been getting much
better performance than these results.
At present I can load about 10,000 records, each in about 60 secs. This
result is obtained using:
Linux RH 6.2
Open LDAP 2.0.6
Berkeley DB 3.2.9 (make sure you are using BDB for the best results)
Also, I am using the C API to add the entries. I open a single connection
and then repeatedly add the entries. Are you calling ldapadd repeatedly
(i.e. 10k times) to add your entries? If you are, that might explain your
results as ldapadd opens a new connection each time.
The hardware I use is a PIII 750 server, 500MB, w. SCSI disks. However, I
get similar results (maybe 10k in 100 secs) with an off the shelf PIII 500
client, 128MB, EIDE drive.
A couple of things which give huge write performance boosts (which your
results confirm):
- turning off db syncing
- turning off substring indexes (equality indexes don't make much
difference)
Mark
-----Original Message-----
From: owner-openldap-software@OpenLDAP.org
[mailto:owner-openldap-software@OpenLDAP.org]On Behalf Of Ricky Charlet
Sent: Tuesday, June 12, 2001 12:11 PM
To: openldap-software@OpenLDAP.org
Subject: Re: why is adding records slow?
Howdy,
I made a cut and past error in the names of my tests. The sequece of
tests should be named First Second Third and Fourth, instead of First
Second Second and Second.
>
> Test = add 10K records to an empty ldbm database through ldapadd.
> Measure time to completion.
>
> slapd.conf for First run:
> -----------------------
> schemacheck on
> cachesize 100000
> dbcachesize 15000000
> dbnosync
> #dbcachenosync
> pidfile /var/run/slapd.pid
> argsfile /var/run/slapd.args
> database ldbm
> suffix "dc=ricky, dc=com"
> rootdn "cn=Manager, dc=ricky, dc=com"
> rootpw "******"
> directory /var/lib/ldap
> # Indices to maintain
> index objectClass,uid,uidNumber,gidNumber eq
> index cn,mail,surname,givenname eq,subinitial
>
> time to add 10K records = 7692 seconds
>
> ======================================================
>
> slapd.conf for Second run: (turn off indexing)
> -----------------------
> schemacheck on
> cachesize 100000
> dbcachesize 15000000
> dbnosync
> #dbcachenosync
> pidfile /var/run/slapd.pid
> argsfile /var/run/slapd.args
> database ldbm
> suffix "dc=ricky, dc=com"
> rootdn "cn=Manager, dc=ricky, dc=com"
> rootpw "******"
> directory /var/lib/ldap
> # Indices to maintain eq
> #index objectClass,uid,uidNumber,gidNumber eq
> #index cn,mail,surname,givenname eq,subinitial
>
> time to add 10K records = 2516 seconds
>
> ======================================================
>
> slapd.conf for Second run: (turn on dbcachenosync)
> -----------------------
> schemacheck on
> cachesize 100000
> dbcachesize 15000000
> dbnosync
> dbcachenosync
> pidfile /var/run/slapd.pid
> argsfile /var/run/slapd.args
> database ldbm
> suffix "dc=ricky, dc=com"
> rootdn "cn=Manager, dc=ricky, dc=com"
> rootpw "******"
> directory /var/lib/ldap
> # Indices to maintain eq
> #index objectClass,uid,uidNumber,gidNumber eq
> #index cn,mail,surname,givenname eq,subinitial
>
> time to add 10K records = 1900 seconds
>
> ======================================================
>
> slapd.conf for Second run: (turn schemacheck to off)
> -----------------------
> schemacheck off
> cachesize 100000
> dbcachesize 15000000
> dbnosync
> dbcachenosync
> pidfile /var/run/slapd.pid
> argsfile /var/run/slapd.args
> database ldbm
> suffix "dc=ricky, dc=com"
> rootdn "cn=Manager, dc=ricky, dc=com"
> rootpw "******"
> directory /var/lib/ldap
> # Indices to maintain eq
> #index objectClass,uid,uidNumber,gidNumber eq
> #index cn,mail,surname,givenname eq,subinitial
>
> time to add 10K records = 1900 seconds (no change from last test)
>
> ======================================================
>
> --
> Ricky Charlet : Redcreek Communications : usa (510) 795-6903
--
Ricky Charlet : Redcreek Communications : usa (510) 795-6903