[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: why is adding records slow?
Randy Kunkee wrote:
>
> "Mark H. Wood" wrote:
>
> > On Mon, 11 Jun 2001, Ricky Charlet wrote:
> > > Thanks for your response, Gabor. Yes, I have misjudged the appropriate
> > > tool for my job. I will have to find something other than LDAP. But on
> > > my way out the door, I was just wondering if anyone could tell me where
> > > that computing time is being spent during ldapadds. I'm curious you see.
> >
> > I too have trouble believing that network delays and protocol processing
> > are taking the bulk of that 0.77sec/transaction, but we need evidence.
> > You're free to build the package with profiling enabled and find out for
> > yourself where the time is going, and if you would publish the results I'm
> > sure we'd all appreciate knowing where the hot spots are.
> >
Thanks Mark:
Might be a little advanced for me, but I'll see if I can find time and
give it a whirl.
> > For a quick-n-dirty estimate, you could just run your load job again
> > (after recreating the empty DIB!) while monitoring the server with 'top'
> > or whatever is equivalent, seeing whether the machine is CPU-bound or
> > I/O-bound, and whether there is free memory available. That will give you
> > a *very* rough idea of whether the time is going to the protocol, the
> > backing store, or an underconfigured server.
Neither slapd nor ldapadd taxed either my memory or my cpu to any
noteworthy degree. The load on my machien stated below 1. slapd took at
most 1.7%CPU/ 1.5%MEM, and ldapadd took at most .7%CPU/.5%MEM. Sorry
that I don't have stats about how IO bound the processes are, I'm not
sure how to find that info.
> >
> > --
> > Mark H. Wood, Lead System Programmer mwood@IUPUI.Edu
> > Make a good day.
>
> I haven't seen anybody suggest turning on "dbcachenosync" yet. By default,
> OpenLDAP syncs the db to disk after every update, which will certainly slow
> things down. If you disable this, you should get much better performance.
>
> Randy
Thanks Randy:
This helped a bit. I got another significant speed increase. Here is a
summary of my results so far:
Test = add 10K records to an empty ldbm database through ldapadd.
Measure time to completion.
slapd.conf for First run:
-----------------------
schemacheck on
cachesize 100000
dbcachesize 15000000
dbnosync
#dbcachenosync
pidfile /var/run/slapd.pid
argsfile /var/run/slapd.args
database ldbm
suffix "dc=ricky, dc=com"
rootdn "cn=Manager, dc=ricky, dc=com"
rootpw "******"
directory /var/lib/ldap
# Indices to maintain
index objectClass,uid,uidNumber,gidNumber eq
index cn,mail,surname,givenname eq,subinitial
time to add 10K records = 7692 seconds
======================================================
slapd.conf for Second run: (turn off indexing)
-----------------------
schemacheck on
cachesize 100000
dbcachesize 15000000
dbnosync
#dbcachenosync
pidfile /var/run/slapd.pid
argsfile /var/run/slapd.args
database ldbm
suffix "dc=ricky, dc=com"
rootdn "cn=Manager, dc=ricky, dc=com"
rootpw "******"
directory /var/lib/ldap
# Indices to maintain eq
#index objectClass,uid,uidNumber,gidNumber eq
#index cn,mail,surname,givenname eq,subinitial
time to add 10K records = 2516 seconds
======================================================
slapd.conf for Second run: (turn on dbcachenosync)
-----------------------
schemacheck on
cachesize 100000
dbcachesize 15000000
dbnosync
dbcachenosync
pidfile /var/run/slapd.pid
argsfile /var/run/slapd.args
database ldbm
suffix "dc=ricky, dc=com"
rootdn "cn=Manager, dc=ricky, dc=com"
rootpw "******"
directory /var/lib/ldap
# Indices to maintain eq
#index objectClass,uid,uidNumber,gidNumber eq
#index cn,mail,surname,givenname eq,subinitial
time to add 10K records = 1900 seconds
======================================================
slapd.conf for Second run: (turn schemacheck to off)
-----------------------
schemacheck off
cachesize 100000
dbcachesize 15000000
dbnosync
dbcachenosync
pidfile /var/run/slapd.pid
argsfile /var/run/slapd.args
database ldbm
suffix "dc=ricky, dc=com"
rootdn "cn=Manager, dc=ricky, dc=com"
rootpw "******"
directory /var/lib/ldap
# Indices to maintain eq
#index objectClass,uid,uidNumber,gidNumber eq
#index cn,mail,surname,givenname eq,subinitial
time to add 10K records = 1900 seconds (no change from last test)
======================================================
--
Ricky Charlet : Redcreek Communications : usa (510) 795-6903