[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: slapadd *very* slow: tuning advice?

Dear Quanah,

Thank you for your generous help.

On 21/08/12 07:32 -0700, Quanah Gibson-Mount wrote:
--On Tuesday, August 21, 2012 4:03 PM +1000 Nick Urbanik <nick.urbanik@optusnet.com.au> wrote:

Dear Folks,

I'm upgrading a cluster of OpenLDAP servers from 2.3.43-25.el5 to
2.4.32 with BDB 4.8.30 on CentOS 5, x86_64, on HP BL460cG6 blades
with two 4-core CPUs, and 12GB RAM.  These are slaves, and have
eleven trees on them.  I have dumped and restored six of the LDAP
databases in reasonable time, but the seventh is taking a long time.
Here are the sizes of the slapcatted LDIF files:

It's the 2.6G LDIF file that's taking the time to slapadd:
-############          63.34% eta 02h09m elapsed       03h43m30s spd
2.7 k/s 	

As you can see, it has slowed to a crawl.

set_cachesize 0 286162472 0

# egrep 'tool-threads|cachesize' /etc/openldap/slapd.conf
tool-threads 8

cachesize and idlcachesize have no bearing on slapadd.  Only
toolthreads does.

Thank you.

You do not mention if you are using slapadd -q or not.

Yes, this is with the options -q -w.

Any suggestions on how to optimise this a little more towards slapadd?

If the LDIF file is 2.6GB, I fully assume the BDB database will be
significantly larger than 2.6 GB.  Probably closer to 6 or 8 GB,
depending on your indexing.  What you really need to look at from
your 2.3 installation is the total size of that database (du -c -h

In the biggest LDAP directory on the (now finally converted) machine:

Over all eleven LDAP directories on that same machine:

Right now, you only have a 272MB BDB cache set... which is definitely going to be too small. I would change DB_CONFIG For that database to be something like:

set_cachesize 6 0 0

Okay; with 12G total RAM, and 6.9G of other directories, am I likely
to overcommit memory?

I would strongly advise examining using a shared memory key for that
DB as well.

Okay: I could create that with a command like
ipcmk -M 16777216

But what size would be appropriate with 12G of RAM, eleven LDAP
databases, and 13G of bdb files?

Finally, look at using MDB, which has zero tuning bits required, and only requires a tool-threads of 2.

So with eight cores (and 16 hyperthreaded siblings) I am better off
specifying tool-threads 2?

Okay, my next one I'll convert to mdb and see how that behaves.

slapd-mdb(5) seems to indicate that mdb needs no shared memory
configuration.   Is that correct?
Nick Urbanik             http://nicku.org           nicku@nicku.org
GPG: 7FFA CDC7 5A77 0558 DC7A 790A 16DF EC5B BB9D 2C24 ID: BB9D2C24