[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: slapadd *very* slow: tuning advice?

--On Wednesday, August 22, 2012 7:15 AM +1000 Nick Urbanik <nicku@nicku.org> wrote:

Dear Quanah,

Thank you for your generous help.

In the biggest LDAP directory on the (now finally converted) machine:

So for optimal *slapadd*, this should be your DB_CONFIG setting. You can reduce it down to just the size of id2entry.bdb + dn2id.bdb + some room for growth (10-20% of that dn2id.bdb +id2entry.bdb) after the slapadd completes.

So if id2entry.bdb is 1 GB and dn2id is 1 GB, I would probably use a 2.5 GB cachesize after slapadd completes.

Over all eleven LDAP directories on that same machine:

For running slapd, the DB_CONFIG sizes should match the above formula.

Right now, you only have a 272MB BDB cache set... which is definitely
going to be too small. I would change DB_CONFIG For that database to
be something like:

set_cachesize 6 0 0

Okay; with 12G total RAM, and 6.9G of other directories, am I likely
to overcommit memory?

Not for the slapadd.  See above.

I would strongly advise examining using a shared memory key for that
DB as well.

Okay: I could create that with a command like
ipcmk -M 16777216

*No*.  Please see the shm_key parameter in the manual pages for slapd-bdb.

Finally, look at using MDB, which has zero tuning bits required, and
only requires a tool-threads of 2.

So with eight cores (and 16 hyperthreaded siblings) I am better off
specifying tool-threads 2?

No. tool-threads is only used by slapadd. Since no other databases are being accessed while you are performing the load, you would want all CPUs dedicated to this slapadd process.

Okay, my next one I'll convert to mdb and see how that behaves.

slapd-mdb(5) seems to indicate that mdb needs no shared memory
configuration.   Is that correct?

Correct.  There is no tuning for MDB at all, other than:

a) Ensuring either noatime or relatime is set on the filesystem hosting the DB

b) Ensuring you have a maxsize value that is large enough. This is for the maximum size you ever want the DB to grow, not a performance tuning value. It needs to be large enough to contain your DB and have room for growth. It cannot be larger than the size of the disk it is located on.

c) toolthreads for slapadd with mdb should be set to 2.  No more, no less.



Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
Zimbra ::  the leader in open source messaging and collaboration