[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: slow slapadd?





--On May 15, 2009 2:10:25 PM -0400 Diego Figueroa <dfiguero@yorku.ca> wrote:


Hi all,

I'm trying to load ldap with several ldif files. The biggest one
containing 500k records of approx 700 bytes in size each took slapadd a
little over 14 hours to load. Would this be considered normal given the
amount of records and size?
---------
DB_CONFIG
---------
set_lg_max         209715200
set_lg_bsize       52428800
set_tmp_dir        /data/ldap/tmp
set_cachesize      0 209715200 2
set_lk_max_locks   4000
set_lk_max_lockers 4000
set_lk_max_objects 4000


I'd guess your cachesize is miniscule compared to what you need. What was the resulting size of the database (du -c -h *.bdb)? Your cachesize needs to be that big, at the least.

Second, don't use multiple BDB cache segments, it slows things down.

----------
slapd.conf
----------
allow bind_v2
include         /etc/ldap/schema/core.schema
include         /etc/ldap/schema/cosine.schema
include         /etc/ldap/schema/nis.schema
include         /etc/ldap/schema/inetorgperson.schema
include         /etc/ldap/schema/my.schema
pidfile         /var/run/slapd/build-slapd.pid
argsfile        /var/run/slapd/build-slapd.args
modulepath            /usr/lib/ldap
moduleload            back_hdb
password-hash   {SSHA}
disallow        bind_anon
backend         hdb
database        hdb
suffix          "dc=mydomain,dc=com"
directory       "/var/lib/build-ldap"
lastmod         on


You should configure the tool-threads option to match how many real cores your system has. Then it can multi-thread any indices.

--Quanah


--

Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra ::  the leader in open source messaging and collaboration