[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: slow queries on huge directory or slapd crash

1) When starting out with a new installation, use the newest stable release available. OpenLDAP 2.1 is no longer supported, 2.1.30 was released last April and there have been numerous features and fixes added since then. The current release is always linked from the www.openldap.org web site. Use it.

2) Read the documentation on the www.openldap.org web site including the Administrator's Guide and the FAQ. There is plenty of information and advice on how to configure the software for best performance. First tip, don't use back-ldbm, use back-bdb or back-hdb. Either one can handle a 4GB database with no slowdown, once properly configured.

Probst, Carsten wrote:

hello guys,

i'm running some tests with openldap 2.1.30 on redhat 9 (2.4.20-30.9smp)
box with 4gig memory and 4 logical cpus (hypertreading enabled, dell
poweredge 2650) to find out if it meets the requirements from our
development team.

configure string:
env CPPFLAGS="-I/usr/local/BerkeleyDB.4.2/include -
I/usr/kerberos/include" LDFLAGS="-
L/usr/local/BerkeleyDB.4.2/lib" ./configure --prefix=/opt/LDAP-
Server-2.1.30 --enable-ldbm=yes --enable-ldap=yes --enable-meta=yes --
enable-rewrite=yes --with-tls --enable-shared=no

i had to use workaround
http://www.openldap.org/its/index.cgi/Archive.Software%20Bugs?
id=1134;selectid=1134;usearchives=1;statetype=-1 to avoid ITS issue 1134
(segfault during make test) in order to build openldap.

i don't know if a id2entry.dbb file of almost 4gb size would mean a big
directory, but this is the situation i have:

# du -sh /opt/LDAP-Server/var/openldap-data/*
45M     /opt/LDAP-Server/var/openldap-data/birthday.dbb
108M    /opt/LDAP-Server/var/openldap-data/cn.dbb
233M    /opt/LDAP-Server/var/openldap-data/dn2id.dbb
132K    /opt/LDAP-Server/var/openldap-data/gender.dbb
3.9G    /opt/LDAP-Server/var/openldap-data/id2entry.dbb
8.0K    /opt/LDAP-Server/var/openldap-data/nextid.dbb
452K    /opt/LDAP-Server/var/openldap-data/objectClass.dbb
17M     /opt/LDAP-Server/var/openldap-data/postalCode.dbb
240K    /opt/LDAP-Server/var/openldap-data/preferredRestaurant.dbb
108M    /opt/LDAP-Server/var/openldap-data/uid.dbb

this was created by a dummy script to simulate approx. 1.000.000 user
profiles for a webportal we run for a customer.

queries like:

/opt/LDAP-Server/bin/ldapsearch -x -D cn=Admin,o=netmldap -W -b
ou=campaigntool,ou=netm,o=netmldap -H "ldap://0.0.0.0:390"; -LLL
postalcode=50000 email

took more than six minutes and returning approx. 110.000 email adresses.

the index in slapd.conf looks like this:

index   cn,uid               pres,eq,approx,sub
index   objectClass          eq
index   gender               eq
index   birthday             eq,sub,subinitial
index   postalCode           eq,sub,subinitial
index   preferredRestaurant  eq

the cachesize and dbcachesize are both 100.000 at the moment. i
experienced slapd to crash when the value is to big.

i used e.g. dbcachesize 45.000.000 and cachesize 1.000.000. when usig
ldapsearch the slapd takes up to 2.9gig memory and then exits.

is there some idea to improve the performace for queries like above on
this hardware and to avoid this crashes? i read that the dbcachesize
should be as large as the biggest index file.

is ldap designed to return such a large number of entries (sizelimit -1)
or would a database fit more to this requirement? developers expect less
than 10sec even for the most complex query...

any ideas? thanx for reading up to here.

regards,
carsten







--
 -- Howard Chu
 Chief Architect, Symas Corp.       Director, Highland Sun
 http://www.symas.com               http://highlandsun.com/hyc
 Symas: Premier OpenSource Development and Support