Re: Upgrade openldap, os and backend - pointers for best performance ?

[Jonathan Clarke <jonathan@phillipoux.net>]

Le 10/02/2010 10:57, Leon de Rooij a écrit :
> Hi all,
>
> I recently decided it was time for some upgrades, so I installed a new
> OpenLDAP server with the following changes:
>
> Old New
>
> os: Ubuntu Hardy Server 32bit FreeBSD 8.0-Release-p2
> kernel: 32bit 2.6.24-24-server 32bit 8.0-Release-p2
> fs: XFS UFS (soft-updates)
> ldap: 2.3.37 2.4.21
> bdb: 4.6.21-6 4.7.25.4
>
> The rest is the same, dual 3.06Ghz P4 Xeon with only 4GB RAM. I'm using
> hdb as backend and have about 170k entries in the directory. Slapd
> configuration is the same. Oh and I'm only using syncprov overlay.
>
> DB_CONFIG is the same on both machines, only containing this:
>
> set_cachesize 0 268435456 1
> set_lg_regionmax 262144
> set_lg_bsize 2097152
> set_lk_detect DB_LOCK_EXPIRE
>
> On the new server, slapcat is faster, search returning single entry is
> slower, but search returning ~10k entries is faster (these are all
> second tries so cached data is used):
>
> time ldapsearch filter=(uid=someusername) > /dev/null, returning one
> entry on localhost
>
> old (on linux):
>
> real 0m0.006s
> user 0m0.000s
> sys 0m0.000s
>
> new (on freebsd):
>
> real 0m0.033s
> user 0m0.010s
> sys 0m0.008s
>
> ---
>
> time ldapsearch filter=(uid=a*) > /dev/null, returning ~10k entries on
> localhost
>
> old (on linux):
>
> real 0m1.728s
> user 0m0.870s
> sys 0m0.220s
>
> new (on freebsd):
>
> real 0m0.611s
> user 0m0.074s
> sys 0m0.015s
>
> ---
>
> time slapcat > /dev/null
>
> old (on linux):
>
> real 0m26.834s
> user 0m17.220s
> sys 0m1.080s
>
> new (on freebsd):
>
> real 0m9.589s
> user 0m8.884s
> sys 0m0.699s
>
> ---
>
> Now I know that too many variables have changed to point exactly at
> what's the reason for the speed changes, but does anyone else have
> experience with these version differences ? I need to be able to do
> about 100 searches/sec which will probably not be a problem, but I am
> curious whether someone has some pointers on how I can optimize for
> example some sysctl's, UFS mount settings, DB_CONFIG or something else
> on this new setup to get the best performance possible ?

Hi,

The differences you're seeing may be due to any number of reasons, as 
you suggest. I think they may also be related to the fact that your new 
server may not have loaded enough entries and indexes into cache, yet.

Unless your new OS is introducing some other bottleneck (unlikely, if 
it's a fresh install), your performance should mostly be tied to cache.

Regarding optimizing the caches, you should configure in detail:

- The size of the BerkeleyDB cache, see:
http://www.openldap.org/faq/data/cache/1075.html

- The numbers of (complete) entries cached by slapd, see:
http://www.openldap.org/faq/data/cache/1076.html

Hope this helps,
Jonathan

--
--------------------------------------------------------------
Jonathan Clarke - jonathan@phillipoux.net
--------------------------------------------------------------
Ldap Synchronization Connector (LSC) - http://lsc-project.org
--------------------------------------------------------------