[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: sanity check on updating from HDB backend to MDB

Brian Reichert wrote:
I have an experiment, that may have too many moving parts to tease apart.

We're exploring migrating our LDAP server from a 32-bit OS to a 64-bit OS,
and I've also explored the port to the mdb back end.  My first shot at this
does not show the performance the write-ups on MDB would have me expect.

If it matters, my old environment is a Dell R610 server, with 6G of RAM.

   # uname -r
   # cat /etc/redhat-release
   CentOS release 5.8 (Final)
   # rpm -qf `which slapd`

My ldapsearch invocations on this host take ~120 seconds.

I set up two Dell R610s, each with 12G or memory.

One environment uses the RPM produced by the LTB project, here named 'ltb':

   root@ltb# uname -r
   root@ltb# cat /etc/redhat-release
   CentOS release 6.5 (Final)
   root@ltb# rpm -qf /usr/local/openldap/libexec/slapd

And the other uses the same OS, but uses CentOS's stock OpenLDAP
RPM; here named 'stock':

   root@stock# rpm -qf `which slapd`

I migrated a database of ~360k DNs, and underwent the cache tuning,
following the suggestions here:


And with those tunings, my ldapsearch command does outperform our old 32-but

   stock# time ldapsearch -x -w XXXXX -D "cn=manager,orgid=MyOrgID" -b orgid=MyOrgID dn | grep ^dn: | wc -l
   real    0m27.147s
   user    0m19.216s
   sys     0m10.585s

However, my efforts to use the MDB backend yield slower responses, on the
order of 175 seconds.

How I set up the MDB backend:

I copied my old slapd.conf over, and replaced my bdb backend with mdb

- I set 'tool-threads' to 2
- I set 'maxsize' to 21474836480 [ 20G = 20 * ( 1024 * 1024 * 1024 ) ]

   ltb# /usr/local/openldap/sbin/slaptest -f /etc/openldap/slapd.conf -F /etc/openldap/slapd.d
   config file testing succeeded
   ltb# chown -R ldap:ldap /etc/openldap/slapd.d

I was apparently able to import all 360k DNs without an error:

   ltb# time setuidgid ldap /usr/local/openldap/sbin/slapadd \
       -q -v -F /etc/openldap/slapd.d -b orgId=MyOrgID \
       -l export_from_OLD.ldif >& log; echo $?

   real    16m46.924s
   user    8m52.165s
   sys     3m44.859s

   ltb# ls -ld data.mdb
   -rw------- 1 ldap ldap 21474836480 Jul 30 20:16 data.mdb
   ltb# du -c -h data.mdb
   6.3G    data.mdb
   6.3G    total

But, the same search looks worse here.

   ltb# time ldapsearch -x -w XXXXX -D "cn=manager,orgid=MyOrgID" -b orgid=MyOrgID dn | grep ^dn: | wc -l
   real    2m55.482s
   user    0m25.459s
   sys     0m23.948s

Both 64-bit hosts show no swapping, and minimal CPU load.  Can
anyone point out what I've missed?

While that search is running you should see slapd at 100% CPU. If not, then something in your system is throttling your connection.

  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/