Re: How to improve performance with MDB backend?

How do you measure or compare speed?

I have a Python program which add a predefined number of users (1mil and each user consists of 8 DNs and 3 alias DNs) and for Modify test I have also Python program which modify 1 attribute with random value under one DN changed sequentially from 1 to 1mil)

Now I can add those 1 million of users in 5508 sec what amounts to 181,55 users per second.

Modify program can do 2141,33 modifications per sec.

Do you find those numbers reasonable or are they nonsense?

Is there a way to get 10000 modifications per second or I am asking a stupid question?

Sasa

On Mon, Apr 8, 2013 at 4:40 PM, Quanah Gibson-Mount <quanah@zimbra.com> wrote:

--On Monday, April 08, 2013 4:49 AM -0700 Howard Chu <hyc@symas.com> wrote:

SaÅa-Stjepan BakÅa wrote:

Hi,

Maybe this is not the best way to ask but I would like to get some
performance expectations or maybe suggestions how to improve
performance. I do have relatively long experience with OpenLDAP as a
precompiled package and with much less users so performance was not an
issue for those installations.

Now I need to put few million users (now one million for test), custom
tailored schema and search performance is crucial but also modify
performance is big issue. Before MDB I have used HDB as backend.

What to do to improve write part of performance or performance in general
(when adding data â from time to time â OpenLDAP stalls in a way)?
Just around 5 add/mod operations during that time then it continue with
much higher speed.

I have used many different sources to find out other peoples experience
and I didn't choose to write to list lightly but I really need some
help/hints.

There's nothing particular to LMDB to tune. But if you're seeing pauses
due to disk I/O, as your iotop output seems to indicate, you might want
to look into using a different I/O scheduler.

Not quite true. ÂOn Linux, I suggest setting the olcDbFlags as follows:

olcDbEnvFlags: writemap
olcDbEnvFlags: nometasync

With those flags set, writes are 65x faster for me with back-mdb than they are with back-hdb/back-bdb.

--Quanah

--

Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra :: Âthe leader in open source messaging and collaboration