Re: strange performance problem

[xoror@infuse.org]

On Sun, 23 Mar 2003, Peter Marschall wrote:

hi, 

No i've only tested with back-bdb. I can test it with ldbm also.


> Hi,
> 
> have you only tred it with back-dbd od with back-ldbm too ?
> Just to get an idea if it is more related to the core of slapd or to a
> specific backend.
> 
> Peter
> 
> 
> 
> On Sunday 23 March 2003 14:47, Cuong bui wrote:
> > Hello all,
> >
> > I'm doing some research on openldap to see if it's suitable for a certain
> > kind of application. This application requires more write operations from
> > the ldap server then an 'average' usage of ldap.
> >
> > For this purpose i've written an multithreaded application to simulate
> > simultaneous access to the ldap server. This application creates an
> > variable number of threads that simulate an client. There's a central queue
> > of task that have to be performed. The queue has 3 kind of task (for now).
> >
> > 1. filter task. Simple search on a certain key
> > 2. modify task. modification of a 'record'.
> > 3. authentication task. (simple bind and unbind, simple authentication for
> > now)
> >
> > The task queue is built in such a way that the type of tasks are mixed with
> > eachother.
> >
> > i've generated a test dataset of 50.000 records and the .bdb files have
> > been copied to an other location (ofcourse  when the server is offline) in
> > order to restore data (instead of regenerating the test dataset, it can now
> > be copied back).
> >
> > The testruns were very promising. for instance with 1000 tasks (300 filter,
> > 600 modification, 100 authentication) and 32 threads only took 9 seconds to
> > complete. However performing this same test a few times gives me a rather
> > surprise effect. the first 4 runs all took about 9-13 seconds to complete.
> > The 5th run and up takes up to 5 mins (!) or more.  Simple searches with
> > ldapsearch takes up a few mins (after 4 testruns). (before that they only
> > took a fraction of a second).
> >
> > during the first 4 testruns the openldapserver used 98% cpu processing
> > time. (seen in top) After the initial 4 runs, slapd only consumes 40%-75%
> > cpu processing time. somethimes it's even dropping to a few %.
> >
> > The only way to get slapd working 'normal' again, is to shutdown the server
> > and restore the dataset. I have defined an index on  the field that i'm
> > filtering on. (without it searches will take a very long time to complete).
> >
> > The test server consist of a pentium 4 2,53 ghz running gentoo linux with
> > kernel 2.4.20 and ultra dma (enabled) hdd   with 512 MB ram. Openldap
> > 2.1.16 was running on this machine. (but also tested with 2.1.12 and 2.1.15
> > they all gave this result) BDB was chosen as backend for this test.)
> >
> > The testclient consist of an pentium 4 1,8ghz with gentoo linux running
> > kernel 2.4.19. The 2 systems were connected on a 100 mbit (switched)
> > network. All the software were compiled with g++ 3.2.1
> >
> > Has anyone an explanation for this behaviour ? Is this a known problem ?
> > All hints are welcome.
> >
> >
> > Thank in advance (and for your time)
> > Cuong
> >
> > btw: are there any documentation on implementations details of openldap ?
> > if yes, where can i find them :) Other documentation that might explain
> > what i'm seeing is also (very) welcome.
> 
> -- 
> Peter Marschall
> eMail: peter@adpm.de
> 
>