[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: scaling openldap for performance



Hi!

I think the discussion Dave mentioned was my question about "BDB locker" and
deadlocks occuring from time to time. Let me make one thing clear: A SAN
(Storage Area Network) normally consists of a SAN controller, a SAN switch
and a HDD array (available from different companys like EMC, HP, Hitachi Data
Storage,...) "behind" the SAN switch. It's like having directly connected a SCSI
HDD on a Adaptec SCSI controller. Do not confuse this with a NAS (Network
Attached Storage). We have a Compaq EVA SAN and beleave me this is fast
as hell ;-) especially the wait time's are almost not measurable and the throughput
is fantastic. If it's good for some big Oracle databases it should be good for
OpenLDAP ;-)


I have done the same tests as Trevor and can comfirm his observations. I've loaded
OpenLDAP 2.2.11 with BDB 4.2.52 (WITHOUT the two patches available from
Sleepycat which solves some locking issues) on a Fujitsu Siemens RX300 (
2 x 2.8 GHz, 2 GB RAM, Redhat ES 3) with about 1 mio. real entries. If have
done some performance testing with Apache JMeter. I've also adjusted the parameters
in DB_CONFIG accordingly.


Reading is simply really fast. But if you are doing a lot of modify's and/or add's on
our system if the server has really a lot to do I've also seen that OpenLDAP
crashes and database files could'nt be recoverd. Not even with the Sleepycat
tools. I was forced to recreate the database. This caused me sleepless nights.
But we have a replica so we can switch quickly if this happens. As far as I
remember the log output wasn't very helpfull. But I will try this on monday
again.


This week I've compiled a new BDB 4.2.52 with the two patches from Sleepycat.
On monday I will do the performance test again. Maybe this patches will solve
the problem.


Cheers,
Robert


Dave Lewney wrote:

Trevor Warren wrote:

Hey there Fellas,

  Having tread through the archives for a long time i
have concluded with some load/performance tests.

 Openldap scales *wonderfully* with respect to read
performance. Towards testing the same for write
performance i performed the following:


With Zero think time we hit openldap with 20


simultaenous users. The db has 1million records.  5
mins thro the test i have a seg fault. I try
restarting the same but the server refuses to re-read
the db unless i re-create the same. Find this very
strange. Can i avoid this In-consistency????


Me brought this DB size to 100 Addressbook records


and pounded the server with 20users and Zero think
time. Still server crashes.


Brought the users to 10 users and 100 records.


Bombarding the server with Zero think time. It now
survives and pulls through the test.

 HW Arch is : HP BLade20p, 2 CPU, 1GB RAM, RHEL AS2.1,
RAID 0+1 on a SAN.

Am quite confused to be true. Logging is disabled
cause we found its over head is simply too much and
hinders response times.
Lemme know of any runtime/compile time options
towards boosting performance please. Thanks in
advance.
Trevor



If you are using the BDB backend then apparently there are problems with using this on a SAN. I seem to remember this being discussed on this list fairly recently.


Dave
--
Dave Lewney
Principal Systems Programmer, IT Services
University of Sussex, Brighton BN1 9QJ. Tel: 01273 678354 Fax: 01273 271956