[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: scaling openldap for performance
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
RW wrote:
| Hi!
|
| I think the discussion Dave mentioned was my question about "BDB locker"
| and
| deadlocks occuring from time to time. Let me make one thing clear: A SAN
| (Storage Area Network) normally consists of a SAN controller, a SAN switch
| and a HDD array (available from different companys like EMC, HP, Hitachi
| Data
| Storage,...) "behind" the SAN switch. It's like having directly
| connected a SCSI
| HDD on a Adaptec SCSI controller. Do not confuse this with a NAS (Network
| Attached Storage). We have a Compaq EVA SAN and beleave me this is fast
| as hell ;-) especially the wait time's are almost not measurable and the
| throughput
| is fantastic. If it's good for some big Oracle databases it should be
| good for
| OpenLDAP ;-)
Yep, we were running RHEL2.1 with Red Hat Cluster Manager on Dell 1650's
on an EMC Clariion with no problems (including graceful failover and
hard reboots of cluster members - of course running database recovery at
startup of the ldap service to ensure we don't have a corrupt database).
We're currently setting up pre-production (on Dell 1750s), so the
prototype 1650s don't have SAN connectivity.
|
| I have done the same tests as Trevor and can comfirm his observations.
| I've loaded
| OpenLDAP 2.2.11 with BDB 4.2.52 (WITHOUT the two patches available from
| Sleepycat which solves some locking issues)
So, add the patches (at *least* 4.2.52.1).
| on a Fujitsu Siemens RX300 (
| 2 x 2.8 GHz, 2 GB RAM, Redhat ES 3) with about 1 mio. real entries. If
have
| done some performance testing with Apache JMeter. I've also adjusted the
| parameters
| in DB_CONFIG accordingly.
|
| Reading is simply really fast. But if you are doing a lot of modify's
| and/or add's on
| our system if the server has really a lot to do I've also seen that
| OpenLDAP
| crashes and database files could'nt be recoverd. Not even with the
| Sleepycat
| tools. I was forced to recreate the database. This caused me sleepless
| nights.
| But we have a replica so we can switch quickly if this happens. As far
as I
| remember the log output wasn't very helpfull. But I will try this on
monday
| again.
The way we are running is having a HA cluster running as master, with
transaction logging enabled, and two slaves running without transaction
logging.
I was doing batch imports of a 250k entry database and an 80k entry
database, followed by queries by 12 simultaneous clients with forced
failover, with no problems.
But, performance is really bad on a server which has transaction logging
enabled while under heavy read load (so, configure your system to not do
this, ensure that servers with transaction logging on have a minimal
read load).
Also, ensure you have checkpointing enabled, on Linux slapd will get
very unhappy when it has 2GB of transaction log files open ...
We hope to be in a position to go into production in 3 weeks time (if
the developers of some software doing non-schema-compliant changes on
the current system fix their code in time), but so far things are
looking good (just need to finish kickstart configurations with all the
necessary drivers etc which aren't up-to-date enough in the RHEL2.1
kernels).
Regards,
Buchan
- --
Buchan Milne Senior Support Technician
Obsidian Systems http://www.obsidian.co.za
B.Eng RHCE (803004789010797)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQFAwML5rJK6UGDSBKcRAkveAJ9VkdnsBosUdHnnLKgoP85/QXLSygCdEbUM
71vQc7r1AJA5D3jHs6LnDpQ=
=l64P
-----END PGP SIGNATURE-----