[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
RE: growth pains, ldbm or bdb?
I know this doesn't speak directly to the original question about
backends, but couldn't you use nscd to cache results from ldap queries
on your mail servers? This would seem to me to solve a lot of
performance issues related to mail and ldap or at least to reduce the
number of ldap replicas one would need to support a large mail
operation.
Mike
-----Original Message-----
From: owner-openldap-software@OpenLDAP.org
[mailto:owner-openldap-software@OpenLDAP.org] On Behalf Of Quanah
Gibson-Mount
Sent: Friday, January 16, 2004 10:10 PM
To: Micah Anderson; openldap-software@OpenLDAP.org
Subject: Re: growth pains, ldbm or bdb?
--On Friday, January 16, 2004 11:27 PM -0600 Micah Anderson
<micah@riseup.net> wrote:
> We've been merrily going about our way with our
> pam/courier/postfix/nss ldap implementation using ldbm, until recently
> when things started to get nasty. I dont know if we reached a growth
> point, or if we have some internal datastructure corruption, but our
> slapd keeps dying on us, and I am guessing that our ldbm files get
> corrupted when the machine crashes (I run db3_recover on them
> afterwards, but it always is pretty nondescript messages:
> db_recover: Recovery complete at Fri Jan 16 21:19:18 2004
> db_recover: Maximum transaction id 80000000 Recovery checkpoint [0][0]
> making me feel like nothing is happening).
>
> So, I'm wondering if we should be switching back-ends to better deal
> with the increased load (we add 10-20 new accounts a day, and have
> over 4,000 now churning several hundred thousand messages a day
> incoming (each is a ldap lookup!) and pop/imap connections ranging
> from about 90,000 a day (each one is an ldap lookup!), does one or the
> other scale better? Have I hit the upper limit with ldbm?
>
> Anyways, I am sure someone has written a document about which one you
> should chose for what scenarios, and maybe a tuning document for each?
> If so I would be eternally grateful if someone might point this out to
> me (or offer any suggestions!).
Micah,
This always seems to be an open debate, but...
We use BDB here at Stanford. We have 9 replica's. 3 are dedicated
entirely to mail delivery. They handle between 333k and 500k request
*each* every day (so 1 mil to 1.5 mil requests a day). We've never had
any
DB corruption. Our environment is modified much more heavily than
yours...
The other 6 handle web authorizations, general lookup information, and
posixAccount (NIS replacenet) ldap queries for the general campus. At
this
time, we are very over-scoped hardware wise. ;)
--Quanah
--
Quanah Gibson-Mount
Principal Software Developer
ITSS/TSS/Computing Systems
ITSS/TSS/Infrastructure Operations
Stanford University
GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html