[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: growth pains, ldbm or bdb?





--On Friday, January 16, 2004 11:27 PM -0600 Micah Anderson <micah@riseup.net> wrote:

We've been merrily going about our way with our
pam/courier/postfix/nss ldap implementation using ldbm, until recently
when things started to get nasty. I dont know if we reached a growth
point, or if we have some internal datastructure corruption, but our
slapd keeps dying on us, and I am guessing that our ldbm files get
corrupted when the machine crashes (I run db3_recover on them
afterwards, but it always is pretty nondescript messages:
db_recover: Recovery complete at Fri Jan 16 21:19:18 2004
db_recover: Maximum transaction id 80000000 Recovery checkpoint [0][0]
making me feel like nothing is happening).

So, I'm wondering if we should be switching back-ends to better deal
with the increased load (we add 10-20 new accounts a day, and have
over 4,000 now churning several hundred thousand messages a day
incoming (each is a ldap lookup!) and pop/imap connections ranging
from about 90,000 a day (each one is an ldap lookup!), does one or the
other scale better? Have I hit the upper limit with ldbm?

Anyways, I am sure someone has written a document about which one you
should chose for what scenarios, and maybe a tuning document for each?
If so I would be eternally grateful if someone might point this out to
me (or offer any suggestions!).

Micah,

This always seems to be an open debate, but...

We use BDB here at Stanford. We have 9 replica's. 3 are dedicated entirely to mail delivery. They handle between 333k and 500k request *each* every day (so 1 mil to 1.5 mil requests a day). We've never had any DB corruption. Our environment is modified much more heavily than yours... The other 6 handle web authorizations, general lookup information, and posixAccount (NIS replacenet) ldap queries for the general campus. At this time, we are very over-scoped hardware wise. ;)

--Quanah

--
Quanah Gibson-Mount
Principal Software Developer
ITSS/TSS/Computing Systems
ITSS/TSS/Infrastructure Operations
Stanford University
GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html