growth pains, ldbm or bdb?

[Micah Anderson <micah@riseup.net>]

We've been merrily going about our way with our
pam/courier/postfix/nss ldap implementation using ldbm, until recently
when things started to get nasty. I dont know if we reached a growth
point, or if we have some internal datastructure corruption, but our
slapd keeps dying on us, and I am guessing that our ldbm files get
corrupted when the machine crashes (I run db3_recover on them
afterwards, but it always is pretty nondescript messages:
db_recover: Recovery complete at Fri Jan 16 21:19:18 2004
db_recover: Maximum transaction id 80000000 Recovery checkpoint [0][0]
making me feel like nothing is happening).

So, I'm wondering if we should be switching back-ends to better deal
with the increased load (we add 10-20 new accounts a day, and have
over 4,000 now churning several hundred thousand messages a day
incoming (each is a ldap lookup!) and pop/imap connections ranging
from about 90,000 a day (each one is an ldap lookup!), does one or the
other scale better? Have I hit the upper limit with ldbm?

Anyways, I am sure someone has written a document about which one you
should chose for what scenarios, and maybe a tuning document for each?
If so I would be eternally grateful if someone might point this out to
me (or offer any suggestions!).

Thanks,
micah