[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: bdb backend: memp_trickle

--On Tuesday, February 15, 2005 2:56 PM -0600 Eric Irrgang <erici@motown.cc.utexas.edu> wrote:

On Tue, 15 Feb 2005, Quanah Gibson-Mount wrote:

I'm using OpenLDAP 2.2.23 and Berkeley DB 4.3.27.

Okay, I strongly suggest not using BDB 4.3 with OpenLDAP 2.2.

I notice you've made that suggestion before but can't find the original post. You've said that the logging is broken? Are there fundamental problems with db4.3 or is it just some bugs that are likely to get fixed in upcoming releases? Is it a big enough issue that someone should fix the 2.2.x sources so that the configure script doesn't looke for db4.3 first?

BDB 4.3 does not allow you to disable the BDB logs generated when doing a slapadd, which will significantly slow down any large DB import. The alternative solution to this provided by Sleepycat is to use in-memory logs, and if you go that route, BDB blows up after a while.

hdb certainly is not abandoned... It is actively maintained.  No idea why
the man page isn't there.

Yeah, I was surprised to notice the source code seemingly up to date. I guess the man page got lost... I don't feel like prowling CVS right now...

I asked the developer about this. There is no manpage for back-hdb because it configures exactly the same as back-bdb. I suggest a symlink to the back-bdb manpage. ;)

It configures the same as back-bdb.  If you have back-bdb configured
already, and you want to try hdb, slapcat, clean up the old db, change
slapd.conf to read "database hdb", slapadd.

"Okay, but what does it DO?" I guess I need to just look through that code after all. I'm curious as to what it does to reduce writes. I normally get drastically improved throughput just by using mp_mmapsize large enough to map all of my database and indexes into process space and checkpointing infrequently, but sooner or later that data has got to go to disk, and I'd rather it didn't happen all at once.

Here's another thought for the BDB brains out there: any reason not to
issue a db_checkpoint from the command line while slapd is running?  From
what I can tell, db4 is meant for this type of concurrent activity.  I
tried it for grins and it seemed to work fine (afterwards slapd was able
to shut down almost immediately because it had nothing left to flush), but
it creeped me out and I'm reluctant to make a habit of it.  :)

You can certainly run db_checkpoint from cron.

In OL 2.3, the checkpoint option in slapd.conf works correctly in that it will checkpoint every X amount of data change or Y minutes, whereas right now, it will only checkpoint if a new change has come in, and X or Y has elapsed.

I assume you are setting the checkpoint command in your slapd.conf file so that the DB is regularly checkpointed.


-- Quanah Gibson-Mount Principal Software Developer ITSS/Shared Services Stanford University GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html

"These censorship operations against schools and libraries are stronger
than ever in the present religio-political climate. They often focus on
fantasy and sf books, which foster that deadly enemy to bigotry and blind
faith, the imagination." -- Ursula K. Le Guin