[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: bdb backend: memp_trickle



Eric Irrgang wrote:

On Tue, 15 Feb 2005, Quanah Gibson-Mount wrote:



I'm using OpenLDAP 2.2.23 and Berkeley DB 4.3.27.


Okay, I strongly suggest not using BDB 4.3 with OpenLDAP 2.2.



I notice you've made that suggestion before but can't find the original post. You've said that the logging is broken? Are there fundamental problems with db4.3 or is it just some bugs that are likely to get fixed in upcoming releases? Is it a big enough issue that someone should fix the 2.2.x sources so that the configure script doesn't looke for db4.3 first?



In typical use there's nothing wrong with BDB 4.3. If you try to run with transaction logging disabled, then you'll have problems since the option that worked in 4.2 was removed from 4.3.

It configures the same as back-bdb. If you have back-bdb configured
already, and you want to try hdb, slapcat, clean up the old db, change
slapd.conf to read "database hdb", slapadd.



"Okay, but what does it DO?" I guess I need to just look through that
code after all. I'm curious as to what it does to reduce writes. I
normally get drastically improved throughput just by using mp_mmapsize
large enough to map all of my database and indexes into process space and
checkpointing infrequently, but sooner or later that data has got to go to
disk, and I'd rather it didn't happen all at once.


mp_mmapsize is only relevant for databases opened in Read-Only mode. slapd has to open everything in Read-Write mode, so this option usually has no effect. It can be useful for slapcat and slapindex, but only if none of the database was already loaded into the cache. I.e., even if a database is opened Read-Only, if it already got into the cache (because of a prior Read-Write usage) then mp_mmapsize is also ignored. With that said, you can insure that mp_mmapsize takes effect for slapcat/slapindex by running db_recover beforehand to flush the cache. Obviously slapd must be shutdown first.

Here's another thought for the BDB brains out there: any reason not to
issue a db_checkpoint from the command line while slapd is running?  From
what I can tell, db4 is meant for this type of concurrent activity.  I
tried it for grins and it seemed to work fine (afterwards slapd was able
to shut down almost immediately because it had nothing left to flush), but
it creeped me out and I'm reluctant to make a habit of it.  :)

That works just fine, as long as you have shared mutexes. By default, BDB on Linux used assembly-code atomic locks, so that usually works. On newer Linux systems that have NPTL, BDB prefers to use POSIX mutexes. That also works fine, unless you try to run with the older LinuxThreads library, in which case it breaks. I test in both LinuxThreads and NPTL environments, so I configure BDB not to use POSIX mutexes.

--
 -- Howard Chu
 Chief Architect, Symas Corp.       Director, Highland Sun
 http://www.symas.com               http://highlandsun.com/hyc
 Symas: Premier OpenSource Development and Support