[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: slow slapadd?

On Mon, May 18, 2009 at 7:56 PM, Quanah Gibson-Mount <quanah@zimbra.com> wrote:
--On Monday, May 18, 2009 7:55 AM -0700 Bill MacAllister <whm@stanford.edu> wrote:

However, on BDB 4.7 it seems the default behavior on Linux is also to
do synchronous flushes of the cache. As such, one approach to getting
consistent performance is to configure the backend to use shared
memory for the BDB cache instead of mmap'd files. That way incidental
page updates don't sync to anything, and the BDB library has full
control over when pages get flushed back to disk.

We can confirm that setting shm_key on 4.7 dramatically affects the load
time.  In our initial tests of 4.7 we just pulled our old 4.2 BDB
parameters forward.  We never saw the slapadd of a 4.6 gbyte database
complete.  We killed it after 4 hours.  After changing the shm_key
setting the load time dropped to the more normal 30 minutes.

To expand on this slightly.  BDB has no difference in perf for small databases between disk and shm.  It's only once your database is past some 6GB in size that shm vs disk starts to make a difference on Linux.

Could someone elaborate on the setting (shared memory instead of mmap) that is best suited for each major OS? Like Linux, FreeBSD, Solaris?



Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
Zimbra ::  the leader in open source messaging and collaboration