Re: Memory Usage Spikes

[Mathew Rowley <mathew_rowley@cable.comcast.com>]

Title: Re: Memory Usage Spikes

Thanks for the suggestions...

Bdb, BerkelyDB-4.7.25, yes, fully patched.

MAT


On 3/18/10 1:56 PM, "Quanah Gibson-Mount" <quanah@zimbra.com> wrote:

> --On Thursday, March 18, 2010 3:47 PM -0400 "Beuerlein, Edward"
> <Edward_Beuerlein@cable.comcast.com> wrote:
>
> > Hi Quanah,
> > Thanks for replying!  Please let us know what other info you might need
> > to help us figure out these large increases in memory usage.
> >
> > Here's the info you requested:
> >
> > I did a dump of the database and then ran this on it(please let me know
> > if that isn't what you wanted):
> >
> ># cat auth01.031610.ldif |grep dn |wc -l
> > 60753
>
> So you have 60,753 entries.  Just to be sure, I would have used grep ^dn:
> to be exact. ;)
>
> > set_cachesize 0 268435456 1
>
> You have 256MB allocated to BDB
>
> > 69M     cn.bdb
> > 26M     dn2id.bdb
> > 3.7M    entryCSN.bdb
> > 2.2M    entryUUID.bdb
> > 160K    gidNumber.bdb
> > 2.1G    id2entry.bdb
> > 840K    ipHostNumber.bdb
> > 44K     memberNisNetgroup.bdb
> > 304K    memberUid.bdb
> > 3.1M    nisNetgroupTriple.bdb
> > 2.9M    objectClass.bdb
> > 8.0K    ou.bdb
> > 8.0K    sudoUser.bdb
> > 3.0M    uid.bdb
> > 364K    uidNumber.bdb
> > 792M    uniqueMember.bdb
> > 2.9G    total
>
> Your BDB database is 2.9GB.
>
> > olcDbCacheSize 1000
> > olcDbCacheFree 1
> > olcDbDNcacheSize 0
> > olcDBIDLcacheSize 1000
>
> You're only allowing the first 1000 entries to be cached in OpenLDAP, and
> your IdlCache is quite small as well.  dncachesize is fine (unlimited).
>
> While none of this explains your memory spikes, your server is definitely
> poorly tuned.  It may be that your massive discrepancy between BDB cache
> allocation and actual BDB size may be causing the problem (in looking at
> the bdb error messages in your log).
>
> I would highly advise the following changes.
>
> For DB_CONFIG
>
> set_cachesize 4 0 1
>
> This will increase the BDB cache to 4GB (which can hold your 2.9GB DB
> easily)
>
> For cn=config:
>
> olcDbCacheSize 80000
> olcDbCacheFree 1000
> olcDBIDLcacheSize 240000
>
> This will allow all entries to be cached in OpenLDAP, free up a reasonable
> number if you exceed the cache, and allow a decent IDL size.
>
> I forgot to ask which slapd backend you are using (hdb or bdb), and which
> version of BDB you are using (and if it is fully patched) which is also
> useful information.
>
> You may also wish to read over
> <http://wiki.zimbra.com/index.php?title=OpenLDAP_Performance_Tuning_6.0>, I
> think you guys have some experience with Zimbra... ;)
>
> --Quanah
>
> --
>
> Quanah Gibson-Mount
> Principal Software Engineer
> Zimbra, Inc
> --------------------
> Zimbra ::  the leader in open source messaging and collaboration