Re: (ITS#5860) slapd memeory leak under openldap 2.4

[rlvcosta@yahoo.com]

Quanah,

That's is exactly the point. My system is a 32 bits and with 64 bits if there are enough physical memory available even slapd cache grows without never release it will be difficult(or very long) to cache enough entrances before system consumes all memory.

With the 32 bits 3GB maximum, depend on OS too, this issue appears more often. My impression is if you ldapsearch all DB in a 32 bits HW/OS slapd will consume all possible memory unless to keep boundaries for its cache(erase and overwrite).

In 64 bits with a lot of memory this will be more difficult to happen but eventually will happen if the logic is the same.

I was wondering if in a sittuation where this cache memory issue would appear, like ldapsearch, if there is a feature in openldap where it would guarantee the cache boundaries and the reutilization of the space after the limits are reached.

Looks like even with the slapd.conf configurations it doesn't keeps its limits.

Regards,

Rodrigo.


--- On Wed, 1/21/09, Quanah Gibson-Mount <quanah@zimbra.com> wrote:

> From: Quanah Gibson-Mount <quanah@zimbra.com>
> Subject: Re: (ITS#5860) slapd memeory leak under openldap 2.4
> To: rlvcosta@yahoo.com, openldap-its@openldap.org
> Date: Wednesday, January 21, 2009, 6:41 PM
> --On Wednesday, January 21, 2009 12:23 PM -0800 Rodrigo
> Costa <rlvcosta@yahoo.com> wrote:
> 
> > Quanah,
> > 
> > My initial attempt was using openldap2.4.11 since this
> is the stable
> > version. At this version Berkeley DB 4.7 isn't
> supported and then I used
> > the BDB 4.6 with all patches.
> 
> I hardly consider 2.4.11 stable, regardless of what the tag
> is. I would advise trying something more current (such as
> 2.4.13).  And with BDB 4.7 + patches.
> 
> > At this configuration, having the DB_CONFIG setup for
> 50MB, and even
> > having the idlcache and cache sizes defined I continue
> to see the memory
> > being allocated and never released by slapd.
> 
> What about dncachesize as well?
> 
> > 2.3.27-8.el5_2.4 where the CentOS5.4 uses a built-in
> backend library for
> > BDB4.4.
> 
> If you are looking for stability, I would hardly go with
> the crappy release CentOS ships.  The final 2.3 release was
> 2.3.43.  Use it instead if you want to use 2.3.
> 
> Also, are you on a 32 or 64-bit system?  I certainly have
> no problems going over 3GB on my 64-bit systems.
> 
> For example:
> [zimbra@ldap openldap-data]$ du -c -h *.bdb
> 1.1G    cn.bdb
> 564M    displayName.bdb
> 702M    dn2id.bdb
> 158M    entryCSN.bdb
> 122M    entryUUID.bdb
> 335M    givenName.bdb
> 6.5G    id2entry.bdb
> 1.1G    mail.bdb
> 5.5M    objectClass.bdb
> 855M    sn.bdb
> 121M    uid.bdb
> 8.0K    zimbraDomainName.bdb
> 122M    zimbraId.bdb
> 20K     zimbraMailAlias.bdb
> 1.1G    zimbraMailDeliveryAddress.bdb
> 2.1M    zimbraMailForwardingAddress.bdb
> 3.1M    zimbraMailTransport.bdb
> 8.0K    zimbraVirtualHostname.bdb
> 8.0K    zimbraVirtualIPAddress.bdb
> 13G     total
> 
> 
> DB is 13GB in size.
> 
> [zimbra@ldap openldap-data]$ cat DB_CONFIG
> set_cachesize 8 0 2
> set_lg_regionmax 262144
> set_lg_bsize 2097152
> set_lk_max_locks 1500
> set_flags DB_LOG_AUTOREMOVE
> set_lg_dir /var/log/zimbra-ldap-journals
> 
> Process size:
> 
> 8447 zimbra    17   0  9.9g 8.9g 8.1g S   34 57.5  39:20.57
> slapd
> 
> 
> Using OpenLDAP 2.3.42.
> 
> 
> --Quanah
> 
> 
> 
> --
> 
> Quanah Gibson-Mount
> Principal Software Engineer
> Zimbra, Inc
> --------------------
> Zimbra ::  the leader in open source messaging and
> collaboration