Re: (ITS#5860) slapd memeory leak under openldap 2.4

[rlvcosta@yahoo.com]

Quanah,

I just used tcmalloc but I still with the same behavior. slapd doesn't have limit for the cache and then it cannot run large databases.

My DB entrance is not large. One example is :

dn: uid=1219843773896\2B554184XXXXXX,ou=REPOSITORY,ou=CONTENT,o=domain,c=country
creationdate: 200808271329Z
localelist: en,fr
weblogin: +554184XXXXXX
expirationdate: 200810161329Z
profileid: 0
mail: +554184XXXXXX/type=PLMN@<domain>
uid: 1219843773896+554184XXXXXX
objectClass: commonuser
objectClass: user
accountStatus: 1
webpassword: 0000
cn: +554184XXXXXX
optionvalues:: AAEAAAcBAgEAAQEBAQEBAAAAAQEBAAEBAQEAAQAAAAAAAAMAAAAAAAAAAAAAAAA
 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==
optionstates:: AAAAAAABAwAAAAAAAAABAAAAAQIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==
isintercepted: FALSE

dn: pnnumber=\2B554184XXXXXX,uid=1219843773896\2B554184XXXXXX,ou=REPOSITORY,ou
 =CONTENT,o=domain,c=country
subpnid: 0
pntype: 2
pncaps: 7
objectClass: phoneinfo
pnnumber: +554184XXXXXX

dn: submxid=0,uid=1219843773896\2B554184XXXXXX,ou=REPOSITORY,ou=CONTENT,o=domain,c=country
secretcode: 0000
fraudaccessnb: 0
mailMessageStore: ./Maildir
maillogin: +554184XXXXXX/type=PLMN@<domain>
objectClass: commonmailbox
objectClass: courierinfo
submxid: 0
mailQuota: 1048576
mailpassword: <secret>
homeDirectory: /var/mms/mail/0/00/0000000000/0

See above I have 3 DN's.

In 64 bits systems with more memory and small dn's this could be hidden but with the time all physical memory will be used.

So, I made an ldapsearch as (searching a single DN):

time ldapsearch -LLL -x -D "cn=admin,ou=CONTENT,o=domain,c=country" -w secret  -b "ou=CONTENT,o=domain,c=country" -H ldap://10.142.15.42:389 'pnnumber=*'| egrep -e '^pnnumber*' |wc -l
1000003

real    6m56.321s
user    0m25.617s
sys     0m8.392s

So just one of the 3 DNs this DB has. Since the filesize is :

1.9G    dn2id.bdb
3.1G    id2entry.bdb
29M     maillogin.bdb
3.3M    objectClass.bdb
28M     pnnumber.bdb
1.3M    submxid.bdb
84M     uid.bdb

I have part of size load into cache(only for pnnnumber).

But slapd doesn't respect cache limits and load all into memory. It also never release it.

The entrances are not large and caches in the order of 1000 entrances should not make slapd process stay with almost 1GB of memory.

Also once it occupies this memory it never release anymore.

I use lsof and the tcmalloc is being used.

Regards,

Rodrigo.

--- On Thu, 1/22/09, Quanah Gibson-Mount <quanah@zimbra.com> wrote:

> From: Quanah Gibson-Mount <quanah@zimbra.com>
> Subject: Re: (ITS#5860) slapd memeory leak under openldap 2.4
> To: rlvcosta@yahoo.com, openldap-its@openldap.org
> Date: Thursday, January 22, 2009, 4:04 PM
> --On Thursday, January 22, 2009 10:01 AM -0800 Rodrigo Costa
> <rlvcosta@yahoo.com> wrote:
> 
> > Quanah,
> > 
> > Just a correction about the man, it was incorrect
> since my man was
> > getting from cache(and so from 2.3 version). Now it is
> ok.
> > 
> > I would like a little more directions about how can be
> compiled openldap
> > 2.4.13 with tcmalloc. Just need to include a lib path?
> Some configure
> > parameter in openldap?
> 
> I use tcmalloc myself, so basically you go and build
> google-perf-tools, and then you preload the library before
> starting slapd.  I have a little startup script for slapd:
> 
> #!/bin/bash
> ulimit -n 32768
> ulimit -c unlimited
> export LD_PRELOAD=/opt/zimbra/tcmalloc/lib/libtcmalloc.so
> exec /opt/zimbra/openldap/sbin/slapd "$@"
> 
> 
> --Quanah
> 
> 
> --
> 
> Quanah Gibson-Mount
> Principal Software Engineer
> Zimbra, Inc
> --------------------
> Zimbra ::  the leader in open source messaging and
> collaboration