(ITS#5860) slapd memeory leak under openldap 2.4

[rlvcosta@yahoo.com]

Full_Name: Rodrigo Costa
Version: 2.4.11
OS: RHEL4 U4
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (201.82.161.220)


Opeldap dev team,

I'm facing an issue and after many tentatives I believe this is really a bug
that maybe could be identified and solved by openldap community. I tried to
include dmalloc into openldap 2.4.11 code but even I could compile with it any
tentative to execute the code  generated "Segmentation Fault" so I could not use
dmalloc to identify where the memory leak is happening.

I compiled both Berkeley DB 4.6.21 with its patches from source and then
compiled openldap 2.4.11 also from source. I download both from links provided
at Oracle and Openldap so I think the code is ok.

For reference I compiled Berkeley DB with the following configuration :

env CFLAGS="-march=pentium4 -O3 -pipe -fomit-frame-pointer" \
../dist/configure --enable-largefiles --enable-java --prefix=/usr

It compiled without problems. The openldap is compiled with the following
configuration :

env CFLAGS="-march=pentium4 -O3 -pipe -fomit-frame-pointer" ./configure
--with-tls --enable-crypt --enable-syslog --enable-modules --enable-bdb=mod
--enable-monitor=mod --enable-shared --with-gnu-ld --enable-overlays=mod
--sysconfdir=/etc --prefix=/usr

It identifies correctly bdb libraries and compile without any problem. The
CHANGES file make reference for ITS#5557 which I saw as possible candidate for
the problem I'm having. Even with it looks like there is memory leak.

My system is a HP DL380-G3 32 bits with 12GB of memory. The OS is Red Hat
Enterprise Linux 4 Update 4. I have actually an old version of OpenLdap(version
2.1.30 with bdb 4.2.52) running in a similar machine without any problem.

The specific details are "Linux brtldp11 2.6.9-42.ELsmp #1 SMP Wed Jul 12
23:27:17 EDT 2006 i686 i686 i386 GNU/Linux".

I made several slapd.conf files trying to control cache and see this could be a
configuration problem. I put into debug 64 and the file is read correctly
without any problem. Please see output below :

all schema stuff...
line 23 (pidfile        /var/run/slapd.pid)
line 24 (argsfile  /var/run/slapd.args)
line 26 (allow bind_v2)
line 28 (sizelimit       10000)
line 29 (timelimit       5400)
line 31 (loglevel  0)
line 34 (modulepath     /usr/libexec/openldap)
line 35 (moduleload     back_bdb.la)
loaded module back_bdb.la
module back_bdb.la: null module registered
line 36 (moduleload     back_monitor.la)
loaded module back_monitor.la
module back_monitor.la: null module registered
line 78 (database  bdb)
line 79 (suffix    "ou=CONTENT,o=alcatel,c=fr")
line 80 (rootdn    "cn=admin,ou=CONTENT,o=alcatel,c=fr")
line 84 (rootpw ***)
line 87 (directory /var/openldap-data/bdb1)
line 90 (index userid,pnnumber,submxid,maillogin,abookaliasid eq,pres)
index uid 0x0006
index pnnumber 0x0006
index submxid 0x0006
index maillogin 0x0006
index abookaliasid 0x0006
line 91 (index objectClass eq)
index objectClass 0x0004
line 93 (cachesize       45000)
line 94 (dncachesize     45000)
line 95 (idlcachesize    45000)
line 96 (cachefree       9000)
line 104 (database bdb)
line 105 (suffix        "ou=INDEXES,o=alcatel,c=fr")
line 106 (rootdn        "cn=admin,ou=INDEXES,o=alcatel,c=fr")
line 110 (rootpw ***)
line 113 (directory     /var/openldap-data/bdb2)
line 116 (index weblogin,maillogin,mail,pnnumber eq,pres)
index weblogin 0x0006
index maillogin 0x0006
index mail 0x0006
index pnnumber 0x0006
line 117 (index profileid,loginname eq,pres)
index profileid 0x0006
index loginname 0x0006
line 118 (index objectClass eq)
index objectClass 0x0004
line 120 (cachesize       45000)
line 121 (dncachesize     45000)
line 122 (idlcachesize    45000)
line 123 (cachefree       9000)
line 131 (database        monitor)
slapd starting


My DB has around 4 million entrances and the bdb DB file size is around 15GB.
The DB itself is not big since the entrances are not soo big.

By openLDAP documents a DB with 4million entrances should be supported correctly
into a 32 bits system.

The issue becomes clear when I try to make a ldapsearch in these DBs(bdb1 and
bdb2). If I do it the memory usage by slapd start to increase until all possible
memory be allocated and system start to paging. After sometime slapd cannot
allocate anymore memory using malloc and slapd crashes.

After I put the cache this behavior is taking a little longer but not much
longer. It ends memory in around 1 hour.

Even if I stop the ldapsearch the memory previously allocated by slad isn't
released and the process can be seen in the machine top with a lot of allocated
memory. The only way to release this memory is to stop slapd process manually.

I beleive there is a memory leakage but I could not identify where. My
tentatives to used dmalloc were not successful and then I would like to report
this bug.

Your expertise identifying if the correct patches are really applied into the
openldap2.4.11 is really important to solve this issue.

Please let me know if you need more information.

Regards,

Rodrigo Costa.