Re: (ITS#7299) Memory leaks when using ldapmodify to add members to groupOfNames objectclass

[jsynacek@redhat.com]

On 06/18/2012 01:55 PM, hyc@symas.com wrote:
> Jan Synacek wrote:
>> On 06/15/2012 03:16 PM, hyc@symas.com wrote:
>>> jsynacek@redhat.com wrote:
>>>> Full_Name: Jan Synacek
>>>> Version: git (c73ec15)
>>>> OS: linux-fedora17
>>>> URL: http://jsynacek.fedorapeople.org/openldap/leak/openldap-mmr-leak.tar.gz
>>>> Submission from: (NULL) (209.132.186.34)
>>>>
>>>>
>>>> I'm using a 2-node mmr setup on my local machine - configuration files and
>>>> 'uploader' scripts are provided in the archive.
>>>>
>>>> 1) I have the two nodes running.
>>>> 2) Execute run.sh (only a wrapper for ldapusradm.sh) and start monitoring
>>>> slapd's memory usage.
>>>> 3) After some time (at about 2k users on my system), slapd consumes a large
>>>> amount of memory which is still growing
>>>>
>>>> Note that not using ldapmodify to add members to 'cn=users,dc=yes,dc=my', but
>>>> using it e.g. for modifying each user's email, does NOT result in any memory
>>>> leakage.
>>>>
>>>> I have also created a massif output using valgrind's massif tool:
>>>> http://jsynacek.fedorapeople.org/openldap/leak/massif.out.17906
>>>>
>>>> I found a very similar bug (#7292), but I'm not sure if it's related.
>>>
>>> Running RE24 with valgrind --leak-check=full I see no leak when running your
>>> test. That should be the same as git c73ec15. No idea what leak you're seeing.
>>>
>>
>> The memory consumption still grows and slapd eventually gets killed by oom-killer.
> 
> Perhaps you're seeing fragmentation in the glibc malloc library. Please test 
> RE24, and try with some other malloc (e.g. tcmalloc). I've run your entire 
> test with both valgrind and with tcmalloc's heap checker and neither one shows 
> any leak.
> 

I've done more tests and have some additional observations to add:

* tcmalloc doesn't make any difference

* if syncing finishes, memory usage drops considerably

* putting sleep between ldapadd and ldapmodify in the script seems to stop the memory consumption

* also, generating one big ldif, that is used modify all the users in groupOfNames just once (calling
ldapmodify only once that is, not calling it once for every user), seems to help as well; the ldif is
formatted like so:

dn: cn=users,dc=yes,dc=my
changetype:modify
add: member
member: uid=test-1@yes1.my,dc=yes,dc=my
-
add: member
member: uid=test-2@yes1.my,dc=yes,dc=my
-
...

I'm also including my configure setup:

export CFLAGS="$(rpm --eval %{optflags}) -O2 -fPIC -D_REENTRANT -DHAVE_TLS -DHAVE_MOZNSS"
export LIBS="-lpthread -lssl3 -lsmime3 -lnss3 -lnssutil3 -lplds4 -lplc4 -lnspr4"

CFLAGS="${CFLAGS:--g -pipe -Wall }"; export CFLAGS;
CXXFLAGS="${CXXFLAGS:--g -pipe -Wall }"; export CXXFLAGS;
FFLAGS="${FFLAGS:--g -pipe -Wall }"; export FFLAGS;


./configure \
	--with-threads=posix --disable-static --with-tls=no \
   	--with-pic --with-cyrus-sasl \
	--enable-bdb \
	--enable-overlays --enable-slapd --enable-spasswd --enable-modules --enable-slapi \
	--enable-accesslog \
	--enable-wrappers



-- 
Jan Synacek
Software Engineer, BaseOS team Brno, Red Hat