Re: (ITS#5860) slapd memeory leak under openldap 2.4

[hyc@symas.com]

rlvcosta@yahoo.com wrote:
> Quanah,
>
> In this specific test I reported in my previous e-mail it was used glibc.
>
> Now I tested with tcmalloc and the memory again was much more consumed and never released.
>
>    PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> 10652 ldap      18   0 2006m 1.8g  68m S  100 15.8   8:36.75 slapd
>
> Once it get this much memory it then stabilize but it is much more than before that was 385m.

We may need to have sample data from you to reproduce this situation. I 
haven't seen anything like it on my tests. Running two searches simultaneously 
still behaves normally for me.
>
> The scenario was :

Likewise, the exact commands and arguments will be needed...
>
> 1) Start a full ldapsearch for one specific dn(the DB has 4 dn's);
> 2) After sometime start a new full ldapsearch to behave as queries to DB when replication for LDIF backup is being done;
> 3) Export one of the queries to a file so speed verification can be done with grep and wc.
>
> After sometime the memory starts to be consumed in large chunks until it stopped in around 2G.
>
> Since one of the queries is being redirect to a file I can tail to have a visual rate of the entrances being returned by slapd. As the issue before after sometime the rate appears to get stuck and come in small 16 entrances chunks but only after some minutes.
>
> This behavior above appears to happen after the second ldapsearch fills its own dn cache even monitor always shows only one information as I could see.
>
> [root@brtldp11 ~]# date; grep -e '^pnnum*' /backup/temp.ldif |wc -l
> Mon Jan 26 21:43:31 BRST 2009
> 482432
> [root@brtldp11 ~]# date; grep -e '^pnnum*' /backup/temp.ldif |wc -l
> Mon Jan 26 21:44:56 BRST 2009
> 482448
>
> This behavior is similar as before with the initial issue with a single process.
>
> I just notice when slapd starts there are these threads :
>
> [root@brtldp12 ~]# ps -mu ldap
>    PID TTY          TIME CMD
> 10638 pts/2    00:00:00 slapd
>      - -        00:00:00 -
>      - -        00:00:00 -
>
> Then after the 2 queries are started I have :
>
> [root@brtldp12 ~]# ps -mu ldap
>    PID TTY          TIME CMD
> 10195 pts/2    00:44:22 slapd
>      - -        00:00:00 -
>      - -        00:00:00 -
>      - -        00:26:09 -
>      - -        00:18:13 -
>      - -        00:00:00 -
>
> So 3 new threads. Looks like for some reason with multiples queries the issue with response getting stuck repeats.
>
> Even one ldapsearch is stopped the number of threads continues the same, memory continues allocated and the other search get with this slow rate.
>
> These outputs are using now tcmalloc but similar as with glibc.

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/