Re: (ITS#5860) slapd memeory leak under openldap 2.4

[rlvcosta@yahoo.com]

Howard,

One more comment. If you take a longer time between the ldapsearches the issue doesn't appear sometimes.

So you need to start the first ldapsearh and like 15 seconds later start the second one. With this scenario the problem will for sure appear.

In this way we will see large chunks of memory being allocated and after it stabilizes the query get stuck returning as the issue before 16 records per minute, or very slow and getting stuck all the time.

Regards,

Rodrigo.


--- On Mon, 1/26/09, Howard Chu <hyc@symas.com> wrote:

> From: Howard Chu <hyc@symas.com>
> Subject: Re: (ITS#5860) slapd memeory leak under openldap 2.4
> To: rlvcosta@yahoo.com
> Cc: openldap-its@openldap.org
> Date: Monday, January 26, 2009, 11:16 PM
> rlvcosta@yahoo.com wrote:
> > Quanah,
> >
> > In this specific test I reported in my previous e-mail
> it was used glibc.
> >
> > Now I tested with tcmalloc and the memory again was
> much more consumed and never released.
> >
> >    PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM   
> TIME+  COMMAND
> > 10652 ldap      18   0 2006m 1.8g  68m S  100 15.8  
> 8:36.75 slapd
> >
> > Once it get this much memory it then stabilize but it
> is much more than before that was 385m.
> 
> We may need to have sample data from you to reproduce this
> situation. I 
> haven't seen anything like it on my tests. Running two
> searches simultaneously 
> still behaves normally for me.
> >
> > The scenario was :
> 
> Likewise, the exact commands and arguments will be
> needed...
> >
> > 1) Start a full ldapsearch for one specific dn(the DB
> has 4 dn's);
> > 2) After sometime start a new full ldapsearch to
> behave as queries to DB when replication for LDIF backup is
> being done;
> > 3) Export one of the queries to a file so speed
> verification can be done with grep and wc.
> >
> > After sometime the memory starts to be consumed in
> large chunks until it stopped in around 2G.
> >
> > Since one of the queries is being redirect to a file I
> can tail to have a visual rate of the entrances being
> returned by slapd. As the issue before after sometime the
> rate appears to get stuck and come in small 16 entrances
> chunks but only after some minutes.
> >
> > This behavior above appears to happen after the second
> ldapsearch fills its own dn cache even monitor always shows
> only one information as I could see.
> >
> > [root@brtldp11 ~]# date; grep -e '^pnnum*'
> /backup/temp.ldif |wc -l
> > Mon Jan 26 21:43:31 BRST 2009
> > 482432
> > [root@brtldp11 ~]# date; grep -e '^pnnum*'
> /backup/temp.ldif |wc -l
> > Mon Jan 26 21:44:56 BRST 2009
> > 482448
> >
> > This behavior is similar as before with the initial
> issue with a single process.
> >
> > I just notice when slapd starts there are these
> threads :
> >
> > [root@brtldp12 ~]# ps -mu ldap
> >    PID TTY          TIME CMD
> > 10638 pts/2    00:00:00 slapd
> >      - -        00:00:00 -
> >      - -        00:00:00 -
> >
> > Then after the 2 queries are started I have :
> >
> > [root@brtldp12 ~]# ps -mu ldap
> >    PID TTY          TIME CMD
> > 10195 pts/2    00:44:22 slapd
> >      - -        00:00:00 -
> >      - -        00:00:00 -
> >      - -        00:26:09 -
> >      - -        00:18:13 -
> >      - -        00:00:00 -
> >
> > So 3 new threads. Looks like for some reason with
> multiples queries the issue with response getting stuck
> repeats.
> >
> > Even one ldapsearch is stopped the number of threads
> continues the same, memory continues allocated and the other
> search get with this slow rate.
> >
> > These outputs are using now tcmalloc but similar as
> with glibc.
> 
> -- 
>    -- Howard Chu
>    CTO, Symas Corp.           http://www.symas.com
>    Director, Highland Sun     http://highlandsun.com/hyc/
>    Chief Architect, OpenLDAP 
> http://www.openldap.org/project/