Re: (ITS#5860) slapd memeory leak under openldap 2.4

[hyc@symas.com]

Rodrigo Costa wrote:
> Howard,
>
> I download the new HEAD version and made some testing. Now the
olmBDBEntryCache is following the cache configuration.

Good, then this ITS is resolved. Usage questions should be directed to the 
-software mailing list.

> I also made some small change in slapd.conf including cachefree
configuration. Please see below in the end how slapd is configured related to
bdb cache :

> line 123 (cachesize       1000)
> line 124 (cachefree     1000)
> line 125 (idlcachesize    1000)
> line 126 (dncachesize     1000000)

Setting cachefree equal to cachesize will effectively cause the entire entry 
cache to be dumped each time it reaches its maximum size. That's clearly not a 
good idea.

> I increase the dncachesize since with a small value the search takes
considerable more time.
>
> First I tested with a value greater than the value that was consuming
> before
without the memory boundary. In this way I was expecting some order of
magnitude as before, what it achieved (line 126 (dncachesize 5000000)):

> BEFORE CACHED:
> 1000000
>
> real    5m23.084s
> user    0m28.026s
> sys     0m6.686s
>
> Then Cache has :
>
> olmBDBEntryCache: 1001
> olmBDBDNCache: 4000264
> olmBDBIDLCache: 1
>
> AFTER CACHED:
>
> 1000000
>
> real    2m31.623s
> user    0m28.145s
> sys     0m8.637s
>
> Just to let clear these tests above where with the new logic and where the
DN Cache Size is bigger than the final number I had with the old logic where
all information is cached into memory.
>
> For each entrance I have 4 dn's that compose the entrance. Since my filter
is for only one of the dn I was expecting this cache to be only related with
the filter and then in 1,000,000.

> Then I change the dncachesize to 1000000 (line 126 (dncachesize 1000000) :
>
> The total time for a search using a filter for one index dn became :
>
> ldap_result: Can't contact LDAP server (-1)
> 255801
>
> real    469m26.619s
> user    0m7.027s
> sys     0m1.756s
>
> I needed to kill the server so I would have an idea about how many
> entrances
it searched. The time is too long and I'm not sure if it would even end. I let
all night but the process did not end. This was the same ldapsearch as when
all entrances would be allocated in a DN cache into memory.
>
> But the cache process was something I wasn't expecting. Before the
dncachesize was reached I had at monitor :
>
> olmBDBEntryCache: 1001
> olmBDBDNCache: 921296
> olmBDBIDLCache: 1
>
> Then after the boundary was reached it became :
>
> olmBDBEntryCache: 1
> olmBDBDNCache: 1000262
> olmBDBIDLCache: 1
>
> Where these numbers didn't change anymore. Not sure why after the
dncachesize is reached the cachesize(or olmBDBEntryCache) became 1.

Because you set cachefree to 1000. 1001 - 1000 = 1.

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/