[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: (ITS#5860) slapd memeory leak under openldap 2.4



hyc@symas.com wrote:
> rlvcosta@yahoo.com wrote:
>
>> See that I have 2 DBs and in each DB area in slapd.conf, read without error by slapd as seen above, I have the cachesize, idlcachesize and dncachesize specified with small values just to monitor the memory usage by slapd.
>>
>> Unfortunately even with this configuration I still having slapd consuming memory without release it. So if a new entrance is queried by some LDAP client, slapd will consume more memory. Since my DB dn's are bigger than 3GB then soon or later the slapd will crash.
>>
>> See my DB information :
>>
>> 7.2G    dn2id.bdb
>> 12G     id2entry.bdb
>> 110M    maillogin.bdb
>> 3.3M    objectClass.bdb
>> 108M    pnnumber.bdb
>> 1.3M    submxid.bdb
>> 323M    uid.bdb

> You're mistaken, dncachesize is in the 2.4 manpages. You seem to not be
> reading the documentation that came with the software release you're using.
>
>> Just as example, with the DB I loaded for tests with 1 million entrances, with the cache sizes definitions as in the slapd.conf described above and making a ldapsearch on all entrances from the CONTENT DB, in the end I have :
>>
>> PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
>> 3189 ldap      15   0  842m 724m  67m S   99  6.1   7:00.48 slapd
>>
>> See the slapd process already consumed 842m and I just read 1 million entrances, even dncachesize is defined to be 1000.
>>
>> I do not see anyway to control the memory usage by slapd.
>>
>> Is this "disrespect" to slapd.conf directives a possible problem?
>
> Looks like slapd is behaving normally so far, though it's hard to tell without
> also seeing your DB_CONFIG settings. Test again with your full database and
> report the results.
>
I'll note that this is a pathological use case for glibc malloc. With a cache 
of only 1000 entries, searching 1 million entries will result in several 
million malloc/free calls which will fragment the heap, and that will cause 
results like what you're seeing. You'll have better luck with hoard or tcmalloc.

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/