Re: (ITS#6660) paged result searches fail to deallocate memory until slapd shutdown

[hyc@symas.com]

masarati@aero.polimi.it wrote:
>> --On Wednesday, September 29, 2010 8:34 PM +0200 masarati@aero.polimi.it
>> wrote:
>>
>>>> --On Wednesday, September 29, 2010 12:38 AM +0000 quanah@zimbra.com
>>>> wrote:
>>>>
>>>>> It appears to be a problem with the entry cache, which is set to
>>>>> 25,000:
>>>>
>>>> Running with a fix from Howard, the entry cache behaves correctly.
>>>> However, slapd still grows at the same rate.
>>>>
>>>> If I limit to only 10 paged results searches, slapd grows at a rate of
>>>> 300MB Virtual and 300MB Resident for every set of 10 paged results
>>>> searches
>>>> I do concurrently, up until I run slapd out of memory.  There's
>>>> something
>>>> very wrong with paged results searches.
>>>
>>> Could it be configuration-specific?  I tested with a plain configuration
>>> resulting from test003; maybe some player in the middle, say, is causing
>>> entries to be duplicated and leaked, or read-locks on originals are not
>>> released correctly?  Can you post a configuration that shows the issue?
>>
>> Hi Pierangelo,
>>
>> My testing shows the issue is only really visible with large databases
>> that
>> return giant result sets.  I don't expect you to see it with a small
>> database and test003, because the amount of "lost" memory will be a few
>> bytes at best.
>
> If it's something related to bdb's cache size I agree; if it's related to
> paged results I'd expect to notice it anyway using valgrind or so.
>
There's no actual leak, so valgrind won't point anything out. It's simply an 
issue with the entry cache not running its purge in some cases where it needs 
to. cn=monitor shows that the entry cache grows far beyond its configured 
size. Still looking into a proper fix.

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/