[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: openldap 2.4.16 hangs with dncachesize smaller than max number of records

Howard,

I tried bigger caches but I do not have enough memory to apply them. 
This was the reason I only tried the dncachesize to speed up search queries.

I also have this same database running in a old openldap version(2.1.30) 
even with a little more records on it. In this version, as I understand, 
there isn't any cache at openLDAP but only at BDB. See how it is 
behaving in terms of memory and CPU from the openLDAP 2.1.30:

 PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME CPU COMMAND
 3651 root      24   0  136M 135M  126M S     0.0  1.1   0:00   1 slapd

See a really small memory consumption and I really reasonable 
performance. The only issue I have with this version is about the 
replication mechanism which I would like to increase its availability 
using syncrepl unless slurpd.

The problem is that for version after 2.1 looks like we need to have 
enough memory to cache all database since there are many situations 
where slapd appear increase CPU or memory usage and considerably reduce 
performance.

I tried to remove from slapd.conf any cache constraint seeing if the 
previous version performance directly from disk reads would be 
reproduced. I saw some good startup, like 500 returns per second, but 
after sometime slapd hanged and did not return any more records to 
ldapseach. Also it consumed all 4 CPUs :

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  
COMMAND          
 4409 ldap      15   0  400m 183m 132m S  400  1.5  27:22.70 slapd

And even after I stop the ldapsearch the CPU consumption continues 
consuming all CPUs processing. I believe it entered in a dead loop.

I do not have a heavily load system but based in records and DBs I have 
some memory resource constraints.

I also tried some smaller caches, like :

cachesize         500000
dncachesize     500000
idlcachesize    500000
cachefree       10000

But it also hangs the search after sometime.

I was wondering if there is a way to run slapd without cache, reading 
from disk(like first time read to insert record in cache), what is 
enough for small/medium systems in terms of consulting. In this way I 
could use the behavior as the 2.1.30 system and the new syncrepl 
replication.

Thanks a lot !

Rodrigo.

Howard Chu wrote:
> Quanah Gibson-Mount wrote:
>> --On Wednesday, June 17, 2009 5:25 AM -0700 Rodrigo Costa
>> <rlvcosta@yahoo.com>  wrote:
>>
>>> Could this be a configuration issue? I do not think but I'm putting
>>> below my cache configuration :
>
> Clearly it is a configuration issue.
>
>>> # Cache values
>>> cachesize       10000
>>> dncachesize     3000000
>>> idlcachesize    10000
>>> cachefree       10
>>
>> These values are extremely low (except for dncachesize) for a system 
>> with 4
>> million records.  I'd expect something more like:
>>
>> cachesize 4000000
>> dncachesize 3000000
>
> dncachesize must always be >= cachesize
>
>> idlcachesize 8000000
>> cachefree 100000
>>
>> or something along those lines.  Particularly in the case of your
>> ldapsearch, cachesize is likely the most relevant.  Try playing with the
>> settings a bit more.
>
> --   -- Howard Chu
>  CTO, Symas Corp.           http://www.symas.com
>  Director, Highland Sun     http://highlandsun.com/hyc/
>  Chief Architect, OpenLDAP  http://www.openldap.org/project/
>
>