[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: restricting slapd memory consumption

Quanah Gibson-Mount schrieb:
--On Wednesday, April 02, 2008 10:41 AM +0200 Ralf Narozny <rnarozny@web.de> wrote:

Yep, as I wrote in my initial mail, we are using 2.3.32 (for testing so

The current release is 2.3.41. Why are you using such an old release? There've been many known problems fixed since that release, including security issues, and possibly some memory leaks.

We are using that version, because it is a project that is not highly priorized and the package was compiled long a ago. But I just initialized an upgrade. Memory leaks are for sure a show stopper.

What version of BDB are you using? Have you patched it with all the relevant patches?

We are using BDB 4.4.20. With as far as I know no patches. Any recommended version or neccessary patches?

And I wrote that we are using BDB which is configured to use 4GB of
shared mem. The only problem I have is that with 1000000 entries
configured as entry cache, slapd uses 11GB out of 16 GB of RAM after the
insert with ldapadd. Which makes it use 7GB for entry cache (and whatever

What's the contents of your DB_CONFIG file? What's the size du -c -h *.bdb in your database directory? What is your idlcachesize setting? How many file descriptors do you allocate to slapd?

DB_CONFIG: set_flags DB_LOG_AUTOREMOVE set_cachesize 4 0 2 set_lg_max 524288000 set_lg_regionmax 512000 set_lg_bsize 268435456

du -c -h *.bdb:
792M    cid.bdb
5,7G    cn.bdb
4,1G    dn2id.bdb
36M     folderName.bdb
15G     id2entry.bdb
948K    locked.bdb
1,9M    objectClass.bdb

idlcachesize: 0, because we only use it for customer logins, so there
are no searches besides the ones for the customer id

If you mean the number of file descriptors ready to be used, it is 4096
currently, but might be raised, if neccessary.

Since you are using ldapadd to populate the database (rather than slapadd), the entry cache is getting immediately allocated. As I described in a previous email writeup I did on performant tuning of slapd, the DB_CONFIG setting is the most important (to hold the size of your entire DB), followed by the entry cache size.

Yep, that is my problem, we will surely need a lot of entries in cache, because, we got more than 2 million customers logging in per day.

Can you tell me the subject of the mail writeup so I can find it in the
list archive?

The entry cache really only needs to be the size of your active set of entries, rather than all entries. For example, at a previous position, even though we had about 500,000 entries in our database, our entry cache was only 20,000. And it was very performant.

We got 23 million entries of which more than 2 million are active entries...we can not really reduce the size of our database. Maybe a few thousands, but not millions ;-)

Our entries have (in LDIF of course) an average size of below 200 Bytes.
So taking 6GB out of the 7GB used as size of the entry cache, it would
mean that each entry consumes about 6K of RAM. Is that correct?

If so, is there any documentation on how to configure the slapd for a
larger amount of entries like ours?

How many entries do you have?