[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: Maximum size of the database.

matthew sporleder wrote:
On 8/11/07, Howard Chu <hyc@symas.com> wrote:
Sumith Narayanan wrote: 
Hi Group,

We have a coorporate openldap database in production which has more
than 4 million entries. The slapd process serves three different
physical dabases of sizes 4 GB , 12 GB and 24 GB respecievely.
Earlier it was runnning in ldab backend and was crashing once in 3 -4
days. We upgraded to OpenLDAP 2.3.27 and BDB backend 4.4.20. Mac OS
Tiger with 4GB RAM in the master and 8 GB ram in the slave. Now it
crashes often may be couple of times in a day or sometimes once in 2
days. There is nothing much written in the log files but at times it
shows out of memory exception. However , the search performance is
still good.


- What is the maximum size of the database that OpenLDAP can support ?
- What is the maxmum number of dns it can hold ?
- Is there any solution for the above problem ?

For a machine with 4GB of RAM I think it will max out at around 1-2 million
DNs. Unfortunately there is no way to control the size of the DN cache in
OpenLDAP 2.3, it simply grows without bound. A config keyword for the DN cache
will be in OpenLDAP 2.4.
--


This is news to me.  Is there a discussion somewhere of the different
caches in openldap/bdb, which ones are tunable and which ones are not?


That 1-2 million estimate was wrong. This post was closer:
http://www.openldap.org/lists/openldap-software/200611/msg00051.html

The limit isn't really the 4GB total RAM, but rather the available address space in a 32 bit process (usually only 2GB, sometimes as much as 3GB depending on the OS and any tweaks in the OS).

As already noted above, the DN cache is not tunable in 2.3. I frankly didn't see any wisdom in accommodating extremely large databases on 32-bit servers.

Could we get an idea with the bdb cache (DB_CONFIG) were set to the
lowest values calculated from db_stat and (FAQ) "How do I determine
the proper BDB/HDB database cache size?" and then working out a
slapd.conf cachesize/idlcachesize?

I think Gavin Henry has written up a tuning section in the 2.4 Admin Guide. Balancing the slapd caches vs the BDB caches is already addressed in the FAQ article. Bottom of the page http://www.openldap.org/faq/index.cgi?file=1075 though of course it doesn't talk about the DN cache.
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/