[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: Maximum size of the database.
Hi All,
Thanks for the suggestions :
I will look forward to upgrade the systems to 64 bit processors and
increase the memory to 16 GB, since it will take time , I am doing
some more tweaking to see whether it will help.
I reduced the set_cachesize in the DB_CONFIG by 10 times and also in
the slapd.conf file. Restarted the process and it was consuming less
memory now. But the master from whether the updates flow to slave is
still slowing using up the memory and crashing , but the slave seems
to be stable with a usage of around 200 MB of memory. I don't think I
will be able to avoid that with the current situation.
I have few questions now :
- Any idea on how big the openldap DB can be ?
- I see some DB transaction files in the db directory , which are of
same size and more than one is created in a day :
-rw------- 1 ldapp ldapp 20485760 Aug 13 21:40 log.0000001070
-rw------- 1 ldapp ldapp 20485760 Aug 14 05:52 log.0000001071
-rw------- 1 ldapp ldapp 20485760 Aug 14 16:08 log.0000001072
-rw------- 1 ldapp ldapp 20485760 Aug 14 23:06 log.0000001073
I took a backup of a lot of this inorder for the slapd process to
start faster. It keeps on getting accumulated. I guess these are DB
transaction log files. Is it supposed to get deleted automatically ?
Why is it accumulating ?
Thanks, Sumith.
On 8/11/07, Howard Chu <hyc@symas.com> wrote:
> matthew sporleder wrote:
> > On 8/11/07, Howard Chu <hyc@symas.com> wrote:
> >> Sumith Narayanan wrote:
> >>> Hi Group,
> >>>
> >>> We have a coorporate openldap database in production which has more
> >>> than 4 million entries. The slapd process serves three different
> >>> physical dabases of sizes 4 GB , 12 GB and 24 GB respecievely.
> >>> Earlier it was runnning in ldab backend and was crashing once in 3 -4
> >>> days. We upgraded to OpenLDAP 2.3.27 and BDB backend 4.4.20. Mac OS
> >>> Tiger with 4GB RAM in the master and 8 GB ram in the slave. Now it
> >>> crashes often may be couple of times in a day or sometimes once in 2
> >>> days. There is nothing much written in the log files but at times it
> >>> shows out of memory exception. However , the search performance is
> >>> still good.
> >>>
> >>>
> >>> - What is the maximum size of the database that OpenLDAP can support ?
> >>> - What is the maxmum number of dns it can hold ?
> >>> - Is there any solution for the above problem ?
> >> For a machine with 4GB of RAM I think it will max out at around 1-2 million
> >> DNs. Unfortunately there is no way to control the size of the DN cache in
> >> OpenLDAP 2.3, it simply grows without bound. A config keyword for the DN cache
> >> will be in OpenLDAP 2.4.
> >> --
> >
> > This is news to me. Is there a discussion somewhere of the different
> > caches in openldap/bdb, which ones are tunable and which ones are not?
>
> That 1-2 million estimate was wrong. This post was closer:
> http://www.openldap.org/lists/openldap-software/200611/msg00051.html
>
> The limit isn't really the 4GB total RAM, but rather the available address
> space in a 32 bit process (usually only 2GB, sometimes as much as 3GB depending
> on the OS and any tweaks in the OS).
>
> As already noted above, the DN cache is not tunable in 2.3. I frankly didn't
> see any wisdom in accommodating extremely large databases on 32-bit servers.
>
> > Could we get an idea with the bdb cache (DB_CONFIG) were set to the
> > lowest values calculated from db_stat and (FAQ) "How do I determine
> > the proper BDB/HDB database cache size?" and then working out a
> > slapd.conf cachesize/idlcachesize?
>
> I think Gavin Henry has written up a tuning section in the 2.4 Admin Guide.
> Balancing the slapd caches vs the BDB caches is already addressed in the FAQ
> article. Bottom of the page http://www.openldap.org/faq/index.cgi?file=1075
> though of course it doesn't talk about the DN cache.
> --
> -- Howard Chu
> Chief Architect, Symas Corp. http://www.symas.com
> Director, Highland Sun http://highlandsun.com/hyc/
> Chief Architect, OpenLDAP http://www.openldap.org/project/
>