RE: FW: OpenLDAP as an enterprise level LDAP provider (try II)

[Quanah Gibson-Mount <quanah@stanford.edu>]

--On Wednesday, November 24, 2004 3:11 PM -0600 "Fortin, John {PBG}" 
<John.Fortin@pepsi.com> wrote:

> Quanah, here you go, in order:
>
> 	1) Doing the initial slapdd with indexing on never completed  We
> ended up turning off indexing just to load the initial ldif file.  We then
> ran slapindex after enabling the indexes.

This indicates that you have not configured a large enough BDB DB cache 
size for your database.

> 	2) Dell 2650, dual 2.1 GHz Intel Zeon processors with Hyperthreading
> support, 2 Gig Ram, 76 Gig HD with Raid 5.

You likely need more RAM.

> 	3) As I indicated, hot backups are not possible due to the
> slapadd/slapindex issues.  If I can make this perform better then I can do
> it.
>       4)DB_CONFIG during load:	
> 		set_cachesize   0      52428800        0
> 		set_lg_regionmax        1048576
> 		set_lg_max              10485760
> 		set_lg_bsize            1097152
> 		set_lg_dir              /var/ldapdb/slave2/logs
> 		set_tmp_dir             /tmp
> 		set_flags DB_LOG_AUTOREMOVE

Your DB Cache Size (500MB) is probably too small.

> I'm tryin to make this work.  I'm surprised at the results too.  If I'm
> not tuning this properly, I can adjust.  These were values suggested in
> one of the links in the FAQ, though I did increase set_cachesize.
>
> I do vaguely remember someone complaining about performance on RedHat,
> but I cannot find the reference in the archives.


Here is what I do to tune the DB size when adding a large DB:

1) Set it to something rather large (like 1.5 GB), and perform a slapadd 
with indexing, and see how long that takes.  If it takes a very very long 
time, it is still going to need more space.  Note that the more indices you 
have, the larger the cachesize you will need.

2) You don't seem to have the BDB flags used for doing a fast slapadd in 
your DB_CONFIG file.  You should add:

# Just use these settings when doing slapadd...
#set_flags DB_TXN_NOSYNC
#set_flags DB_TXN_NOT_DURABLE

to your DB_CONFIG, and unset the last two when doing a slapadd.  That will 
speed up slapadd's enormously, while still performing indexing.  Once the 
slapadd is completed, simply comment them out again.

3) Once you know how long it takes with a large DB Cache, drop some off of 
it, and slapadd again.  I would guestimate that if it takes more than 30 
minutes with the above settings & a large cache size on your box, you 
likely need more DB cachesize.

--Quanah


--
Quanah Gibson-Mount
Principal Software Developer
ITSS/Shared Services
Stanford University
GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html