Re: need suggestion on indexing big directory

[vadim tarassov <vadim.tarassov@swissonline.ch>]

Hi Quanah,

thanx a lot for replying me. Could you please tell me what is difference
between slapindex and slapadd in terms of disk usage? Do they work so
differently? I just want to understand what is happening ...

Thanx a lot, vadim tarassov.

On Sat, 2004-06-05 at 22:45, Quanah Gibson-Mount wrote:
> --On Saturday, June 05, 2004 10:23 PM +0200 vadim tarassov 
> <vadim.tarassov@swissonline.ch> wrote:
> 
> > I have started with DB_CONFIG containing only set_cachesize parameter,
> > nothing about checkpoints and dbnosync in slapd.conf. I tried ldapadd,
> 
> ldapadd is really the wrong way to populate a database.  man "slapadd".
> 
> > set_lg_dir
> > set_flags DB_LOG_AUTOREMOVE
> >
> > to my DB_CONFIG and started to write log files on file system in RAM.
> > With checkpoint in slapd.conf I made sure that I was not running out of
> > memory due to log files.
> 
> You need to fully configure your DB_CONFIG file.
> 
> > slapdindex was running rather well first 1 Gb of log files, however it
> > started to run rather slow afterwards, and I have noticed that db was
> > writing on disk like crazy, causing again huge iowait.
> 
> slapindex is generally worthless for indexing a large data load.  Use 
> slapadd.
> 
> See:
> 
> <http://www.stanford.edu/services/directory/openldap/configuration/>
> 
> In particular, read, and *pay attention to the notes*:
> <http://www.stanford.edu/services/directory/openldap/configuration/bdb-config.html>
> 
> You may also find this useful:
> 
> <http://www.stanford.edu/services/directory/openldap/configuration/slapd-conf-replica.html>
> 
> --Quanah
> 
> --
> Quanah Gibson-Mount
> Principal Software Developer
> ITSS/Shared Services
> Stanford University
> GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html