[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: need suggestion on indexing big directory





--On Saturday, June 05, 2004 10:23 PM +0200 vadim tarassov <vadim.tarassov@swissonline.ch> wrote:

I have started with DB_CONFIG containing only set_cachesize parameter,
nothing about checkpoints and dbnosync in slapd.conf. I tried ldapadd,

ldapadd is really the wrong way to populate a database. man "slapadd".

set_lg_dir
set_flags DB_LOG_AUTOREMOVE

to my DB_CONFIG and started to write log files on file system in RAM.
With checkpoint in slapd.conf I made sure that I was not running out of
memory due to log files.

You need to fully configure your DB_CONFIG file.

slapdindex was running rather well first 1 Gb of log files, however it
started to run rather slow afterwards, and I have noticed that db was
writing on disk like crazy, causing again huge iowait.

slapindex is generally worthless for indexing a large data load. Use slapadd.


See:

<http://www.stanford.edu/services/directory/openldap/configuration/>

In particular, read, and *pay attention to the notes*:
<http://www.stanford.edu/services/directory/openldap/configuration/bdb-config.html>

You may also find this useful:

<http://www.stanford.edu/services/directory/openldap/configuration/slapd-conf-replica.html>

--Quanah

--
Quanah Gibson-Mount
Principal Software Developer
ITSS/Shared Services
Stanford University
GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html