Re: slapadd, 1 million entries, some numbers

[Andreas <andreas@conectiva.com.br>]

On Wed, May 28, 2003 at 10:36:45AM -0300, Andreas wrote:
> > > Is cache relevant for bulk loading data?
> > 
> > Read this:
> > http://www.openldap.org/faq/data/cache/893.html
> 
> I'll play with cache and buffer sizes and see how much it improves. I see
> that the default buffer size is really too smal, thanks for the tip.

ok, after tuning some parameters, I got the following:

# time slapadd -l saida.ldif

real    31m14.530s
user    8m32.710s
sys     1m30.840s

This is a huge improvement, about 530 adds/s. Just to remember, the previous run was:
# time slapadd -l saida.ldif

real    148m49.360s
user    7m54.460s
sys     1m54.810s

This is what I added to my DB_CONFIG:
set_flags       DB_TXN_NOSYNC
set_lg_dir      /data/ldap
set_lg_max      104857600
set_lg_bsize    26214400  <----------- added
set_cachesize   0 1048576 0 <--------- added

These numbers could probably be further tuned by analyzing the output
of db_stat -m, but, nonetheless, this is already a huge improvement.
Now to the indexes :)