[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: slapadd performance

Pete Giesin wrote: 
We are using Symas OpenLDAP 2.4.12.1 on a  Red Hat EL 5.2 64-bit with
8GB of memory.

My DB_CONFIG settings are:
set_cachesize 3 0 2
set_lk_max_locks   3000
set_lk_max_objects 1500
set_lk_max_lockers 1500
set_lg_regionmax 262144
set_lg_bsize 2097152
set_lg_dir /opt/symas/var/openldap-logs/amwater
set_flags       DB_LOG_AUTOREMOVE
set_tas_spins 1

The 10 million user load finished in about 18 hours. I believe the
primary issue is the fact that we have a single disk on the system. I
assume that if I were to move the database and log files off to separate
disks I would get much better performance.

With slapadd -q there are no transactions, so there will not be any log file traffic. It sounds like your DB is very much larger than your available memory, so you're simply being limited by your disk speed. Running vmstat while you load would confirm this.

Thanks,
Pete


On Wed, Mar 18, 2009 at 11:26 PM, Quanah Gibson-Mount <quanah@zimbra.com
<mailto:quanah@zimbra.com>> wrote:

    --On Wednesday, March 18, 2009 1:25 PM -0400 Pete Giesin
    <pgiesin@hubcitymedia.com <mailto:pgiesin@hubcitymedia.com>> wrote:

        I am trying to perform some benchmarking against OpenLDAP. So
        far I have
        ran my tests against a 100K and 1Million entry database and I
        have had
        rather decent numbers. My final set of tests were to be ran
        against a
        10Million entry database. Unfortunately, I am having difficult
        loading
        the database with this many entries. I have generated 10
        1Million LDIF
        files. I am using "slapadd -c -q -v -l <file>" to import each
        file. The
        first 2 files took approximately 15 minutes each to load. The
        remaining 8
        files are taking progressively longer and longer. So much longer
        that I
        anticipate the entire proceess to take well over 24 hours. My
        question is
        is there anything I can do to increase the performance of slapadd. I
        assume that since slapd is not running at this point that the normal
        DB_CONFIG and slapd.conf settings do not have much affect.


    What are the settings in your DB_CONFIG file?  It is absolutely
    critical to the performance of slapadd.  What version of OpenLDAP
    are you using?  What version of BDB?  What operating system?

    --Quanah

    --

    Quanah Gibson-Mount
    Principal Software Engineer
    Zimbra, Inc
    --------------------
    Zimbra ::  the leader in open source messaging and collaboration




--
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/