[Date Prev][Date Next] [Chronological] [Thread] [Top]

ldapadd still slow, even with tweaks?



I'm trying to import a large number of entries (~200,000) into an OpenLDAP
2.0.7 directory (installed from RPMs, openldap-servers-2.0.7-14) under Linux
2.4.x. I'm basically feeding a very large ldif to ldapadd. However,
performance is less than spectacular; about 6,000 entries every half-hour.
CPU util stays low (10-15% cpu util on an otherwise-quiescent machine) and
the machine's got plenty of RAM for db caching.

After a little STFWing, I added 'dbnosync' and 'dbnolocking' to my
slapd.conf. This didn't improve performance appreciably. I tried cranking
the entry and index caches sky-high and that didn't make any difference in
performance, either. As a last-ditch effort, I tried splitting the ldif and
parallelizing four instances of ldapadd; the aggregate performance in this
case was similar to a single ldapadd.

I'm at a bit of a loss here. (Empirically,) I've gotten better performance
previously with large imports to an OpenLDAP 1.x directory with
'dbcachenowsync,' but turning off syncing doesn't seem to have much of an
impact in this case. I've considered feeding my large ldif to ldif2ldbm to
generate an initial database, but I don't want to do that Just In Case
everything isn't perfect in my ldif.

Any suggestions? I really need to get this import going faster than it's
going now and I've exhausted my bag of tricks in this situation.

thanks!
john
-- 
John Morrissey          _o            /\         ----  __o
jwm@horde.net        _-< \_          /  \       ----  <  \,
www.horde.net/    __(_)/_(_)________/    \_______(_) /_(_)__