[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: write performance (was Re: LDIF parser performance)
Howard Chu wrote:
Howard Chu wrote:
With synchronous writes and the original code in HEAD, 1GB BDB cache,
it took 1:33:08.74 to ldapadd the database. With the patched add.c and
the quicksort it took 1:14:52.11. As a point of reference, it took
only 2:42.64 for slapadd -q.
Unfortunately the bulk of the ldapadd time is really determined by
BerkeleyDB. With DB_TXN_NOSYNC and transaction logs written to a
tmpfs, the original code in HEAD completes the ldapadd in 18:54.56.
With the add.c patch that goes down to 13:35.82. With the quicksort in
slap_mods_check that drops down further to 7:19.52.
One further refinement - omitting checkpoint calls from the main
operation, and leaving them to occur only in the checkpoint thread,
drops the time down to 6:16.55. I'd say this is the desired behavior
when you only configure a checkpoint time and not a checkpoint size, but
if you configure a size then you really need to call after each commit.
With the silly quadratic stuff out of the way, there's only small
incremental gains to be made. The oc_bvfind change got things down to
5:55. Optimizing entry_dup will probably get it down to 5:20 or so. At
this point, having come down from 1:33:09 to 6:00 or so, within about a
2x factor of slapadd -q is probably good enough. The remaining lber
overhead isn't going to go away very easily...
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc
OpenLDAP Core Team http://www.openldap.org/project/