[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: write performance (was Re: LDIF parser performance)

Howard Chu wrote:
With synchronous writes and the original code in HEAD, 1GB BDB cache, it took 1:33:08.74 to ldapadd the database. With the patched add.c and the quicksort it took 1:14:52.11. As a point of reference, it took only 2:42.64 for slapadd -q.

Unfortunately the bulk of the ldapadd time is really determined by BerkeleyDB. With DB_TXN_NOSYNC and transaction logs written to a tmpfs, the original code in HEAD completes the ldapadd in 18:54.56. With the add.c patch that goes down to 13:35.82. With the quicksort in slap_mods_check that drops down further to 7:19.52.

One further refinement - omitting checkpoint calls from the main operation, and leaving them to occur only in the checkpoint thread, drops the time down to 6:16.55. I'd say this is the desired behavior when you only configure a checkpoint time and not a checkpoint size, but if you configure a size then you really need to call after each commit.

I'm not sure how realistic any of these configurations are, but I suppose with a fast enough disk subsystem synchronous writes wouldn't take such a heavy toll.

  -- Howard Chu
  Chief Architect, Symas Corp.  http://www.symas.com
  Director, Highland Sun        http://highlandsun.com/hyc
  OpenLDAP Core Team            http://www.openldap.org/project/