[Date Prev][Date Next] [Chronological] [Thread] [Top]

RE: Questions...

> -----Original Message-----
> From: owner-openldap-devel@OpenLDAP.org
> [mailto:owner-openldap-devel@OpenLDAP.org]On Behalf Of Jong

> > The high write concurrency in back-hdb has been largely defeated by the
> > syncrepl entryCSN update code, which serializes all modifications in both
> > back-bdb and back-hdb.
>
> This seems inevitable if we store the context CSN in the database,
> because the DB pages containing the context CSN entry will be
> a hot spot.
> One option would be to make it optional to maintain the context CSN.
> Another option would be to maintain the context CSN only in memory
> (not in database) to relieve the contention, but even in this case
> the BDB locking for the memory location would still remain contended.

The BDB nextid is already maintained purely in memory, using a regular mutex.
As long as the entryCSN critical section is small, it will be an improvement
over the current implementation.

> BTW, I didn't observe a noticeable performance difference
> when I ran test008.
> Also, I want to see how much performance degradation the current
> context CSN design is imposing on the update performance, because
> the context CSN update code is placed at a point close to the
> transaction
> commit so the performance impact may not be as large as it seems to.
> It would be very helpful to see performance data if you have some.

I created a 1 million entry database using slapadd from the various releases.
Here are the results. The base DN was o=Test,c=US and there were 50 nodes
under there (st=Alabama to st=Wyoming) and 1,000,000 person objects
distributed randomly under those. The entries consisted only of a cn and sn,
in addition to the dn and objectClass.

The results here show the run time and the database size for the various
datbase versions. All of the  code was built on the same machine, with BDB
4.2.52, so the only variable is the OpenLDAP code. Notice that 2.2.5 is
slightly less efficient than RE21, with all else identical. No indexing was
done.

REL_ENG_2 ldbm (2.0.27 + idl.c fixes)
879.130u 565.180s 48:35.92 49.5%        0+0k 0+0io 105330pf+0w
total 332611
-rw-------   1 hyc      users    187510784 Jan 26 02:09 dn2id.dbb
-rw-------   1 hyc      users    151740416 Jan 26 02:08 id2entry.dbb
-rw-------   1 hyc      users        8192 Jan 26 02:07 nextid.dbb
REL_ENG_2_1 ldbm
801.210u 459.460s 29:13.49 71.8%        0+0k 0+0io 69899pf+0w
total 587342
-rw-------   1 hyc      users    187617280 Jan 26 02:47 dn2id.dbb
-rw-------   1 hyc      users    411459584 Jan 26 02:46 id2entry.dbb
-rw-------   1 hyc      users        8192 Jan 26 02:45 nextid.dbb
REL_ENG_2_2_5 ldbm
842.710u 463.140s 29:23.94 74.0%        0+0k 0+0io 70607pf+0w
total 587342
-rw-------   1 hyc      users    187617280 Jan 26 03:21 dn2id.dbb
-rw-------   1 hyc      users    411459584 Jan 26 03:19 id2entry.dbb
-rw-------   1 hyc      users        8192 Jan 26 03:18 nextid.dbb
REL_ENG_2_1 bdb
689.950u 56.070s 17:20.57 71.6% 0+0k 0+0io 276994pf+0w
total 1111581
-rw-r--r--   2 hyc      users          98 Jan 26 03:31 DB_CONFIG
-rw-------   1 hyc      users        8192 Jan 26 03:49 __db.001
-rw-------   1 hyc      users    503324672 Jan 26 03:49 __db.002
-rw-------   1 hyc      users     2162688 Jan 26 03:49 __db.003
-rw-------   1 hyc      users      368640 Jan 26 03:49 __db.004
-rw-------   1 hyc      users       16384 Jan 26 03:49 __db.005
-rw-------   1 hyc      users    196280320 Jan 26 03:48 dn2id.bdb
-rw-------   1 hyc      users    431816704 Jan 26 03:49 id2entry.bdb
REL_ENG_2_2_5 bdb
714.440u 58.610s 18:19.99 70.2% 0+0k 0+0io 280852pf+0w
total 1198287
-rw-r--r--   2 hyc      users          98 Jan 26 03:31 DB_CONFIG
-rw-------   1 hyc      users        8192 Jan 26 04:10 __db.001
-rw-------   1 hyc      users    503324672 Jan 26 04:10 __db.002
-rw-------   1 hyc      users     2162688 Jan 26 04:10 __db.003
-rw-------   1 hyc      users      368640 Jan 26 04:10 __db.004
-rw-------   1 hyc      users       16384 Jan 26 04:10 __db.005
-rw-------   1 hyc      users    196280320 Jan 26 04:09 dn2id.bdb
-rw-------   1 hyc      users    520257536 Jan 26 04:10 id2entry.bdb
REL_ENG_2_2_5 hdb
473.320u 48.230s 12:40.35 68.5% 0+0k 0+0io 273735pf+0w
total 1048434
-rw-r--r--   2 hyc      users          98 Jan 26 03:31 DB_CONFIG
-rw-------   1 hyc      users        8192 Jan 26 04:40 __db.001
-rw-------   1 hyc      users    503324672 Jan 26 04:40 __db.002
-rw-------   1 hyc      users     2162688 Jan 26 04:40 __db.003
-rw-------   1 hyc      users      368640 Jan 26 04:40 __db.004
-rw-------   1 hyc      users       16384 Jan 26 04:40 __db.005
-rw-------   1 hyc      users    131985408 Jan 26 04:39 dn2id.bdb
-rw-------   1 hyc      users    431702016 Jan 26 04:39 id2entry.bdb

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support