[Date Prev][Date Next] [Chronological] [Thread] [Top]

Berkeley DB congestion control patch (RE: commit: ldap/servers/slapd/back-bdb trans.c ...)


This patch relieves the deadlock contention of Berkeley DB significantly.
The patch is based on exponential backoff widely used in the multiple access control.

The Berkeley DB congestion behavior is observed in one (1-way PIII 1G) of my development boxes,
but not observed in another (2-way Athlon 1700+).
With the congestion control patch, test008-concurrency becomes stable in both boxes.

The following is the execution time and number of deadlocks of test008-concurrency on the first box.
----------------------------------------------------------------------------------------------------------------------------
                                                                            Execution Time              # Deadlocks
----------------------------------------------------------------------------------------------------------------------------
1. Cong Ctrl (exp backoff)                             5m29s                                    303    (avg of 3 runs)
2. Cong Ctrl (fixed 1sec backoff)                6m17s                                     357   (avg of 3 runs)
3. No Cong Ctrl                                               51m28s                                  5075   (1 run)
----------------------------------------------------------------------------------------------------------------------------

Under high contention, DB_LOCK_YOUNGEST does not solve the problem,
especially when there are multiple rolled-back threads.
In a quick experiment, the execution time and # of deadlocks
when DB_LOCK_YOUNGEST is set are observed unstable as in the above table :

----------------------------------------------------------------------------------------------------------------------------
4. No Cong Ctrl with DB_LOCK_YOUNGEST           43m52s             4501   (1 run)
----------------------------------------------------------------------------------------------------------------------------

- Jong

------------------------
Jong Hyuk Choi
IBM Thomas J. Watson Research Center - Enterprise Linux Group
P. O. Box 218, Yorktown Heights, NY 10598
email: jongchoi@us.ibm.com
(phone) 914-945-3979    (fax) 914-945-4425   TL: 862-3979


"Howard Chu" <hyc@highlandsun.com>
Sent by: owner-openldap-devel@OpenLDAP.org

07/01/2003 06:22 AM

       
        To:        <jongchoi@OpenLDAP.org>, "'OpenLDAP Commit'" <openldap-commit2devel@OpenLDAP.org>
        cc:        
        Subject:        RE: commit: ldap/servers/slapd/back-bdb trans.c Makefile.in add.c delete.c modify.c modrdn.c passwd.c proto-bdb.h



This sounds a bit on the heavyweight side. I think using DB_LOCK_YOUNGEST for
deadlock resolution should handle most problems. It should ensure that
operations complete in the same order they were received, and thus should
also fix the out-of-order replog problem as well.

 -- Howard Chu
 Chief Architect, Symas Corp.       Director, Highland Sun
 http://www.symas.com               http://highlandsun.com/hyc
 Symas: Premier OpenSource Development and Support

> -----Original Message-----
> From: owner-openldap-commit@OpenLDAP.org
> [mailto:owner-openldap-commit@OpenLDAP.org]On Behalf Of
> jongchoi@OpenLDAP.org
> Sent: Tuesday, July 01, 2003 2:55 AM
> To: OpenLDAP Commit
> Subject: commit: ldap/servers/slapd/back-bdb trans.c Makefile.in add.c
> delete.c modify.c modrdn.c passwd.c proto-bdb.h
>
>
> Update of /repo/OpenLDAP/pkg/ldap/servers/slapd/back-bdb
>
> Modified Files:
>                  Makefile.in  1.21 -> 1.22
>                  add.c  1.78 -> 1.79
>                  delete.c  1.82 -> 1.83
>                  modify.c  1.80 -> 1.81
>                  modrdn.c  1.113 -> 1.114
>                  passwd.c  1.56 -> 1.57
>                  proto-bdb.h  1.79 -> 1.80
> Added Files:
>                  trans.c  NONE -> 1.1
>
> Log Message:
> Berkeley DB congestion avoidance
>
>
> CVS Web URLs:
>   http://www.openldap.org/devel/cvsweb.cgi/servers/slapd/back-bdb/
>
http://www.openldap.org/devel/cvsweb.cgi/servers/slapd/back-bdb/trans.c

http://www.openldap.org/devel/cvsweb.cgi/servers/slapd/back-bdb/Makefile.in
   http://www.openldap.org/devel/cvsweb.cgi/servers/slapd/back-bdb/add.c
   http://www.openldap.org/devel/cvsweb.cgi/servers/slapd/back-bdb/delete.c
   http://www.openldap.org/devel/cvsweb.cgi/servers/slapd/back-bdb/modify.c
   http://www.openldap.org/devel/cvsweb.cgi/servers/slapd/back-bdb/modrdn.c
   http://www.openldap.org/devel/cvsweb.cgi/servers/slapd/back-bdb/passwd.c

http://www.openldap.org/devel/cvsweb.cgi/servers/slapd/back-bdb/proto-bdb.h

Changes are generally available on cvs.openldap.org (and CVSweb)
within 30 minutes of being committed.