RE: back-bdb deadlocks

["Kurt D. Zeilenga" <Kurt@OpenLDAP.org>]

At 04:49 AM 2002-01-18, Howard Chu wrote:
>> -----Original Message-----
>> From: Kurt D. Zeilenga [mailto:Kurt@OpenLDAP.org]
>
>> Even that's pretty slow.  I see on my old FreeBSD devbox (2*P350)
>> with userland threads (one execution context).  For -008:
>>
>>   1.323u 1.586s 1:18.83 3.6%      222+3232k 68+265io 148pf+0w
>>
>> Have you updated glibc/pthreads, did you completely rebuild bdb?
>> I wonder if NGpthreads for Linux (from IBM) would help.
>
>I've updated to glibc 2.2.4/linuxthreads 0.9, did not rebuild bdb after that.
>
>Your db_stat results are interesting, 0 deadlocks. I can now get this
>sometimes, but other times there are a few. I would expect this to be totally
>non-deterministic.

That's exactly what I see. I've attached numbers for a run which
did have a few conflicts.

>Must have something to do with thread scheduling
>algorithms... With usermode threads you also need aio support, otherwise
>any blocking I/O blocks all threads, making you effectively single-threaded.

Not when using FreeBSD pthreads...  the library properly transfers
control to another thread if the I/O operation would block.
(Of course, if the kernel blocks where it shouldn't then the
process will block, but as far as I can tell.)

>That would certainly give deadlock-free execution.

Well, I do occasionally see deadlock, but I haven't seen
things degrade into what you are seeing.

Until I added a fourth writer...

I killed slapd(8) before completion as I got tired of waiting
for it to complete:

1/1991648       File/offset for last checkpoint LSN.
0/0     No pending checkpoint LSN.
Fri Jan 18 07:48:23 2002        Checkpoint timestamp.
80000227        Last transaction ID allocated.
20      Maximum number of active transactions possible.
0       Active transactions.
3       Maximum active transactions.
551     Number of transactions begun.
351     Number of transactions aborted.
200     Number of transactions committed.
0       Number of transactions restored.
16KB    Transaction region size (16384 bytes).
0       The number of region locks granted after waiting.
1119    The number of region locks granted without waiting.

78 Last allocated locker ID.
9       Number of lock modes.
1000    Maximum number of locks possible.
1000    Maximum number of lockers possible.
1000    Maximum number of objects possible.
100852  Current locks.
100872  Maximum number of locks so far.
0       Current number of lockers.
28      Maximum number  lockers so far.
0       Current number lock objects.
18      Maximum number of lock objects so far.
130227  Number of lock requests.
129863  Number of lock releases.
0       Number of lock requests that would have waited.
1881    Number of lock conflicts.
364     Number of deadlocks.
0       Number of transaction timeouts.
0       Number of lock timeouts.
352KB   Lock region size (360448 bytes).
1       The number of region locks granted after waiting.
150758  The number of region locks granted without waiting.


>> 1/2743070       File/offset for last checkpoint LSN.
>> 0/0     No pending checkpoint LSN.
>> Thu Jan 17 21:01:23 2002        Checkpoint timestamp.
>> 80000136        Last transaction ID allocated.
>> 20      Maximum number of active transactions possible.
>> 0       Active transactions.
>> 3       Maximum active transactions.
>> 310     Number of transactions begun.
>> 5       Number of transactions aborted.
>> 305     Number of transactions committed.
>> 0       Number of transactions restored.
>> 16KB    Transaction region size (16384 bytes).
>> 0       The number of region locks granted after waiting.
>> 637     The number of region locks granted without waiting.
>>
>> 73 Last allocated locker ID.
>> 9       Number of lock modes.
>> 1000    Maximum number of locks possible.
>> 1000    Maximum number of lockers possible.
>> 1000    Maximum number of objects possible.
>> 150846  Current locks.
>> 150864  Maximum number of locks so far.
>> 0       Current number of lockers.
>> 22      Maximum number  lockers so far.
>> 0       Current number lock objects.
>> 16      Maximum number of lock objects so far.
>> 178824  Number of lock requests.
>> 178824  Number of lock releases.
>> 0       Number of lock requests that would have waited.
>> 699     Number of lock conflicts.
>> 0       Number of deadlocks.
>> 0       Number of transaction timeouts.
>> 0       Number of lock timeouts.
>> 352KB   Lock region size (360448 bytes).
>> 19      The number of region locks granted after waiting.
>> 199355  The number of region locks granted without waiting.
>>
>
>  -- Howard Chu
>  Chief Architect, Symas Corp.       Director, Highland Sun
>  http://www.symas.com               http://highlandsun.com/hyc
>  Symas: Premier OpenSource Development and Support