[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: [LMDB] Lockups with robust mutexes and crashing processes

Marcos-David Dione wrote:
         I already posted this to the IRC channel, but there was no
response, so I repost this here.

... already followed up in IRC.

         I'm trying out lmdb from master, including the robust mutex
code. We're experiencing lock ups after the process holding the lock
dies, as if the robust lock was not recovered. I tried to come up with
an lmdb example that shows it and I got it, just a few lines. It uses
fork() just to automate it; see that the environment is opened in both
children. Here's the code:


The example is broken; it does not mimic the behavior of a crashed process. In particular it does a clean call to mdb_env_close() but doesn't call mdb_txn_abort() first. An actual crashing process would not make the call to mdb_env_close(), and a cleanly exiting process would close all outstanding transactions before calling env_close.

         If I run this, I see that one of the children waits for the
write lock and is not awakened when the other child dies without closing
the txn (but notice I close the env). This is on purpose, to simulate a
crashing process.The worst part is that I can't reproduce it using
directly libpthread and mmap. Here is the code I came up with:


         It's a little bit more verbose because I based it on a glibc
test case.

         Are we missing anything? It seems to us that the code follows
does not break any of LMDB's caveats (specially the one about creating
the envs before fork()'ing. Is it wrong to assume that the waiting
process should recover the lock from staleness?

env_close does an munmap of the memory containing the mutex. According to the manpages, a robust mutex is supposed to automatically unlock when unmapped. Since this is not happening, it appears you've found a kernel bug. Regardless, the example is invalid. If you modify the code to just exit/abort/die without the bogus call to env_close, the other process wakes up correctly. E.g.


Marcos Dione
Astek Sud-Est
for Amadeus SAS
T: +33 (4)4 9704 1727

  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/