LMDB Hanging Problem in Multi-Threaded Application

[Yahoo! <e_ensafi@yahoo.com>]

Hello,

I successfully ran a multi-threaded application using LMDB 0.9.10-1 on a 64-bit Ubuntu 14.04 machine with 4x10-core Xeon E5-2660 v3 CPUs and 132 GB RAM.  I then attempted to run the same application on a CentOS 6.7 box with 2x4-core i7-4790K CPUs and 32 GB RAM, where I tried two versions of LMDB: first EPEL RPM package version 0.9.18-1, and then the latest version 0.9.70 which I downloaded and compiled from GitHub sources on 4/27/2016.  On the CentOS system with fewer cores, my application started to hang.  I have traced the problem to a periodic commit, after which a new transaction must be created.  I commit periodically to avoid a long delay when the application shuts down, at which point a 2 GB database is flushed to disk.

I have a single write transaction, which is protected by a mutex when calling mdb_txn_commit(), mdb_txn_begin() and mdb_put().  The threads are created by TBB using parallel_for, and each thread has access to the single MDB_dbi and associated MDB_txn for writing.  When a counter hits a certain value, we lock a mutex and call mdb_txn_commit(), and then we create a new MDB_txn to replace the invalidated one.  Other threads try to acquire the same mutex before attempting to call mdb_put().  After a new write transaction is created, the next call the mdb_put() works fine on my 40-core Ubuntu box and the database is created successfully, but the application hangs on my 8-core CentOS box.  My only guess is that I am encountering a race condition that is somehow avoided on the faster machine.

Is the above valid LMDB usage?  I have tried to find information about multi-threaded writing, and I have not found any definitive answers.  I would be happy to share the relevant code, but I need to first strip out a great deal of project-specific ancillary code, which will take some time.

Thanks,

Alex