(ITS#8582) LMDB lockfile/mutex arch dependency

[hyc@openldap.org]

Full_Name: Howard Chu
Version: HEAD
OS: Linux
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (195.235.15.200)
Submitted by: hyc


I ran into some corruption problems when running a 64bit LMDB process concurrent
with a 32bit one using MDB_VL32. While LMDB's own structures are all 64bit clean
in a VL32 build, unfortunately pthread_mutex_t structures are not. In
particular, in glibc a pthread_mutex_t can be one of 3 different sizes - 24
bytes for 32bit runtime on 32bit processor, 32 bytes for 32bit runtime on
x86-64, and 40 bytes for 64bit runtime on x86-64. As such, we cannot safely
share an LMDB environment between 32 and 64bit Linux processes, even when using
MDB_VL32.

This problem doesn't affect Windows, platforms using SysV semaphores, or
platforms using POSIX named semaphores. I don't currently know whether this is a
problem on other platforms that support POSIX process-shared mutexes.


Hallvard has suggested that we add some additional LOCK_FORMAT bits to protect
against incompatible accesses. This would primarily be useful for the mdb_stat
command, which can still safely retrieve some useful information from an
environment without acquiring any mutexes. E.g., an MDB_VL32 build of mdb_stat
could retrieve most envinfo, statinfo, and readerinfo from a running 64bit
environment. It could not scan the freelist, scan all the names of named DBs, or
clean the reader list, since these operations all require taking the reader
mutex.