[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: Using LMDB safely with fork() and exec()




On 24 August 2016 at 06:27, Hallvard Breien Furuseth <h.b.furuseth@usit.uio.no> wrote:
On 22/08/16 12:47, Lorenz Bauer wrote:
The LMDB documentation says the following in its section on caveats:

* Use an MDB_env* in the process which opened it, without fork()ing.
* Do not have open an LMDB database twice in the same process at the same time. Not even from a plain open() call - close()ing it breaks flock() advisory locking.

This seems contrary to an earlier thread on this list (1), which
suggests that fork/execing a process using LMDB is OK so long as the
MDB_env is not used in the forked process.

It's just poorly worded.  The point of the first sentence is,
"do not use the MDB_env after forking".  It's OK to fork if you
do not use it afterwards.  A patch with a better wording might
be in order:-)

Happy to do that.
 
MDB was originally written with Unix in mind.  I'm guessing the "no
multiple handles" restriction is only relevant on Unix: Hopefully
Windows locking doesn't have flock's insane semantics.
But I don't know Windows.

According to the CreateFileW docs, calling the it without a security descriptor (which is what LMDB does) makes it behave like O_CLOEXEC. So it's only the POSIX code that is affected.

OTOH it won't hurt to add close-on-fork/exec flags for everything, not
just the Unix lockfile descriptor.  Would need to factor the opens out
to a separate function to avoid excessive code and #ifdef duplication,

I've got a patch for this which I'll submit once I get approval.
 
Regarding your next message:

The unix version only uses O_DIRECT if psize >= OS psize
because O_DIRECT typically requires alignment on OS page
boundaries, or something like that.  Should be commented.
Didn't find anything similar in the Windows doc, but again,
I don't know Windows.  Maybe Howard knows more.

According to man 2 open O_DIRECT alignment is file system specific from 2.6 onwards, but "usually" 512bytes FWIW. Not sure how that would affect this code.