[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: lmdb

Patrick Hawley wrote:



I've been puzzled by lmdb's free list use. In some experiments, I've
been seeing that it sometimes grabs pages from the end instead of
using free list pages. This means that the number of pages used
tends to grow until it uses them all. I've tried increasing the map size
(to even quite a large value), and ensuring that write transactions are
not held open long but that doesn't seem to help.
Length of write transactions is irrelevant. Nothing in the LMDB documentation talks about length of write transactions; perhaps you need to reread the docs. 

The issue is keeping a lot of read transactions open when writes are in progress.


 I've tried with

subdatabases and without subdatabases. It does appear to increase
faster when overflow pages are used, but even without overflow
pages the effect occurs. I can see in the code some places where it
might choose not to use the free list, but I haven't been able to
identify a problem in the code. Wondering whether anyone else
has observed this also. The library otherwise looks quite useful
and performant. Using lmdb via python (0.92). Thanks.

Patrick



--
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/