[Date Prev][Date Next]
Re: (ITS#8777) [LMDB] Closing read cursor uses already freed transaction (MDB_VL32)
- To: openldap-its@OpenLDAP.org
- Subject: Re: (ITS#8777) [LMDB] Closing read cursor uses already freed transaction (MDB_VL32)
- From: firstname.lastname@example.org
- Date: Mon, 27 Nov 2017 04:37:55 +0000
- Auto-submitted: auto-generated (OpenLDAP-ITS)
Markus Junginger wrote:
> Thanks for the report. Unfortunately your suggested fix means the rpage
> test will always be skipped on read-only cursors, which will break the
> rpage tracking if the txn is still alive.
> Could you maybe explain a bit about rpages? Why does mdb_cursor_close need to
> unref pages, but not mdb_cursor_renew? If closing a transaction does not clean
> up rpages, I would have expected mdb_cursor_renew would have to do. And why
> does setting MDB_VL32 affect rpages in the first place? To my understanding,
> MDB_VL32 is about using 64 bits also on 32 bit CPUs, so a differences on page
> handling seem surprising at first sight.
Think about how LMDB works.
LMDB was originally written for use on 64bit CPUs, with 64bit virtual address
spaces. It expects to map an entire DB into virtual memory using a single mmap
Obviously this cannot be done when trying to use a DB larger than 2GB on a
machine with only a 32bit virtual address space.
When using MDB_VL32 to access a 64bit DB on a 32bit CPU, the DB cannot fit
into a single mmap. It must be mapped in chunks, and the chunks must be
mapped, unmapped, and remapped on demand. The rpages thus need to be tracked,
to know which are currently mapped. None of this rpage tracking exists in the
native non-VL32 build because it's completely unnecessary.
Also note that the MDB_VL32 feature is not officially released. You're free to
use it if you want, but there is no expectation that it is suitable for
general use. When you use unreleased features, you're expected to know how to
support yourself. When you use LMDB at all, you're expected to know how
virtual addresses work.
Ideally, you should quit using obsolete 32bit CPUs.
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/