Full_Name: Sergej Jurecko Version: lmdb latest OS: osx 10.9.5 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (193.189.172.218) Using a dupsort and writing a large value (which is still bellow maxkeysize) will result in MDB_PAGE_FULL. Program here: http://pastebin.com/Mu6bp1DR Compiling with: gcc pagefull.c mdb.c midl.c -g -DMDB_MAXKEYSIZE=10000 -o pagefull Running ./pagefull maxkeysize=10000 writing keysize=12, datasize=4063 Cursor put eror=-30786
On 27/05/15 09:00, sergej.jurecko@gmail.com wrote: > Using a dupsort and writing a large value (which is still bellow maxkeysize) > will result in MDB_PAGE_FULL. > (...) > maxkeysize=10000 Too big MDB_MAXKEYSIZE. Its doc says: "Keys and MDB_DUPSORT data items must fit on a node in a regular page." But the doc should clarify that MDB_MAXKEYSIZE can be set big enough to break lmdb. Or mdb_env_open() can fail if the computed me_maxkey would be smaller. Or compute a safe MIN_PAGESIZE from MDB_MAXKEYSIZE and use that, and then fail if the page size would be unsupported. (MIN_PAGESIZE can be computed at compile time, with some ugly macros.) lmdb sets the max safe maxkeysize if you define MDB_MAXKEYSIZE=0. Some time ago, that was 1/3 of the size you could reasonably use (to allow 1 key + sub-page with 2 dupsort values), so it could make sense to define MDB_MAXKEYSIZE above the safe max. Today lmdb uses a sub-DB at once if a sub-page would be too small, though. So I don't see much need to support maxkeysize > safe max.
On 27/05/15 10:33, h.b.furuseth@usit.uio.no wrote: > (blah...) > So I don't see much need to support maxkeysize > safe max. Sorry, I should I said there's no need to recommend changing MDB_MAXKEYSIZE as anything else than 0, except if you need to make sure old liblmdb versions can use the database. I don't want to restart the discussion about which errors to catch yet again.
Thank you for the explanation. Sergej On Wed, May 27, 2015 at 10:32 AM, Hallvard Breien Furuseth < h.b.furuseth@usit.uio.no> wrote: > On 27/05/15 09:00, sergej.jurecko@gmail.com wrote: > >> Using a dupsort and writing a large value (which is still bellow >> maxkeysize) >> will result in MDB_PAGE_FULL. >> (...) >> maxkeysize=10000 >> > > Too big MDB_MAXKEYSIZE. Its doc says: > "Keys and MDB_DUPSORT data items must fit on a node in a regular page." > > But the doc should clarify that MDB_MAXKEYSIZE can be set big enough > to break lmdb. > > Or mdb_env_open() can fail if the computed me_maxkey would be smaller. > > Or compute a safe MIN_PAGESIZE from MDB_MAXKEYSIZE and use that, > and then fail if the page size would be unsupported. (MIN_PAGESIZE > can be computed at compile time, with some ugly macros.) > > lmdb sets the max safe maxkeysize if you define MDB_MAXKEYSIZE=0. > Some time ago, that was 1/3 of the size you could reasonably use > (to allow 1 key + sub-page with 2 dupsort values), so it could make > sense to define MDB_MAXKEYSIZE above the safe max. Today lmdb > uses a sub-DB at once if a sub-page would be too small, though. > So I don't see much need to support maxkeysize > safe max. >
changed notes changed state Open to Test moved from Incoming to Documentation
Fixed in mdb.master and mdb.RE/0.9
changed notes changed state Test to Release
changed state Release to Closed