[Date Prev][Date Next] [Chronological] [Thread] [Top]

LMBD questions (stat & copy)



Hi,

I am trying to detect MDB_MAP_FULL based on mdb_stat(txn, dbi, &stat) results and I am wondering if I am on the right track or doing things correctly.
I have
a small and simple test program (http://pastebin.com/SPCYgWMC) exhibiting the following behavior (consistent between Windows 7 64-bit and Ubuntu 15.10 64-bit - latest version of lmdb cloned and recompiled on Linux):

  • Inserting 20990 key value pairs in a database is OK when using one transaction. Just before commiting the write transaction, mdb_stat(txn, dbi, &stat) shows that the total number of pages (253) is very close the the number of pages of the map (256). Indeed, adding another entry leads to MDB_MAP_FULL. To put it shortly, this scenario looks OK (I am assuming there might be 3 "internal-purpose" pages not shown by mdb_stat) and can be tested by setting the stat_test variable to 0 at line 20 of the test program.
  • Inserting same 20990 key value pairs in a database fails (MDB_MAP_FULL encountered at 19357) when using two consecutive write transactions. Just before MDB_MAP_FULL, mdb_stat(txn, dbi, &stat) shows that the total number of pages (232) is "quite far" from the the number of pages of the map (256), leading my MDB_MAP_FULL detection heuristic to fail. It means that only 92% of the total amount of data can be sucessfully added to the database. Other scenarii (with same key/value lengths though) stop at 85% only. This can be tested by setting the stat_test variable to 1 (line 20).

By "total number of pages" I mean ms_branch_pages + ms_leaf_pages + ms_overflow_pages.

Do you think I am on a wrong track? (maybe mdb_stat does not show 24 "internal-purpose" pages in the second case? or I am forgetting something?).

 

One last remark: it seems that mdb_env_copy(db, copy_dir) is possible during a write transaction on Windows (like stated in the documentation), but not on Linux (the test program seems to be stopped on a mutex at line 62). This can be tested by changing the copy_test variable to 1 at line 19. Is it expected given my program?

 

Best regards,

 

Bruno.