[Date Prev][Date Next] [Chronological] [Thread] [Top]

Growing an LMDB database after MDB_MAP_FULL



Hi,

 

I have a question regarding growing an LMDB database when a write transaction hits MDB_MAP_FULL.

I would like to avoid defining a high mapsize value because my application will contain many MDB_envs, and because I have Windows users (Windows allocates the whole file on the disk).

 

Based on the intuition that MDB_MAP_FULL should not leave the database in a weird state, I have made the following little experiment. When MDB_MAP_FULL is encountered I tried to:

·         copy the current env (mdb_env_copy) into another directory (fine: it does not seem to contain uncommited data)

·         reset the transaction « error  bit » (modified LMDB code to introduce a « txn->mt_flags &= ~MDB_TXN_ERROR » somewhere)

·         commit the transaction

·         close the database

·         close the env

·         reopen it with a higher mapsize value

·         reopen the database

·         create another transaction

·         continue writing

… and it seems to be working pretty well.

 

Assuming I am ready to « relax » some of the ACID requirements, does it sound reasonable to think that MDB_MAP_FULL does not leave LMDB is a weird state? And that the « trick » described above should always be working? By « working » I mean: the copied environment will never contain uncommitted data (so I can rely on it to implement a kind of rollback), the reopened environment will always be valid and contain the expected data (data written before hitting MDB_MAP_FULL)?

 

Thanks in advance for any insight,

Best regards,

 

Bruno.