[Date Prev][Date Next] [Chronological] [Thread] [Top]

lmdb VL32 high virtual address usage

This sort of coincides with ITS issue 8347 I just reported.


With ldmb VL32, I’ve noticed that it can eat up a lot of memory space.  Sometimes ~500mb for an 800mb db after reading every record, one by one using the same readonly transaction/cursor.  I found that modifying MDB_ERPAGE_SIZE from 16384 to 256 brought virtual memory usage down substantially (~30megs).  I have not noticed any adverse effects from this change or MDB_MAP_FULL.  The mapped address spaces to seem to get deallocated on the close of the environment, not the transaction, which sounds like its being cached in the env->me_rpages for reasons explained in mdb_rpage_get comments.


Is the mem map cache supposed to be this aggressive by design, or just because this area of the code is so new...or simply a usage error on my part?  If it’s by design, could there be a consideration to make MDB_ERPAGE_SIZE a #define’able value in the public lmdb.h so one could tweak the behavior?