[Date Prev][Date Next]
Incremental backup in LMDB
A preliminary branch has been available on gitorious for a few months. It has
some issues though, which still need to be resolved.
The biggest problem is that scanning every page of the DB is unsafe - if a
page was an overflow page, and was freed and partially reused, then a scan may
walk into intermediate chunks of data (which won't have valid page headers,
although the data might coincidentally appear to have them).
Two approaches to solving this are to perform the scan, but ignore pages that
are currently listed in the freeDB, or to do a regular tree walk instead. At
the moment I'm leaning toward just walking the trees. If the freeDB is large
then comparing every scanned page to the freeDB could be expensive. Also,
newer pages of the freeDB itself must be copied as well, and this would mean
freeDB pages get read twice.
Just thinking out loud here to document what we discussed before in IRC, so it
doesn't get lost in the meantime.
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/