[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: LMDB stuff
- To: openldap-devel@openldap.org
- Subject: Re: LMDB stuff
- From: Emmanuel LÃcharny <elecharny@gmail.com>
- Date: Tue, 04 Feb 2014 08:08:20 +0100
- Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=knlHeARpVX1bYPFbxR6vn5qEs+JDcoLdLWeMfQCZJ1U=; b=Z+QzWznwfYqNsPFC3BUZvcHd48e07hquQgjcVTrMFjHmUH2dbSs7BNUjFfCR02YeXj OgPshWWNKFYww6Uu0ImgqFfWs66ctTh209McOfadSt3yh4ClpHQk7bUJyE83Yy4DbsUT YMwpkyRDeng5jI4eSCAcgTGiiHHvp4OOIhmnCNu/XbHJW2UoWJ49gfH4B3rlxJAmTKbC cUSz4SxKUh4w9T960KYoh/lBt5Vt2uWoU6tO4HrSbsmgURFcuqcSMRghGqD+HegxJ1hg N6NXqxc8sDdU53q6AjSL7lSY7tIjwwcQTFGoL7JpjaQ3V9q+z+nHA3UucxQloVSz6IHF srJg==
- In-reply-to: <5e0d90d8ec7d6cfd947c18215659ffdb@ulrik.uio.no>
- References: <52F006AF.4030104@symas.com> <5e0d90d8ec7d6cfd947c18215659ffdb@ulrik.uio.no>
- User-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:24.0) Gecko/20100101 Thunderbird/24.2.0
Le 2/4/14 3:29 AM, Hallvard Breien Furuseth a Ãcrit :
> On 2014-02-03 22:14, Howard Chu wrote:
>> Was chatting with Emmanuel Lecharny (who is currently working on
>> Mavibot for ApacheDS, an MVCC backend similar to LMDB) and had an
>> interesting realization: we can avoid the current issue of long-lived
>> reader txns preventing page reclamation.
>>
>> (...) since we're already going to add a txnID to every page's
>> page header, we can simply add a 2nd txnID, recording the txnID of the
>> previous change to this page's ancestor. Then, any page where this
>> prevTxnID is >= the outstanding reader's txnID can be reclaimed.
>
> Nice. But: Seems to me the freelist needs to know when the page
> was written, and when it was freed. A reader older than the txn
> which wrote the current contents of a page, is irrelevant to
> whether the page can be overwritten. How do ancestors matter? (Do
> you mean a branch page? The age of the previous page contents?)
>
>> Still thinking about the actual implementation of this, it may make
>> more sense to store the prevTxnID in the freeDB than in each page
>> header. Ideally we want to be able to grab a chunk of pageIDs
>> unambiguously, instead of having to iterate thru each page and read
>> its header to determine if it's safe.
>
> Yes, re-reading lots of pages just to find they can't be used does
> not sound fun. And we can't look for page headers in a broken-up
> overflow page. Unless we either quit breaking up freed ovpages,
> or write a page header to the unused chunk when breaking it up.
>
> Can we grab Mavibot's freeDB structure?
The free page structure is irrelevant. What is important is the copied
page structure, and we have the exact same data structure here : a
B-tree of <txn ID, list of copiedPage>.
The key is to leverage this B-tree to free the pages that we know won't
be used when we release a txn.
--
Regards,
Cordialement,
Emmanuel LÃcharny
www.iktek.com