[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: LMDB: "non-cached" reads

Dmitri Shubin wrote:
> Hi,
> 
> Our application uses LMDB and sometimes we need to perform scan of old ("cold") data.
> The problem here is that this data can/will remove "hot" data from page-cache.
> 
> Ideally to prevent this from happening we'd like to somehow mark cursor as "streaming"/"non-temporal".
> But AFAICS there is no such feature currently.
> 
> We're experimenting with marking pages returned by cursor with madvise(MADV_DONTNEED) and this at least prevents RSS from growing.
> But maybe there are some downsides of this approach?

You obviously can only mark the pages after you've used the result. If you mark it
before returning the data to the caller, some other thread may cause those pages
to be reused before you get a chance to see the data, causing a wasted re-fetch.

-- 
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/