Re: Increasing memory usage on successive LMDB reads

[Rohit Girdhar <rohit.iiith@gmail.com>]

Thanks, that helps!

On Fri, Jul 31, 2015 at 10:02 AM, Howard Chu <hyc@symas.com> wrote:

Rohit Girdhar wrote:

Hi

I'm using LMDB (original C interface) to store and read some data. However, I
notice that my resident memory usage (RES in `htop`) keeps rising with each
call to mdb_get with unique keys. It does not change if I do a repeated call
to mdb_get with a key I've used before.

It looks like mdb_get is caching the reads,

LMDB does no caching of its own. Read the docs. Learn how mmap works.

but I want to make sure what
effect it is really having on my system. Is there some way to limit the amount
of memory lmdb can use for this? Or is it something strange and should not be
happening?

There's nothing strange going on. LMDB's memory use is entirely constrained by how much demand for memory other processes on the system are imposing, i.e. system memory pressure. If there is no pressure, LMDB will use all available RAM (or up to the configured mapsize, whichever is less). If there is memory pressure, LMDB will use less. Memory used by LMDB grows and shrinks dynamically and automatically with the system load.

--
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/