[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: (ITS#7651) LMDB: Uncontrolled database when opened from multiple processes
- To: openldap-its@OpenLDAP.org
- Subject: Re: (ITS#7651) LMDB: Uncontrolled database when opened from multiple processes
- From: hyc@symas.com
- Date: Tue, 30 Jul 2013 20:19:33 GMT
- Auto-submitted: auto-generated (OpenLDAP-ITS)
dimitrij.denissenko@blacksquaremedia.com wrote:
> Full_Name: Dimitrij Denissenko
> Version:
> OS: Ubuntu 12.04
> URL:
> Submission from: (NULL) (62.30.100.0)
>
>
> Hi,
>
> I found an interesting issue with LMDB. I have populated the DB with a bunch of
> records and it uses ~30M on disk (after sync). Then I added a background process
> to my app and populated the database again with the same record set.
> Surprisingly. the resulting size on disk was >70M.
>
> The background process is forked periodically to perform some maintenance tasks,
> here is my (simplified) code:
>
> /* Close env before forking */
> mdb_env_close(env);
>
> if ((childpid = fork()) == 0) {
> /* Child */
> rc = mdb_env_open(env, ".", MDB_NOSYNC, 0644);
> ...
> } else {
> /* Parent */
> rc = mdb_env_open(env, ".", MDB_NOSYNC, 0644);
> ...
> }
>
> I could narrow it down to the mdb_env_open call in the child. If I add exit(0)
> before the mdb_env_open line, the DB size remains consistently at ~30M. The data
> size seems to grow proportionally to the number of forks performed during data
> load. What could be causing the growth? What can I do to prevent it?
>
> Thanks in advance
>
> PS: I tried it with MDB_FIXMAP and without, same result.
Without seeing more of your code, it's impossible to tell. Are you adding the
data on both sides of the fork? In the above code snippet, where are your
mdb_put calls occurring? Are both the parent and child processes writing
identical data?
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/