[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: (ITS#7432) slapd crashes when removing members from a large group

--On Monday, November 05, 2012 10:10 AM -0800 Howard Chu <hyc@symas.com> wrote:

hyc@symas.com wrote:
marvin.mundry@uni-hamburg.de wrote:
Full_Name: Marvin Mundry
Version: 2.4.33
OS: Ubuntu 12.10
URL: https://idmswiki.rrz.uni-hamburg.de:8005/debug.tar.bz
Submission from: (NULL) (

Thanks for the report. The crash has been fixed in git, but your test
runs into another (known) issue in MDB.

You're working with a very large entry, which libmdb stores in overflow
pages. In the current version of libmdb, freespace management for
overflow pages is not fully implemented, so every time you update the
entry libmdb will always use new pages (instead of reusing old pages).
Thus, after a few hundred operations, your 1GB map will be exhausted.

It looks like you won't be able to use back-mdb until this feature is
fully implemented in libmdb.

So the issue is how to find a contiguous run of pages large enough to
satisfy the overflow page, in the current freelist. This takes us into
the realm of malloc algorithms, first-fit/best-fit/..., etc.

I think first we scan whatever freelist we have in memory, to see if a
suitable run of pages is already present.

If not, and there are additional freelists still available:
   1) we could just merge all of them, and then search again
   2) merge one at a time, and search again

Leaning toward #2, I suspect we don't need to coalesce all freelists all
the time.

I like the sound of #2 as well.  If you come up with a patch, I can test. ;)



Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
Zimbra ::  the leader in open source messaging and collaboration