[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: malloc/libumem
Aaron Richton wrote:
DB. Really fixing that requires a smarter malloc.
You've mentioned libumem as that "smarter malloc" along with
portability concerns: are you using libumem as a drop-in, or are you
seeing performance improvements based off umem_alloc(3MALLOC) use?
If it's a drop-in, I can't see anything bad about an autoconf check as
Quanah suggested. Take performance if you can get it, take portability
when you can't. Portable performance should help both cases. At the
cost of seconds of compile-time decision making, I can't see the flip
side of the coin. (Indeed, if this is the case, I have half a mind to
LD_PRELOAD in my slapd init scripts today.)
At the moment we're using it as a drop-in, with LD_PRELOAD. In my first
test I modified liblber to call it explicitly, because it didn't work as
an LD_PRELOAD drop-in, but I've managed to coerce umem into behaving a
little better. (It's still likely to crash when preloaded with arbitrary
programs, but it manages to survive its initialization when preloaded
into slapd.)
If it's umem_alloc, I don't see that much bad, although it certainly
could introduce new issues...but it'd probably be worth it in the long
run, as (a) libumem becomes increasingly portable, assuming an active
community there and/or (b) OpenLDAP's use of umem_alloc and friends
gets field tested by the OpenLDAP community.
umem seems to yield extremely consistent performance, with an above
average constant overhead. In contrast, glibc malloc can yield slightly
higher performance (smaller constant) but sometimes goes pathological on
us. I'll have to re-test with libhoard to get a feel for how it behaves
on the current data set. (The difference in performance, when all is
going well, is only 3% in glibc's favor.)
slapd is fairly brutal on malloc implementations because the range of
sizes of data objects is wide, and objects in the entry cache are most
often freed by a different thread than the one that allocated it. This
pattern is the most pessimal for libhoard, for example.
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc
OpenLDAP Core Team http://www.openldap.org/project/