Aaron Richton wrote:
DB. Really fixing that requires a smarter malloc. I've tested again with libhoard 3.5.1, and it's actually superior to libumem for both speed and fragmentation. Here are some results against glibc 2.3.3, libhoard, and libumem:
The initial size is the size of the slapd process right after startup, with the process totally idle. The id2entry DB is 1.3GB with about 360,000 entries, BDB cache at 512M, entry cache at 70,000 entries, cachefree 7000. The subsequent statistics are the size of the slapd process after running a single search filtering on an unindexed attribute, basically spanning the entire DB. The entries range in size from a few K to a couple megabytes. Since not everything fits in RAM, obviously the disk speed is a factor here but the disk and DB are identical from run to run. After running the single ldapsearch 4 times, I then ran the same search again with 4 jobs in parallel. There should of course be some process growth for the resources for 3 additional threads (about 60MB is about right since this is an x86_64 system). The machine only had 2GB of RAM, and you can see that with glibc malloc the kswapd got really busy in the 4-way run. The times might improve slightly after I add more RAM to the box. But clearly glibc malloc is fragmenting the heap like crazy. The current version of libhoard looks like the winner here. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc OpenLDAP Core Team http://www.openldap.org/project/ |