[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: commit: ldap/servers/slapd entry.c
Howard Chu wrote:
While profiling a large (2 million entry) slapadd run we found that
43% of the total runtime was spent in str2entry, only 29% was being
consumed by database operations (bdb_tool_entry_put). For those 2
million entries, str2entry called ch_free 198 million times. This
patch reduced the malloc churn a fair bit, str2entry now only calls
ch_free 70 million times, which resulted in saving about 12.5% of the
total slapadd runtime. Now str2entry vs DB operations is only 34.8% vs
34.2% of the total runtime.
It still seems rather pathetic though. This is a large chunk of time,
and it surely cannot be as complex a job as all of the rest of the
backend + database work combined.
After rewriting str2entry, some more profiling, and various other
tweaks, it looks like about 33% of slapadd's runtime has been trimmed
off. Turning off transactions didn't have a huge effect, but it helped some.
Also did some profiles of slapindex to see why it performs so poorly. No
great surprise, it has to read every entry from id2entry, which consumes
the majority of the BDB cache, so there's not much cache left over for
writing the index databases. If you bump up the memory-map limit in BDB
so that id2entry can be mapped instead of cached, then the cache can be
used more effectively for the indexing. All of this presumes there's
enough free RAM; otherwise the page faults will kill the performance. On
a large slapadd that took about 3 minutes runtime, slapindex took over 6
minutes. With the memory map tweaks it took only 30 seconds.
--
-- Howard Chu
Chief Architect, Symas Corp. Director, Highland Sun
http://www.symas.com http://highlandsun.com/hyc
Symas: Premier OpenSource Development and Support