[Date Prev][Date Next]
Re: server startup overhead
Searching for objectclass=* and only asking for the entryDN attribute is
almost an order of magnitude faster than searching for everything. I like
the idea of searching for a non-existent value of an unindexed attribute.
It is 40% faster than the objectclass=* search (down to 16 minutes) but
does not fix the searches made outside of the directory's root search
base. At least not quite. The test search I tried took four minutes
(about an order of magnitude better than no priming search) and the
following test searches were under a second. Any thoughts on what extra
magic might be coming out of the objectclass=* into the caches?
My DB cache is just barely sufficient but it seems to be large enough.
Examining the db_stat output, iostat, and vmstat, all indications are that
by the time I've got right about the time my DB cache approaches full my
DB cache hits rapidly approach 100% and I'm neither paging in or out. I
can't really go any bigger.
dbconfig set_cachesize 10 0 1
dbconfig set_shm_key 7
A freshly restarted server quickly ends up with 11 or 12 Gigs resident
with a VM size of 13+ gigs and within a week or so is up to 16 Gig VM size
and 13 or 14 Gigs resident. On a 16Gig machine, an inch beyond that and
the OS starts to run out of room and I risk swap madness.
I suppose I ought to track down where the CPU cycles are going, whether it
is ACL processing or just the overhead of getting all of the attributes
from DB and building LDIF.
On Fri, 25 May 2007, Howard Chu wrote:
Eric Irrgang wrote:
I continue to have trouble with getting a freshly started server to be
responsive. One problem in particular is one that I thought had been
resolved some time ago but is apparently biting me right now...
With the hdb backend (at least in OL 2.3.34 and OL 2.3.35) if you perform a
search with a search base deeper than the root suffix, the search takes a
very long time to complete if the cache hasn't been established. In my case
the difference is less than a second versus several hours. I'm not sure yet
which bit of cache needs to be primed. I can switch back and forth
searching with the same filter in the root and then a child search base with
the same results.
If it takes several hours, then most likely your BDB cache is too small.
As for which cache, it's either the DN cache (aka EntryInfo in the code) or
the IDL cache. (Currently the DN cache size is not configurable, will probably
add a keyword for that in 2.4.)
Is this a bug recursion or something that I just hadn't been noticing?
What would be the best search to perform to prepare whatever cache is
getting hit to make searches outside of the root DN faster?
Priming the caches will only help if you actually have sufficient RAM
available. If the DB is too large, then there's not much you can do about it.
If you have sufficient RAM, then doing a subtree search from the root, on an
unindexed attribute, looking for a value that doesn't exist, will hit every
entry in the DB and fully prime the DN cache (and the DN-related info in the
IDL cache). It will cycle the full contents of the dn2id and id2entry DBs
through the BDB cache as well.
Eric Irrgang - UT Austin ITS Unix Systems - (512)475-9342