RE: adding entry, indexing takes too long (ITS#3081)

[jmcclure@lucent.com]

> The BDB cache is only about 128MB and the database is over 5GB. He just needs
> more BDB cache.

I am going to start increasing the cache size beginning at 512M until no noticable improvment in behavior can be seen.  Unfortunately this is not task is not an exact science as decribed by BDB documentation.  From viewing the statistics of this paritcular database, the high hit rates did not suggest to me that cache was the bottle neck in this instance (see below).

125MB 3KB 184B  Total cache size.
1       Number of caches.
125MB 8KB       Pool individual cache size.
0       Requested pages mapped into the process' address space.
7415741 Requested pages found in the cache (99%).
47734   Requested pages not found in the cache.
5672    Pages created in the cache.
47734   Pages read into the cache.
77259   Pages written from the cache to the backing file.
27697   Clean pages forced from the cache.
3       Dirty pages forced from the cache.
0       Dirty pages written by trickle-sync thread.
25706   Current total page count.
17648   Current clean page count.
8058    Current dirty page count.
16381   Number of hash buckets used for page location.
7516881 Total number of times hash chains searched for a page.
7       The longest hash chain searched for a page.
17M     Total number of hash buckets examined for page location.
19M     The number of hash bucket locks granted without waiting.
132120  The number of hash bucket locks granted after waiting.
60      The maximum number of times any hash bucket lock was waited for.
166249  The number of region locks granted without waiting.
60      The number of region locks granted after waiting.
53446   The number of page allocations.
60757   The number of hash buckets examined during allocations
4694    The max number of hash buckets examined for an allocation
27700   The number of pages examined during allocations
2028    The max number of pages examined for an allocation
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Pool File: id2entry.bdb
16384   Page size.
0       Requested pages mapped into the process' address space.
169008  Requested pages found in the cache (100%).
6       Requested pages not found in the cache.
2685    Pages created in the cache.
6       Pages read into the cache.
2500    Pages written from the cache to the backing file.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Pool File: dn2id.bdb
4096    Page size.
0       Requested pages mapped into the process' address space.
1790556 Requested pages found in the cache (100%).
21      Requested pages not found in the cache.
2841    Pages created in the cache.
21      Pages read into the cache.
2818    Pages written from the cache to the backing file.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Pool File: objectClass.bdb
4096    Page size.
0       Requested pages mapped into the process' address space.
2027813 Requested pages found in the cache (100%).
8       Requested pages not found in the cache.
0       Pages created in the cache.
8       Pages read into the cache.
195     Pages written from the cache to the backing file.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Pool File: msgEventDescriptor.bdb
4096    Page size.
0       Requested pages mapped into the process' address space.
671728  Requested pages found in the cache (100%).
22      Requested pages not found in the cache.
0       Pages created in the cache.
22      Pages read into the cache.
251     Pages written from the cache to the backing file.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Pool File: msgEventTimestamp.bdb
4096    Page size.
0       Requested pages mapped into the process' address space.
850863  Requested pages found in the cache (98%).
18122   Requested pages not found in the cache.
0       Pages created in the cache.
18122   Pages read into the cache.
23983   Pages written from the cache to the backing file.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Pool File: msgMessageUID.bdb
4096    Page size.
0       Requested pages mapped into the process' address space.
186912  Requested pages found in the cache (96%).
8505    Requested pages not found in the cache.
15      Pages created in the cache.
8505    Pages read into the cache.
11991   Pages written from the cache to the backing file.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Pool File: msgRecipientAddress.bdb
4096    Page size.
0       Requested pages mapped into the process' address space.
190337  Requested pages found in the cache (94%).
11869   Requested pages not found in the cache.
0       Pages created in the cache.
11869   Pages read into the cache.
16392   Pages written from the cache to the backing file.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Pool File: msgSenderAddress.bdb
4096    Page size.
0       Requested pages mapped into the process' address space.
292072  Requested pages found in the cache (97%).
8150    Requested pages not found in the cache.
85      Pages created in the cache.
8150    Pages read into the cache.
12443   Pages written from the cache to the backing file.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Pool File: msgStatusCode.bdb
4096    Page size.
0       Requested pages mapped into the process' address space.
668509  Requested pages found in the cache (100%).
73      Requested pages not found in the cache.
0       Pages created in the cache.
73      Pages read into the cache.
458     Pages written from the cache to the backing file.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Pool File: msgVMUID.bdb
4096    Page size.
0       Requested pages mapped into the process' address space.
567943  Requested pages found in the cache (100%).
958     Requested pages not found in the cache.
46      Pages created in the cache.
958     Pages read into the cache.
6228    Pages written from the cache to the backing file.