[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: (ITS#3851) Berkeley DB Scalability Patch
--On Wednesday, August 10, 2005 11:55 AM -0400 Jong-Hyuk
<jongchoi@watson.ibm.com> wrote:
> Here is a result with various degrees of indexing for 2 million entries :
>
> ftp://ftp.openldap.org/incoming/ScalabilityWithIndexing.png
>
> As shown in the graph, for 13 indexing (6 equality, 2 equality &
> substring, 5 equality, substring & approximate), it took almost a day
> (only less one hour) without the Berkeley DB scalability patch while it
> took only six hours with the patch. It would have been decreased further
> if a separate indexing were performed by using slapindex after the
> initial slapadd. Although you tested with 21 indexing, the number of
> substring/approximate indexing are the same at seven in both cases. The
> effect of additional equality indexing was found negligible through my
> performance evaluation.
>
> As a result, I would say my results contradict yours. With my performance
> evaluation configuration, I could not find any tendency that might give
> me a hint why you had experienced such a problem. One thing I noticed
> from your last result is that you did not run the experiment to
> completion. I think it is required to run each run to completion to draw
> a meaningful result.
Hi Jong,
I've run the test to the finish for the unpatched version, and have the
patched version running. The unpatched version will beat the patched
version by several days, at the current rate things are going.
For the unpatched load, the results were:
cgi3-linux:/tmp/ldif# time slapadd -q -l 2M-peons.ldif
[########################################] (100%) 06d08h 2000002 obj
277927.430u 2905.190s 152:22:40.95 51.1% 0+0k 0+0io 684572pf+0w
Right now, the *patched* load is at:
cgi3-linux:/tmp/ldif# time slapadd -q -l 2M-peons.ldif
[########################## ] ( 66%) 06d13h 1331216 obj
and doing a few percent every day.
I think you are not doing a broad enough test to really understand the
validity of your patch, and I suggest testing with more than 13 indices,
and letting it run to completion.
My indices are:
# Indices to maintain
index default eq
index objectClass
index uid
index departmentNumber
index cn eq,sub,approx
index sn eq,sub,approx
index l
index ou
index telephonenumber eq,sub
index userpassword
index givenName eq,sub,approx
index mail
index carLicense
index employeeType
index homephone eq,sub
index mobile eq,sub
index pager eq,sub
index o
index roomNumber
index preferredLanguage
index postalCode
index st
and DB_CONFIG is:
# $Id: DB_CONFIG,v 1.1 2004/04/02 19:27:11 quanah Exp $
set_cachesize 1 713031680 0
set_lg_regionmax 262144
set_lg_bsize 2097152
set_lg_dir /var/log/bdb
Regards,
Quanah
--
Quanah Gibson-Mount
Product Engineer
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
<http://www.symas.com>