[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: (ITS#3851) Berkeley DB Scalability Patch
Quanah,
Do you also have a result without "-q" option ?
- Jong-Hyuk
Quanah Gibson-Mount wrote:
>
>
> --On Wednesday, August 10, 2005 11:55 AM -0400 Jong-Hyuk
> <jongchoi@watson.ibm.com> wrote:
>
>> Here is a result with various degrees of indexing for 2 million
>> entries :
>>
>> ftp://ftp.openldap.org/incoming/ScalabilityWithIndexing.png
>>
>> As shown in the graph, for 13 indexing (6 equality, 2 equality &
>> substring, 5 equality, substring & approximate), it took almost a day
>> (only less one hour) without the Berkeley DB scalability patch while it
>> took only six hours with the patch. It would have been decreased further
>> if a separate indexing were performed by using slapindex after the
>> initial slapadd. Although you tested with 21 indexing, the number of
>> substring/approximate indexing are the same at seven in both cases. The
>> effect of additional equality indexing was found negligible through my
>> performance evaluation.
>>
>> As a result, I would say my results contradict yours. With my
>> performance
>> evaluation configuration, I could not find any tendency that might give
>> me a hint why you had experienced such a problem. One thing I noticed
>> from your last result is that you did not run the experiment to
>> completion. I think it is required to run each run to completion to draw
>> a meaningful result.
>
>
> Hi Jong,
>
> I've run the test to the finish for the unpatched version, and have
> the patched version running. The unpatched version will beat the
> patched version by several days, at the current rate things are going.
>
> For the unpatched load, the results were:
>
> cgi3-linux:/tmp/ldif# time slapadd -q -l 2M-peons.ldif
> [########################################] (100%) 06d08h 2000002 obj
> 277927.430u 2905.190s 152:22:40.95 51.1% 0+0k 0+0io 684572pf+0w
>
>
> Right now, the *patched* load is at:
>
> cgi3-linux:/tmp/ldif# time slapadd -q -l 2M-peons.ldif
> [########################## ] ( 66%) 06d13h 1331216 obj
>
> and doing a few percent every day.
>
> I think you are not doing a broad enough test to really understand the
> validity of your patch, and I suggest testing with more than 13
> indices, and letting it run to completion.
>
> My indices are:
>
> # Indices to maintain
> index default eq
> index objectClass
> index uid
> index departmentNumber
> index cn eq,sub,approx
> index sn eq,sub,approx
> index l
> index ou
> index telephonenumber eq,sub
> index userpassword
> index givenName eq,sub,approx
> index mail
> index carLicense
> index employeeType
> index homephone eq,sub
> index mobile eq,sub
> index pager eq,sub
> index o
> index roomNumber
> index preferredLanguage
> index postalCode
> index st
>
> and DB_CONFIG is:
>
> # $Id: DB_CONFIG,v 1.1 2004/04/02 19:27:11 quanah Exp $
> set_cachesize 1 713031680 0
> set_lg_regionmax 262144
> set_lg_bsize 2097152
> set_lg_dir /var/log/bdb
>
> Regards,
> Quanah
>
> --
> Quanah Gibson-Mount
> Product Engineer
> Symas Corporation
> Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
> <http://www.symas.com>
>
>
>
>