[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: MDB, read/write mmap
Howard Chu wrote:
> Howard Chu wrote:
>> It seems to me this can only be a viable mode of operation if you're always
>> going to run asynch and don't care much about transaction durability or DB
>> recoverability. Running in this mode offers absolutely zero crash resistance;
>> the entire DB will almost always be irreparably damaged after a system crash.
>>
>> Would you run like that, if it offered you the potential of maybe 10x faster
>> write performance?ï (It could be useful for slapadd -q, certainly.)
>
> OK, 10x speedup was far too optimistic. Quickly cobbling the changes together,
> it looks more like about a 70% speedup.
>
> slapadd -q with 5 million entries took 24m16s as originally reported at
> LDAPCon last year.
>
> With current mdb.master it takes 22m24s:
>
> real 22m23.984s
> user 26m1.658s
> sys 8m17.415s
>
> With the writable mmap and no msyncs it took 13m17s.
>
> real 13m17.225s
> user 22m15.511s
> sys 1m12.533s
>
> This code is currently available on the map2 branch of my git repo on
> ada.openldap.org. I'll clean it up a bit further then push it to mdb.master
> after some more testing.
The speedup seems to be proportional to the number of indices that are defined
on the database. With Quanah's torture-test LDIF (~6 million entries, 4.9GB),
there's only a small difference between the two when no indices are defined:
Original mdb.master:
real 11m27.385s
user 15m5.489s
sys 6m47.825s
Writemap:
real 10m27.447s
user 14m35.663s
sys 6m10.767s
But with 31 indices defined...
Original:
real 94m35.862s
user 93m31.755s
sys 20m39.693s
Writemap:
real 42m53.499s
user 54m35.509s
sys 7m23.364s
Over a 2:1 speedup.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/