[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: LMDB ITS 8324 WriteMap performance on Windows



Victor Baybekov wrote:
Hi,

Thanks a lot for ITS#8324! For embedded, not server, use case that change adds
much convenience.

I have tested the master from .NET via P/Invoke and do not see any major
slowdown with default options. To insert 10M <int32,int32> pairs inside a
single transaction v.0.9.14 takes minimum 3400 msec, latest master takes
minimum 3750 msec. This is not scientific, just best result from 10 runs.
Sometimes both timings increase to 5000+ msecs. On average slowdown is visible
but tolerable - from 2.9 Mops to 2.6 Mops (absolute numbers are still
awesome!). With Append and NoSync I could get 3.45 Mops on the same test with
master build.

However, with WriteMap performance of |master| drops 3x to 10000 msec or just
1 Mops, while for the |v.0.9.14| performance with WriteMap improvesto 2350
msec or 4.25 Mops.

Is this the cost of convenience or it could be fixed so that WriteMap
still "is faster and uses fewer mallocs" as the docs say?

That's pretty much the cost of this patch, it has the biggest impact on WriteMap usage. In default mode, regular Writes are done to grow the file so the code path is basically unchanged from before. In WriteMap mode the file has to be grown explicitly, right before accessing a new page and apparently the VirtualAlloc call that does this is expensive. Since it's the equivalent of both a malloc and a write together, it's actually more expensive than the default mode.

Please followup to the ITS so this conversation stays with the ticket.

--
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/