[Date Prev][Date Next]
Re: write-scaling problems in LMDB
Luke Kenneth Casson Leighton wrote:
On Mon, Oct 20, 2014 at 3:25 PM, Howard Chu <firstname.lastname@example.org> wrote:
The timings on the left are for the writer process; the reader process
is on the right. You were right, the number of writers is largely
irrelevant because they're all waiting most of the time. There's a
huge jump in System CPU time from 1 writer to 2 writers, because with
only 1 writer the mutex is uncontended and essentially zero-cost. As
more writer threads are added, the System CPU time will rise.
by some 15 to 20% or so, from 2 writers going up to 32 writers.
which is not unreasonable - nothing like what i saw (CPU usage
dropping to 40%, loadavg jumping to over 8x the number of cores).
ok... what's the number of writes being carried out in each
transaction? the original scenario in which i saw the issue was when
there was one (!!). the design at the time was not optimised for
batches of reads followed by batches of writes.
This is 1 write per txn.
a single write per transaction would mean quite a lot more mutex usage.
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/