[Date Prev][Date Next]
- To: firstname.lastname@example.org
- Subject: Parallel traversal
- From: Gabriel Tanase <email@example.com>
- Date: Thu, 30 Mar 2017 10:05:54 -0500
- Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:reply-to:from:date:message-id:subject:to; bh=tH9dS1MN2CCA9smak+VTkiHkcwC0Xo0y6Epej60RUKA=; b=GU59HMwByY2E+p4DM6GD0vxdc6C3tdz8b4gCM80OF2oK/Nczylxp2WbdIuwIR1bOzG 5CZb8L16TxmUt4SG/poCvOFYISGMq+7CY7xd/1bzX1gvgec3mNYAxfKdixZtKMQSSwGz 0X62ltDoK5jb8fstTsBjroantA3lRxR/a052jM1ILYU7w06s6I53lG64Ra4ICbz91I1r Y8QdNGOlzFnPlBbsdkAVm8WDxcsBBGrhRxkoCV4rieat3JrJltq3vAURywqBpEMjbdiw UVsWboA8/HzxKtE+srA/MVa47+wxBdPZowgXDef09jmDe7s+afZzLvWwId2+fIuaIkPU mzUw==
Assuming I have an LMDB database: to traverse all data in a table I open a cursor and go over each element in a loop using MDB_NEXT or MDB_NEXT_DUP
: :mdb_cursor_get (cur, &_entry.first, &_entry.second, MDB_NEXT_DUP);
How can one traverse the data in an LMDB table using multiple threads for a faster overall traversal?
I am assuming a read only transaction and I would like each thread to work on distinct entries; Like if there are 100 [key, value] and I use 4 threads, thread zero will work on first 25 elements, thread one will work on the next 25 and so on. 25 doesn't have to be exactly 25 but ideally the partitioned should be balanced.
A simple solution I designed is to read all keys into a vector first and then subsequently parallelize with each thread working on a subset of key from this vector. Each thread will traverse a subset of keys, and for each key we find the corresponding [key, value] and work on the value.
Is there an alternative ?