(ITS#8418) mdb_load performance

[aschorr@telemetry-investments.com]

Full_Name: Andrew Schorr
Version: lmdb 0.9.18
OS: Linux
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (38.76.0.51)


I patched mdb_load to add an option to set the mapsize with -T, as per
open issue #8417. I then used it to load a 7 GB text file, and I found that
the calls to mdb_txn_commit after every 100 mdb_cursor_put calls really slows
down performance. If one comments out the batching, it goes much faster.
And even faster is to avoid the cursor and just call mdb_put in a loop.
Is there a reason mdb_load commits after every 100 puts?