[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: Issues arising from creating powerdns backend based on LMDB

On 23/08/13 14:02, Howard Chu wrote:
Mark Zealey wrote:

4) Creating database with non-sequential keys is very bad (on 4gb
databases, 2* slower than kyoto - about 1h30 and uses more memory).

This was actually a typo - kyoto only takes about 20 minutes to generate
it so 4* slower. However, using a commit every 1m inserts (and because

Try again with commit every 100K inserts.

Thanks, that's much quicker again - for commit every 30k or 100k inserts time is about 12m30s. This is probably more limited by out pulling the data now. Of course this is still using memdisk

I've found another weird - I have now converted the database to use duplicates. Typically when I do mdb_cursor_get(... MDB_NEXT ) it will set the key and value but I've found 1 place so far where I do it and on the duplicate's second entry the value is set but the key is empty. However in other cases with duplicates both key and value are returned. Also, using MDB_NEXT_DUP seems to return key/value in all cases. Could this be another feature caused by a page boundary ? (I'm assuming here that MDB_NEXT is meant to go through dups as well although the docs don't specifically say that)