[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: Issues arising from creating powerdns backend based on LMDB

Mark Zealey wrote:

On 23/08/13 14:02, Howard Chu wrote:

Mark Zealey wrote:



4) Creating database with non-sequential keys is very bad (on 4gb
databases, 2* slower than kyoto - about 1h30 and uses more memory).


This was actually a typo - kyoto only takes about 20 minutes to generate
it so 4* slower. However, using a commit every 1m inserts (and because


Try again with commit every 100K inserts.


Thanks, that's much quicker again - for commit every 30k or 100k inserts
time is about 12m30s. This is probably more limited by out pulling the
data now. Of course this is still using memdisk

I've found another weird - I have now converted the database to use
duplicates. Typically when I do mdb_cursor_get(... MDB_NEXT ) it will
set the key and value but I've found 1 place so far where I do it and on
the duplicate's second entry the value is set but the key is empty.
I don't see how this can happen; the only time we don't return the key is if some operation actually failed. Can you send test code to reproduce this? 


However in other cases with duplicates both key and value are returned.
Also, using MDB_NEXT_DUP seems to return key/value in all cases. Could
this be another feature caused by a page boundary ? (I'm assuming here
that MDB_NEXT is meant to go through dups as well although the docs
don't specifically say that)


Next means next. The docs shouldn't have to say anything more specific than that.

--
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/