[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: Issues arising from creating powerdns backend based on LMDB

On 23/08/13 00:07, Howard Chu wrote:
... as I said already, it does exactly what you said. When you've deleted the last item on the page the cursor no longer points at a valid node, so GET_CURRENT returns EINVAL.

OK I think that's a pretty big gotcha - it would be great to either make the cursor behaviour consistent (ie automatically skip to next record at end of a page) or to very clearly document this! It's pretty different from kyoto/bdb interfaces in that (to my mind at least) it requires a bit too much knowledge from the developer about the underlying structure of the database which should be abstracted by the interface.

I'm not doing *any* commits just one big txn for all the data...

The below C works fine up until i=4m (ie 500mb of residential memory
shown in top), then has massive slowdown, shared memory (again, as seen
in top) increases, waits about 20-30 seconds and then disks get hammered
writing 10mb/sec (200txns) when they are capable of 100-200mb/sec
streaming writes... Does it do the same for you?

Kyoto writes async by default. You should do the same here, use MDB_NOSYNC on the env_open.

MDB_NOSYNC makes no difference in my test case above - seeing exactly the same memory, speed and disk patterns. Are you able to reproduce it?