Re: fast delete a lot of entries

[Derek Zhou <derek@shannon-data.com>]

Howard and others:

Again on deleting lots of entries. I have 2 experiments:

1, in a fresh db insert 10 million entries. let's call this state A. then delete 9 million entries over night. let's call this state B
2, in a fresh db insert 1 million entries I call this state B'

In B and B' even the 1 million entries are the same; so from user's perspective B and B' are indistinguishable. However, deleting entries from B is much slower than deleting entries from B', like 10x slower. It seems like deleting speed depends on the peak db size, and how full the db currently is. 

My question is: is this wide a deletion performance gap expected?         

By the way, querying and adding speed are also different among A, B and B', but the gap is much smaller. also deletion speed gap between A and B' is not large. restarting daemon at state B does not improve deletion speed (so the difference between B and B' are persistent). 

In the meantime I will do another experiment: add back 9 million entries to state B and call it A'. then compare performance between A and A'. 
   
-- 
Derek Zhou
Shannon Systems
http://www.shannon-sys.com