On Thursday, December 13, 2018 05:24:53 PM Derek Zhou wrote: > Howard and others: > > Again on deleting lots of entries. I have 2 experiments: > > 1, in a fresh db insert 10 million entries. let's call this state A. then delete 9 million entries over night. let's call this state B > 2, in a fresh db insert 1 million entries I call this state B' > > In B and B' even the 1 million entries are the same; so from user's perspective B and B' are indistinguishable. However, deleting entries from B is much slower than deleting entries from B', like 10x slower. It seems like deleting speed depends on the peak db size, and how full the db currently is. > > My question is: is this wide a deletion performance gap expected? > Further testing shows that this deletion speed drop only happens in writemap mode. default mode is much better. See the following plots: 1, adding 10 million entries, default vs writemap As you can see, writemap is marginally faster, with curious period performance drop. 2, deleting all 10 million entries previously added, default vs writemap As you can see, in writemap mode, after ~10% entries are deleted, the performance went down hard. The test was not finished, would probably take hours. Whereas in default mode, delete speed was better in the beginning, there is also a performance drop after ~10% entries are deleted but much less severe. The test finished in ~4800 seconds. -- Derek Zhou
Attachment:
add_tps.png
Description: PNG image
Attachment:
delete_tps.png
Description: PNG image
