[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: (ITS#8813) MDB_VL32 causes MDB_TXN_FULL
--001a114390bed95e010567661549
Content-Type: text/plain; charset="UTF-8"
Some more details:
Transaction read only pages are running out in mdb_rpage_get(). MDB_txn.
mt_rpages[0] is at 4096 and thus MDB_TXN_FULL is returned. This happens
after around 100K mdb_cursor_del() calls.
How can we get millions of deletes to work in a single transaction?
It's an essential feature to us and I am happy to contribute under some
guidance.
So, here's my hope: given that the cleanup involves only consecutive keys,
we could consider a bulk delete function (delete an entire range from
position 1 to position 2). This should be much more efficient. Basically,
for most of the data, there's no need to operate on the node level.
Deleting entire leaf pages should be relatively simple (rebalance on the
parent branch page)? Not sure if rebalancing could also cope with cutting
off entire branches (superefficient if that would work).
Thanks,
Markus
--001a114390bed95e010567661549
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; charset=
=3Dus-ascii"><meta name=3D"Generator" content=3D"Microsoft Word 14 (filtere=
d medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.E-MailFormatvorlage17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 2.0cm 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style></head><body lang=3D"DE" link=3D"blue" vlink=3D"purple"><div cla=
ss=3D"WordSection1"><p class=3D"MsoNormal"><span lang=3D"EN-US">Some more d=
etails:</span></p><p class=3D"MsoNormal"><span lang=3D"EN-US">Transaction r=
ead only pages are running out in mdb_rpage_get(). MDB_txn. mt_rpages[0] is=
at 4096 and thus MDB_TXN_FULL is returned. This happens after around 100K =
mdb_cursor_del() calls.</span></p><p class=3D"MsoNormal"><span lang=3D"EN-U=
S"> </span></p><p class=3D"MsoNormal"><span lang=3D"EN-US">How can we get m=
illions of deletes to work in a single transaction?</span></p><p class=3D"M=
soNormal"><span lang=3D"EN-US">=C2=A0</span></p><p class=3D"MsoNormal"><spa=
n lang=3D"EN-US">It's an essential feature to us and I am happy to cont=
ribute under some guidance.</span></p><p class=3D"MsoNormal"><span lang=3D"=
EN-US">=C2=A0</span></p><p class=3D"MsoNormal"><span lang=3D"EN-US">So, her=
e's my hope: given that the cleanup involves only consecutive keys, we =
could consider a bulk delete function (delete an entire range from position=
1 to position 2). This should be much more efficient. Basically, for most =
of the data, there's no need to operate on the node level. Deleting ent=
ire leaf pages should be relatively simple (rebalance on the parent branch =
page)? Not sure if rebalancing could also cope with cutting off entire bran=
ches (superefficient if that would work).</span></p><p class=3D"MsoNormal">=
<span lang=3D"EN-US">=C2=A0</span></p><p class=3D"MsoNormal"><span lang=3D"=
EN-US">Thanks,</span></p><p class=3D"MsoNormal"><span lang=3D"EN-US">Markus=
</span></p></div></body></html>
--001a114390bed95e010567661549--