Re: (ITS#8777) [LMDB] Closing read cursor uses already freed transaction (MDB_VL32)

[markus@greenrobot.de]

--001a114db4265d5f2405669693c3
Content-Type: text/plain; charset="UTF-8"

Because of another MDB_VL32 issue (
http://www.openldap.org/its/index.cgi?findid=8813), I would like to come
back the MDB_SIZE64 patch suggested here.

I understand that you have no intention to merge this in. For us, applying
the patch makes perfectly sense nevertheless: It fixes a severe problem and
the 2 GB limit on 32 bits is fine for our use case.

Do you see any side effects the proposed patch might have? I'm aware you
cannot give guarantees, but a simple "should work" from your side would be
appreciated!

Thanks!
Markus


I'd like to propose a different approach to 64 bits on 32 bit CPUs;
> something like this:
> [...]
> lmdb.h
> @@ -187,7 +187,15 @@
>  # define MDB_FMT_Z     "z"                     /**< printf/scanf format
> modifier for size_t */
>  #endif
>
> -#ifndef MDB_VL32
> +#if defined(MDB_VL32) && defined(MDB_SIZE64)
> +#error either define MDB_VL32 OR MDB_SIZE64
> +#endif
> +#ifdef MDB_SIZE64
> +typedef uint64_t       mdb_size_t;
> +# define MDB_SIZE_MAX  SIZE_MAX        /**< max #mdb_size_t */
> +# define MDB_PRIy(t)   PRI##t##64
> +# define MDB_SCNy(t)   SCN##t##64
> +#elif !defined(MDB_VL32))
>  /** Unsigned type used for mapsize, entry counts and page/transaction IDs.
>   *
>   *     It is normally size_t, hence the name. Defining MDB_VL32 makes it
>

--001a114db4265d5f2405669693c3
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote"><div=
>Because of another=C2=A0<span style=3D"color:rgb(34,34,34);font-family:ari=
al,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:norm=
al;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-alig=
n:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing=
:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-d=
ecoration-color:initial;float:none;display:inline">MDB_VL32 issue (<a href=
=3D"http://www.openldap.org/its/index.cgi?findid=3D8813";>http://www.openlda=
p.org/its/index.cgi?findid=3D8813</a>), I would like to come back the=C2=A0=
<span style=3D"color:rgb(34,34,34);font-family:arial,sans-serif;font-size:s=
mall;font-style:normal;font-variant-ligatures:normal;font-variant-caps:norm=
al;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;t=
ext-transform:none;white-space:normal;word-spacing:0px;background-color:rgb=
(255,255,255);text-decoration-style:initial;text-decoration-color:initial;f=
loat:none;display:inline">MDB_SIZE64 patch <span style=3D"color:rgb(34,34,3=
4);font-family:arial,sans-serif;font-size:small;font-style:normal;font-vari=
ant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacin=
g:normal;text-align:start;text-indent:0px;text-transform:none;white-space:n=
ormal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-st=
yle:initial;text-decoration-color:initial;float:none;display:inline">sugges=
ted<span>=C2=A0</span></span>here.</span></span></div><div><span style=3D"c=
olor:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:=
normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:4=
00;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:no=
ne;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);te=
xt-decoration-style:initial;text-decoration-color:initial;float:none;displa=
y:inline"><span style=3D"color:rgb(34,34,34);font-family:arial,sans-serif;f=
ont-size:small;font-style:normal;font-variant-ligatures:normal;font-variant=
-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-in=
dent:0px;text-transform:none;white-space:normal;word-spacing:0px;background=
-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color=
:initial;float:none;display:inline"><br></span></span></div><div><span styl=
e=3D"color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-=
style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-we=
ight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transf=
orm:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,2=
55);text-decoration-style:initial;text-decoration-color:initial;float:none;=
display:inline"><span style=3D"color:rgb(34,34,34);font-family:arial,sans-s=
erif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-v=
ariant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;t=
ext-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;back=
ground-color:rgb(255,255,255);text-decoration-style:initial;text-decoration=
-color:initial;float:none;display:inline">I understand that you have no int=
ention to merge this in. For us, applying the patch makes perfectly sense n=
evertheless: It fixes a severe problem and the 2 GB limit on 32 bits is fin=
e for our use case.=C2=A0</span></span></div><div><span style=3D"color:rgb(=
34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;fo=
nt-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter=
-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-=
space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decora=
tion-style:initial;text-decoration-color:initial;float:none;display:inline"=
><span style=3D"color:rgb(34,34,34);font-family:arial,sans-serif;font-size:=
small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:nor=
mal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;=
text-transform:none;white-space:normal;word-spacing:0px;background-color:rg=
b(255,255,255);text-decoration-style:initial;text-decoration-color:initial;=
float:none;display:inline"><br></span></span></div><div><span style=3D"colo=
r:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:nor=
mal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;=
letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;=
white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-=
decoration-style:initial;text-decoration-color:initial;float:none;display:i=
nline"><span style=3D"color:rgb(34,34,34);font-family:arial,sans-serif;font=
-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-ca=
ps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-inden=
t:0px;text-transform:none;white-space:normal;word-spacing:0px;background-co=
lor:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:in=
itial;float:none;display:inline">Do you see any side effects the proposed p=
atch might have? I&#39;m aware you cannot give guarantees, but a simple &qu=
ot;should work&quot; from your side would be appreciated!</span></span></di=
v><div><span style=3D"color:rgb(34,34,34);font-family:arial,sans-serif;font=
-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-ca=
ps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-inden=
t:0px;text-transform:none;white-space:normal;word-spacing:0px;background-co=
lor:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:in=
itial;float:none;display:inline"><span style=3D"color:rgb(34,34,34);font-fa=
mily:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatu=
res:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;t=
ext-align:start;text-indent:0px;text-transform:none;white-space:normal;word=
-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initia=
l;text-decoration-color:initial;float:none;display:inline"><br></span></spa=
n></div><div><span style=3D"color:rgb(34,34,34);font-family:arial,sans-seri=
f;font-size:small;font-style:normal;font-variant-ligatures:normal;font-vari=
ant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text=
-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;backgro=
und-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-co=
lor:initial;float:none;display:inline"><span style=3D"color:rgb(34,34,34);f=
ont-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-=
ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:no=
rmal;text-align:start;text-indent:0px;text-transform:none;white-space:norma=
l;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:=
initial;text-decoration-color:initial;float:none;display:inline">Thanks!</s=
pan></span></div><div><span style=3D"color:rgb(34,34,34);font-family:arial,=
sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;=
font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:s=
tart;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0p=
x;background-color:rgb(255,255,255);text-decoration-style:initial;text-deco=
ration-color:initial;float:none;display:inline"><span style=3D"color:rgb(34=
,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font=
-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-s=
pacing:normal;text-align:start;text-indent:0px;text-transform:none;white-sp=
ace:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decorati=
on-style:initial;text-decoration-color:initial;float:none;display:inline">M=
arkus</span></span></div><div><span style=3D"color:rgb(34,34,34);font-famil=
y:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures=
:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text=
-align:start;text-indent:0px;text-transform:none;white-space:normal;word-sp=
acing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;t=
ext-decoration-color:initial;float:none;display:inline"><span style=3D"colo=
r:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:nor=
mal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;=
letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;=
white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-=
decoration-style:initial;text-decoration-color:initial;float:none;display:i=
nline"><br></span></span></div><div><br></div><blockquote class=3D"gmail_qu=
ote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,20=
4);padding-left:1ex">I&#39;d like to propose a different approach to 64 bit=
s on 32 bit CPUs;<br>
something like this:<br>[...]<br>lmdb.h<br>
@@ -187,7 +187,15 @@<br>
=C2=A0# define MDB_FMT_Z=C2=A0 =C2=A0 =C2=A0&quot;z&quot;=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0/**&lt; printf/s=
canf format modifier for size_t */<br>
=C2=A0#endif<br>
<br>
-#ifndef MDB_VL32<br>
+#if defined(MDB_VL32) &amp;&amp; defined(MDB_SIZE64)<br>
+#error either define MDB_VL32 OR MDB_SIZE64<br>
+#endif<br>
+#ifdef MDB_SIZE64<br>
+typedef uint64_t=C2=A0 =C2=A0 =C2=A0 =C2=A0mdb_size_t;<br>
+# define MDB_SIZE_MAX=C2=A0 SIZE_MAX=C2=A0 =C2=A0 =C2=A0 =C2=A0 /**&lt; ma=
x #mdb_size_t */<br>
+# define MDB_PRIy(t)=C2=A0 =C2=A0PRI##t##64<br>
+# define MDB_SCNy(t)=C2=A0 =C2=A0SCN##t##64<br>
+#elif !defined(MDB_VL32))<br>
=C2=A0/** Unsigned type used for mapsize, entry counts and page/transaction=
 IDs.<br>
=C2=A0 *<br>
=C2=A0 *=C2=A0 =C2=A0 =C2=A0It is normally size_t, hence the name. Defining=
 MDB_VL32 makes it<br>
</blockquote></div><br></div></div>

--001a114db4265d5f2405669693c3--