Re: (ITS#8916) MDB_VL32 in readonly causes MDB_CORRUPTED

[pierre.chambart@ocamlpro.com]

On 19/09/2018 17:35, Howard Chu wrote:
> pierre.chambart@ocamlpro.com wrote:
>> Full_Name: Pierre Chambart
>> Version:=20
>> OS:=20
>> URL:=20
>> Submission from: (NULL) (185.194.234.133)
>>
>>
>> We have been trying LMDB compiled with the MDB_VL32 flag and encounter=
ed some
>> MDB_CORRUPTED error that didn't occur without MDB_VL32.
>> We could produce a minimal reproduction case:
>> https://raw.githubusercontent.com/chambart/lmdb/vl32_readonly_fix/libr=
aries/liblmdb/vl32_failure.c
>> This is basically writing a key, reading it, writing something else th=
en reading
>> it again.
>>
>> After some investigation, this is how I understand what is happening:
>>
>> * The chunk being mapped for the first read contains 3 pages instead o=
f the 16
>> pages that would be mapped without readonly.
>> * The second write extend the size of the base to 5 pages
>> * Finally the last read retrieve the mapped chunk from the first read,=
 which has
>> size 3, but assumes that it has size 5, hence reading some random valu=
e in
>> memory past the end of the pages.
>>
>> It can be verified by asserting that id3.mcnt is larger than rem at th=
e end of
>> mdb_rpage_get, around this line:
>>
>> ok:
>>     p =3D (MDB_page *)((char *)id3.mptr + rem * env->me_psize);
>>
>> This does not seem to be the case.
>>
>> I made a dirty fix that seems to work. It's simply to remove those lin=
es:
>>
>>         /* don't map past last written page in read-only envs */
>>         if ((env->me_flags & MDB_RDONLY) && pgno + MDB_RPAGE_CHUNK-1 >=

>> txn->mt_last_pgno)
>>             id3.mcnt =3D txn->mt_last_pgno + 1 - pgno;
>>         else
>>
>> This seems to be some optimization that avoids mapping the 16 pages of=
 the
>> chunk. But given that those pages are
>> initialized, there doesn't seem to be much harm in mapping a few more =
pages in
>> that relatively rare situation.
> Please check git history before assuming the code is an optimization.
>
> These lines prevent a crash on Windows.
>
Sorry, about that.

Also I'm not suggesting that this is a proper fix, rather that this is so=
me
clue to understand the problem. I'm happy producing a patch if you
indicate what kind of fix you would like.
--=20
Pierre