Re: (ITS#9007) In writemap mode, a loose page can end up in me_dpages, causing a segfault on mdb_env_close.

[hyc@symas.com]

kriszyp@gmail.com wrote:
> Full_Name: Kristopher William Zyp
> Version: LMDB 0.9.23
> OS: Windows
> URL: fhttps://github.com/kriszyp/node-lmdb/commit/dc290553acb57fa3f2d6d88a0d5e02006d3879d0.patch
> Submission from: (NULL) (71.199.6.148)
> 
> 
> In writemap mode in LMDB, it seems that a loose page can end up in me_dpages,
> causing a segfault on mdb_env_close or on writes after mdb_env_set_mapsize.

Thanks for the report, fix applied to mdb.master
> 
> If I understand correctly, in MDB_WRITEMAP mode, dirty pages should never enter
> into me_dpages, since dirty pages directly refer to the mapped memory, and
> me_dpages are allocated from the heap (for reuse). However, I think a dirty page
> that has become a loose page can end up in me_dpages; it seems that when
> mdb_freelist_save() is called, it iterates through the loose pages and can end
> up calling mdb_dpage_free at
> https://github.com/LMDB/lmdb/blob/mdb.RE/0.9/libraries/liblmdb/mdb.c#L3114,
> allowing the loose page to end up in me_dpages. This means that me_dpages ends
> up with a reference to data in the memory map. And this becomes apparent if you
> call mdb_env_close or do a mdb_env_set_mapsize triggering a segfault when it
> attempts to free a memory map'ed page or access previously unmapped page.
> 
> Everywhere else in the code, the MDB_WRITEMAP flag prevents any calls to
> mdb_dpage_free (or mdb_dlist_free) in writemap mode (which I assume is
> intentional), except in mdb_freelist_save. It seems like the mdb_dpage_free call
> just needs to be moved up a few lines to the else block of the MDB_TXN_WRITEMAP
> conditional, so that it also won't be called in writemap mode.
> 
> I apologize I can't provide a more isolated test case. This combination actually
> seems to be pretty rare occurrence, and very difficult to reproduce, I only
> occasionally observe it happening with large amounts of operations with frequent
> closing/resizing of the env. And I don't understand the internals to be 100%
> confident of this, and perhaps I am misunderstanding this code path. But, it
> does seems like this fix prevents these crashes for us, and seems like ensuring
> there are no mdb_dpage_free calls in writemap mode (only mdb_page_free calls) is
> the legitimate intention of the code.
> 
> The attached URL contains the fix as a patch (against the node-lmdb project).
> 
> 


-- 
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/