(ITS#8431) LMDB: Access newly opened database from another transaction

[juerg.bircher@helmedica.com]

Full_Name: Jeerg Bircher
Version: lmdb (master)
OS: MacOS / Linux
URL: ftp://ftp.openldap.org/incoming/Juerg_Bircher_160527-Access-newly-opened-database-from-another-transactio.patch
Submission from: (NULL) (178.82.37.195)


A transaction tracks newly opened databases and if the transaction is committed
the newly opened databases are propagated to the list of open databases of the
environment. However if the read only transaction is aborted the databases are
not propagated.
If database handles (MDB_dbi) are cached in the application to avoid calling
mdb_dbi_open() there might be the situation of two threads running a read only
transaction concurrently.

First threads opens the database and commits the read-only transaction. The
database is added to the list of open databases in the environment. The returned
dbi is globally cached in the application.

The second thread also wants to access the same database and finds the database
handle in the global application cache. The database handle however is not valid
as the transaction only uses a snapshot of open databases. So the second thread
gets an EINVAL when using that database handle.
This should not happen as the database is open and added to the environment.

The following should fix this issue by updating the mt_numdbs and marking the
delta with DB_UNUSED

static int mdb_update_db_info(MDB_txn *txn, MDB_dbi dbi) {
    /* propagate newly opened db from env to current txn. Mark them as unused
*/
    if (txn->mt_numdbs < txn->mt_env->me_numdbs) {
        memset(txn->mt_dbflags + txn->mt_numdbs, DB_UNUSED,
txn->mt_env->me_numdbs - txn->mt_numdbs);
    }
    txn->mt_numdbs = txn->mt_env->me_numdbs;

    return dbi < txn->mt_numdbs;
}

    /** Check \b txn and \b dbi arguments to a function and initialize db info
if needed */
#define TXN_DBI_EXIST(txn, dbi, validity) \
    ((txn) && ((dbi)<(txn)->mt_numdbs || mdb_update_db_info((txn), (dbi))) &&
(((txn)->mt_dbflags[dbi] & (validity)) || (((txn)->mt_dbflags[dbi] & DB_UNUSED)
&& mdb_setup_db_info((txn), (dbi), (validity)))))


IMPORTANT depends ITS#8430
See also mailing list openldap-technical thread: Improved handling for large
number of databases / Access newly opened database from another transaction