(ITS#8452) LMDB: mdb_env_copyfd2 can deadlock due to missing mdb_env_cthr_toggle check

[lmb@cloudflare.com]

Full_Name: Lorenz Bauer
Version: git c367c1f69685
OS: OS X
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (2a06:98c0:1000:1200:156f:df88:ee9a:7775)


The function mdb_env_copyfd2 with MDB_CP_COMPACT enabled can currently deadlock,
since a call to mdb_env_cthr_toggle is not checked.

The testcase at https://gist.github.com/lmb/17a528cdadde73fb231a2a1ed84714f7
reproduces this issue as follows:

- Use pthread_sigmask to ignore SIGPIPE
- Create new pipe
- Spawn thread with mdb_env_copyfd2 using MDB_CP_COMPACT
- Close read side of the pipe in main thread
- Join thread and check rc code

On LMDB master the testase e hangs instead of returning EPIPE.

The following patch fixes the deadlock, but since I'm not very familiar with the
codebase it's likely incorrect.

diff --git a/libraries/liblmdb/mdb.c b/libraries/liblmdb/mdb.c
index 79a958b..97ff7c7 100644
--- a/libraries/liblmdb/mdb.c
+++ b/libraries/liblmdb/mdb.c
@@ -9988,11 +9988,15 @@ mdb_env_copyfd1(MDB_env *env, HANDLE fd)
 	rc = mdb_env_cwalk(&my, &txn->mt_dbs[MAIN_DBI].md_root, 0);
 	if (rc == MDB_SUCCESS && my.mc_wlen[my.mc_toggle])
 		rc = mdb_env_cthr_toggle(&my, 1);
-	mdb_env_cthr_toggle(&my, -1);
+	rc = mdb_env_cthr_toggle(&my, -1);
+	if (rc)
+		goto done;
 	pthread_mutex_lock(&my.mc_mutex);
 	while(my.mc_new)
 		pthread_cond_wait(&my.mc_cond, &my.mc_mutex);
 	pthread_mutex_uocock(&my.mc_mutex);
+
+done:
 	THREAD_FINISH(thr);
 
 	mdb_txn_abort(txn);