(ITS#7816) Bug in LMDB's mdb_cmp_memn[r] functions

[dror.harari@gmail.com]

Full_Name: Dror Harari
Version: 
OS: Windows 7 64bit
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (79.181.198.22)


The default compare functions start with:

mdb_cmp_memn(const MDB_val *a, const MDB_val *b)
{
	int diff;
	ssize_t len_diff;
	unsigned int len;

	len = a->mv_size;
	len_diff = (ssize_t) a->mv_size - (ssize_t) b->mv_size;
	if (len_diff > 0) {

This code assumes that ssize_t is signed and in all normal platforms it is.
However, on Windows Visal Studio 2010 (did not check others) it is defined as:

#ifndef _SSIZE_T_DEFINED
#ifdef  _WIN64
typedef unsigned __int64    ssize_t;
#else
typedef _W64 unsigned int   ssize_t;
#endif
#define _SSIZE_T_DEFINED
#endif

So it is unsigned and len_diff would never become <0.

There's a workaround to defined a custom compare function but one can rewrite
the default compare around this bug (that is, avoiding subtraction).

This issue was detected with SET_KEY_RANGE on a key value of 00 00 00 02 where
the database had keys:
00 00 00 01 65
00 00 00 02 65
00 00 00 03 65

Instead of getting the 00 00 00 02 65, the code returns 00 00 00 03 65