Re: (ITS#9039) key corruption using mdb_put and differing data

[hyc@symas.com]

lmdb@pluckeye.net wrote:
> Full_Name: Jonny Wilkes
> Version: 2.4.2
> OS: Linux
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (67.160.82.31)
> 
> 
> The following code shows key corruption.

Yes, because you've misused the API.
> 
> I am probably using the API wrong, but I don't see how.  I apologize in advance
> for not having a clue.

Read the Note http://www.lmdb.tech/doc/group__mdb.html#ga8bf10cd91d3f3a83a34d04ce6b07992d

Further comments below.

> And now, on to the test.  It can be compiled and run like so:
> 
> gcc t1.c -llmdb && ./a.out
> 
> --------- t1.c starts here ---------
> /*
> 
> The output of this program using liblmdb0 v0.9.21-1 on Ubuntu is:
> 
> --- dump ---
> key12345
> key67890
> --- dump ---
> key12345
> y12345
> 
> Why?  What happened to key67890?
> 
>  */
> 
> #include <lmdb.h>
> #if USE_SOURCE
> #include "midl.c"
> #include "mdb.c"
> #endif
> #include <stdio.h>
> #include <string.h>
> #include <unistd.h>
> 
> #define P(K) printf("%s\n", (char *)K.mv_data)
> 
> #define X(x) do { int _x = x; if (_x) fprintf(stderr, "%d %s\n", _x,
> mdb_strerror(_x)); } while (0)
> 
> const char *path = "/tmp/lmdb-why.tmp";
> MDB_env *env;
> 
> /*
>  * setup creates /tmp/lmdb-why.tmp with 2 keys with the data "hey":
>  *
>  * 1. key12345 : hey
>  * 2. key67890 : hey
>  */
> int
> setup(void)
> {
>   unlink(path);
>   if (!env) {
>     X(mdb_env_create(&env));
>     X(mdb_env_open(env, path, MDB_NOSUBDIR, 0664));
>   }
> 
>   MDB_txn *txn; X(mdb_txn_begin(env, NULL, 0, &txn));
>   MDB_dbi dbi;  X(mdb_dbi_open(txn, NULL, 0, &dbi));
> 
>   const char *keys[] = {
>     "key12345",
>     "key67890"
>   };
>   for (unsigned i = 0; i < 2; ++i) {
>     MDB_val key, data;
>     key.mv_data = (void *)keys[i];
>     key.mv_size = strlen(keys[i])+1;
>     data.mv_data = "hey";
>     data.mv_size = 4;
>     X(mdb_put(txn, dbi, &key, &data, 0));
>   }
>   X(mdb_txn_commit(txn));
>   return 0;
> }
> 
> int
> dump(void)
> {
>   printf("--- dump ---\n");
> 
>   MDB_txn *txn;       X(mdb_txn_begin(env, NULL, 0, &txn));
>   MDB_dbi dbi;        X(mdb_dbi_open(txn, NULL, 0, &dbi));
>   MDB_cursor *cursor; X(mdb_cursor_open(txn, dbi, &cursor));
> 
>   MDB_val key;
>   int rc;
>   while ((rc = mdb_cursor_get(cursor, &key, NULL, MDB_NEXT)) == 0) {
>     P(key);
>   }
> 
>   mdb_cursor_close(cursor);
>   if (MDB_NOTFOUND != rc)
>     return rc;
>   mdb_txn_abort(txn);
>   return 0;
> }
> 
> /*
>  * change_data change the data for each key to "there" using mdb_put.
>  * After it completes, the keys are corrupt.
>  */
> int
> change_data(void)
> {
>   MDB_txn *txn;       X(mdb_txn_begin(env, NULL, 0, &txn));
>   MDB_dbi dbi;        X(mdb_dbi_open(txn, NULL, 0, &dbi));
>   MDB_cursor *cursor; X(mdb_cursor_open(txn, dbi, &cursor));
> 
>   MDB_val key;
>   int rc;
>   while ((rc = mdb_cursor_get(cursor, &key, NULL, MDB_NEXT)) == 0) {
>     MDB_val data;
>     data.mv_data = (void *)"there";
>     data.mv_size = 6;           /* ! */
>     X(mdb_put(txn, dbi, &key, &data, 0));

As documented, the key returned by mdb_cursor_get is only valid until the next update operation.
Since you've called mdb_put here, the key is altered while the put is executing, and so you get
your corruption. You have to copy the key if you want this particular code to work. But even so,
you're not using the API optimally. Since you have a cursor already, you should be using
mdb_cursor_put instead.

>   }
>   mdb_cursor_close(cursor);
>   if (MDB_NOTFOUND != rc)
>     return rc;
> 
>   X(mdb_txn_commit(txn));
>   return 0;
> }
> 
> int main(void)
> {
>   X(setup());
>   X(dump());
>   X(change_data());
>   X(dump());
>   return 0;
> }

Closing this ITS.

-- 
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/