Re: LibLmdb non string data

[Hallvard Breien Furuseth <h.b.furuseth@usit.uio.no>]

On 02. des. 2016 12:26, Tim Uckun wrote:
> I am writing a wrapper for liblmdb and one thing I was able to do was to
> store simple integers and floats as the value into the database.  This seems
> to work pretty well but I am wondering if this is somehow against best practice.
>
> One bad thing about this is that when you are doing a seek or a cursor
> operation the code does not know what kind of data is at the record. I have
> some routines to cast them back from the pointers but the user has to know
> what to cast them to. It would be very handy if the MDB_val struct had a
> field for metadata, even if it just a byte.

LMDB handles binary data fine.  The problem is ordering:

Use a single datatype for each DB, and mdb_set_compare()
mdb_set_dupsort() to order data in that DB properly.  Or
MDB_INTEGERKEY/MDB_INTEGERDUP which do this for unsigned int/size_t.

If that is not feasible and you are mixing different datatypes in a
single database, you need to tag each data item so the compare function
can figure out how to parse and order them all.  Note that a slow
compare function can degrade LMDB's performance.

Note that the compare functions may receive unaligned values.  Though
you can hack alignment 2, 4, or sizeof(size_t) = 4 or 8 by ensuring
*every* key and value in a DB has size = multiple of that alignment.
Keys always have alignment of at least 2.  Hmm, I should add some
code so data will too except when both key and data have uneven size.

Requests to add a user-defined argument to the compare functions have
been rejected.  If you need that, maybe your wrapper can emulate it by
setting thread-specific data before calling LMDB, and then your sort
function can read that data.  That gets cumbersome, though.

If this is all generic, maybe define a DB with meta-info where your
wrapper can look up which datatypes (sizes, compare functions, whatever)
are in which database.

--
Hallvard