[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: [LMDB] What pointer is returned with combination of MDB_WRITEMAP and MDB_RESERVE?

Victor Baybekov wrote:

Thank you! I understand this copy-on-write behavior, but am interested if I
could control it a little. What if I use records that are always much bigger
than a single page, e.g. 100 kb with 4kb pages, and make sure that a record is
never updated (via LMDB means) during a lifetime of an environment, - is there
any scenario that the location of such a big record could be changed during a
lifetime of an environment, without updating the record?
At this point in time, no, if you don't update a large record there is no reason that it will move. That is not to say that this won't change in the future. The documentation tells you what promises we are willing to make. Relying on any non-documented behavior is your own responsibility. 



On Fri, Oct 2, 2015 at 4:38 PM, Howard Chu <hyc@symas.com
<mailto:hyc@symas.com>> wrote:

    Victor Baybekov wrote:

        Hi,

        Docs for MDB_RESERVE say that a returned pointer to the reserved space is
        valid "before the next update operation or the transaction ends." Docs
        for MDB_WRITEMAP say that it "writes directly to the mmap instead of using
        malloc for pages." Does combining the two options return a pointer
        directly to
        a place in a mmap


    Yes.

        so that this pointer could be used after a transaction ends
        or after the next update?


    No.

    Longer answer: maybe.

    Full answer: LMDB is copy-on-write. If you update another record on the
    same page, in a later transaction, the contents of that page will be
    copied to a new page and the original page will go onto the freelist. In
    that case, the pointer you got must not be used again.

    If you don't directly update that page and cause it to be copied, then you
    might get lucky and be able to use the pointer for a while. It all depends
    on what other modifications you do and how they affect that node or
    neighboring nodes.


        I have a use case where I want to somewhat abuse LMDB safety for
        convenience.
        If I could get a pointer to a place inside a mmap I could work with
        LMDB value
        as opaque blob or as a region inside the single big mmap. This could
        be more
        convenient than creating and opening hundreds of temporary memory
        mapped files
        and keeping open handles to them. For example, Aeron terms could be stored
        like this: a stream id per an LMDB db and a term id for a key in the db.


        Thanks!
        Victor



    --
       -- Howard Chu
       CTO, Symas Corp. http://www.symas.com
       Director, Highland Sun http://highlandsun.com/hyc/
       Chief Architect, OpenLDAP http://www.openldap.org/project/





--
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/