[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: How to calculate maxsize for an MDB [was: 2.4.32: mdb stable enough?]
Peter Marschall wrote:
> Hi,
>
> On Saturday, 18. August 2012, Howard Chu wrote:
>> Peter Marschall wrote:
>>> But this brought up a question:
>>> Given an existing HDB database, is there a formula or something to
>>> calculate the 'maxsize' config option of MDB from the existing
>>> information.
>>
>> In my testing, MDB typically uses about 60% as much space as HDB. Factor in
>> however much future growth you anticipate and go from there.
>>
>> As an example, I have a test LDIF that's 558694630 bytes, containing 380836
>> entries. Looking at info from [m]db_stat, we can compare the number of
>> pages used for each index:
>>
>> hdb mdb
>> branch leaf overflow branch leaf overflow
>> dn2id 328 8097 0 67 7625 0
>> id2e 344 249856 59368 263 29681 293169
>> oc 11 154 0 1 3 0
>> uid 2487 26392 0 65 10895 0
>>
>> (page sizes are normalized here; hdb id2entry uses 16K pages while all
>> other databases use 4K pages)
>>
>> This is with
>> index objectclass eq
>> index uid eq,sub
>>
>> The dn2id, oc, and uid database formats are logically identical between hdb
>> and mdb, so the difference in size is due to the difference in BDB and MDB.
>> The id2entry database in mdb uses a slightly different encoding than hdb,
>> so there are both library and backend format differences there.
>>
>> As you can see, the more indexing you use, the bigger the difference
>> between mdb and hdb.
>
> Thanks for the explanation, Howard.
>
> I may be a bit thick today, but I do not see, how I can determine a minimal
> value for MDB's *maxsize* parameter from the values given above.
> (I do not want to waste memory ;-)
Your comment makes no sense. There is no memory being wasted. Setting the size
of the memory map only reserves the address space, and sets an upper limit on
the size of the DB file on disk. If the DB never grows to the size you
configure, then who cares. There is no point to minimizing the configured
size, you're not saving anything.
Set it to a few hundred GB. If the DB only contains 1MB of data then that's
all it will use. You only need to be concerned if in fact your DB will grow
large enough to overflow your disks. Another possibility is that you're
running multiple databases at once, and you've given them a terabyte each;
you'll only be able to create 100 or so databases before you start impacting
on shared library address space, etc. (On a contemporary 64 bit CPU with only
48 bit virtual address space.)
I'm pretty sure, if you're making comments like this, that this scenario
doesn't apply to you.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/