[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: mdb expected growth

Paul B. Henson wrote:

So after updating our dev ldap environment to use mdb, and slapadd'ing a
fresh copy from our production environment into it, the database was using
828M:

Environment Info
   Map address: (nil)
   Map size: 2147483648
   Page size: 4096
   Max pages: 524288
   Number of pages used: 211612
   Last transaction ID: 1333
   Max readers: 126
   Number of readers used: 6
Status of Main DB
   Tree depth: 1
   Branch pages: 0
   Leaf pages: 1
   Overflow pages: 0
   Entries: 29

I then proceeded to run a test load on it (basically, I had a script I put
together when I added the memberof overlay that ripped through all of our
groups, removing and then re-adding all members). After this test run, the
use jumped to 1.9G:

Environment Info
   Map address: (nil)
   Map size: 2147483648
   Page size: 4096
   Max pages: 524288
   Number of pages used: 479536
   Last transaction ID: 1075380
   Max readers: 126
   Number of readers used: 11
Status of Main DB
   Tree depth: 1
   Branch pages: 0
   Leaf pages: 1
   Overflow pages: 0
   Entries: 29

There is no new data in the database, all that happened was existing data
was removed and then re-added. Is this drastic increase in space utilization
expected in such a scenario?
From the sound of your quite vague test description, sure. As it states in the LMDB doc, long-lived reader transactions prevent reuse of freed pages. 
http://symas.com/mdb/doc/
You have a long search operation running, and are issuing writes while the search is in progress. 

--
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/