Re: BDB fragmented

[Howard Chu <hyc@symas.com>]

Eric Irrgang wrote:
> It looks to me like DB 4.4 has a db->compact method that doesn't seem to
> have been available in earlier versions.  I would think that this could
> come in handy for collapsing btrees into fewer levels.  (back-bdb uses
> btrees, right?)  This ought to reclaim storage and performance, even when
> there haven't been any entries removed, shouldn't it?
>
> Has anyone played with this yet or would it be worth experimenting with?
>   

It will probably reclaim storage; it will not necessarily improve 
performance. It could just as easily have no effect, or a negative effect.

First of all, if no entries have been removed, then there is nothing to 
reclaim.

When entries are removed, if the number of data items on a data page 
falls below the minimum items per page, BDB will automatically do a 
reverse-split and join items to another page. If a page is emptied as a 
result, it will get recorded on the DB's free list. If there are few 
enough nodes to allow the total depth of the tree to be collapsed, then 
BDB must already do so; that's a requirement of the self-balancing 
nature of a Btree.

The performance aspect comes in here - if you're going to be adding new 
elements to the DB anyway, then it will be faster to reuse a page from 
the free list than to ask the OS/filesystem for a brand new page.

None of this is really relevant for the OpenLDAP-software mailing list; 
feel free to discuss the benefits of db->compact on the BerkeleyDB 
newsgroup if you want to followup.
> On Wed, 8 Feb 2006, Howard Chu wrote:
>
>   
> > Ansar Mohammed wrote:
> >     
> > > Is there anyway to de-fragment the BDB backend?
> > > I recently cleared out 40k objects and the database is the same size.
> > >
> > >       
> > Sounds like a question you'll find answers for in the BDB documentation.
> > Generally there's no point in freeing up the space since it will just
> > get reused the next time you add data.
> >     
>
>   


--
 -- Howard Chu
 Chief Architect, Symas Corp.  http://www.symas.com
 Director, Highland Sun        http://highlandsun.com/hyc
 OpenLDAP Core Team            http://www.openldap.org/project/