Re: (ITS#6116) BDB 4.8 support

[hyc@symas.com]

quanah@OpenLDAP.org wrote:
> Full_Name: Quanah Gibson-Mount
> Version: 2.4.16
> OS: Linux 2.6
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (209.131.62.113)
>
>
> Oracle has kindly provided Howard and I access to the early BDB 4.8 code.
> Filing this ITS so that we can track modifications to support new features in
> BDB 4.8, such as the bulk add/delete capabilities.
>
After a quick read-thru of the Changes in 4.8, I'm a bit skeptical that we'll 
be able to use the bulk add/delete feature. It simply allows multiple 
key/value pairs to be written in one request and in general, since we only 
operate on individual entries, we seldom operate on more than one k/v pair at 
a time. The main exception is when generating a substring index. But in that 
case, we generate pairs in arbitrary (hashed) order, and BDB expects them in 
sorted order.

We could try modifying slapadd to accumulate multiple entries in memory and 
then add them all at once, but that would only help the adds to the id2entry 
DB. For the dn2id and all the other index DBs, we again cannot guarantee that 
inserts will occur in any particular order. As such it seems like we'll be 
creating quite a bit of complexity, batching up id2entry writes but not 
batching up the other writes.

On a separate front, we should probably look into adding support for 
partitioned databases. I think that has the potential to be pretty useful, 
although we'll have to continue to train admins to stop RAIDing all of their 
disks in order to see any benefit from this.

It looks like most of the other changes in 4.8 are internal/transparent, so no 
action is required of us to take advantage.

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/