Re: OpenLDAP sync code

["Kurt D. Zeilenga" <Kurt@OpenLDAP.org>]

At 03:53 PM 6/14/2001, Randy Kunkee wrote:
>"Kurt D. Zeilenga" wrote:
>
>> At 01:27 PM 6/14/2001, Randy Kunkee wrote:
>> >Mark Whitehouse has taken my old periodic sync patch and implemented it
>> >for 2.0.7, and we are currently working to get it working for 2.0
>> >Engineering (and eventually the head development branch).
>>
>> >In this work, he has also corrected the fact that dbcachesize does
>> >not work correctly with Berkeley DB.
>>
>> This should be fixed in HEAD and OPENLDAP_REL_ENG_2.
>>
>> >He is also working on code
>> >to make use of transactions in Berkeley/Sleepycat DB.
>
>> I hope this is based upon the back-bdb work...  the main thing
>> needed is indexing support.  I made a start of rewriting the
>> index code, but got bogged with other stuff.  Likely the best
>> thing to do now is just to port the LDBM indexing code over
>> to back-bdb.
>>
>
>Sorry, it's all work in back-ldbm.
>
>>
>> >I have several questions I want to ask about how to proceed.
>> >
>> >1. Is there any objection to adding the periodic sync feature to the
>> >code base?  The new version is enabled via the config file.
>>
>> Well, I would rather see back-bdb worked on... but if there are
>> simple changes we can make to prolong the usefulness of back-ldbm,
>> I won't object.
>>
>
>Part of our problem is that we need production systems with this feature
>right away, and back-bdb is a longer term approach.

That's why I won't object.

>>
>> I would also prefer the sync changes not require --with-threads.
>
>Understood.  Can take a look at that.  At this point, I'd be willing to make
>it at least not break a build with --without-threads, ie. disable the
>feature.  Would that be sufficient to get it in?

Yes.


>>
>>
>> >2. I understand that patches want to be kept separate.  However, in this
>> >case, since they are related, is there any objection to submitting the
>> >sync patch together with the fix for calling set_cachesize?
>>
>> Given that set_cachesize should be fixed, this should be moot.
>> However, if still broken, I prefer separate patches as I rather
>> no have to split them later for release engineering needs.
>> Any set_cachesize bug fix is likely to progress into OPENLDAP_REL_ENG_2
>> much faster than a new sync feature (and certainly faster than
>> transaction code).
>>
>> >3. The patch is somewhat complicated with getting all the #if
>> >DB_VERSION_MAJOR tests etc. working correctly and testing with earlier
>> >versions of Berkeley DB.  In my ideal world, I'd prefer to copy the
>> >back-ldbm work into a new directory, ie. back-bdb3, rip out all of the
>> >#ifdef's for < version 3, and march on from there.  I've heard of work
>> >before on a bdb3 version, perhaps intending to write it from scratch.
>> >Would we be duplicating work by doing this?
>>
>> If you are not enhancing back-bdb, then yes.

At the moment, no.  I'm swamped with other things right now.

>> >Is there any objection to
>> >using the "back-bdb3" subdirectory off of servers?  I've been away from
>> >the project for a while so forgive me if this has been discussed in the
>> >mailing lists.
>>
>> As we already have a back-bdb which is Berkeley DB 3 (latest)
>> specific, so, yes.  Let's figure out how you and Mark can pick up
>> the back-bdb work.
>
>Okay.  I haven't looked at the indexing code in a long time.  Hopefully it's
>simpler to write now.

Well, what bogged me was a rewrite of the LDBM indexing code to
replace the ALLIDS fallback with a range fallback.  I can send
you a copy of my work in progress, but at this point, I think
it might be best to avoid the ALLIDS to range redesign for now.