[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: Syncrepl: full sync vs. delta

Quanah Gibson-Mount wrote:

--On Wednesday, October 12, 2005 9:35 AM -0700 Howard Chu <hyc@symas.com> wrote:

Howard Chu wrote:
In fact, running test008 with an accesslog overlay that uses a single
mutex on the write ops actually completes faster than normal, because
back-bdb never hits any deadlocks. (Quite a bit faster, 2x at least.)
So I guess this approach is acceptable.
The assignment of CSNs to operations needs to occur later though. ModRdn
is fine, because it doesn't get a CSN until the backend calls
slap_modrdn2mods. But Add, Delete, and Modify all assign the CSN in the
frontend, before the accesslog's mutex is locked, so CSNs are still
allocated in an uncontrolled order. I'm thinking that all of the
assignments should be invoked from the backend. But that means that other
modules that call into backends would need to do the graduate() call that
the frontend does right now. I guess the simplest would be for the
accesslog overlay to just replace the CSN the frontend assigned with a
new one. Or add a backend flag "SLAP_DBFLAG_DEFER_CSN" that tells the
frontend not to assign it, because someone else will. ...?

If someone is going to be using delta-syncrepl with a fallback of a full refresh, don't the CSN's need to be coordinated between the accesslog backend and their main (bdb/hdb) backend?

Yes, that would be best. Currently they are.

I'm guessing that is what you mean will happen in the SLAP_DBFLAG_DEFER_CSN bit.

Yes. But that's still too messy; currently only the frontend de-queues CSNs but there are several places that en-queue them (passwd.c, sasl.c, various overlays) which means right now those places cause a CSN memory leak. The cleanest solution now appears to be invoking the CSN management directly from the backend (like ModRdn does) and removing the responsibility from the frontend. This seems to be the only way to make sure slap_graduate_commit_csn() will get called when it's needed.

 -- Howard Chu
 Chief Architect, Symas Corp.  http://www.symas.com
 Director, Highland Sun        http://highlandsun.com/hyc
 OpenLDAP Core Team            http://www.openldap.org/project/