[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: syncrepl simplification



OK, these changes are complete and working in HEAD:
syncrepl consumer only uses be_rootdn
only one consumer per database allowed (as it turns out, config.c already returned an error if you tried to specify more, so apparently no one was ever using multiple contexts anyway.)
consumer context is stored in suffix contextCSN attribute, same as the provider - no more syncrepl subentries.
slapadd options -p, -r, and -i removed.
-w option causes the suffix contextCSN to be updated with the greater of:
old value, if any
greatest entryCSN seen in the current Add session
slapcat options -k and -m removed.
provider uses a single sessionlog for the entire context, it is used automatically for all sync operations. No more sid in sync cookie.


The slapadd -w option is really just a convenience now. Since the syncprov overlay will scan the database on startup, it will work fine if you load a provider without it. (But using -w in this case will of course allow syncprov to startup quicker.) Also since the -w option compares the old max with the current max, you can run incremental slapadd's and still get the right result. (It's important not to just override the old contextCSN with the greatest entryCSN in the current database; if the last operation on a database was a delete then the contextCSN will be greater than all the entryCSNs in the database and just using the greatest entryCSN would therefore cause a rollback.)

So now any provider database can be used as a consumer and vice versa, without any tweaking in between.

There was a bug where deletes from a sessionlog didn't propagate in a cascaded setup, but that all works now. We may need to extend the syncrepl test scripts to test with and without a sessionlog, although it looks like test019 does some of this already.

Howard Chu wrote:

Kurt D. Zeilenga wrote:

At 09:19 PM 1/12/2005, Howard Chu wrote:


Kurt D. Zeilenga wrote:



The rationale, I believe, for multiple consumer contexts
within a database context was that administrators could state
different consumer policies for different areas of the DIT.


I'll have to think about that some more.


I've been, too. In particular, I think the above rationale more
applies to provider contexts, where one might want to have different
server handling of different requests. For instance, one might
want to have one area covered by a session log and one area not.


That actually makes sense, although the current code doesn't allow for this. It would require the sessionlog config directive to take a search specifier (baseDN of a subtree at least) to differentiate logs. Again this is problematic when you're dealing with ModDN requests; if an entry moves across a sessionlog boundary then one or both logs becomes useless.

At the moment all updates in a context are recorded in all the active sessionlogs, so the only difference between the multiple logs is their size. This also seems unnecessary, it would be best to have a single log of the largest practical size. I have a feeling this sessionlog stuff was not thoroughly thought out; the fact that there is no config keyword for the sid on the consumer implies to me that it was never even tested before.

Given all of this, I think it would be best to have only one sessionlog per provider, and we can forget about the sid parameter entirely.

On the consumer side, we just just require additional
database contexts where the admin desires different consumer
policies.



To me, it still sounds like a recipe for disaster. If the separate consumer contexts are talking to different providers, then you have the cascade problem I mentioned before. If the contexts are all pulling from the same provider, then I guess it works. I still don't see any rationale for the updatedn on the consumer; surely any data that the provider returns must satisfy the consumer criteria and therefore must be reflected into the consumer DB.


I concur with regards to the updatedn. Also, we need to revisit
consumer ACLs. Seems odd to me that syncrepl consumer needs
permission to write into its own database.


Yes. OK, I'm going to delete the updatedn stuff then.



--
 -- Howard Chu
 Chief Architect, Symas Corp.       Director, Highland Sun
 http://www.symas.com               http://highlandsun.com/hyc
 Symas: Premier OpenSource Development and Support