syncrepl

[Howard Chu <hyc@symas.com>]

ITS#4618, 4623, 4626 and 4703 all basically have to do with trying to 
use multiple replication contexts with a single provider. This is a 
behavior that the 2.3 syncprov implementation just wasn't designed for; 
it was meant to handle only a single context.

Looking at the ideas in 2.2's syncrepl, it might have gone in the 
direction of solving these problems if it weren't weighed down by so 
many insurmountable design and implementation flaws. 2.2 probably tried 
to do too much too soon, and got waylaid by the devil in the details.

At this point, this solution for multiple contexts presents itself:
1) We assign distinct searchbases to each context.
2) Every distinct source of changes must have its own unique rid. E.g., 
if a database is a provider for a context, it must have an rid. Every 
consumer within its namingContext must have their own rid's just as 
before. (The new requirement here is assigning rid's to providers that 
are masters of their data.)
3) Currently the provider hands a consumer a cookie consisting of the 
rid that the consumer supplied, plus a single contextCSN from the 
provider. This single contextCSN is inadequate for accurately capturing 
all of the changes that may come from multiple sources in a 
namingContext. Instead, the provider will send out a cookie consisting 
of multiple rid,CSN pairs - one for every rid of the provider's that 
resides in the consumer's search space. This is the only reliable way to 
make sure that all changes are tracked and propagated.

This says that in general, rids should not need to be configured on 
consumers - they should be dictated solely by the providers. It may be a 
good idea to allow them to be configured on consumers as an override, 
but for now that seems unimportant.

So:
1) the provider must have its own unique rid configured
2) the consumer's rid is optional
3) the provider must be told about all of the consumers living under it
4) the provider must aggregate all of the consumer cookies under it with 
its own context info when generating a cookie for its own consumers

Currently slapd treats an entire database as read-only when it has a 
consumer configured on it. This raises the question of how to allow 
multiple consumers in a single context - should we allow multiple 
consumers per DB, as 2.2 tried (and failed) to do, or should we continue 
with the current approach of one consumer per DB, and use glue to 
collect multiple consumers under one roof?

--
  -- Howard Chu
  Chief Architect, Symas Corp.  http://www.symas.com
  Director, Highland Sun        http://highlandsun.com/hyc
  OpenLDAP Core Team            http://www.openldap.org/project/