Issue 4261 - [enhancement] Factor replication code out of slapd
Summary: [enhancement] Factor replication code out of slapd
Status: VERIFIED SUSPENDED
Alias: None
Product: OpenLDAP
Classification: Unclassified
Component: slapd (show other issues)
Version: unspecified
Hardware: All All
: --- normal
Target Milestone: ---
Assignee: OpenLDAP project
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2005-12-15 19:42 UTC by ando@openldap.org
Modified: 2021-08-03 17:59 UTC (History)
0 users

See Also:


Attachments
dif.txt (9.65 KB, text/plain)
2006-02-05 09:29 UTC, Howard Chu
Details
mandif.txt (960 bytes, text/plain)
2006-02-05 09:29 UTC, Howard Chu
Details

Note You need to log in before you can comment on or make changes to this issue.
Description ando@openldap.org 2005-12-15 19:42:33 UTC
Full_Name: Pierangelo Masarati
Version: HEAD
OS: irrelevant
URL: http://www.sys-net.it/~ando/Download/slurpover-2005-12-15.1.patch.bz2
Submission from: (NULL) (81.72.89.40)
Submitted by: ando


This patch, for HEAD as of December 15th, 2005, factors replication code out of
slapd.  It's actually composed of two parts, which are tightly coupled and thus
it's submitted as just one.

It provides two overlays:

"slurprov" takes care of logging modifications to the replog, and of those
config statements that are actually used by slurpd: replica-{args,pid}file,
replica, replicationinterval.  It should be instantiated inside any master
database.

"consumer" is a generic consumer interface.  It handles generic consumer issues,
like rejecting non-replication modifications (unless set up as so-called
"multimaster") and, for syncrepl, it accounts for the configuration.

Both overlays support back-config.

Current issues:
- some existing overlays that are replog-aware are not handled yet.  As such,
slapo-refint(5) and slapo-ppolicy(5) cannot be compiled along with
slapo-slurprov(5);
- the overlays need to be explicitly instantiated; the rest of the configuration
is fully compatible with the current.  It would be nice to have them
automatically instantiated when the appropriate replication statements occur in
slapd.conf(5), I don't think it's going to be a big deal;
- slapo-consumer(5) must be global; thus, a single slapo-consumer(5) takes care
of all consumer database in the DSA.  As a consequence, the related
configuration is held in subordinate entries of the global overlay rather than
inside (or subordinate to) the related database.  This makes configuration via
back-config little intuitive
- slapo-consumer(5) doesn't yet support run-time configuration via back-config;
only EMIT is available.

The patch is provided for initial discussion, development and testing.  I plan
to commit it when it's a bit more mature, i.e. when the above issues are closed.
 Note though that (at least on my laptop ;) it passes all tests, and
significantly all replication-related ones.

p.

PS: no space left on device on ftp.openldap.org; I've placed it on my website.

Comment 1 ando@openldap.org 2005-12-15 20:12:20 UTC
changed notes
moved from Incoming to Software Enhancements
Comment 2 Howard Chu 2005-12-16 00:25:01 UTC
ando@sys-net.it wrote:
> This patch, for HEAD as of December 15th, 2005, factors replication code out of
> slapd.  It's actually composed of two parts, which are tightly coupled and thus
> it's submitted as just one.
>
> It provides two overlays:
>
> "slurprov" takes care of logging modifications to the replog, and of those
> config statements that are actually used by slurpd: replica-{args,pid}file,
> replica, replicationinterval.  It should be instantiated inside any master
> database.
>
> "consumer" is a generic consumer interface.  It handles generic consumer issues,
> like rejecting non-replication modifications (unless set up as so-called
> "multimaster") and, for syncrepl, it accounts for the configuration.
>
> Both overlays support back-config.
>
> Current issues:
>   

There's a lot of misc cleanup in the diff that appears unrelated to the 
current feature, perhaps you should commit those bits in advance. Also, 
don't include the diff for configure since that script will just get 
regenerated from configure.in.

> - some existing overlays that are replog-aware are not handled yet.  As such,
> slapo-refint(5) and slapo-ppolicy(5) cannot be compiled along with
> slapo-slurprov(5);
>   

I've looked at refint and ppolicy; the reason they invoke replog_cb 
directly is due to the order of processing between that callback and the 
overlay callbacks. With slurprov moved into an overlay, the references 
to slap_replog_cb can be removed from these overlays. Then it will just 
be a matter of configuring the overlays in the proper order to insure 
that refint and ppolicy's own updates are propagated as desired. (In 
this case, slurprov must be below refint/ppolicy in the overlay stack.) 
That takes care of one issue; the other is that these overlays may need 
to know whether the current database is a shadow or not, and alter their 
behavior accordingly.

> - the overlays need to be explicitly instantiated; the rest of the configuration
> is fully compatible with the current.  It would be nice to have them
> automatically instantiated when the appropriate replication statements occur in
> slapd.conf(5), I don't think it's going to be a big deal;
> - slapo-consumer(5) must be global; thus, a single slapo-consumer(5) takes care
> of all consumer database in the DSA.  As a consequence, the related
> configuration is held in subordinate entries of the global overlay rather than
> inside (or subordinate to) the related database.  This makes configuration via
> back-config little intuitive
> - slapo-consumer(5) doesn't yet support run-time configuration via back-config;
> only EMIT is available.
>
> The patch is provided for initial discussion, development and testing.  I plan
> to commit it when it's a bit more mature, i.e. when the above issues are closed.
>  Note though that (at least on my laptop ;) it passes all tests, and
> significantly all replication-related ones.
-- 
  -- Howard Chu
  Chief Architect, Symas Corp.  http://www.symas.com
  Director, Highland Sun        http://highlandsun.com/hyc
  OpenLDAP Core Team            http://www.openldap.org/project/

Comment 3 Howard Chu 2006-02-05 09:29:06 UTC
While waiting for this patch to get cleaned up, I wrote this up to allow 
configuring the multimaster mode.

-- 
  -- Howard Chu
  Chief Architect, Symas Corp.  http://www.symas.com
  Director, Highland Sun        http://highlandsun.com/hyc
  OpenLDAP Core Team            http://www.openldap.org/project/

Comment 4 Howard Chu 2007-02-16 05:58:53 UTC
changed state Open to Suspended
Comment 5 OpenLDAP project 2014-08-01 21:04:52 UTC
replication