[Date Prev][Date Next] [Chronological] [Thread] [Top]
RE: A couple of slapd/slurpd replication suggestions.

To: Openldap-devel@OpenLDAP.org
Subject: RE: A couple of slapd/slurpd replication suggestions.
From: Robert Rothlisberger <rwroth@netdox.com>
Date: Fri, 15 Jan 1999 10:12:19 -0600
I have also added multiple master capability.  But did it by using
a second port for all replication updates.  This makes sure that
multiple masters/slaves are duplicates of each other regardless of where
the original update was done.

	Bob Rothlisberger
	rwroth@netdox.com

> -----Original Message-----
> From:	Stephen Rush [SMTP:Stephen.Rush@liffe.com]
> Sent:	Friday, January 15, 1999 9:11 AM
> To:	openldap-devel@OpenLDAP.org
> Subject:	A couple of slapd/slurpd replication suggestions.
> 
> Hi, 
> 
> After a couple of weeks configuring/playing around with slapd and
> slurpd I
> have a couple of suggestions: 
> 
> Firstly: Whilst writing some scripts to generate slapd configuration
> files
> for a number of machines I was quite surprised to find that slurpd
> does not
> read include files. Hence, if you have a configuration file of the
> form:
> 
> 	# Main Configuration File 
> 	database ldbm
> 	#    :
> 	#    :
> 	include    /usr/local/LDAP/slapd_replication_config
> 	#    :
> 
> together with a /usr/local/LDAP/slapd_replication_config file 
> 
> 		# Definition of replication details
> 		replica	host=xxx .......
> 
> Then slapd will read the replication configuration file but slurpd
> will not,
> hence slurpd will complain that there are no replicas specified in the
> config file and will abort. The changes to allow slurpd to read
> include
> files appear pretty simple and are based on the slapd configuration
> routines
> (see below).
> Is there any reason for not allowing this to be done? 
> 
> Secondly: Replication in LDAP has an obvious master slave relationship
> between a master who stores the data and a number of slaves which can
> be
> used for read access to it. slurpd allows the master to replicate data
> to
> the slaves, but there appears to be no way for slaves to replicate
> data back
> to the master if a client performs an update on one of the slaves
> rather
> than on the master. (If they use slurpd then the updates end up in an
> infinite cycle).
> However, for clients to use this model in write mode they must always
> be
> aware of which server is the master and which ones are not (otherwise
> their
> updates will not be replicated to the other servers). In our
> application at
> least, this seems to be a rather unnecessary restriction. 
> Whilst I'm sure the slaves could generate a referral and it could all
> be
> made to work, this seems rather a complicated way to do things. 
> Instead, slurpd/slapd currently have a configuration parameter which
> indicates the dn to use when performing replication. Would it not be
> reasonable to have a further configuration item which indicates a dn
> for a
> user who can perform updates to the database, but who's updates are
> not
> themselves replicated. 
> Again, the changes are not large (though they touch a few more files,
> see
> below) and perhaps other people may also find them useful. Are there
> any
> pressing reasons why this kind of functionality should not be allowed?
> 
> If this were allowed then the following configuration files would
> allow two
> slapd servers to replicate changes to each other. 
> 
> --------------- SLAPD_CONFIG_HOST_1
> -------------------------------------
> # SLAPD CONFIG FILE FOR HOST1 - REPLICTES TO HOST2
> # The backup database will replicate to us as this user, do NOT
> replicate 
> # these changes on to anyone else or we get in a loop. 
> updatenorepdn	"cn=replicator2,o=organisation,c=UK"
> 	
> # The root user can access anything, as can the replication users. 
> access to *	by "dn=cn=admin,o=organisation,c=UK" write 
> 				by
> "dn=cn=replicator2,o=organisation,c=UK"
> write 
> 	
> # Nobody else needs to be able to do anything at all.
> defaultaccess 	none
> 
> replica		host=host2
> 
> "binddn=cn=replicator1,o=organisation,c=UK"
> 				bindmethod=simple
> 				credentials=password
> # END
> 
> --------------- SLAPD_CONFIG_HOST_2
> -------------------------------------
> # SLAPD CONFIG FILE FOR HOST2 - REPLICTES TO HOST1
> # The backup database will replicate to us as this user, do NOT
> replicate 
> # these changes on to anyone else or we get in a loop. 
> updatenorepdn	"cn=replicator1,o=organisation,c=UK"
> 
> # The root user can access anything, as can the replication users. 
> access to *	by "dn=cn=admin,o=organisation,c=UK" write 
> 				by
> "dn=cn=replicator1,o=organisation,c=UK"
> write 
> 	
> 	# Nobody else needs to be able to do anything at all.
> defaultaccess 	none
> 
> replica		host=host1
> 
> "binddn=cn=replicator2,o=organisation,c=UK"
> 				bindmethod=simple
> 				credentials=password
> # END
> 
> 
> Any comments on either of these suggestions ? 
> 
> Steve R
> 
> 
> 1)	Changes to allow slurpd to read nested include files:
> 
> 	*** ldap/servers/slurpd/config.c	Fri Jan 15 14:27:30 1999
> 	--- ldap/servers/slurpd/config.c.save	Fri Jan 15 14:27:20 1999
> 	***************
> 	*** 56,61 ****
> 	--- 56,62 ----
> 	  {
> 	      FILE	*fp;
> 	      char	*line;
> 	+     char	*savefname;
> 	      int		cargc;
> 	      char	*cargv[MAXARGS];
> 	  
> 	***************
> 	*** 107,112 ****
> 	--- 108,123 ----
> 	  	    }
> 	  	} else if ( strcasecmp( cargv[0], "replica" ) == 0 ) {
> 	  	    add_replica( cargv, cargc );
> 	+ 	} else if ( strcasecmp( cargv[0], "include" ) == 0 ) {
> 	+ 		if ( cargc < 2 ) {
> 	+ 			Debug( LDAP_DEBUG_ANY,
> 	+ 			       "%s: missing filename in
> \"include
> <filename>\" line\n",
> 	+ 			       fname, 0, 0 );
> 	+ 			exit( 1 );
> 	+ 		}
> 	+ 		savefname = strdup( cargv[1] );
> 	+ 		slurpd_read_config( savefname);
> 	+ 		free( savefname );
> 	  	}
> 	      }
> 	      fclose( fp );
> 
> 2)	Changes to allow replication from slaves back to the master
> server.
> 
> *** ldap/servers/slapd/add.c	Fri Jan 15 15:00:14 1999
> --- ldap/servers/slapd/add.c.save	Fri Jan 15 14:57:37 1999
> ***************
> *** 119,125 ****
>   	if ( be->be_add != NULL ) {
>   		/* do the update here */
>   		if ( be->be_updatedn == NULL ||
> ! 			strcasecmp( be->be_updatedn, op->o_dn ) == 0 ) {
>   
>   			if ( (be->be_lastmod == ON || (be->be_lastmod ==
> UNDEFINED &&
>   				global_lastmod == ON)) &&
> be->be_updatedn ==
> NULL ) {
> --- 119,127 ----
>   	if ( be->be_add != NULL ) {
>   		/* do the update here */
>   		if ( be->be_updatedn == NULL ||
> ! 		     strcasecmp( be->be_updatedn, op->o_dn ) == 0 ||
> ! 		     (be->be_updatenorepdn && 
> ! 		      strcasecmp( be->be_updatenorepdn, op->o_dn ) == 0
> )) {
>   
>   			if ( (be->be_lastmod == ON || (be->be_lastmod ==
> UNDEFINED &&
>   				global_lastmod == ON)) &&
> be->be_updatedn ==
> NULL ) {
> ***************
> *** 127,133 ****
>   				add_created_attrs( op, e );
>   			}
>   			if ( (*be->be_add)( be, conn, op, e ) == 0 ) {
> ! 				replog( be, LDAP_REQ_ADD, e->e_dn, e, 0
> );
>   			}
>   
>   		} else {
> --- 129,138 ----
>   				add_created_attrs( op, e );
>   			}
>   			if ( (*be->be_add)( be, conn, op, e ) == 0 ) {
> ! 			  	if (!be->be_updatenorepdn ||
> ! 				    strcasecmp( be->be_updatenorepdn,
> op->o_dn ) != 0) {
> ! 				  	replog( be, LDAP_REQ_ADD,
> e->e_dn,
> e, 0 );
> ! 				}
>   			}
>   
>   		} else {
> *** ldap/servers/slapd/config.c	Fri Jan 15 15:00:14 1999
> --- ldap/servers/slapd/config.c.save	Fri Jan 15 14:57:37 1999
> ***************
> *** 355,360 ****
> --- 355,375 ----
>   				be->be_updatedn = ch_strdup( cargv[1] );
>   				(void) dn_normalize( be->be_updatedn );
>   			}
> + 		} else if ( strcasecmp( cargv[0], "updatenorepdn" ) == 0
> ) {
> + 			if ( cargc < 2 ) {
> + 				Debug( LDAP_DEBUG_ANY,
> + 		    "%s: line %d: missing dn in \"updatenorepdn <dn>\"
> line\n",
> + 				    fname, lineno, 0 );
> + 				exit( 1 );
> + 			}
> + 			if ( be == NULL ) {
> + 				Debug( LDAP_DEBUG_ANY,
> + "%s: line %d: updatenorepdn line must appear inside a database
> definition
> (ignored)\n",
> + 				    fname, lineno, 0 );
> + 			} else {
> + 				be->be_updatenorepdn = ch_strdup(
> cargv[1]
> );
> + 				(void) dn_normalize(
> be->be_updatenorepdn );
> + 			}
>   
>   		/* replication log file to which changes are appended */
>   		} else if ( strcasecmp( cargv[0], "replogfile" ) == 0 )
> {
> *** ldap/servers/slapd/delete.c	Fri Jan 15 15:00:14 1999
> --- ldap/servers/slapd/delete.c.save	Fri Jan 15 14:57:37 1999
> ***************
> *** 72,81 ****
>   	 */
>   	if ( be->be_delete != NULL ) {
>   		/* do the update here */
> ! 		if ( be->be_updatedn == NULL || strcasecmp(
> be->be_updatedn,
> ! 		    op->o_dn ) == 0 ) {
>   			if ( (*be->be_delete)( be, conn, op, dn ) == 0 )
> {
> ! 				replog( be, LDAP_REQ_DELETE, odn, NULL,
> 0 );
>   			}
>   		} else {
>   			send_ldap_result( conn, op,
> LDAP_PARTIAL_RESULTS,
> NULL,
> --- 72,85 ----
>   	 */
>   	if ( be->be_delete != NULL ) {
>   		/* do the update here */
> ! 		if ( be->be_updatedn == NULL || 
> ! 		     strcasecmp( be->be_updatedn, op->o_dn ) == 0 ||
> ! 		     (be->be_updatenorepdn && strcasecmp(
> be->be_updatenorepdn, op->o_dn ) == 0)) {
>   			if ( (*be->be_delete)( be, conn, op, dn ) == 0 )
> {
> ! 			  	if (!be->be_updatenorepdn || 
> ! 				    strcasecmp( be->be_updatenorepdn,
> op->o_dn ) != 0) {
> ! 				  	replog( be, LDAP_REQ_DELETE,
> odn,
> NULL, 0 );
> ! 				}
>   			}
>   		} else {
>   			send_ldap_result( conn, op,
> LDAP_PARTIAL_RESULTS,
> NULL,
> *** ldap/servers/slapd/modify.c	Fri Jan 15 15:00:14 1999
> --- ldap/servers/slapd/modify.c.save	Fri Jan 15 14:57:37 1999
> ***************
> *** 156,162 ****
>   	if ( be->be_modify != NULL ) {
>   		/* do the update here */
>   		if ( be->be_updatedn == NULL ||
> ! 			strcasecmp( be->be_updatedn, op->o_dn ) == 0 ) {
>   
>   			if ( (be->be_lastmod == ON || ( be->be_lastmod
> ==
> UNDEFINED &&
>   				global_lastmod == ON ) ) &&
> be->be_updatedn
> == NULL ) {
> --- 156,164 ----
>   	if ( be->be_modify != NULL ) {
>   		/* do the update here */
>   		if ( be->be_updatedn == NULL ||
> ! 		     strcasecmp( be->be_updatedn, op->o_dn ) == 0 ||
> ! 		     (be->be_updatenorepdn && 
> ! 		      strcasecmp( be->be_updatenorepdn, op->o_dn ) ==
> 0)) {
>   
>   			if ( (be->be_lastmod == ON || ( be->be_lastmod
> ==
> UNDEFINED &&
>   				global_lastmod == ON ) ) &&
> be->be_updatedn
> == NULL ) {
> ***************
> *** 163,171 ****
>   				add_lastmods( op, &mods );
>   			}
>   			if ( (*be->be_modify)( be, conn, op, odn, mods )
> ==
> 0 ) {
> ! 				replog( be, LDAP_REQ_MODIFY, dn, mods, 0
> );
>   			}
> ! 
>   		/* send a referral */
>   		} else {
>   			send_ldap_result( conn, op,
> LDAP_PARTIAL_RESULTS,
> NULL,
> --- 165,176 ----
>   				add_lastmods( op, &mods );
>   			}
>   			if ( (*be->be_modify)( be, conn, op, odn, mods )
> ==
> 0 ) {
> ! 			  	if (!be->be_updatenorepdn ||
> ! 				    strcasecmp( be->be_updatenorepdn,
> op->o_dn ) != 0) {
> ! 				  	replog( be, LDAP_REQ_MODIFY, dn,
> mods, 0 );
> ! 				}
>   			}
> ! 			
>   		/* send a referral */
>   		} else {
>   			send_ldap_result( conn, op,
> LDAP_PARTIAL_RESULTS,
> NULL,
> *** ldap/servers/slapd/modrdn.c	Fri Jan 15 15:00:14 1999
> --- ldap/servers/slapd/modrdn.c.save	Fri Jan 15 14:57:38 1999
> ***************
> *** 79,90 ****
>   	 */
>   	if ( be->be_modrdn != NULL ) {
>   		/* do the update here */
> ! 		if ( be->be_updatedn == NULL || strcasecmp(
> be->be_updatedn,
> ! 		    op->o_dn ) == 0 ) {
> ! 			if ( (*be->be_modrdn)( be, conn, op, dn, newrdn,
> ! 			    deloldrdn ) == 0 ) {
> ! 				replog( be, LDAP_REQ_MODRDN, odn,
> newrdn,
> ! 				    deloldrdn );
>   			}
>   		} else {
>   			send_ldap_result( conn, op,
> LDAP_PARTIAL_RESULTS,
> NULL,
> --- 79,93 ----
>   	 */
>   	if ( be->be_modrdn != NULL ) {
>   		/* do the update here */
> ! 	  	if ( be->be_updatedn == NULL || 
> ! 		     strcasecmp( be->be_updatedn, op->o_dn ) == 0 ||
> ! 		     (be->be_updatenorepdn && 
> ! 		      strcasecmp( be->be_updatenorepdn, op->o_dn ) == 0
> )) {
> ! 		  	if ( (*be->be_modrdn)( be, conn, op, dn, newrdn,
> deloldrdn ) == 0 ) {
> ! 			  	if (!be->be_updatenorepdn ||
> ! 				    strcasecmp( be->be_updatenorepdn,
> op->o_dn ) != 0) {
> ! 				  	replog( be, LDAP_REQ_MODRDN,
> odn,
> newrdn, deloldrdn );
> ! 				}
>   			}
>   		} else {
>   			send_ldap_result( conn, op,
> LDAP_PARTIAL_RESULTS,
> NULL,
> *** ldap/servers/slapd/slap.h	Fri Jan 15 15:00:14 1999
> --- ldap/servers/slapd/slap.h.save	Fri Jan 15 14:57:38 1999
> ***************
> *** 206,211 ****
> --- 206,212 ----
>   	char	**be_replica;	/* replicas of this backend (in master)
> */
>   	char	*be_replogfile;	/* replication log file (in master)
> */
>   	char	*be_updatedn;	/* allowed to make changes (in replicas)
> */
> + 	char	*be_updatenorepdn;	/* allowed to make changes no
> replication */
>   	int	be_lastmod;	/* keep track of lastmodified{by,time}
> */
>   	char	*be_type;	/* type of database
> */
>   
> 
> 
>
Prev by Date: A couple of slapd/slurpd replication suggestions.
Next by Date: Re: Replication test on Solaris 2.5
Index(es):
- Chronological
- Thread