[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: (ITS#6671) multi-master replication fails to delete objects from consumer
- To: openldap-its@OpenLDAP.org
- Subject: Re: (ITS#6671) multi-master replication fails to delete objects from consumer
- From: hyc@symas.com
- Date: Thu, 9 Jun 2011 06:45:52 GMT
- Auto-submitted: auto-generated (OpenLDAP-ITS)
hyc@symas.com wrote:
> bcolston@xtec.com wrote:
>> Full_Name: Barry Colston
>> Version: 2.4.23
>> OS: Fedora 10
>> URL: ftp://ftp.openldap.org/incoming/
>> Submission from: (NULL) (209.255.208.200)
>
> I'm looking at your test case now. One thing to note, serverIDs must all be
> non-zero in a multi-master configuration. serverID == 0 is only valid in
> single-master replication.
>
>> When a "Refresh Present" phase is performed at a multi-master consumer, objects
>> that were deleted at the provider
>> while the consumer was down are not deleted from the multi-master consumer (if
>> the provider is brought down and back up after the consumer is down). This
>> problem can be duplicated as follows:
>> 1. Start provider
>> 2. Start consumer, which is configured as a multi-master server
>> 3. Add 10 records to the provider and wait until the records are replicated to
>> the multi-master consumer
>> 4. Stop the consumer
>> 5. Delete 3 of the records that were added to the provider
>> 6. Stop the provider
>> 7. Start the provider
>> 8. Start the consumer
>> 9. After giving the consumer time to perform the refresh present phase, the 3
>> records that were deleted at the provider while the consumer was down
>> are still present on the consumer and were not deleted during the refresh
>> present phase
>>
>> Note: the consumer database must have a contextCSN attribute value present for
>> itself (e.g., there should be 2 contextCSN attribute values
>> present in the consumer database, one for the provider (in my configuration,
>> this is serverID 000), and one value for the consumer
>> (in my configuration, this is serverID 001). An example of the contextCSN
>> values follows:
I see. This configuration is not supported. "Multi-Master" requires the
servers to be full peers. In your configuration, you have a dedicated
provider, it never pulls changes from the consumer, while your consumer is
pulling from the provider and accepting local changes. Since the consumer has
changes that the provider doesn't know about, the sync protocol is broken. The
consumer cannot conclusively state that all entries the provider doesn't know
about must be deleted, since some of those entries may have legitimately been
created on the consumer, so it cannot execute the delete-nonpresent step.
Closing this ITS.
>> dn: dc=authentx
>> objectClass: top
>> entryUUID: 8eb6b259-ab66-49bd-b012-674c4f0fba8f
>> contextCSN: 20101012154030.379053Z#000000#000#000000
>> contextCSN: 20101012141508.956053Z#000000#001#000000
>>
>> OpenLDAP 2.4.23 is being used, along with Berkeley 4.6.21 (plus patches).
>>
>> Consumer log file from -d sync logging option:
>> @(#) $OpenLDAP: slapd 2.4.23 (Oct 11 2010 18:00:59) $
>> Barry@XTecVisaAdmin:/usr/src/openldap-2.4.23/servers/slapd
>> slapd starting
>> do_syncrep2: rid=001 LDAP_RES_INTERMEDIATE - SYNC_ID_SET
>> do_syncrep2: rid=001 LDAP_RES_INTERMEDIATE - SYNC_ID_SET
>> do_syncrep2: rid=001 LDAP_RES_INTERMEDIATE - SYNC_ID_SET
>> do_syncrep2: rid=001 LDAP_RES_INTERMEDIATE - SYNC_ID_SET
>> do_syncrep2: rid=001 LDAP_RES_INTERMEDIATE - SYNC_ID_SET
>> do_syncrep2: rid=001 LDAP_RES_INTERMEDIATE - SYNC_ID_SET
>> do_syncrep2: rid=001 LDAP_RES_INTERMEDIATE - SYNC_ID_SET
>> do_syncrep2: rid=001 LDAP_RES_INTERMEDIATE - SYNC_ID_SET
>> do_syncrep2: rid=001 LDAP_RES_INTERMEDIATE - SYNC_ID_SET
>> do_syncrep2: rid=001 LDAP_RES_INTERMEDIATE - SYNC_ID_SET
>> do_syncrep2: rid=001 LDAP_RES_INTERMEDIATE - SYNC_ID_SET
>> do_syncrep2: rid=001 LDAP_RES_INTERMEDIATE - SYNC_ID_SET
>> do_syncrep2: rid=001 LDAP_RES_INTERMEDIATE - REFRESH_PRESENT
>> do_syncrep2: rid=001
>> cookie=rid=001,csn=20101012180219.072053Z#000000#000#000000
>> slap_queue_csn: queing 0x1841a30 20101012180219.072053Z#000000#000#000000
>> slap_graduate_commit_csn: removing 0x1841b18
>> 20101012180219.072053Z#000000#000#000000
>> daemon: shutdown requested and initiated.
>> slapd shutdown: waiting for 1 operations/tasks to finish
>> slapd stopped.
>>
>> Provider slapd.conf file:
>> #
>> include /usr/local/etc/openldap/schema/core.schema
>> include /usr/local/etc/openldap/schema/cosine.schema
>> include /usr/local/etc/openldap/schema/inetorgperson.schema
>> include /usr/local/etc/openldap/schema/nis.schema
>>
>> include /usr/local/etc/openldap/schema/authentx.schema
>>
>> pidfile /usr/local/var/slapd.sync1.pid
>> argsfile /usr/local/var/slapd.sync1.args
>>
>> allow bind_v2
>>
>> threads 32
>>
>> #loglevel 416
>> password-hash {SSHA}
>>
>> serverID 000
>>
>> database bdb
>> suffix "dc=authentx"
>> rootdn "cn=xroot,dc=authentx"
>> rootpw SECRET
>> directory /authentx/db/ldap/authentx-sync1
>>
>> overlay syncprov
>> syncprov-checkpoint 100 15
>> syncprov-sessionlog 150000
>>
>> # checkpoint<kbytes> <min>
>> checkpoint 128 20
>>
>> #index for the objectclass
>> index objectClass eq,pres
>> index cid,eid,pid eq,pres
>> index gcoid eq,pres
>> index sysid,certid eq,pres
>> index imageid,bioid eq,pres
>> index addrid,emplid eq,pres
>> index acid,apid,aeid eq,pres
>> index acpid,agpid eq,pres
>> index deviceid eq,pres
>> index qid eq,pres
>> index scid eq,pres
>> index docid eq,pres
>> index procid eq,pres
>> index prochandlerid eq,pres
>> index ounit eq,pres
>> index credentials eq,pres
>> index entities eq,pres
>> index permissions eq,pres
>> index region eq,pres
>> index aliasedObjectName eq,pres
>> index proctype eq,pres
>> index procname eq,pres
>> index status eq,pres
>> index upi,ediident eq,pres
>> index xsync eq,pres
>> index role,xrole eq,pres
>> index eroid eq,pres
>> index esfunction eq,pres
>> index nippsector eq,pres
>> index ercog,ercoop eq,pres
>> index eropron eq,pres
>> index xsyncid eq,pres
>> index stockid eq,pres
>> index stocktypecode eq,pres
>> index lotserialin eq,pres
>> index lotserialout eq,pres
>> index entryCSN,entryUUID eq
>> index incl eq,pres
>>
>> cachesize 10000
>> dncachesize 20000
>> idlcachesize 1000
>>
>> sizelimit 500000
>> timelimit 36000
>> include /usr/local/etc/openldap/acl.authentx
>>
>> database monitor
>>
>> Consumer slapd.conf file:
>> #
>> include /usr/local/etc/openldap/schema/core.schema
>> include /usr/local/etc/openldap/schema/cosine.schema
>> include /usr/local/etc/openldap/schema/inetorgperson.schema
>> include /usr/local/etc/openldap/schema/nis.schema
>>
>> include /usr/local/etc/openldap/schema/authentx.schema
>>
>> pidfile /usr/local/var/slapd.sync2.pid
>> argsfile /usr/local/var/slapd.sync2.args
>>
>> allow bind_v2
>>
>> threads 32
>>
>> #loglevel 416
>> password-hash {SSHA}
>>
>> serverID 001
>>
>> database bdb
>> suffix "dc=authentx"
>> rootdn "cn=xroot,dc=authentx"
>> rootpw SECRET
>> directory /authentx/db/ldap/authentx-sync2
>>
>> overlay syncprov
>> syncprov-checkpoint 100 15
>> syncprov-sessionlog 5000
>>
>> # SLAVE server replication section
>> syncrepl rid=001
>> provider=ldap://localhost:3891
>> type=refreshAndPersist
>> retry="30 60 60 +"
>> searchbase="dc=authentx"
>> scope=sub
>> schemachecking=off
>> bindmethod=simple
>> binddn="cn=xroot,dc=authentx"
>> credentials="SECRET"
>>
>> mirrormode on
>>
>> # checkpoint<kbytes> <min>
>> checkpoint 128 20
>>
>> #index for the objectclass
>> index objectClass eq,pres
>> index cid,eid,pid eq,pres
>> index gcoid eq,pres
>> index sysid,certid eq,pres
>> index imageid,bioid eq,pres
>> index addrid,emplid eq,pres
>> index acid,apid,aeid eq,pres
>> index acpid,agpid eq,pres
>> index deviceid eq,pres
>> index qid eq,pres
>> index scid eq,pres
>> index docid eq,pres
>> index procid eq,pres
>> index prochandlerid eq,pres
>> index ounit eq,pres
>> index credentials eq,pres
>> index entities eq,pres
>> index permissions eq,pres
>> index region eq,pres
>> index aliasedObjectName eq,pres
>> index proctype eq,pres
>> index procname eq,pres
>> index status eq,pres
>> index upi,ediident eq,pres
>> index xsync eq,pres
>> index role,xrole eq,pres
>> index eroid eq,pres
>> index esfunction eq,pres
>> index nippsector eq,pres
>> index ercog,ercoop eq,pres
>> index eropron eq,pres
>> index xsyncid eq,pres
>> index stockid eq,pres
>> index stocktypecode eq,pres
>> index lotserialin eq,pres
>> index lotserialout eq,pres
>> index entryCSN,entryUUID eq
>> index incl eq,pres
>>
>> cachesize 10000
>> dncachesize 20000
>> idlcachesize 1000
>>
>> sizelimit 100000
>> timelimit 36000
>> # the authentx database access control directives
>> include /usr/local/etc/openldap/acl.authentx
>>
>> database monitor
>>
>>
>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/