Issue 5385 - contextCSN not propagated in cascading syncrepl refreshAndPersist
Summary: contextCSN not propagated in cascading syncrepl refreshAndPersist
Status: VERIFIED FIXED
Alias: None
Product: OpenLDAP
Classification: Unclassified
Component: historical (show other issues)
Version: unspecified
Hardware: All All
: --- normal
Target Milestone: ---
Assignee: OpenLDAP project
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-02-18 10:15 UTC by Buchan Milne
Modified: 2019-11-20 18:03 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description Buchan Milne 2008-02-18 10:15:39 UTC
Full_Name: Buchan Milne
Version: OPENLDAP_REL_ENG_2_3
OS: Linux 2.6
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (196.15.129.131)


One of our environments does cascading replication of one database (the master
for the other databases in this environment is a syncrepl refreshAndPersist
syncrepl client to a database in another environment).

We monitor the replication status of all our slaves by comparing the contextCSN
on each database with the contextCSN of the updateref, and I have noticed that
the contextCSN on the cascaded replicas is not being updated, even though the
entries have been replicated (according to logging at sync level on these
slaves). All other replication (including non-cascaded refreshAndPersist
databases) works fine. All the servers in question are currently on 2.3.40.

It seems that the issue can be reproduced with current REL_ENG_2_3, with a
slight modification to test019 (see patch below) to include the contextCSN in
the comparison, which will then fail as follows:

$ ./run test019
Running ./scripts/test019-syncreplication-cascade...
running defines.sh
Starting master slapd on TCP/IP port 9011...
Using ldapsearch to check that master slapd (pid=23363) is running...
Using ldapadd to create the context prefix entry in the master...
Starting R1 slave slapd on TCP/IP port 9012...
Using ldapsearch to check that R1 slave slapd (pid=23370) is running...
Starting R2 slave slapd on TCP/IP port 9013...
Using ldapsearch to check that R2 slave slapd (pid=23376) is running...
Starting P1 slave slapd on TCP/IP port 9014...
Using ldapsearch to check that P1 slave slapd (pid=23382) is running...
Starting P2 slave slapd on TCP/IP port 9015...
Using ldapsearch to check that P2 slave slapd (pid=23388) is running...
Starting P3 slave slapd on TCP/IP port 9016...
Using ldapsearch to check that P3 slave slapd (pid=23395) is running...
Using ldapadd to populate the master directory...
Waiting 25 seconds for syncrepl to receive changes...
Using ldapmodify to modify master directory...
Waiting 25 seconds for syncrepl to receive changes...
Using ldapsearch to read all the entries from the master...
Using ldapsearch to read all the entries from the R1 slave...
Using ldapsearch to read all the entries from the R2 slave...
Using ldapsearch to read all the entries from the P1 slave...
Using ldapsearch to read all the entries from the P2 slave...
Using ldapsearch to read all the entries from the P3 slave...
Filtering master ldapsearch results...
Filtering R1 slave ldapsearch results...
Filtering R2 slave ldapsearch results...
Filtering P1 slave ldapsearch results...
Filtering P2 slave ldapsearch results...
Filtering P3 slave ldapsearch results...
Comparing retrieved entries from master and R1 slave...
Comparing retrieved entries from master and R2 slave...
Comparing retrieved entries from master and P1 slave...
Comparing retrieved entries from master and P2 slave...
test failed - master and P2 slave databases differ
$ diff -u testrun/server{1,5}.flt
--- testrun/server1.flt 2008-02-18 12:00:52.000000000 +0200
+++ testrun/server5.flt 2008-02-18 12:00:52.000000000 +0200
@@ -108,7 +108,7 @@
 postalAddress: Example, Inc. $ 535 W. William St. $ Anytown, MI 48109 $ US
 telephoneNumber: +1 313 555 1817
 associatedDomain: example.com
-contextCSN: 20080218100027Z#000002#00#000000
+contextCSN: 20080218100027Z#000000#00#000000

 dn: cn=Gern Jensen,ou=Information Technology Division,ou=People,dc=example,dc=
  com


Index: scripts/test019-syncreplication-cascade
===================================================================
RCS file: /repo/OpenLDAP/pkg/ldap/tests/scripts/test019-syncreplication-cascade,v
retrieving revision 1.13.2.7
diff -u -r1.13.2.7 test019-syncreplication-cascade
--- scripts/test019-syncreplication-cascade     11 Feb 2008 23:24:28 -0000    
1.13.2.7
+++ scripts/test019-syncreplication-cascade     18 Feb 2008 10:02:51 -0000
@@ -331,7 +331,7 @@

 echo "Using ldapsearch to read all the entries from the master..."
 $LDAPSEARCH -S "" -b "$BASEDN" -h $LOCALHOST -p $PORT1 \
-       'objectclass=*' > $MASTEROUT 2>&1
+       'objectclass=*' '*' contextCSN > $MASTEROUT 2>&1
 RC=$?

 if test $RC != 0 ; then
@@ -342,7 +342,7 @@

 echo "Using ldapsearch to read all the entries from the R1 slave..."
 $LDAPSEARCH -S "" -b "$BASEDN" -h $LOCALHOST -p $PORT2 \
-       'objectclass=*' > $SERVER2OUT 2>&1
+       'objectclass=*' '*' contextCSN > $SERVER2OUT 2>&1
 RC=$?

 if test $RC != 0 ; then
@@ -353,7 +353,7 @@

 echo "Using ldapsearch to read all the entries from the R2 slave..."
 $LDAPSEARCH -S "" -b "$BASEDN" -h $LOCALHOST -p $PORT3 \
-       'objectclass=*' > $SERVER3OUT 2>&1
+       'objectclass=*' '*' contextCSN > $SERVER3OUT 2>&1
 RC=$?

 if test $RC != 0 ; then
@@ -364,7 +364,7 @@

 echo "Using ldapsearch to read all the entries from the P1 slave..."
 $LDAPSEARCH -S "" -b "$BASEDN" -h $LOCALHOST -p $PORT4 \
-       'objectclass=*' > $SERVER4OUT 2>&1
+       'objectclass=*' '*' contextCSN > $SERVER4OUT 2>&1
 RC=$?

 if test $RC != 0 ; then
@@ -375,7 +375,7 @@

 echo "Using ldapsearch to read all the entries from the P2 slave..."
 $LDAPSEARCH -S "" -b "$BASEDN" -h $LOCALHOST -p $PORT5 \
-       'objectclass=*' > $SERVER5OUT 2>&1
+       'objectclass=*' '*' contextCSN > $SERVER5OUT 2>&1
 RC=$?

 if test $RC != 0 ; then
@@ -386,7 +386,7 @@

 echo "Using ldapsearch to read all the entries from the P3 slave..."
 $LDAPSEARCH -S "" -b "$BASEDN" -h $LOCALHOST -p $PORT6 \
-       'objectclass=*' > $SERVER6OUT 2>&1
+       'objectclass=*' '*' contextCSN > $SERVER6OUT 2>&1
 RC=$?

 if test $RC != 0 ; then

Comment 1 Howard Chu 2008-02-18 17:10:49 UTC
bgmilne@staff.telkomsa.net wrote:
> Full_Name: Buchan Milne
> Version: OPENLDAP_REL_ENG_2_3
> OS: Linux 2.6
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (196.15.129.131)
>
>
> One of our environments does cascading replication of one database (the master
> for the other databases in this environment is a syncrepl refreshAndPersist
> syncrepl client to a database in another environment).
>
> We monitor the replication status of all our slaves by comparing the contextCSN
> on each database with the contextCSN of the updateref, and I have noticed that
> the contextCSN on the cascaded replicas is not being updated, even though the
> entries have been replicated (according to logging at sync level on these
> slaves). All other replication (including non-cascaded refreshAndPersist
> databases) works fine. All the servers in question are currently on 2.3.40.
>
> It seems that the issue can be reproduced with current REL_ENG_2_3, with a
> slight modification to test019 (see patch below) to include the contextCSN in
> the comparison, which will then fail as follows:

I've reproduced this issue in RE23 as well. I note that there's no such 
problem with RE24. If we can't track this down easily I'm inclined to just 
ignore it.
-- 
   -- Howard Chu
   Chief Architect, Symas Corp.  http://www.symas.com
   Director, Highland Sun        http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP     http://www.openldap.org/project/

Comment 2 Howard Chu 2008-03-20 23:14:09 UTC
changed notes
changed state Open to Suspended
Comment 3 Howard Chu 2009-01-26 09:02:10 UTC
changed notes
moved from Incoming to Historical
Comment 4 OpenLDAP project 2014-08-01 21:05:04 UTC
2.3 wontfix
Comment 5 Quanah Gibson-Mount 2019-11-20 18:03:38 UTC
changed state Suspended to Closed