Re: (ITS#6864) delta-sync - contextCSN not updating on provider

[hyc@symas.com]

yuribank@gmail.com wrote:
> Full_Name: Yuri Bank
> Version: 2.4.23-24
> OS: Linux Ubuntu 10.10
> URL: http://yuri.easytospell.net/consumer.provider.txt
> Submission from: (NULL) (67.180.182.165)
>
>
> This issue exists in both versions 2.4.23 and 2.4.24
>
> I've found an interesting problem with delta-sync replication in which the
> ContextCSN on my consumers is higher than the contextCSN on my provider. This is
> because the Provider does not properly update its contextCSN for its base suffix
> (dc=test,dc=com) when changes are made to group membership. Of couse, if it so
> happens that the last change in my database was not a group membership change,
> then the contextCSNs will be consistent between my consumers and provider.

Sounds like this bug was introduced by patches for ITS#6766 or #6329. Both of 
those patches have been reverted for ITS#6915. This bug should be fixed in the 
current git HEAD.
>
>
> I use the following command to check the ContextCSN on each consumer:
>
> Consumer: 1
> root@neteng1.oak:/etc/ldap# ldapsearch -Y EXTERNAL -H ldapi:/// -s base -b
> "dc=test,dc=com" contextCSN dn: dc=test,dc=com
> dn: dc=test,dc=com
> contextCSN: 20110313041653.752098Z#000000#000#000000
>
> Consumer: 2
> ybank@neteng1.iad:~$ ldapsearch -Y EXTERNAL -H ldapi:/// -s base -b
> "dc=test,dc=com" contextCSN dn: dc=test,dc=com
> dn: dc=test,dc=com
> contextCSN: 20110313041653.752098Z#000000#000#000000
>
> So we can see that the two consumers have matching contextCSNs:
> ContextCSN. 20110313041653.752098Z#000000#000#000000
>
> Lets check the Provider now.
>
> Provider:
> root@neteng0.iad:~# ldapsearch -Y EXTERNAL -H ldapi:/// -s base -b
> "dc=test,dc=com" ContextCSN dn: dc=test,dc=com
> dn: dc=test,dc=com
> contextCSN: 20110313041653.709140Z#000000#000#000000
>
> The providers CSN is smaller!?
>
> Lets take a closer look and search cn=accesslog
>
> These are the last two entries: ( first the user was added, and then the user
> was added to a group)
>
> # 20110313041653.000003Z, accesslog
> dn: reqStart=20110313041653.000003Z,cn=accesslog
> objectClass: auditAdd
> reqStart: 20110313041653.000003Z
> reqEnd: 20110313041653.000004Z
> reqType: add
> reqSession: 34633
> reqAuthzID: cn=admin,dc=test,dc=com
> reqDN: cn=Bank\2C Yuri(banky),o=UserAccounts,dc=test,dc=com
> reqResult: 0
> reqMod: sn:+ Bank
> reqMod: userPassword:+ {SASL}banky
> reqMod: uid:+ banky
> reqMod: objectClass:+ top
> reqMod: objectClass:+ person
> reqMod: objectClass:+ shadowAccount
> reqMod: structuralObjectClass:+ person
> reqMod: cn:+ Bank, Yuri(banky)
> reqMod: entryUUID:+ 78a75ef6-e174-102f-9571-ffecbfef68e5
> reqMod: creatorsName:+ cn=admin,dc=test,dc=com
> reqMod: createTimestamp:+ 20110313041653Z
> reqMod: entryCSN:+ 20110313041653.709140Z#000000#000#000000
> reqMod: modifiersName:+ cn=admin,dc=test,dc=com
> reqMod: modifyTimestamp:+ 20110313041653Z
>
> # 20110313041653.000006Z, accesslog
> dn: reqStart=20110313041653.000006Z,cn=accesslog
> objectClass: auditModify
> reqStart: 20110313041653.000006Z
> reqEnd: 20110313041653.000007Z
> reqType: modify
> reqSession: 34633
> reqAuthzID: cn=admin,dc=test,dc=com
> reqDN: cn=SSLVPN,o=Groups,dc=test,dc=com
> reqResult: 0
> reqMod: member:+ cn=Bank\2C Yuri(banky),o=UserAccounts,dc=test,dc=com
> reqMod: entryCSN:= 20110313041653.752098Z#000000#000#000000
> reqMod: modifiersName:= cn=admin,dc=test,dc=com
> reqMod: modifyTimestamp:= 20110313041653Z
>
> You can see that the consumers have the latest entryCSN (20110313041653.752098Z)
> as their contextCSN, but the provider has the entryCSN (20110313041653.709140Z)
> before that as its contextCSN:
>
> If I search the contextCSN on -s base -b cn=accesslog it yields correctly, the
> same result as the consumers:
> 20110313041653.752098Z#000000#000#000000
>
>
> As you can see, the provider is not using the latest entryCSN as its ContextCSN,
> where as the consumer nodes are. Also notice that the last modification was to
> group membership. This problem only seems to exist when Adding/Removing users
> from a group.
>
> A side effect of this issue causes brand new consumers to get stuck in an
> infinite loop while syncing for the first time.
>
> A work around is to make a random change to a User/Person, such as injecting a
> random number into their description field AFTER making a change to a group
> membership. Such a change will:  A. cause the Provider to correctly update its
> contextCSN, B. provider and all consumer[s] will have the same contextCSN C.
> brand new consumers can be added without getting stuck in an infinite loop when
> syncing the database. ( they seem to get stuck on the last entry which makes
> sense if the last entry has a higher entryCSN than that of the Providers
> contextCSN? )
>
> Confgiuration: ( See URL )
> http://yuri.easytospell.net/consumer.provider.txt
>
> Overlays:
> syncprov
> accesslog
> memberof
>
>
> Please feel free to email me about reproducing this problem. I have a lab with
> various configurations and would be happy to give access to anyone interested.
>
> - Yuri Bank
>


-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/