Re: (ITS#7710) contextCSN values not updated by internal non-replicated ops

[ck@cksoft.de]

Hi Michael,

On Sun, 6 Oct 2013, michael@stroeder.com wrote:
> I've updated the test data with LDIF files and a README describing the LDAP
> operations to perform:
>
> http://www.stroeder.com/temp/openldap-testbed-its7710.tar.bz2
>
> I consider this to be a serious issue which could be the cause for other
> replication issues including dead-locks we're experiencing. (Maybe there are
> similar issues in slapo-refint since dead-locks in our deployment arised when
> moving entries into different subtree.)

I can confirm that I can see the same bug with your testcase.

It seems that restarting the second slapd instance fixes the contextCSN
issue just because it scans all it's entries and does the right thing.

Even a slapcat of both directories shows the issue.

Things that I find disturbing:

After applying the group membership the member dn: gets it's memberOf 
attribute updated accordingly. But the entryCSN stays unchanged.

Entry before applying group membership:

     # michael, users, example.com
     dn: uid=michael,ou=users,dc=example,dc=com
     uid: michael
     objectClass: account
     objectClass: simpleSecurityObject
     userPassword:: dGVzdHNlY3JldA==
     description: test
     entryCSN: 20131006193757.912740Z#000000#002#000000

Entry after applying group membership is obviously updated but still has
the same entryCSN:

     # michael, users, example.com
     dn: uid=michael,ou=users,dc=example,dc=com
     uid: michael
     objectClass: account
     objectClass: simpleSecurityObject
     userPassword:: dGVzdHNlY3JldA==
     memberOf: cn=admins,ou=groups,dc=example,dc=com
     memberOf: cn=testgroup1,ou=groups,dc=example,dc=com
     description: test
     entryCSN: 20131006193757.912740Z#000000#002#000000

I can also confirm the subsequent contextCSN mismatch.

I too consider this a serious issue.

If I read Howards reply correctly and from what I see in the source code
of the memberOf overlay the strategy here seems to be not to replicate the
side effect of adding the memberOf: entry.

OpenLDAP relies on the side effect happening on the replicas through the
same codepath when the member: attribute in the group gets updated
through replication.

This sounds very similar to the difference between:

1. statement replication in sql databases ala mysql

2. log based replication ala postgresql

OpenLDAP is doing 1. here while 2. would perhaps have been the better option.

I understand turning this around is propably too late at this point in time.

Looks like we will just have to work out the bugs the hard way.

I am using slapo-lastbind and slapo-ppolicy extensively in my current
project and will keep my eyes open for similar issues resulting from not
replicating the side effects that these overlays produce.

Greetings
Christian

-- 
Christian Kratzer                      CK Software GmbH
Email:   ck@cksoft.de                  Wildberger Weg 24/2
Phone:   +49 7032 893 997 - 0          D-71126 Gaeufelden
Fax:     +49 7032 893 997 - 9          HRB 245288, Amtsgericht Stuttgart
Web:     http://www.cksoft.de/         Geschaeftsfuehrer: Christian Kratzer