ITS#7052, syncrepl, deletes, and MMR

[Howard Chu <hyc@symas.com>]

I'm still seeing cases where deleted entries are getting resurrected when a 
number of concurrent Add/Delete sequences are occurring, with multiple MMR 
servers (4 minimum to show the error).

The problem begins because multiple writes are outstanding, and they are 
replicated in persist mode without a CSN in their syncrepl cookie. This is a 
normal occurrence when the current op does not correspond to the last 
committed CSN.

Because there is no CSN, the consumer doesn't update its cookie state while 
performing a particular op.

As a result, if a client does Add/Delete/Add/Delete of the same DN, it's 
possible for the Adds to propagate several times (more than the client 
actually executed).

Adds and Modifies can usually be rejected if they're too old, because they 
carry an entryCSN attribute which can be compared against the existing entry, 
even if the consumer cookie state has not been updated. But Deletes don't 
carry any attributes, and Deleted entries can't be checked.

So, given a MMR setup like so:

   1 -- 2
   |    |
   3 -- 4

A sequence of Add/Del/Add/Del performed at server 1 will be replicated to both 
2 and 3 immediately. They will then cascade it to server 4. If many other 
writes were occurring at the same time, causing these writes to be propagated 
without a cookie CSN, then server 4 will propagate them back to 3 and 2 
respectively, and 3 and 2 will re-add the deleted entries because they have 
nothing to check that says the Adds are old. This cycle only gets broken if 
server 1 eventually sends an op with accompanying cookie update, so that all 
the downstream servers can see that the ops are old.

...

OK, upon further digging, this appears to be caused by ITS#6024. rein's patch 
prevents the consumer and provider from informing each other of their SIDs 
when no CSN is present; this prevents syncprov's propagation loop detection 
from working. Sigh. Reverting ITS#6024 patch...

--
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/