[Date Prev][Date Next] [Chronological] [Thread] [Top]

ITS#7052, syncrepl, deletes, and MMR

I'm still seeing cases where deleted entries are getting resurrected when a number of concurrent Add/Delete sequences are occurring, with multiple MMR servers (4 minimum to show the error).

The problem begins because multiple writes are outstanding, and they are replicated in persist mode without a CSN in their syncrepl cookie. This is a normal occurrence when the current op does not correspond to the last committed CSN.

Because there is no CSN, the consumer doesn't update its cookie state while performing a particular op.

As a result, if a client does Add/Delete/Add/Delete of the same DN, it's possible for the Adds to propagate several times (more than the client actually executed).

Adds and Modifies can usually be rejected if they're too old, because they carry an entryCSN attribute which can be compared against the existing entry, even if the consumer cookie state has not been updated. But Deletes don't carry any attributes, and Deleted entries can't be checked.

So, given a MMR setup like so:

   1 -- 2
   |    |
   3 -- 4

A sequence of Add/Del/Add/Del performed at server 1 will be replicated to both 2 and 3 immediately. They will then cascade it to server 4. If many other writes were occurring at the same time, causing these writes to be propagated without a cookie CSN, then server 4 will propagate them back to 3 and 2 respectively, and 3 and 2 will re-add the deleted entries because they have nothing to check that says the Adds are old. This cycle only gets broken if server 1 eventually sends an op with accompanying cookie update, so that all the downstream servers can see that the ops are old.


OK, upon further digging, this appears to be caused by ITS#6024. rein's patch prevents the consumer and provider from informing each other of their SIDs when no CSN is present; this prevents syncprov's propagation loop detection from working. Sigh. Reverting ITS#6024 patch...

  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/