[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: delta-syncrepl contextCSN update timing, schema checking

John Morrissey wrote:
To make a long story short, it seems that syncrepl doesn't update the
backend's contextCSN until it's processed its backlog? To check, I stopped
another consumer and let a backlog build, then started it at debuglevel
16384 and watched the backend's contextCSN with ldapsearch(1). contextCSN
didn't increment until the backlog was completely processed, even though I
could see the changes it was processing with ldapsearch(1) as soon as they
were processed.

If a consumer processes replication without updating the backend's
contextCSN, it will try to re-process the same replication entries when it
starts up again, which will generally fail. This seems to leave one in a
bind, either having to manually determine the correct value for contextCSN
and update it manually, or remove the backend's data files and let syncrepl
rebuild them from scratch. If this assessment is correct, this behavior
doesn't seem desirable.

Your description of the behavior is correct. It's required to work that way with regular syncrepl; it probably should work differently for delta-sync but nobody pointed it out before.

In regular syncrepl the refresh phase does a regular LDAP search, whose results are returned in whatever arbitrary order the database normally retrieves things. Since that's pretty much guaranteed not to be the same as the order in which changes were made, we can't update the contextCSN until all of the changes have been received.

In delta-syncrepl the refresh is coming from the log, and unless somebody has been explicitly mucking around in the log DB, the entries will always be returned in order, so it's possible to update the contextCSN after each entry has been received. But it's up to the provider to send the cookie with each entry in this case, and the syncprov overlay doesn't really know the difference between delta-sync and regular sync, so it doesn't do it.

You could file an ITS for this, but I don't think we'll be changing this in 2.3.
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/