moved from Incoming to Software Bugs
Full_Name: Rein Tollevik Version: CVS head OS: Irrelevant URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (84.215.59.121) Submitted by: rein New syncprov consumers connecting to a forwarding server and presenting an apperently up-to-date cookie will loose any mods that have already taken place on the forwarding server if it itself is refreshing from its provider. This should not be a problem if the forwarding server have a sufficiently large syncprov log, but a fix for servers without is coming. -- Rein Tollevik Basefarm AS
changed notes changed state Open to Test
On 11/15/10 16:39 , rein@OpenLDAP.org wrote: > New syncprov consumers connecting to a forwarding server and presenting an > apperently up-to-date cookie will loose any mods that have already taken place > on the forwarding server if it itself is refreshing from its provider. This > should not be a problem if the forwarding server have a sufficiently large > syncprov log, but a fix for servers without is coming. The currently committed fix for this its leaves one problem open. If a forwarding server restarts in the middle of the refresh phase, after making some changes but before updating the csn, new consumers connecting with an apparently up to date csn set after the server comes up again will not know that the context is dirty and will loose these changes. The same problem arise if the server restarts between the time a locally initiated delete operation is performed in the database and the accompanying csn set is saved. A fix could be to always assume the context is dirty after start up, and thereby forcing all clients to undergo the refresh phase even if they are in sync until some operation that updates the csn set is performed. An unnecessary refresh is probably better than loosing changes.. Rein
rein@OpenLDAP.org wrote: > On 11/15/10 16:39 , rein@OpenLDAP.org wrote: >> New syncprov consumers connecting to a forwarding server and presenting an >> apperently up-to-date cookie will loose any mods that have already taken place >> on the forwarding server if it itself is refreshing from its provider. This >> should not be a problem if the forwarding server have a sufficiently large >> syncprov log, but a fix for servers without is coming. > The currently committed fix for this its leaves one problem open. If a > forwarding server restarts in the middle of the refresh phase, after > making some changes but before updating the csn, new consumers > connecting with an apparently up to date csn set after the server comes > up again will not know that the context is dirty and will loose these > changes. The same problem arise if the server restarts between the time > a locally initiated delete operation is performed in the database and > the accompanying csn set is saved. > > A fix could be to always assume the context is dirty after start up, and > thereby forcing all clients to undergo the refresh phase even if they > are in sync until some operation that updates the csn set is performed. > An unnecessary refresh is probably better than loosing changes.. I haven't had the opportunity to review these patches yet, but the bug description sounds a little flaky to me. The original design is this: when a consumer requests a refresh from the provider, the provider uses a snapshot of its current contextCSN. All changes from the consumer's cookie to this snapshot (inclusive) are sent to the consumer. Any changes currently in progress that have not yet been committed will be skipped until the next refresh. Nothing is lost, it's simply delayed, and that's in accordance with syncrepl's "eventual convergence" model. Likewise, the provider only updates its contextCSN when a change is fully committed. syncprov should NOT need to defer any consumer while it has outstanding mods. There is no reason that would be needed by the sync protocol. Servers only update their contextCSN after an entire refresh has completed. If a downstream consumer connects while a forwarder is still refreshing, the consumer should receive nothing. (Or, it should only receive the changes between its cookie and the server's committed contextCSN, if any.) You may very well have found bugs in the implementation, but it sounds to me like you've changed the overall behavior to something outside of the original design. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Just a note that I have switched my reply to -devel. Rein
changed notes changed state Test to Release
changed notes changed state Release to Closed
changed notes changed state Closed to Partial
(partly) fixed in HEAD (partly) fixed in RE24