[Date Prev][Date Next]
Re: (ITS#7104) refreshOnly syncrepl out of sync during bulk update/delete operations
- To: openldap-its@OpenLDAP.org
- Subject: Re: (ITS#7104) refreshOnly syncrepl out of sync during bulk update/delete operations
- From: firstname.lastname@example.org
- Date: Thu, 8 Dec 2011 21:37:54 GMT
- Auto-submitted: auto-generated (OpenLDAP-ITS)
> Full_Name: Martin Krzaczek
> Version: 2.4.28
> OS: CentOS 5.7
> Submission from: (NULL) (188.8.131.52)
> I noticed an issue with "refreshOnly" syncrepl when a large number of records is
> added, updated and deleted. At the end of synchronization the contextCSN is
> equal on all servers, but the data is out of synchronization for some deleted
> records that remains undeleted on consumer/slave servers.
> Restarting the consumer or forcing a synchronization with "slapd ... -c
> 'rid=1,csn=0' " does not help.
This appears to caused by the sessionlog overflowing due to incoming modifies
that occur after the refresh operation has begun. I would call this a
misconfigured server, since the sessionlog can only ever be useful if you size
it large enough to accomodate all of your update traffic.
I don't have any plans to fix this issue; fix the server configuration.
> A similar issue was submitted few weeks ago (ITS#7052), but I'm not sure the
> problem is the same.
> My configuration :
> - CentOS 5.7 + OpenLDAP 2.4.28 + BDB 5.1.29
> (the same issue occurs with CentOS 5.7 + OpenLDAP 2.4.26 + BDB 5.1.25)
> - one main provider where all modification operations are performed
> - one 'first level' consumer synchronizing data on the main provider
> and providing data for 'second level' consumers
> - one 'second level' consumer synchronizing data on the 'first level'
> I tried several combinations of configurations (refreshOnly/refreshAndPersist,
> syncprov-sessionlog) and noticed that when refreshOnly mode is used, some
> records may be not deleted on consumer/slave servers. The number of undeleted
> records varies depending on the configuration.
> So with a higher value of "syncprov-sessionlog" (for example 5000 instead of
> 500) the number of lost records is reduced.
> I have an application that sequentially (i.e. in a single thread) adds about
> 12000 records, modifies about 25000 records and deletes about 1000 records.
> Add/update/delete operations are mixed i.e. it is possible a have sequences such
> as add/add/delete/update/update/update/delete/add/...
> With a small amount of records to add/update/delete the synchronization properly
> works without any data loss.
> So it seams that if all modifications may be cached and transmitted to consumers
> then deleted records are not lost.
> There is no data loss for added and updated records whatever is the number of
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/