[Date Prev][Date Next] [Chronological] [Thread] [Top]

(ITS#7104) refreshOnly syncrepl out of sync during bulk update/delete operations



Full_Name: Martin Krzaczek
Version: 2.4.28
OS: CentOS 5.7
URL: 
Submission from: (NULL) (83.202.204.201)


Hello,

I noticed an issue with "refreshOnly" syncrepl when a large number of records is
added, updated and deleted. At the end of synchronization the contextCSN is
equal on all servers, but the data is out of synchronization for some deleted
records that remains undeleted on consumer/slave servers.

Restarting the consumer or forcing a synchronization with "slapd ... -c
'rid=1,csn=0' " does not help.


A similar issue was submitted few weeks ago (ITS#7052), but I'm not sure the
problem is the same.

My configuration :
- CentOS 5.7 + OpenLDAP 2.4.28 + BDB 5.1.29
  (the same issue occurs with CentOS 5.7 + OpenLDAP 2.4.26 + BDB 5.1.25)
- one main provider where all modification operations are performed
- one 'first level' consumer synchronizing data on the main provider 
  and providing data for 'second level' consumers
- one 'second level' consumer synchronizing data on the 'first level'
  consumer/provider

I tried several combinations of configurations (refreshOnly/refreshAndPersist,
syncprov-sessionlog) and noticed that when refreshOnly mode is used, some
records may be not deleted on consumer/slave servers. The number of undeleted
records varies depending on the configuration.
So with a higher value of "syncprov-sessionlog" (for example 5000 instead of
500) the number of lost records is reduced.

I have an application that sequentially (i.e. in a single thread) adds about
12000 records, modifies about 25000 records and deletes about 1000 records.
Add/update/delete operations are mixed i.e. it is possible a have sequences such
as add/add/delete/update/update/update/delete/add/...

With a small amount of records to add/update/delete the synchronization properly
works without any data loss.
So it seams that if all modifications may be cached and transmitted to consumers
then deleted records are not lost.
There is no data loss for added and updated records whatever is the number of
operations.


Regards,
Martin