Viewing Incoming/7498
Full headers

Major security issue: yes  no

Notes:

Notification:

Date: Wed, 23 Jan 2013 11:59:55 +0000
From: arunkumar_1123@yahoo.com
To: openldap-its@OpenLDAP.org
Subject: Openldap syncrepl issue

Full_Name: Arunkumar shanmugam
Version: 2.4.33
OS: rhel5
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (203.83.248.32)


Hi,

I've been having issues with syncrepl in openldap. This is plain 
syncrepl. I'm not using delta-syncrepl as I run with mirror mode on.

I've noticed that when there are a large number of writes on the 
writemaster (Typically around 10 per second) the data on the readslave 
goes out of sync with the master, typically having entries which have 
been deleted on the master. The contextCSNs on both the readslaves and 
writemaster match eventually, but the data doesn't. I'm running the 
readslave in refreshOnly mode with an interval of 1 minute. I'm using a 
reasonably large DB of size ~ 2 GB. I turned on the sync logs and found 
lots of entries with "nonpresent_callback" and "syncrepl_del_nonpresent".

Tested this and found it occurring in both 2.4.29 and 2.4.33 (Latest as 
of this time), and with BDB 4.2.36 as the backend.

I was wondering if anyone has faced similar issues before, and if there 
exists a solution to this problem. I will send any relevant 
configuration and log snippets as needed.

Thanks

Followup 1

Date: Wed, 23 Jan 2013 10:02:58 -0800
From: Quanah Gibson-Mount <quanah@zimbra.com>
To: arunkumar_1123@yahoo.com, openldap-its@openldap.org
Subject: Re: (ITS#7498) Openldap syncrepl issue

--On Wednesday, January 23, 2013 11:59 AM +0000 arunkumar_1123@yahoo.com 
wrote:

> Full_Name: Arunkumar shanmugam
> Version: 2.4.33
> OS: rhel5
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (203.83.248.32)
>
>
> Hi,
>
> I've been having issues with syncrepl in openldap. This is plain
> syncrepl. I'm not using delta-syncrepl as I run with mirror mode on.

MMR has supported delta-syncrepl since 2.4.27.  You should use it.

--Quanah



--

Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra ::  the leader in open source messaging and collaboration

Followup 2

Date: Thu, 24 Jan 2013 13:24:28 +0800 (SGT)
From: arun s <arunkumar_1123@yahoo.com>
Subject: Re: (ITS#7498) Openldap syncrepl issue
To: Quanah Gibson-Mount <quanah@zimbra.com>,
        "openldap-its@openldap.org" <openldap-its@openldap.org>

> MMR has supported delta-syncrepl since 2.4.27.. You should use it.

I did consider delta-syncrepl, but given the high write load expected, I.
was not sure if it was the right decision. I am expecting an average.
load of 50-100 writes per second, and with 1 backup provider, and 10.
consumers, it might get heavy on the provider which is serving the.
writes, given that OpenLDAP would be maintaining another DB for the.
access log.

But with this syncrepl issue, it looks like I need to change the.
replication mode to refreshAndPersist anyway. As such it looks like.
delta syncrepl is also refreshAndPersist too.

Is there any recommendation between simple refreshAndPersist and.
delta-syncrepl with refreshAndPersist for high write loads?

Thanks


PS: I see that delta syncrepl is compatible with MMR which I presume.
to be MultiMaster. I hope it works with Mirror Mode too. I did not find.
any documents supporting this.