[Date Prev][Date Next]
Re: Will slapadd work with delta-syncrepl?
- To: Howard Chu <email@example.com>
- Subject: Re: Will slapadd work with delta-syncrepl?
- From: Khoa Nguyen <firstname.lastname@example.org>
- Date: Fri, 26 Mar 2010 20:43:08 -0400
- Cc: email@example.com
- Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:received:message-id:subject:from:to:cc:content-type; bh=TGHKajNSJwnOBHgCI2F+2/P2HJcYDlfFLBkAMRXc8Hw=; b=kfPohGLZAmJvr9/elTOamV43LR9jHFYRXDPOLoHL7TuIQJlZDw8JAs89u8zDr8lhc4 HAQVL60qXbKUw2RjjDzBbPBZtz0pVUk+mi2drXzCGOFFmfNRw3o390R06VK5g8vF2fCq va9fLiU7wd1Lr88mEeBZ+4s10OWpNbI6RFhrA=
- Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=m3dS69u4QUM3eVM290fxcnhAP6dVBuKH60I17FhZa8CYOgnuEm5NPzoNn1eXRqbtLi KulxRsjfJTHLKqVzx9bLu5/fDn+/jgrrP/Eo+5eS4cakm59A1Z5Yu4Oj5PbFu7U77AAl +/ecOyKysM0DkJfZyiMVNLe4NVnb1ji1Xcar0=
- In-reply-to: <4BAD0F9E.firstname.lastname@example.org>
- References: <email@example.com> <4BAD0F9E.firstname.lastname@example.org>
Thank you all for your insights, which make me think slapadding may not be a good companion with delta-syncrepl. So I plan to do all the updates online since I managed to get close to 3000 updates per second on my single-disk server.
Now, my colleague doesn't agree with me on the delta-syncrepl approach, and prefers to update A and B independently. His argument is that with delta-syncrepl, B is dependent on A, so if A's databases (main + log) are corrupted, and we have to restore A to a previous checkpoint, B would automatically rollback, and we would lost the latest data. I still prefer delta-syncrepl approach, since if updated independently, A and B can be out-of-synch over time and we wouldn't know it.
I also looked at other replication modes (mirror, n-way master, etc.), but since we only have 2 servers to work with, and our openldap version is still at 2.3, our choices are limited.
Your advices and suggestions on what should be the best approach are appreciated.
On Fri, Mar 26, 2010 at 3:48 PM, Howard Chu <email@example.com>
Delta-syncrepl works by writing a log of all your main database changes into a log database. When you add entries using slapadd, nothing is added to the log database, therefore delta-sync cannot replicate those changes.
Khoa Nguyen wrote:
I have set up delta-syncrepl between provider A and consumer B, and it seems
to work ok. Online updates to A are synch'ed to B. However, about once a
month, there is a large update which contains hundreds of million records.
Online update is going to take days. I tried to bring A offline, do slapdadd,
and bring A back online. But these new entries were not synch'ed to B. Is
there a way I can make this work?
You can force a resync by emptying the log database. When a delta-sync consumer tries to connect and the log no longer contains a record of the consumer's last change, it will automatically fallback to regular syncrepl to resync.
Note that since you're talking about new entries, which need to be replicated in whole anyway, delta-syncrepl offers no benefit over regular syncrepl here.
Also, as Buchan pointed out, replicating hundreds of millions of records will take a long time. You're better off just slapadding on both the provider and the consumer.
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/