Re: backend requirements for syncrepl

[Howard Chu <hyc@symas.com>]

Jong-Hyuk wrote:

> I am currently into
> the design of a changelog-based operatoin mode of syncrepl. Wondering
> whether this would be sufficient for your case.
> - Jong-Hyuk
>  
I'd like to hear more about this, since we are in the process of 
implementing changelog functionality as well.

One of the weaknesses we see in the current syncrepl implementation is 
that the syncrepl consumer must always retrieve complete entries for any 
changes that have occurred on the master. The network overhead can get 
large for a large number of changes to large entries. Our approach to 
solving this is to create a changelog on the master which is a database 
whose entries consist of LDIF modification records. The changelog is 
written using an overlay on the real database of interest, and the 
changelog database becomes the syncrepl provider. So the syncrepl 
protocol doesn't change, but there is a new syncrepl consumer that reads 
these entries and turns them into the appropriate LDAP operations.

--
 -- Howard Chu
 Chief Architect, Symas Corp.       Director, Highland Sun
 http://www.symas.com               http://highlandsun.com/hyc
 Symas: Premier OpenSource Development and Support