[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: Some interesting thoughts

David Boreham wrote:

Quanah Gibson-Mount wrote:

2) Replica initialization/re-initialization from the master - If you create an attribute "ndsReplicaRefresh" with a hostname as the value in the job class, it will reload that replica's DB, by sending binary indexes over the wire, and then removes the attributes from the job scheduler. I can only see this working if the master & replica agree on their indexing, however. It also doesn't replicate the replica specific configuration options.

This is not how it works. It sends the entries over the wire, but feeds them into the
fast concurrent import code on the replica.

That seems simple enough to do (except for the "fast" part. We definitely have work to do on speeding up indexing, and Jong has a number of items addressing that). Of course, unambiguously identifying a replica to update requires more than just a hostname, but that's a minor detail. Also, I have to wonder why such a feature is needed in a properly operating replication environment. For syncrepl, a freshly installed consumer will naturally pull down a complete copy of the database as its first action, so "initialization" should not be an issue. And if the replication mechanism is doing its job correctly, then "re-initialization" is not an issue.

There _is_ an option to initialize a replica using binary indices, but you have to
move the data yourself (with ftp or whatever).

I suppose that could save some time. This is something I've been wondering about (in the context of "centipede") as well, coming up with a portable representation of an index that can be easily distributed. This would allow sparse replicas to fully evaluate search filters locally, then request specific matching entries from the remote masters. (The issue is most apparent in our translucent overlay, but probably has wider application.) If you represent an index as a set of key/value pairs, it's pretty obvious that the values must be entryUUIDs, but what would the keys be? Just AVAs? I guess the problem is that a portable representation would be just as voluminous as the original entry data, thus defeating some of the point of a "sparse" replica.

 -- Howard Chu
 Chief Architect, Symas Corp.       Director, Highland Sun
 http://www.symas.com               http://highlandsun.com/hyc
 Symas: Premier OpenSource Development and Support