[Date Prev][Date Next]
Re: Some interesting thoughts
David Boreham wrote:
Quanah Gibson-Mount wrote:
2) Replica initialization/re-initialization from the master - If you
create an attribute "ndsReplicaRefresh" with a hostname as the value
in the job class, it will reload that replica's DB, by sending binary
indexes over the wire, and then removes the attributes from the job
scheduler. I can only see this working if the master & replica agree
on their indexing, however. It also doesn't replicate the replica
specific configuration options.
This is not how it works. It sends the entries over the wire, but
feeds them into the
fast concurrent import code on the replica.
That seems simple enough to do (except for the "fast" part. We
definitely have work to do on speeding up indexing, and Jong has a
number of items addressing that). Of course, unambiguously identifying a
replica to update requires more than just a hostname, but that's a minor
detail. Also, I have to wonder why such a feature is needed in a
properly operating replication environment. For syncrepl, a freshly
installed consumer will naturally pull down a complete copy of the
database as its first action, so "initialization" should not be an
issue. And if the replication mechanism is doing its job correctly, then
"re-initialization" is not an issue.
There _is_ an option to initialize a replica using binary indices, but
you have to
move the data yourself (with ftp or whatever).
I suppose that could save some time. This is something I've been
wondering about (in the context of "centipede") as well, coming up with
a portable representation of an index that can be easily distributed.
This would allow sparse replicas to fully evaluate search filters
locally, then request specific matching entries from the remote masters.
(The issue is most apparent in our translucent overlay, but probably has
wider application.) If you represent an index as a set of key/value
pairs, it's pretty obvious that the values must be entryUUIDs, but what
would the keys be? Just AVAs? I guess the problem is that a portable
representation would be just as voluminous as the original entry data,
thus defeating some of the point of a "sparse" replica.
-- Howard Chu
Chief Architect, Symas Corp. Director, Highland Sun
Symas: Premier OpenSource Development and Support