[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: Some interesting thoughts

David Boreham wrote:

I suppose that could save some time. This is something I've been wondering about (in the context of "centipede") as well, coming up with a portable representation of an index that can be easily distributed. This would allow sparse replicas to fully evaluate search filters locally, then request specific matching entries from the remote masters. (The issue is most apparent in our translucent overlay, but probably has wider application.) If you represent an index as a set of key/value pairs, it's pretty obvious that the values must be entryUUIDs, but what would the keys be? Just AVAs? I guess the problem is that a portable representation would be just as voluminous as the original entry data, thus defeating some of the point of a "sparse" replica.

Also watch out for big/little endian issues in the BDB metadata.

Interesting, the BDB docs mention that the database files are byte-order independent, but they didn't talk about anything else. I tar'd up a database directory from a Solaris/Sparc system (using BDB 4.2.52) and extracted it onto an x86 box (using BDB 4.3.27), then ran db_recover on the x86 system. It complained about the environment version number (which was obviously mismatched) and the magic number of the log file, (which was clearly in swapped order) but after deleting the log file the database (and all associated indices) was fully usable. This is pretty much what I expected, as the docs state that database file formats didn't change between 4.2 and 4.3, but the environment obviously did. I didn't expect a byte-ordering problem in the log files though. So it would be possible to take an offline backup and use it directly on another machine (with the same word size) but not a hot backup, since you need the log files for the hot backup.

 -- Howard Chu
 Chief Architect, Symas Corp.       Director, Highland Sun
 http://www.symas.com               http://highlandsun.com/hyc
 Symas: Premier OpenSource Development and Support