Hor Meng Yoong wrote:
Dear Howard Thank you for your great effort in creating lmdb. I am looking a solution that can sync multiple lmdb slaves to one lmdb master. Changes from the each lmdb slave are also updated to other lmdb slaves via the lmdb master. Any up-to-date, up-and-running lmdb slaves can vote among themsleves to become he next master in the event that the master fails. To prevent islanding and multiple masters, vote is only allowed if there are at least a minimum numbers (configurable) of interconnected lmdb slaves. To improve efficiency, data protection, a master table inside the lmdb master decides which lmdb slaves should be updated, such as frequency of update, tables to be updated. Drastic database operation (e.g., table creation and deletion) shall only be initiated from he master. Bases on current design architecture and future design plan, what is the best way for this to be implemented.
Replication is a non-goal for LMDB. In OpenLDAP we already have a multimaster replication protocol built on LDAP; we don't need replication in the database engine. Projects that aim to build replication as an LMDB feature are misguided.
An embedded database engine's only job is to store and retrieve bytes using local storage. Nothing else. What you're talking about belongs at a higher abstraction layer and has nothing to do with the DB engine. Whatever system you are trying to design is orthogonal to the storage engine.
I suggest you look into Dynomite using memcacheDB. http://techblog.netflix.com/2014/11/introducing-dynomite.html https://github.com/LMDB/memcachedbNote that any time you start dealing with network services, you immediately incur the need for authentication, authorization, and privacy controls. Please don't be like the rest of the NoSQL crowd and only realize this after the fact. You will need a proper network protocol, one that's designed to be efficient and secure. You'll also need a decent data model for serializing data over the wire. Most of what you'll need has already been invented and formalized in a spec - we call it LDAP.
Besr regards Hor Meng
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/