[Date Prev][Date Next]
Clusters and scaling, again
It's been a little over 4 years since we worked with MySQL using their
NDBCluster engine to develop back-ndb, in the hopes of developing a backend
that could leverage horizontal scaling systems. After Oracle acquired MySQL
(via acquisition of Sun) they terminated the project.
NDB had some promise, although reports from the field indicate that folks felt
it was too difficult to configure and deploy. It appears that NDB has not
gained much traction anywhere in the past few years, and with Oracle cutting
off our access to MySQL/NDB developers, back-ndb was effectively killed.
In OpenLDAP 2.5 we'll be trying again, this time using HyperDex.
HyperDex has a number of desirable properties - trivial deployment, automatic
self-balancing of data, guaranteed data durability, configurable fault
tolerance... There is an extension for it providing distributed transaction
support as well (which I haven't looked into yet). It's also quite fast, much
faster than any of the other commonly known NoSQL systems out there. It's also
reliable by design, unlike systems like MongoDB (whose only consistency is
that it consistently loses data...)
I've been playing with it off and on for the past several months. I've also
written an LMDB backend for it, available on github.
So using HyperDex for an OpenLDAP backend has some greater appeal, because it
would be OpenLDAP code at both ends of the stack.
If you're interested in working with NoSQL and large scale databases with
OpenLDAP you should definitely take a look at this. I suspect it will be
easier to write a slapd backend for this API than for NDB, particularly since
it's free of the rigid column layouts that NDB/MySQL required.
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/