Openldap replication over XMPP

[kalyan <skalyanasundaram@novell.com>]

Hi,

I would like to share some simple idea and find out similar interest.
This is about the ldap replication using the existing xmpp network.
Every server is considered as a single user entity. There is no modes
like refresh or refreshonly or delete phase or current phase instead
every servers acts as receiver or sender at the same time. Its also
easily possible to make roles of the server to read only or master of
that replica and changing them on the fly. 

This idea of replicating ldap data over xmpp is inspired from Isode's 
sodium sync where the  replication is done through series of mail 
exchange and XEP-0072 soap over xmpp protocol. 

The rough idea is,
Each server will have its own jid (Jabber id) and its password mentioned
in slapd.conf. When the server comes if possible or if replication is
required it logins to the jabber server like a normal user. The server 
becomes active.

In slapd.conf each database will mention its replicated or not. After
server logins to the jabber server, during db_open
the server will join the MUC (multi user chat). The room name is same as
the suffix of the database. The other servers who has the same copy of
the database will join the same MUC. 

Then Each server will negotiate with their contextCSN? to find out the 
need to pull the data. (It needs a better way). The MUC can be used as a
command stream to other servers. It would help to find out which servers
are online right now which can do replication or which servers are  busy
right now and cant do replication. (I don't know yet on how to figure out
the offline servers. We can have an attribute which holds the list of JID
of the servers who are all holding the copy of the DIT and this attribute
can be synced like any other attribute and the servers from the list that
not appearing as online can be considered as offline servers).

Now we negotiated, we can send receive signals with the other server.
Though its not technically very clear, its possible to achieve maximum
things. The real data transfer can be done multiple ways. Like the
sodium sync we can generate the ldif file and do a jingle file transfer
to other servers. Or during the onreponse event catch the event and if its
add/modify/delete/moddn generate the changes as xml elements for each
entry and either send it through the MUC itself or each server opens a pubsub
(publish-subscribe) node and publish their changes. The other remaining
server who are all holding the copy will listen to the node get their updates. All the
server plays the role of receiver and sender simultaneously. 
Once the offline servers are identified the updates can be sent over
offline message so that when the server comes up they need not pull
everything, its just has to process the offline messages.  

(Only the required content of Modifications* strcuture orm_modlist
or ora_modlist is sent)

MUC also has the role of Administrator, Owner, Participant, based on the
server roles we can decide server holds master copy or read write copy
but again this needs to be stored insome attribute. This helps regulating
the changes from one server to another, for instance the updates from read
only server can be denied. Change of the role on the fly might be possible.

Since its all either xml or ldif data the other directory server can
talk to openldap as well. The next problem is schema, we could sync the
entire cn=subschema as well. Or whenever the updates goes for an entry,
the corresponding schema information also can be sent over for verification. 
Or simply the receiving server tries to update their DIT and ignore if there
is a failure? 

This is it for now.

There are lot of place it needs a decision. A XEP is needed to define
the format of each command and data. I do have some working code (may be
the worst code). I am writing it as a overlay per database basis.
It logins to the xmpp server and join the MUC and creates the pubsub
node and other server starts listening to the node. Right now I am
trying to convert the data to xml and publish on the node.

I wanted to find out does this idea make any sense at all to someone. I
hope someone finds it interesting. :)

I know I have missed a lot of technical points, its because I dont know
them yet deep enough or I haven't thought about it. It would be nice if the 
discussion could help brushing those gray areas.

Thanks, 
 -Kalyan