[Date Prev][Date Next] [Chronological] [Thread] [Top]

RE: MDB data replication issues

I checked the debugging logs.
Using this command: ldapsearch -H ldap://xx.xx.xx.xx:2016 -LLL -x -s base -b "my-domain,dc=com" contextCSN

I found that contextCSN of the Server (Master 1) is coming as below:
contextCSN: 20160704065252.267837Z#000000#001#000000

and that of replica (Master 2) is:
contextCSN: 20160704065252.267837Z#000000#001#000000

I have some entries in the Server (Master 1) with entryCSN values as given below:
1. entryCSN: 20160704061459.015144Z#000000#000#000000
2. entryCSN: 20160704061459.041359Z#000000#000#000000
3. entryCSN: 20160704061459.052873Z#000000#000#000000

I added these entries using slapadd on Server (Master 1)
These entries don't get replicated on Replica (Master 2) , though contextCSN value is greater than these three entryCSN values.

I have tries starting Replica (Master 2) slapd with option -c "rid"
But it still doesn't get these entries.

Please help in understanding how can I get these entries replicated on my Replica (Master 2) from Server (Master 1).

Kind Regards,
Gurjot Kaur


-----Original Message-----
From: Bastian Tweddell [mailto:b.tweddell@fz-juelich.de]
Sent: Friday, July 01, 2016 6:34 PM
To: Quanah Gibson-Mount <quanah@zimbra.com>
Cc: Gurjot Kaur <gurjot.kaur@aricent.com>; openldap-technical@openldap.org
Subject: Re: MDB data replication issues

On 29Jun16 10:04-0700, Quanah Gibson-Mount wrote:
> --On Wednesday, June 29, 2016 4:21 PM +0200 Bastian Tweddell
> <b.tweddell@fz-juelich.de> wrote:
>
> > On 27Jun16 09:16-0700, Quanah Gibson-Mount wrote:
> > > --On Monday, June 27, 2016 2:00 PM +0000 Gurjot Kaur
> > > <gurjot.kaur@aricent.com> wrote:
> > >
> > > > Thanks Bastian for your response.
> > > >
> > > > But I want to clear that I don't use slapadd while slapd is running. I
> > > > have LDIF data which needs to be imported in the new Multimaster
> > > > servers. That's why I used slapadd. When I did the import (slapadd)
> > > > with the big data, it doesn't get replicated. To check the problems I
> > > > did the import with single entry only.
> > >
> > > It is not now, nor has it ever been, supported to do an offline add via
> > > slapadd on a master, and then magically expect that to replicate out to a
> > > consumer via syncrepl.  Syncrepl is an active protocol (it replicates the
> > > changes that occur while slapd is running).  If you are going to offline
> > > modify slapd on the master, you will have to do the same thing on the
> > > replicas, OR reload them via slapadd, OR trigger a refresh manually on
> > > the replicas.
> >
> > How to trigger a refresh on a replica?
> >
> > One question:
> >
> >   My disaster recovery (and sandbox initialization) is using slapadd to
> >   bring up a fresh instance of my ldap environment. Technically:
> >
> >     - bring up n replicated slapds with an empty db
> >     - stop them
> >     - slapadd the db dump into A
> >     - start A
> >     - start B..
> >
> >   That always worked. By now, I have to say.
> >   I even use slapadd without -w.
> >
> >   Do I understand correctly, that I should prefer the use of ldapadd?
> >   Does ldapadd also support operational and user attributes?
>
> You should just slapadd A & B at the same time.  It's the most efficient way
> to load an OpenLDAP Server.

I would like to elaborate a bit more on that.

>From the manual [1]:
--- paste: 18.1.1.2. Syncrepl Details  ---
[...]
Note that at startup time, if the provider is unable to read a
contextCSN from the suffix entry, it will scan the entire database to
determine the value, and this scan may take quite a long time on a large
database. When a contextCSN value is read, the database will still be
scanned for any entryCSN values greater than it, to make sure the
contextCSN value truly reflects the greatest committed entryCSN in the
database.
[...]
--- eop ---

  1: http://www.openldap.org/doc/admin24/replication.html#LDAP%20Sync%20Replication

A bit more detail about the disaster recovery method I (plan to) use:
 - recreate two slapds A and B (mirrormode)
 - the slapdump file of my data base contains contextCSN, entryCSN
   attributes for all entries
 - slapadd is used without -w to db of slapd A
 - Starting slapd A
 - After that, slapd B is started which has a complete empty DB, even
   without an entry for the suffix
 - A complete synchronization takes place from A to B

After reading the manual again, when B comes up it does not have any
idea in which state the provider engine is, so it has nothing to do. The
consumer facility then fetches all entries from the provider in A.


Further from the manual [2]:
--- paste: 18.3.1.4. Start the provider and the consumer slapd ---
[...]
contextCSN is automatically generated as needed: it might be originally
contained in the LDIF file, generated by slapadd (8), generated upon
changes in the context, or generated when the first LDAP Sync search
arrives at the provider
[...]
--- eop ---

  2: http://www.openldap.org/doc/admin24/replication.html#Syncrepl

This part lets me believe, that even if I do not have contextCSN in my
db in A these would be created on the first sync requested from B.


  - What is the designed behavior for the consumer engine if the db is
    completely empty?

  - How to trigger a refresh on the consumer?
    I have never done that, but I can imagine, that is done by slapd -c?


About the question from Gurjot, where he misses entries on the
replicated side. What about, after he slapadds his entries to the db,
remove the contextCSN entry of his suffix. When slapd starts up, the
syncrepl engine would scan the entire DB.

In any case, my assumptions rely on the admin manual which might not
reflect the code base. But from what I read, I still feel I did it the
right way all by now. So Quanah, if you find some time, please let me
know what I am missing and why my disaster recovery is unsafe.


Cheers,
--
Bastian Tweddell               Juelich Supercomputing Centre
phone: +49 (2461) 61-6586          HPC in Neuroscience
"DISCLAIMER: This message is proprietary to Aricent and is intended solely for the use of the individual to whom it is addressed. It may contain privileged or confidential information and should not be circulated or used for any purpose other than for what it is intended. If you have received this message in error, please notify the originator immediately. If you are not the intended recipient, you are notified that you are strictly prohibited from using, copying, altering, or disclosing the contents of this message. Aricent accepts no responsibility for loss or damage arising from the use of the information transmitted by this email including damage from virus."