[Date Prev][Date Next] [Chronological] [Thread] [Top]

Issue with replication

We are running openldap in cluster mode with MDB setup, and we started second cluster after some time and we observe that data is non synch between those 2 servers.
So how do we synchronize the data.

> On Sep 7, 2018, at 8:00 AM, openldap-technical-request@openldap.org wrote:
> 
> Send openldap-technical mailing list submissions to
> 	openldap-technical@openldap.org
> 
> To subscribe or unsubscribe via the World Wide Web, visit
> 	http://www.openldap.org/lists/mm/listinfo/openldap-technical
> or, via email, send a message with subject or body 'help' to
> 	openldap-technical-request@openldap.org
> 
> You can reach the person managing the list at
> 	openldap-technical-owner@openldap.org
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of openldap-technical digest..."
> 
> 
> Send openldap-technical mailing list submissions to
>       openldap-technical@openldap.org
> When replying, please edit your Subject: header so it is more specific than "Re: openldap-technical digest..."
> 
> Today's Topics:
> 
>   1. Replication issue? Data is different between master and
>      consumer with same entryCSNs (Dave Steiner)
>   2. olcSecurity: tls=1 and olcLocalSSF= : what value should I
>      use? (Jean-Francois Malouin)
>   3. Re: olcSecurity: tls=1 and olcLocalSSF= : what value should I
>      use? (Quanah Gibson-Mount)
>   4. Re: Replication issue? Data is different between master and
>      consumer with same entryCSNs (Frank Swasey)
>   5. Re: Replication issue? Data is different between master and
>      consumer with same entryCSNs (Quanah Gibson-Mount)
>   6. Re: olcSecurity: tls=1 and olcLocalSSF= : what value should I
>      use? (Jean-Francois Malouin)
>   7. Re: Replication issue? Data is different between master and
>      consumer with same entryCSNs (Dave Steiner)
> 
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Wed, 5 Sep 2018 16:49:44 -0400
> From: Dave Steiner <steiner@rutgers.edu>
> To: openldap-technical@openldap.org
> Subject: Replication issue? Data is different between master and
> 	consumer with same entryCSNs
> Message-ID: <129e3614-50fe-ba15-4d4b-5f94d14abcd9@oit.rutgers.edu>
> Content-Type: text/plain; charset="utf-8"; Format="flowed"
> 
> 
> I've been noticing various data discrepancies between our LDAP master and LDAP 
> consumers.? We are running OpenLDAP v2.4.44.? We have two masters running 
> "mirromode TRUE" and all updates go through a VIP that points to the first one 
> unless it's not available (doesn't happen very often except for during patches 
> and restarts). We have 13 consumers that replicate through that same VIP.
> 
> Here's an example of our syncrepl for a client:
> 
> syncrepl rid=221
> ? type=refreshAndPersist
> ? schemachecking=on
> ? provider="ldap://ldapmastervip.rutgers.edu/";
> ? bindmethod=sasl
> ? saslmech=EXTERNAL
> ? starttls=yes
> ? tls_reqcert=demand
> ? tls_protocol_min="3.1"
> ? searchbase="dc=rutgers,dc=edu"
> ? attrs="*,+"
> ? retry="10 10 20 +"
> ? logbase="cn=accesslog"
> ? logfilter="(&(objectClass=auditWriteObject)(reqResult=0))"
> ? syncdata=accesslog
> ? network-timeout=30
> ? keepalive=180:3:60
> 
> I check the contextCSN attributes on all the instances every day and they are 
> all in sync (except during any major changes, of course). But I occasionally 
> notice discrepancies in the data.... even though the contextCSNs and entryCSNs 
> are the same.? For example (note hostnames have been changed):
> 
> $ ldapsearch ... -H ldap://ldapmaster.rutgers.edu uid=XXXX postalAddress 
> createTimestamp modifyTimestamp entryCSN
> dn: uid=XXXX,ou=People,dc=rutgers,dc=edu
> createTimestamp: 20121220100700Z
> postalAddress: Business And Science Bldg$227 Penn Street$Camden, NJ 081021656
> entryCSN: 20180505002024.083133Z#000000#001#000000
> modifyTimestamp: 20180505002024Z
> 
> $ ldapsearch ... -H ldap://ldapconsumer3.rutgers.edu uid=XXXX postalAddress 
> createTimestamp modifyTimestamp entryCSN
> dn: uid=XXXX,ou=People,dc=rutgers,dc=edu
> createTimestamp: 20121220100700Z
> postalAddress: BUSINESS AND SCIENCE BLDG$227 PENN STREET$CAMDEN, NJ 081021656
> entryCSN: 20180505002024.083133Z#000000#001#000000
> modifyTimestamp: 20180505002024Z
> 
> So I'm trying to figure out why this happens (config issue, bug, ???) and 
> second, if I can't use the contextCSN to report that everything is fine, what 
> else can I do besides trying to compare ldif dumps.
> 
> thanks,
> ds
> -- 
> Dave Steiner steiner@rutgers.edu
> IdM, Enterprise Application Services ?? ASB101; 848.445.5433
> Rutgers University, Office of Information Technology
> 
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://www.openldap.org/lists/openldap-technical/attachments/20180905/4e3872b4/attachment.html>
> 
> ------------------------------
> 
> Message: 2
> Date: Thu, 6 Sep 2018 12:40:05 -0400
> From: Jean-Francois Malouin <Jean-Francois.Malouin@bic.mni.mcgill.ca>
> To: OpenLDAP Technical <openldap-technical@openldap.org>
> Subject: olcSecurity: tls=1 and olcLocalSSF= : what value should I
> 	use?
> Message-ID: <20180906164005.GB8166@bic.mni.mcgill.ca>
> Content-Type: text/plain; charset=us-ascii
> 
> Sorry if this is long and naive, I'm making my way with OpenLDAP.
> 
> I have this annoying problem of local access over ldapi:/// of a configured
> mdb database using its rootDN.
> 
> Some details:
> 
> (I typically use ldapvi to access/modify/edit config as I'm an old wolf with vi
> hard-wired in my brain!)
> 
> (Same could be done using native OpenLDAP utilities ldapadd/search/delete/etc:
> just replace the ldapvi '-h' option with '-H' to specify the protocol/host/port).
> 
> Binding using EXTERNAL mech over local ldapi:/// works correctly for 'cn=config'.
> For example, here I made a mod to olcLogLevel for 'cn=config':
> 
> ~# ldapvi -Y EXTERNAL -h ldapi:/// -b 'cn=config'
> SASL/EXTERNAL authentication started
> SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth
> SASL SSF: 0
>     24 entries read                                                                                                                            
> add: 0, rename: 0, modify: 1, delete: 0
> Action? [yYqQvVebB*rsf+?] y
> Done.
> 
> Server logs for slapd show the binding with ssf=71:
> 
> Sep  6 11:40:52 slapd[677]: conn=48667 fd=17 ACCEPT from PATH=/var/run/slapd/ldapi (PATH=/var/run/slapd/ldapi)
> Sep  6 11:40:52 slapd[677]: conn=48667 op=0 BIND dn="" method=163
> Sep  6 11:40:52 slapd[677]: conn=48667 op=0 BIND authcid="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth" authzid="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth"
> Sep  6 11:40:52 slapd[677]: conn=48667 op=0 BIND dn="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth" mech=EXTERNAL sasl_ssf=0 ssf=71
> 
> 
> However for the configured mdb database 'olcDatabase={1}mdb,cn=config' I have set 
> 
>    olcSecurity: tls=1
> 
> to force binding with StartTLS. Here the relevant config piece for it:
> ('--out' makes ldapvi behave like ldapsearch).
> 
> ~# ldapvi --out -Y EXTERNAL -h ldapi:/// -b 'olcDatabase={1}mdb,cn=config'
> SASL/EXTERNAL authentication started
> SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth
> SASL SSF: 0
> 
> dn: olcDatabase={1}mdb,cn=config
> objectClass: olcDatabaseConfig
> objectClass: olcMdbConfig
> olcDatabase: {1}mdb
> olcDbDirectory: /var/lib/ldap
> olcSuffix: dc=example,dc=com
> olcRootDN: cn=admin,dc=example,dc=com
> olcRootPW: {SSHA}XXXXXXXXXXXXXXXXXXXX
> olcSecurity: tls=1
> ...
> 
> However this setting prohibits me from binding to it using ldapi:/// with
> EXTERNAL mech with its rootDN 'cn=admin,dc=example,dc=com' as I then get the
> message:
> 
> ~# ldapsearch -LLL -Y EXTERNAL -H ldapi:/// -D 'cn=admin,dc=example,dc=com' -b 'dc=example,dc=com'
> 
> SASL/EXTERNAL authentication started
> SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth
> SASL SSF: 0
> Confidentiality required (13)
> Additional information: TLS confidentiality required
> 
> 
> I can however do a simple bind over StartTLS with the rootDN of the database
> either over localhost or a remote client:
> 
> ~# ldapsearch -LLL -Z -x -w xxxxxxxx -H ldap://localhost -D 'cn=admin,dc=example,dc=com' -b 'dc=example,dc=com'
> 
> slapd logs show:
> 
> Sep  6 11:54:40 slapd[677]: conn=48699 fd=17 ACCEPT from IP=127.0.0.1:53542 (IP=0.0.0.0:389)
> Sep  6 11:54:40 slapd[677]: conn=48699 op=0 EXT oid=1.3.6.1.4.1.1466.20037
> Sep  6 11:54:40 slapd[677]: conn=48699 op=0 STARTTLS
> Sep  6 11:54:40 slapd[677]: conn=48699 op=0 RESULT oid= err=0 text=
> Sep  6 11:54:40 slapd[677]: conn=48699 fd=17 TLS established tls_ssf=256 ssf=256
> Sep  6 11:54:40 slapd[677]: conn=48699 op=1 BIND dn="cn=admin,dc=example,dc=com" method=128
> Sep  6 11:54:40 slapd[677]: conn=48699 op=1 BIND dn="cn=admin,dc=example,dc=com" mech=SIMPLE ssf=0
> 
> So ssf=265...
> 
> I guess I need to modify either 'olcSecurity: tls=1' in the database config or
> add/insert the proper value for 'olcLocalSSF=' in the cn=config. What value
> should I use in order to still force StartTLS over simple binding and allow
> read/write/modify local access on the ldapi:/// listener.
> 
> Regards!
> jf
> 
> 
> 
> ------------------------------
> 
> Message: 3
> Date: Thu, 06 Sep 2018 11:36:33 -0700
> From: Quanah Gibson-Mount <quanah@symas.com>
> To: Jean-Francois Malouin <Jean-Francois.Malouin@bic.mni.mcgill.ca>,
> 	OpenLDAP Technical <openldap-technical@openldap.org>
> Subject: Re: olcSecurity: tls=1 and olcLocalSSF= : what value should I
> 	use?
> Message-ID: <4CC3F9658701FD268DFEC4C0@[192.168.1.10]>
> Content-Type: text/plain; charset=us-ascii; format=flowed
> 
> --On Thursday, September 06, 2018 1:40 PM -0400 Jean-Francois Malouin 
> <Jean-Francois.Malouin@bic.mni.mcgill.ca> wrote:
> 
>> I guess I need to modify either 'olcSecurity: tls=1' in the database
>> config or add/insert the proper value for 'olcLocalSSF=' in the
>> cn=config. What value should I use in order to still force StartTLS over
>> simple binding and allow read/write/modify local access on the ldapi:///
>> listener.
> 
> Hello,
> 
> Just set:
> 
> olcSecurity: ssf=1
> 
> that will allow either to work as *some* SSF level is then required.
> 
> As long as you have tls=X, then it will always require TLS, regardless of 
> what the LocalSSF setting is configured to be.
> 
> --Quanah
> 
> 
> --
> 
> Quanah Gibson-Mount
> Product Architect
> Symas Corporation
> Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
> <http://www.symas.com>
> 
> 
> 
> 
> ------------------------------
> 
> Message: 4
> Date: Thu, 6 Sep 2018 18:39:03 +0000
> From: Frank Swasey <Frank.Swasey@uvm.edu>
> To: Dave Steiner <steiner@rutgers.edu>
> Cc: "openldap-technical@openldap.org"
> 	<openldap-technical@openldap.org>
> Subject: Re: Replication issue? Data is different between master and
> 	consumer with same entryCSNs
> Message-ID: <8ACC8FD3-D160-4ACC-B933-9A8A5409FBBA@uvm.edu>
> Content-Type: text/plain; charset="us-ascii"
> 
> 
> 
>> On Sep 5, 2018, at 16:49, Dave Steiner <steiner@rutgers.edu> wrote:
>> 
>> 
>>  But I occasionally notice discrepancies in the data.... even though the contextCSNs and entryCSNs are the same.  For example (note hostnames have been changed):
>> 
>> $ ldapsearch ... -H ldap://ldapmaster.rutgers.edu uid=XXXX postalAddress createTimestamp modifyTimestamp entryCSN
>> postalAddress: Business And Science Bldg$227 Penn Street$Camden, NJ 081021656
>> 
>> $ ldapsearch ... -H ldap://ldapconsumer3.rutgers.edu uid=XXXX postalAddress createTimestamp modifyTimestamp entryCSN
>> dn: uid=XXXX,ou=People,dc=rutgers,dc=edu
>> postalAddress: BUSINESS AND SCIENCE BLDG$227 PENN STREET$CAMDEN, NJ 081021656
> 
> You do realize that as far as LDAP is concerned, those two postalAddress values are identical.  The matching rules for postalAddress are case insensitive.
> 
> Assuming that the mixed case one is what you prefer and the upper case one is what you started with - one has to ask, how was it changed on the one server and make certain that it was not done in a way that would not be sent through the syncrepl process.
> 
> - Frank
> 
> 
> ------------------------------
> 
> Message: 5
> Date: Thu, 06 Sep 2018 11:48:06 -0700
> From: Quanah Gibson-Mount <quanah@symas.com>
> To: Dave Steiner <steiner@rutgers.edu>,
> 	openldap-technical@openldap.org
> Subject: Re: Replication issue? Data is different between master and
> 	consumer with same entryCSNs
> Message-ID: <88F55B570F9FE65911DAA8EE@[192.168.1.10]>
> Content-Type: text/plain; charset=us-ascii; format=flowed
> 
> --On Wednesday, September 05, 2018 5:49 PM -0400 Dave Steiner 
> <steiner@rutgers.edu> wrote:
> 
> Hi Dave,
> 
> 
>> $ ldapsearch ... -H ldap://ldapmaster.rutgers.edu uid=XXXX postalAddress
>> createTimestamp modifyTimestamp entryCSN
>> dn: uid=XXXX,ou=People,dc=rutgers,dc=edu
>> createTimestamp: 20121220100700Z
>> postalAddress: Business And Science Bldg$227 Penn Street$Camden, NJ
>> 081021656
>> entryCSN: 20180505002024.083133Z#000000#001#000000
>> modifyTimestamp: 20180505002024Z
>> 
>> $ ldapsearch ... -H ldap://ldapconsumer3.rutgers.edu uid=XXXX
>> postalAddress createTimestamp modifyTimestamp entryCSN
>> dn: uid=XXXX,ou=People,dc=rutgers,dc=edu
>> createTimestamp: 20121220100700Z
>> postalAddress: BUSINESS AND SCIENCE BLDG$227 PENN STREET$CAMDEN, NJ
>> 081021656
>> entryCSN: 20180505002024.083133Z#000000#001#000000
>> modifyTimestamp: 20180505002024Z
> 
> Those entries appear identical to me. I assume you believe they are not 
> identical because the case doesn't match.  However, postalAddress uses 
> caseIgnore normalization rules, so they are in fact identical as far as 
> LDAP is concerned.
> 
> Also, given the entry was created in 2012 and there's no indication of when 
> the postalAddress attribute was modified, it's impossible to determine 
> when/how the divergence in capitalization occurred.  You also don't provide 
> any information about the underlying database in use, which could be very 
> relevant, history wise.
> 
> 
>> So I'm trying to figure out why this happens (config issue, bug, ???) and
>> second, if I can't use the contextCSN to report that everything is fine,
>> what else can I do besides trying to compare ldif dumps.
> 
> So far, everything appears fine from a technical standpoint (the values 
> match per LDAP SYNTAX rules).
> 
> I would suggest the following if you would like the case to match:
> 
> dn: ...
> changetype: modify
> replace: postalAddress
> postalAddress: ...
> 
> That should correct the value on the providers and consumers to match case.
> 
> Warm regards,
> Quanah
> 
> 
> 
> --
> 
> Quanah Gibson-Mount
> Product Architect
> Symas Corporation
> Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
> <http://www.symas.com>
> 
> 
> 
> 
> ------------------------------
> 
> Message: 6
> Date: Thu, 6 Sep 2018 15:59:54 -0400
> From: Jean-Francois Malouin <Jean-Francois.Malouin@bic.mni.mcgill.ca>
> To: Quanah Gibson-Mount <quanah@symas.com>
> Cc: OpenLDAP Technical <openldap-technical@openldap.org>
> Subject: Re: olcSecurity: tls=1 and olcLocalSSF= : what value should I
> 	use?
> Message-ID: <20180906195954.GC8166@bic.mni.mcgill.ca>
> Content-Type: text/plain; charset=us-ascii
> 
> Hi,
> 
> * Quanah Gibson-Mount <quanah@symas.com> [20180906 14:36]:
>> --On Thursday, September 06, 2018 1:40 PM -0400 Jean-Francois
>> Malouin <Jean-Francois.Malouin@bic.mni.mcgill.ca> wrote:
>> 
>>> I guess I need to modify either 'olcSecurity: tls=1' in the database
>>> config or add/insert the proper value for 'olcLocalSSF=' in the
>>> cn=config. What value should I use in order to still force StartTLS over
>>> simple binding and allow read/write/modify local access on the ldapi:///
>>> listener.
>> 
>> Hello,
>> 
>> Just set:
>> 
>> olcSecurity: ssf=1
>> 
>> that will allow either to work as *some* SSF level is then required.
>> 
>> As long as you have tls=X, then it will always require TLS,
>> regardless of what the LocalSSF setting is configured to be.
> 
> Thank you for the pointer!
> 
> jf
> 
>> 
>> --Quanah
>> 
>> 
>> --
>> 
>> Quanah Gibson-Mount
>> Product Architect
>> Symas Corporation
>> Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
>> <http://www.symas.com>
> 
> 
> 
> ------------------------------
> 
> Message: 7
> Date: Thu, 6 Sep 2018 16:43:49 -0400
> From: Dave Steiner <steiner@rutgers.edu>
> To: Quanah Gibson-Mount <quanah@symas.com>, Dave Steiner
> 	<steiner@rutgers.edu>, openldap-technical@openldap.org
> Subject: Re: Replication issue? Data is different between master and
> 	consumer with same entryCSNs
> Message-ID: <4889f305-d336-a139-687b-f1686903ce4b@oit.rutgers.edu>
> Content-Type: text/plain; charset=utf-8; format=flowed
> 
> 
> 
> On 9/6/2018 2:48 PM, Quanah Gibson-Mount wrote:
>> --On Wednesday, September 05, 2018 5:49 PM -0400 Dave Steiner 
>> <steiner@rutgers.edu> wrote:
>> 
>> Hi Dave,
>> 
>> 
>>> $ ldapsearch ... -H ldap://ldapmaster.rutgers.edu uid=XXXX postalAddress
>>> createTimestamp modifyTimestamp entryCSN
>>> ?dn: uid=XXXX,ou=People,dc=rutgers,dc=edu
>>> ?createTimestamp: 20121220100700Z
>>> ?postalAddress: Business And Science Bldg$227 Penn Street$Camden, NJ
>>> 081021656
>>> ?entryCSN: 20180505002024.083133Z#000000#001#000000
>>> ?modifyTimestamp: 20180505002024Z
>>> 
>>> $ ldapsearch ... -H ldap://ldapconsumer3.rutgers.edu uid=XXXX
>>> postalAddress createTimestamp modifyTimestamp entryCSN
>>> ?dn: uid=XXXX,ou=People,dc=rutgers,dc=edu
>>> ?createTimestamp: 20121220100700Z
>>> ?postalAddress: BUSINESS AND SCIENCE BLDG$227 PENN STREET$CAMDEN, NJ
>>> 081021656
>>> ?entryCSN: 20180505002024.083133Z#000000#001#000000
>>> ?modifyTimestamp: 20180505002024Z
>> 
>> Those entries appear identical to me. I assume you believe they are not 
>> identical because the case doesn't match.? However, postalAddress uses 
>> caseIgnore normalization rules, so they are in fact identical as far as LDAP 
>> is concerned.
>> 
>> Also, given the entry was created in 2012 and there's no indication of when 
>> the postalAddress attribute was modified, it's impossible to determine 
>> when/how the divergence in capitalization occurred.? You also don't provide 
>> any information about the underlying database in use, which could be very 
>> relevant, history wise.
> 
> Ok, bad example... Here's another example that's rather different:
> 
> $ searchn -H ldap://ldapmaster.rutgers.edu uid=XXXX mail createTimestamp 
> modifyTimestamp entryCSN
> dn: uid=XXXX,ou=People,dc=rutgers,dc=edu
> createTimestamp: 20180504140010Z
> mail: xxxxxxxxx.xxxx@rutgers.edu
> entryCSN: 20180831205041.549496Z#000000#001#000000
> modifyTimestamp: 20180831205041Z
> 
> $ searchn -H ldap://ldapconsumer1.rutgers.edu uid=XXXX mail createTimestamp 
> modifyTimestamp entryCSN
> dn: uid=XXXX,ou=People,dc=rutgers,dc=edu
> createTimestamp: 20180504140010Z
> mail: yyyyy@rutgers.edu
> entryCSN: 20180831205041.549496Z#000000#001#000000
> modifyTimestamp: 20180831205041Z
> 
> 
>> 
>> 
>>> So I'm trying to figure out why this happens (config issue, bug, ???) and
>>> second, if I can't use the contextCSN to report that everything is fine,
>>> what else can I do besides trying to compare ldif dumps.
>> 
>> So far, everything appears fine from a technical standpoint (the values match 
>> per LDAP SYNTAX rules).
>> 
>> I would suggest the following if you would like the case to match:
>> 
>> dn: ...
>> changetype: modify
>> replace: postalAddress
>> postalAddress: ...
>> 
>> That should correct the value on the providers and consumers to match case.
>> 
>> Warm regards,
>> Quanah
>> 
>> 
>> 
>> -- 
>> 
>> Quanah Gibson-Mount
>> Product Architect
>> Symas Corporation
>> Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
>> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.symas.com&amp;data=02%7C01%7Csteiner%40oit.rutgers.edu%7Ca13f8a75c55b4fdbdb6308d614294c60%7Cb92d2b234d35447093ff69aca6632ffe%7C1%7C0%7C636718564940835797&amp;sdata=pVe%2FLnPDf4Civ13qlr1pXPCb0icy%2BzTZUEAcnViaTCo%3D&amp;reserved=0> 
>> 
>> 
> 
> 
> 
> 
> ------------------------------
> 
> Subject: Digest Footer
> 
> _______________________________________________
> openldap-technical mailing list
> openldap-technical@openldap.org
> http://www.openldap.org/lists/mm/listinfo/openldap-technical
> 
> 
> ------------------------------
> 
> End of openldap-technical Digest, Vol 130, Issue 3
> **************************************************