[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: Two contextCSNs



Hallvard B Furuseth wrote:
Peter Mogensen writes:
I'm trying to understand why changes made to SID 1 in my mirror set while SID 2 is down does not get propagated to SID 2 when it comes up.

Maybe your mirror is configured with refreshAndPersist mode and you have
not specified a retry interval?  Then thed default is 1 hour, according
to the slapd.conf manpage.

No. retry is "60 +" (1 minute from what I read).

Since no one has complained against the 8 step procedure I posted, I will assume that it is the correct way to load an huge LDIF into an empty mirrormode setup. So since, it's not the procedure, it must be either my configuration or a bug. I'll assume it's my configuration though I suspect this message is about the same problem:
http://www.openldap.org/lists/openldap-software/200911/msg00058.html

So here's my configuration in a step-by-step sequence. I do:

* First install openldap 2.4.20 / db 4.8.24 on two debian Lenny systems.

* Set /etc/ldap/slapd.conf to this:
===================================================================
gentlehup on
pidfile         /var/run/slapd/slapd.pid
argsfile        /var/run/slapd/slapd.args
loglevel        none

tool-threads 8

# Modules
modulepath      /usr/lib/ldap
moduleload      back_hdb
moduleload      syncprov

# Schemas
include         /etc/ldap/schema/core.schema
include         /etc/ldap/schema/cosine.schema
include         /etc/ldap/schema/inetorgperson.schema

# Limits
disallow bind_anon
idletimeout 120
sizelimit 2000

# TLS/Auth
TLSCACertificateFile    /etc/ldap/ssl/ca.crt
TLSCertificateFile     /etc/ldap/ssl/server.crt
TLSCertificateKeyFile  /etc/ldap/ssl/server.nopass.key
TLSCipherSuite "NULL-SHA"

# Allow root to configure slapd via ldapi:///
TLSVerifyClient demand
authz-regexp
  "gidNumber=0\\+uidNumber=0,cn=peercred,cn=external,cn=auth"
  "cn=config"

authz-regexp
 "email=root@example.com,cn=config,ou=dev,o=example.com,st=Denmark,c=DK"
 "cn=config"

##### Mirror mode ####
serverID 1

database config

limits dn.exact="cn=config"
  time.soft=unlimited
  time.hard=unlimited
  size.soft=unlimited
  size.hard=unlimited

syncrepl rid=1
   provider=ldaps://server1.example.com:636/
   searchbase="cn=config"
   type=refreshAndPersist
   retry="60 +"
   scope=sub
   schemachecking=on
   bindmethod=sasl
   binddn="cn=config"
   saslmech="EXTERNAL"
   tls_cert=/etc/ldap/ssl/config.crt
   tls_key=/etc/ldap/ssl/config.nopass.key
   tls_cacert=/etc/ldap/ssl/ca.crt
   tls_cipher_suite="NULL-SHA"

syncrepl rid=2
   provider=ldaps://server2.example.com:636/
   searchbase="cn=config"
   type=refreshAndPersist
   retry="60 +"
   scope=sub
   schemachecking=on
   bindmethod=sasl
   binddn="cn=config"
   saslmech="EXTERNAL"
   tls_cert=/etc/ldap/ssl/config.crt
   tls_key=/etc/ldap/ssl/config.nopass.key
   tls_cacert=/etc/ldap/ssl/ca.crt
   tls_cipher_suite="NULL-SHA"

overlay syncprov
syncprov-checkpoint 100 10
syncprov-sessionlog 100
syncprov-reloadhint TRUE

mirrormode on
=====================================================

* Then, I run slaptest -f /etc/ldap/slapd.conf -F /etc/ldap/slapd.d
  to convert the above to a cn=config based setup.

* Then I start slapd on both servers.
  $  /usr/sbin/slapd -h ldapi:/// ldaps://server1.example.com:636/ \
     ldap://server1.example.com/ -g openldap -u openldap \
     -F /etc/ldap/slapd.d -4

  ... all of the above of course different wrt. server1/server2, SID 1/2

* The I load the following LDIF files on server 1 with
  $ ldapadd -YEXTERNAL -H ldapi:/// -f <LDIFFILE>
  In sequence:
==============================
dn: cn=module{0},cn=config
changetype: modify
add: olcModuleLoad
olcModuleLoad: refint
=============================
dn: cn=module{0},cn=config
changetype: modify
add: olcModuleLoad
olcModuleLoad: back_bdb
==============================
... a bunch of schemas, like:
dn: cn=evolutionperson,cn=schema,cn=config
==============================
dn: olcDatabase={1}hdb,cn=config
objectClass: olcHdbConfig
objectClass: olcDatabaseConfig
olcDatabase: hdb
olcSuffix: cn=data,dc=example,dc=com
olcRootDN: cn=config
olcDbDirectory: /var/lib/ldap/cn=data,dc=example,dc=com
olcDbMode: 0660
olcDbConfig: set_cachesize 2 0 0
olcDbConfig: set_lg_bsize 2097512
olcDbConfig: set_lg_dir /var/lib/ldap/cn=data,dc=example,dc=com-log
olcDbConfig: set_flags DB_LOG_AUTOREMOVE
olcDbConfig: set_lk_max_objects 5000
olcDbConfig: set_lk_max_locks   5000
olcDbConfig: set_lk_max_lockers 5000
olcDbCheckpoint: 1024 10
olcDbCachefree: 16
olcDbCachesize: 100000
olcDbIDLcacheSize: 300000
olcDbLinearIndex: FALSE
olcDbIndex: objectClass eq
olcDbIndex: entryUUID eq
olcDbIndex: entryCSN eq
olcDbIndex: cn eq,sub
olcDbIndex: uid eq
olcDbIndex: ou eq
olcDbIndex: o eq
olcDbIndex: givenName eq,sub
olcDbIndex: sn eq,sub
olcDbIndex: mail eq,sub
olcDbIndex: member eq
olcDbIndex: reader eq
olcDbIndex: writer eq
olcDbIndex: admin eq
olcAccess:
 to dn.base="cn=data,dc=example,dc=com"  attrs=userPassword
  by * auth
olcAccess:
 to dn.base="cn=data,dc=example,dc=com"
  by dn.base="cn=data,dc=example,dc=com" search
olcAccess:
 to dn.children="cn=data,dc=example,dc=com"
  by dn.base="cn=data,dc=example,dc=com" write
olcSyncRepl: rid=3
  provider=ldaps://server1.example.com:636/
  searchbase="cn=data,dc=example,dc=com"
  type=refreshAndPersist
  retry="60 +"
  scope=sub
  schemachecking=on
  bindmethod=sasl
  binddn="cn=config"
  saslmech="EXTERNAL"
  tls_cert=/etc/ldap/ssl/config.crt
  tls_key=/etc/ldap/ssl/config.nopass.key
  tls_cacert=/etc/ldap/ssl/ca.crt
  tls_cipher_suite="NULL-SHA"
olcSyncRepl: rid=4
  provider=ldaps://server2.example.com:636/
  searchbase="cn=data,dc=example,dc=com"
  type=refreshAndPersist
  retry="60 +"
  scope=sub
  schemachecking=on
  bindmethod=sasl
  binddn="cn=config"
  saslmech="EXTERNAL"
  tls_cert=/etc/ldap/ssl/config.crt
  tls_key=/etc/ldap/ssl/config.nopass.key
  tls_cacert=/etc/ldap/ssl/ca.crt
  tls_cipher_suite="NULL-SHA"
olcMirrorMode: TRUE
olcLimits: dn.base="cn=config"
  size.soft=unlimited
  size.hard=unlimited
  time.soft=unlimited
  time.hard=unlimited

dn: olcOverlay=syncprov,olcDatabase={1}hdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcSyncProvConfig
olcOverlay: syncprov
olcSpCheckpoint: 100 600
olcSpSessionlog: 100
olcSpReloadHint: TRUE

dn: olcOverlay=refint,olcDatabase={1}hdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcRefintConfig
olcOverlay: refint
olcRefintAttribute: member
===========================================

* All of the above gets properly replicated to server2.

* Then I take an LDIF from slapcat on slapd 2.3.30 and run:
 $ cat dump.ldif | grep -v -E '^(entryCSN:|contextCSN:)' > load_noCSN.ldif

  The data (dump.ldif) looks like this (the root object):
===================================
dn: cn=data,dc=example,dc=com
objectClass: top
objectClass: NamedObject
objectClass: dcObject
objectClass: simpleSecurityObject
cn: data
userPassword:: BASE64
structuralObjectClass: NamedObject
entryUUID: ab7d5590-3e90-102c-8c03-91e70ecd3b46
creatorsName: cn=data,dc=example,dc=com
modifiersName: cn=data,dc=example,dc=com
createTimestamp: 20071214130312Z
modifyTimestamp: 20071214130312Z
entryCSN: 20071214130312Z#000000#00#000000
contextCSN: 20091118105948Z#000001#00#000000
=====================================

* Then I STOP slapd on both servers.

* Then I load the output on server1:
  $ slapadd -S 1 -q -w -l ~/load_noCSN.ldif

* Then I immediately slapcat this and move it to server2:
  $  slapcat > ~/toserver2.ldif

* And load it on server2:
  $ slapadd -q -l ~/toserver2.ldif

* I start server1, but BEFORE I start server2 I make ONE SINGLE CHANGE:
=================
dn: cn=data,dc=example,dc=com
changetype: modify
replace: userPassword
userPassword: NEWBASE64
=================

 * THEN I start server2 and monitor it's data.

What I find is that the contextCSN from server1 gets replicated, but the change doesn't. Also I see a contextCSN on server2 with SID 002 without I've done any operations on server2.

I'm sorry, this was quite a lot. I had hoped not to throw it at the list If my procedure was wrong from the beginning,

regards,
Peter