cn=config replication stops after adding olcAccess entries

[Jan Hugo Prins <jhp@jhprins.org>]

Hello,

I'm trying to setup a new MMR cluster.
I started the installation using the documentation from:
https://www.openldap.org/doc/admin24/replication.html#N-Way%20Multi-Master

OS: CentOS 7.7.
CentOS RPM packages OpenLDAP version 2.4.44-21

The base installation works fine, all schema's loaded etc.

At first I was thinking everything was fine, until I started testing
failure scenario's.
When I tried to rebuild a failed node I noticed that it wouldn't be
populated.
cn=config was empty except for the base config and only some data was
replicated.

After a lot of searching I found that adding any olcAccess rules would
destroy the replication.
With the olcAccess lines in the config below only the schemas are
replicated. When I remove all olcAccess lines all replication works.

The base config I use for a node looks like this:

dn: cn=config
objectClass: olcGlobal
cn: config
olcServerID: 1 ldap://ldap01.internal
olcServerID: 2 ldap://ldap02.internal
olcServerID: 3 ldap://ldap03.internal
olcPidFile: /var/run/openldap/slapd.pid
olcLogLevel: 16512

dn: olcDatabase={0}config,cn=config
objectClass: olcDatabaseConfig
olcDatabase: {0}config
olcRootDN: cn=config
olcRootPW: eich4mieceeB6ma9ehah
olcSyncRepl: rid=001 provider=ldap://ldap01.internal binddn="cn=config"
bindmethod=simple
  credentials=XXXXXX searchbase="cn=config" type=refreshAndPersist
  retry="5 5 300 +" timeout=1
olcSyncRepl: rid=002 provider=ldap://ldap02.internal binddn="cn=config"
bindmethod=simple
  credentials=XXXXXX searchbase="cn=config" type=refreshAndPersist
  retry="5 5 300 +" timeout=1
olcSyncRepl: rid=003 provider=ldap://ldap03.internal binddn="cn=config"
bindmethod=simple
  credentials=XXXXXX searchbase="cn=config" type=refreshAndPersist
  retry="5 5 300 +" timeout=1
olcMirrorMode: TRUE

dn: cn=module,cn=config
objectClass: olcModuleList
cn: module
olcModulePath: /usr/lib64/openldap/
olcModuleLoad: syncprov.la

dn: olcOverlay=syncprov,olcDatabase={0}config,cn=config
objectClass: olcOverlayConfig
objectClass: olcSyncProvConfig
olcOverlay: syncprov


The complete config on a running node looks like this:

dn: cn=config
objectClass: olcGlobal
objectClass: olcConfig
objectClass: top
cn: config
olcLogLevel: 16512
olcPidFile: /var/run/openldap/slapd.pid
olcServerID: 1 ldap://ldap01.internal
olcServerID: 2 ldap://ldap02.internal
olcServerID: 3 ldap://ldap03.internal

dn: cn=module{0},cn=config
objectClass: olcModuleList
objectClass: olcConfig
objectClass: top
cn: module{0}
olcModuleLoad: {0}syncprov.la
olcModulePath: /usr/lib64/openldap/

dn: olcDatabase={-1}frontend,cn=config
objectClass: olcDatabaseConfig
objectClass: olcConfig
objectClass: olcFrontendConfig
objectClass: top
olcAccess: {0}to dn.subtree="dc=domain,dc=com"
attrs=userPassword,shadowLastChange   by self write     by
dn.base="cn=bbsyncrepl, dc=domain,dc=com" read     by anonymous auth    
by * none
olcAccess: {1}to dn.subtree="dc=domain,dc=com" attrs=sambaNTPassword    
by self write     by dn.base="cn=bbsyncrepl, dc=domain,dc=com" read    
by anonymous auth     by * none
olcAccess: {2}to dn.subtree="dc=domain,dc=com" attrs=sambaLMPassword    
by self write     by dn.base="cn=bbsyncrepl, dc=domain,dc=com" read    
by anonymous auth     by * none
olcAccess: {3}to dn.subtree="dc=domain,dc=com"    by
dn.base="cn=bbsyncrepl,dc=domain,dc=com" read    by self write     by * read
olcAddContentAcl: FALSE
olcDatabase: {-1}frontend
olcLastMod: TRUE
olcMaxDerefDepth: 0
olcMonitoring: TRUE
olcReadOnly: FALSE
olcSchemaDN: cn=Subschema
olcSyncUseSubentry: FALSE

dn: olcDatabase={0}config,cn=config
objectClass: olcDatabaseConfig
objectClass: olcConfig
objectClass: top
olcDatabase: {0}config
olcMirrorMode: TRUE
olcMonitoring: TRUE
olcRootDN: cn=config
olcRootPW: XXXXXX
olcSyncrepl: {0}rid=001 provider=ldap://ldap01.internal
binddn="cn=config" bindmethod=simple credentials=XXXXXX
searchbase="cn=config" type=refreshAndPersist retry="5 5 300 +" timeout=1
olcSyncrepl: {1}rid=002 provider=ldap://ldap02.internal
binddn="cn=config" bindmethod=simple credentials=XXXXXX
searchbase="cn=config" type=refreshAndPersist retry="5 5 300 +" timeout=1
olcSyncrepl: {2}rid=003 provider=ldap://ldap03.internal
binddn="cn=config" bindmethod=simple credentials=XXXXXX
searchbase="cn=config" type=refreshAndPersist retry="5 5 300 +" timeout=1

dn: olcOverlay={0}syncprov,olcDatabase={0}config,cn=config
objectClass: olcSyncProvConfig
objectClass: olcOverlayConfig
objectClass: olcConfig
objectClass: top
olcOverlay: {0}syncprov

dn: olcOverlay={1}syncprov,olcDatabase={0}config,cn=config
objectClass: olcSyncProvConfig
objectClass: olcOverlayConfig
objectClass: olcConfig
objectClass: top
olcOverlay: {1}syncprov

dn: olcDatabase={1}bdb,cn=config
objectClass: olcBdbConfig
objectClass: olcDatabaseConfig
objectClass: olcConfig
objectClass: top
olcAddContentAcl: FALSE
olcDatabase: {1}bdb
olcDbCacheFree: 1
olcDbCacheSize: 1000000
olcDbCheckpoint: 512 5
olcDbDirectory: /var/lib/ldap
olcDbDirtyRead: FALSE
olcDbDNcacheSize: 0
olcDbIDLcacheSize: 0
olcDbIndex: default sub
olcDbIndex: objectClass eq
olcDbIndex: entryUUID eq
olcDbIndex: entryCSN eq
olcDbIndex: cn eq,subinitial
olcDbIndex: uid pres,eq
olcDbIndex: uidNumber eq
olcDbIndex: gidNumber eq
olcDbIndex: mailLocalAddress pres,eq
olcDbIndex: memberUid eq
olcDbIndex: mail eq,subinitial
olcDbIndex: mailRoutingAddress pres,eq
olcDbIndex: sambaSID eq
olcDbIndex: sn eq,subinitial
olcDbIndex: sambaPrimaryGroupSID eq
olcDbIndex: givenName eq,subinitial
olcDbIndex: sendmailMTAHost pres,eq,sub
olcDbIndex: sambaDomainName eq
olcDbIndex: sendmailMTAKey pres,eq,sub
olcDbIndex: sendmailMTAMapName pres,eq
olcDbIndex: sendmailMTAAliasValue pres,eq
olcDbIndex: sendmailMTAMapValue pres,eq
olcDbIndex: sendmailMTAAliasGrouping pres,eq
olcDbLinearIndex: FALSE
olcDbMode: 0600
olcDbNoSync: FALSE
olcDbSearchStack: 16
olcDbShmKey: 0
olcLastMod: TRUE
olcLimits: {0}dn.exact="cn=Manager,dc=domain,dc=com" time.soft=unlimited
time.hard=unlimited size.soft=unlimited size.hard=unlimited
olcLimits: {1}dn.exact="cn=bbsyncrepl,dc=domain,dc=com"
size.soft=unlimited size.hard=unlimited time.soft=unlimited
time.hard=unlimited
olcLimits: {2}dn.exact="cn=incontrol,dc=domain,dc=com"
size.soft=unlimited size.hard=unlimited time.soft=unlimited
time.hard=unlimited
olcMaxDerefDepth: 15
olcMirrorMode: TRUE
olcMonitoring: TRUE
olcReadOnly: FALSE
olcRootDN: cn=Manager,dc=domain,dc=com
olcRootPW: XXXXXX
olcSuffix: dc=domain,dc=com
olcSyncrepl: {0}rid=004 provider=ldap://ldap01.internal
binddn="cn=Manager,dc=domain,dc=com" bindmethod=simple
credentials=XXXXXX searchbase="dc=domain,dc=com" type=refreshAndPersist
interval=00:00:00:10 retry="5 5 300 5" timeout=1 network-timeout=0
starttls=no filter="(objectClass=*)" scope=sub attrs="*" schemachecking=off
olcSyncrepl: {1}rid=005 provider=ldap://ldap02.internal
binddn="cn=Manager,dc=domain,dc=com" bindmethod=simple
credentials=XXXXXX searchbase="dc=domain,dc=com" type=refreshAndPersist
interval=00:00:00:10 retry="5 5 300 5" timeout=1 network-timeout=0
starttls=no filter="(objectClass=*)" scope=sub attrs="*" schemachecking=off
olcSyncrepl: {2}rid=006 provider=ldap://ldap03.internal
binddn="cn=Manager,dc=domain,dc=com" bindmethod=simple
credentials=XXXXXX searchbase="dc=domain,dc=com" type=refreshAndPersist
interval=00:00:00:10 retry="5 5 300 5" timeout=1 network-timeout=0
starttls=no filter="(objectClass=*)" scope=sub attrs="*" schemachecking=off

dn: olcOverlay={0}syncprov,olcDatabase={1}bdb,cn=config
objectClass: olcSyncProvConfig
objectClass: olcOverlayConfig
objectClass: olcConfig
objectClass: top
olcOverlay: {0}syncprov

In the logging I see the following:

Jan 27 09:37:34 radius01.internal slapd[16596]: 5e2ea14e =>
access_allowed: search access to "cn=config" "entry" requested
Jan 27 09:37:34 radius01.internal slapd[16596]: 5e2ea14e <= root access
granted
Jan 27 09:37:34 radius01.internal slapd[16596]: 5e2ea14e =>
access_allowed: search access granted by manage(=mwrscxd)
Jan 27 09:37:34 radius01.internal slapd[16596]: 5e2ea14e =>
access_allowed: search access to "cn=config" "objectClass" requested
Jan 27 09:37:34 radius01.internal slapd[16596]: 5e2ea14e <= root access
granted
Jan 27 09:37:34 radius01.internal slapd[16596]: 5e2ea14e =>
access_allowed: search access granted by manage(=mwrscxd)
Jan 27 09:37:34 radius01.internal slapd[16596]: 5e2ea14e =>
access_allowed: search access to "cn=config" "entry" requested
Jan 27 09:37:34 radius01.internal slapd[16596]: 5e2ea14e <= root access
granted
Jan 27 09:37:34 radius01.internal slapd[16596]: 5e2ea14e =>
access_allowed: search access granted by manage(=mwrscxd)
Jan 27 09:37:34 radius01.internal slapd[16596]: 5e2ea14e =>
access_allowed: search access to "cn=config" "objectClass" requested
Jan 27 09:37:34 radius01.internal slapd[16596]: 5e2ea14e <= root access
granted
Jan 27 09:37:34 radius01.internal slapd[16596]: 5e2ea14e =>
access_allowed: search access granted by manage(=mwrscxd)
Jan 27 09:37:34 radius01.internal slapd[16596]: 5e2ea14e =>
access_allowed: search access to "cn=config" "entry" requested
Jan 27 09:37:34 radius01.internal slapd[16596]: 5e2ea14e <= root access
granted
Jan 27 09:37:34 radius01.internal slapd[16596]: 5e2ea14e =>
access_allowed: search access granted by manage(=mwrscxd)
Jan 27 09:37:34 radius01.internal slapd[16596]: 5e2ea14e =>
access_allowed: search access to "cn=config" "objectClass" requested
Jan 27 09:37:34 radius01.internal slapd[16596]: 5e2ea14e <= root access
granted
Jan 27 09:37:34 radius01.internal slapd[16596]: 5e2ea14e =>
access_allowed: search access granted by manage(=mwrscxd)
Jan 27 09:37:34 radius01.internal slapd[16596]: 5e2ea14e =>
access_allowed: read access to "cn=config" "entry" requested
Jan 27 09:37:34 radius01.internal slapd[16596]: 5e2ea14e <= root access
granted

But also:

Jan 27 09:38:36 radius01.internal slapd[16596]: 5e2ea18c do_syncrep2:
rid=003 LDAP_RES_SEARCH_RESULT
Jan 27 09:38:36 radius01.internal slapd[16596]: 5e2ea18c do_syncrep2:
rid=003 LDAP_RES_SEARCH_RESULT (53) Server is unwilling to perform
Jan 27 09:38:36 radius01.internal slapd[16596]: 5e2ea18c do_syncrep2:
rid=003 (53) Server is unwilling to perform
Jan 27 09:38:36 radius01.internal slapd[16596]: 5e2ea18c do_syncrepl:
rid=003 rc -2 retrying
Jan 27 09:39:30 radius01.internal slapd[16596]: 5e2ea1c2 do_syncrep2:
rid=002 LDAP_RES_SEARCH_RESULT
Jan 27 09:39:30 radius01.internal slapd[16596]: 5e2ea1c2 do_syncrep2:
rid=002 LDAP_RES_SEARCH_RESULT (53) Server is unwilling to perform
Jan 27 09:39:30 radius01.internal slapd[16596]: 5e2ea1c2 do_syncrep2:
rid=002 (53) Server is unwilling to perform
Jan 27 09:39:30 radius01.internal slapd[16596]: 5e2ea1c2 do_syncrepl:
rid=002 rc -2 retrying


Something else I see, when I use jxplorer to look at the content of  a
server using the cn=config credentials I would expect to see all values
including the empty values. On a server without olcAccess lines I see
this, but when there are olcAccess lines I only see the configured
values. All unset values are not visible.


Any help is appreciated.

Thank you very much.
Jan Hugo Prins