Issue 7608 - cn=config with modifiersdn outside cn=config breaks recovery using slapadd
Summary: cn=config with modifiersdn outside cn=config breaks recovery using slapadd
Status: VERIFIED FIXED
Alias: None
Product: OpenLDAP
Classification: Unclassified
Component: slapd (show other issues)
Version: 2.4.35
Hardware: All All
: --- normal
Target Milestone: ---
Assignee: OpenLDAP project
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-05-24 19:41 UTC by Christian Kratzer
Modified: 2014-08-01 21:04 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description Christian Kratzer 2013-05-24 19:41:24 UTC
Full_Name: Christian Kratzer
Version: 2.4.35
OS: CentOS 6.4
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (37.80.52.67)


Hi,

I have setup an acl to allow write access to cn=config for users in the main
directory.

This is mainly to have a clean audit trail of config changes using auditlog so
changes can be tracked back to who did them.

We also backup config with slapcat -n0 and data with slapcat.

When restoring configs and data I ran into two spefific problems when cn=config
entries have a modifiersdn outside of cn=config as in:

  dn: cn=config
  objectClass: olcGlobal
  ...
  modifiersName: cn=Alice,ou=People,dc=test
  modifyTimestamp: 20130524161850Z

This is on openldap-2.4.35 on centos-6.4.

I have built a simple test case to illustrate the problem. Should be easily
reproducible using after editing a few paths in below configs.

We setup a new openldap server by using slapadd -n0 of below config and slapadd
of below data in test.ldif.

  http://www.cksoft.de/paste/374f18f905d53f8e6e158702e686b563/config.ldif
  http://www.cksoft.de/paste/374f18f905d53f8e6e158702e686b563/test.ldif

After starting slapd I modify the configuration as cn=Alice,ou=People,dc=test
using below scripts

  http://www.cksoft.de/paste/374f18f905d53f8e6e158702e686b563/olcLoglevel.sh
  http://www.cksoft.de/paste/374f18f905d53f8e6e158702e686b563/olcLoglevel.ldif

I then extract following two backups from cn=config using slapcat -n0 and
dc=test using slapcat

  http://www.cksoft.de/paste/374f18f905d53f8e6e158702e686b563/slapcat-cn=config.ldif
  http://www.cksoft.de/paste/374f18f905d53f8e6e158702e686b563/slapcat-dc=test.ldif

First problem: An attempt to to restore slapcat-dc=test.ldif after deleting data
using "rm -f /var/lib/ldap/*" fails as follows:

  [root@test-centos64 openldap]# slapadd -v -F slapd.d -l slapcat-dc\=test.ldif
  519f93a8 PROXIED attributeDescription "OU" inserted.
  519f93a8 PROXIED attributeDescription "DC" inserted.
  519f93a8 => mdb_tool_entry_put: id2entry_add failed: err=17
  519f93a8 => mdb_tool_entry_put: txn_aborted! File exists (17)
  slapadd: could not add entry dn="dc=test" (line=1): txn_aborted! File exists
(17)
  _####                  24.55% eta   none elapsed            none spd   2.0
M/s
  Closing DB...
  [root@test-centos64 openldap]#

I can successfully import the data if I edit cn=config.ldif and manuelly fix the
modifiersName to cn=config and reimport configuration using slapadd -n0.

This also kills our disaster recovery scenario of deleting data and letting
syncrepl handle the recovery.

Second problem: Recovery of the configuration fails as after "rm -fr slapd.d"

  [root@test-centos64 openldap]# slapadd -v -n0 -F slapd.d -l
slapcat-cn\=config.ldif
  519f95c2 str2entry: invalid value for attributeType modifiersName #0 (syntax
1.3.6.1.4.1.1466.115.121.1.12)
  slapadd: could not parse entry (line=1)
  _#                      5.39% eta   none elapsed            none spd   3.2
M/s
  Closing DB...
  [root@test-centos64 openldap]#

Again I am able to work around the problem by fixing the modifiersdn of the
cn=config dump before slapadding with slapadd -n0.

Any chance of getting above use cases working ?  I would like to provide the
customer in question with a straight forward recovery process for both data and
cn=config.

They would like to avoid using the cn=config rootDN for regular administration
in order to have a clean audit trail of changes using auditlog on cn=config.

Greetings
Christian

Comment 1 ando@openldap.org 2013-05-24 19:56:57 UTC
On 05/24/2013 09:41 PM, ck@cksoft.de wrote:
> Full_Name: Christian Kratzer
> Version: 2.4.35
> OS: CentOS 6.4
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (37.80.52.67)
>
>
> Hi,
>
> I have setup an acl to allow write access to cn=config for users in the main
> directory.
>
> This is mainly to have a clean audit trail of config changes using auditlog so
> changes can be tracked back to who did them.
>
> We also backup config with slapcat -n0 and data with slapcat.
>
> When restoring configs and data I ran into two spefific problems when cn=config
> entries have a modifiersdn outside of cn=config as in:
>
>    dn: cn=config
>    objectClass: olcGlobal
>    ...
>    modifiersName: cn=Alice,ou=People,dc=test
>    modifyTimestamp: 20130524161850Z
>
> This is on openldap-2.4.35 on centos-6.4.
>
> I have built a simple test case to illustrate the problem. Should be easily
> reproducible using after editing a few paths in below configs.
>
> We setup a new openldap server by using slapadd -n0 of below config and slapadd
> of below data in test.ldif.
>
>    http://www.cksoft.de/paste/374f18f905d53f8e6e158702e686b563/config.ldif
>    http://www.cksoft.de/paste/374f18f905d53f8e6e158702e686b563/test.ldif
>
> After starting slapd I modify the configuration as cn=Alice,ou=People,dc=test
> using below scripts
>
>    http://www.cksoft.de/paste/374f18f905d53f8e6e158702e686b563/olcLoglevel.sh
>    http://www.cksoft.de/paste/374f18f905d53f8e6e158702e686b563/olcLoglevel.ldif
>
> I then extract following two backups from cn=config using slapcat -n0 and
> dc=test using slapcat
>
>    http://www.cksoft.de/paste/374f18f905d53f8e6e158702e686b563/slapcat-cn=config.ldif
>    http://www.cksoft.de/paste/374f18f905d53f8e6e158702e686b563/slapcat-dc=test.ldif
>
> First problem: An attempt to to restore slapcat-dc=test.ldif after deleting data
> using "rm -f /var/lib/ldap/*" fails as follows:
>
>    [root@test-centos64 openldap]# slapadd -v -F slapd.d -l slapcat-dc\=test.ldif
>    519f93a8 PROXIED attributeDescription "OU" inserted.
>    519f93a8 PROXIED attributeDescription "DC" inserted.
>    519f93a8 => mdb_tool_entry_put: id2entry_add failed: err=17
>    519f93a8 => mdb_tool_entry_put: txn_aborted! File exists (17)
>    slapadd: could not add entry dn="dc=test" (line=1): txn_aborted! File exists

This is a chicken and egg problem.  OU is not built-in, so it cannot be 
used until it is configured.  Configuration data must only use schema 
items whose definition is already loaded.

I don't see a clear way out of this issue, so I'd pass the ball to 
Howard.  My (humble) suggestion is: delegate modifications to identities 
whose DN is made of hardcoded attribute types: CN, UID (you can find 
what attribute types are hardcoded in schema_prep.c).

p.

> (17)
>    _####                  24.55% eta   none elapsed            none spd   2.0
> M/s
>    Closing DB...
>    [root@test-centos64 openldap]#
>
> I can successfully import the data if I edit cn=config.ldif and manuelly fix the
> modifiersName to cn=config and reimport configuration using slapadd -n0.
>
> This also kills our disaster recovery scenario of deleting data and letting
> syncrepl handle the recovery.
>
> Second problem: Recovery of the configuration fails as after "rm -fr slapd.d"
>
>    [root@test-centos64 openldap]# slapadd -v -n0 -F slapd.d -l
> slapcat-cn\=config.ldif
>    519f95c2 str2entry: invalid value for attributeType modifiersName #0 (syntax
> 1.3.6.1.4.1.1466.115.121.1.12)
>    slapadd: could not parse entry (line=1)
>    _#                      5.39% eta   none elapsed            none spd   3.2
> M/s
>    Closing DB...
>    [root@test-centos64 openldap]#
>
> Again I am able to work around the problem by fixing the modifiersdn of the
> cn=config dump before slapadding with slapadd -n0.
>
> Any chance of getting above use cases working ?  I would like to provide the
> customer in question with a straight forward recovery process for both data and
> cn=config.
>
> They would like to avoid using the cn=config rootDN for regular administration
> in order to have a clean audit trail of changes using auditlog on cn=config.
>
> Greetings
> Christian
>
>
>
>
>


-- 
Pierangelo Masarati
Associate Professor
Dipartimento di Ingegneria Aerospaziale
Politecnico di Milano

Comment 2 Christian Kratzer 2013-05-26 08:20:47 UTC
Hi,

Summary: it seems having a modifiersdn outside of cn=config in cn=config breaks replication once slapd is restarted.

I have a more elaborate test setup with four servers.

Two masters ldaptest1, ldaptest2 using syncrepl in mirrormode .

Two slaves ldaptest3, ldaptest4 that use syncrepl to slave from the masters.

The masters and the slaves also replicate cn=config between themselves.

Replication config is as follows:

on ldaptest1 and ldaptest2:

 	dn: cn=config
 	...
 	olcServerID: 1 ldap://ldaptest1.cksoft.de
 	olcServerID: 2 ldap://ldaptest2.cksoft.de
 	olcServerID: 3 ldap://ldaptest3.cksoft.de
 	olcServerID: 4 ldap://ldaptest4.cksoft.de

 	dn: olcDatabase={1}mdb,cn=config
 	...
 	olcSyncrepl:
 	  rid=001
 	  provider=ldap://ldaptest1.cksoft.de
 	  bindmethod=simple
 	  binddn="cn=replica,ou=system,dc=test"
 	  credentials="secret"
 	  keepalive=0:0:0
 	  starttls=yes
 	  tls_cert="/usr/local/etc/openldap/certs/server.cert"
 	  tls_key="/usr/local/etc/openldap/certs/server.key"
 	  tls_cacert="/usr/local/etc/openldap/certs/cksoftware-gmbh-ca-2011-2031.cert"
 	  tls_reqcert=demand tls_crlcheck=none
 	  filter="(objectclass=*)"
 	  searchbase="dc=test"
 	  scope=sub
 	  type=refreshAndPersist
 	  retry="60 10 300 +"
 	olcSyncrepl:
 	  rid=002
 	  provider=ldap://ldaptest2.cksoft.de
 	  bindmethod=simple
 	  binddn="cn=replica,ou=system,dc=test"
 	  credentials="secret"
 	  keepalive=0:0:0
 	  starttls=yes
 	  tls_cert="/usr/local/etc/openldap/certs/server.cert"
 	  tls_key="/usr/local/etc/openldap/certs/server.key"
 	  tls_cacert="/usr/local/etc/openldap/certs/cksoftware-gmbh-ca-2011-2031.cert"
 	  tls_reqcert=demand tls_crlcheck=none
 	  filter="(objectclass=*)"
 	  searchbase="dc=test"
 	  scope=sub
 	  type=refreshAndPersist
 	  retry="60 10 300 +"
 	olcMirrorMode: TRUE


on ldaptest3 and ldaptest4:

 	dn: cn=config
 	...
 	olcServerID: 1 ldap://ldaptest1.cksoft.de
 	olcServerID: 2 ldap://ldaptest2.cksoft.de
 	olcServerID: 3 ldap://ldaptest3.cksoft.de
 	olcServerID: 4 ldap://ldaptest4.cksoft.de

 	dn: olcDatabase={1}mdb,cn=config
 	...
 	olcSyncrepl:
 	  rid=001
 	  provider=ldap://ldaptest1.cksoft.de
 	  bindmethod=simple
 	  binddn="cn=replica,ou=system,dc=test"
 	  credentials="secret"
 	  keepalive=0:0:0
 	  starttls=yes
 	  tls_cert="/usr/local/etc/openldap/certs/server.cert"
 	  tls_key="/usr/local/etc/openldap/certs/server.key"
 	  tls_cacert="/usr/local/etc/openldap/certs/cksoftware-gmbh-ca-2011-2031.cert"
 	  tls_reqcert=demand tls_crlcheck=none
 	  filter="(objectclass=*)"
 	  searchbase="dc=test"
 	  scope=sub
 	  type=refreshAndPersist
 	  retry="60 10 300 +"
 	olcSyncrepl:
 	  rid=002
 	  provider=ldap://ldaptest2.cksoft.de
 	  bindmethod=simple
 	  binddn="cn=replica,ou=system,dc=test"
 	  credentials="secret"
 	  keepalive=0:0:0
 	  starttls=yes
 	  tls_cert="/usr/local/etc/openldap/certs/server.cert"
 	  tls_key="/usr/local/etc/openldap/certs/server.key"
 	  tls_cacert="/usr/local/etc/openldap/certs/cksoftware-gmbh-ca-2011-2031.cert"
 	  tls_reqcert=demand tls_crlcheck=none
 	  filter="(objectclass=*)"
 	  searchbase="dc=test"
 	  scope=sub
 	  type=refreshAndPersist
 	  retry="60 10 300 +"
 	olcMirrorMode: FALSE

Once I modify something trivial like olcLogLevel on the slaves using my personal dn, cn=config will have following operational attributes:

 	# config
 	dn: cn=config
 	structuralObjectClass: olcGlobal
 	entryUUID: dc78e00a-461a-1032-9454-cd151c72b364
 	creatorsName: cn=config
 	createTimestamp: 20130430194949Z
 	entryCSN: 20130526075059.867187Z#000000#004#000000
 	modifiersName: uid=ckratzer,ou=people,dc=test
 	modifyTimestamp: 20130526075059Z
 	contextCSN: 20130526074631.636527Z#000000#003#000000
 	contextCSN: 20130526075059.867187Z#000000#004#000000
 	entryDN: cn=config
 	subschemaSubentry: cn=Subschema

I then restart slapd on the first slave ldaptest3 and leave it running on ldaptest4.

A change replicated from the masters will appear on ldaptest4 just fine.

The restarted master on ldaptest3 fails to apply the change

 	May 26 10:06:13 ldaptest3 slapd[2410]: syncrepl_message_to_entry: rid=001 DN: cn=test,dc=test, UUID: 3e48a550-4dd1-1032-9f9f-47169a206073
 	May 26 10:06:13 ldaptest3 slapd[2410]: syncrepl_entry: rid=001 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD)
 	May 26 10:06:13 ldaptest3 slapd[2410]: syncrepl_entry: rid=001 inserted UUID 3e48a550-4dd1-1032-9f9f-47169a206073
 	May 26 10:06:13 ldaptest3 slapd[2410]: syncrepl_entry: rid=001 be_search (0)
 	May 26 10:06:13 ldaptest3 slapd[2410]: syncrepl_entry: rid=001 cn=test,dc=test
 	May 26 10:06:13 ldaptest3 slapd[2410]: syncrepl_entry: rid=001 entry unchanged, ignored (cn=test,dc=test,)
 	May 26 10:06:13 ldaptest3 slapd[2410]: syncrepl_updateCookie: rid=001 be_modify failed (17)
 	May 26 10:06:13 ldaptest3 slapd[2410]: do_syncrepl: rid=001 rc 17 retrying
 	May 26 10:06:13 ldaptest3 slapd[2410]: syncrepl_message_to_entry: rid=002 DN: cn=test,dc=test, UUID: 3e48a550-4dd1-1032-9f9f-47169a206073
 	May 26 10:06:13 ldaptest3 slapd[2410]: syncrepl_entry: rid=002 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD)
 	May 26 10:06:13 ldaptest3 slapd[2410]: syncrepl_entry: rid=002 inserted UUID 3e48a550-4dd1-1032-9f9f-47169a206073
 	May 26 10:06:13 ldaptest3 slapd[2410]: syncrepl_entry: rid=002 be_search (0)
 	May 26 10:06:13 ldaptest3 slapd[2410]: syncrepl_entry: rid=002 cn=test,dc=test,
 	May 26 10:06:13 ldaptest3 slapd[2410]: syncrepl_entry: rid=002 entry unchanged, ignored (cn=test,dc=test,)
 	May 26 10:06:13 ldaptest3 slapd[2410]: syncrepl_updateCookie: rid=002 be_modify failed (17)
 	May 26 10:06:13 ldaptest3 slapd[2410]: do_syncrepl: rid=002 rc 17 retrying

I currently have customer data in above test setup so I had to do some obfuscation in above samples and logs. If required I can setup maximally stripped down case that illustrates the problem.

Would it be possible to mask modifiersName inside cn=config to either a fixed value just append something like cn=proxied,cn=config to dns outside of cn=config.

Greetings
Christian

-- 
Christian Kratzer                      CK Software GmbH
Email:   ck@cksoft.de                  Wildberger Weg 24/2
Phone:   +49 7032 893 997 - 0          D-71126 Gaeufelden
Fax:     +49 7032 893 997 - 9          HRB 245288, Amtsgericht Stuttgart
Web:     http://www.cksoft.de/         Geschaeftsfuehrer: Christian Kratzer

Comment 3 Howard Chu 2013-05-27 03:04:07 UTC
ck@cksoft.de wrote:
> Hi,
>
> Summary: it seems having a modifiersdn outside of cn=config in cn=config breaks replication once slapd is restarted.

Yeah, using DNs other than the cn=config rootDN is frequently a problem. This 
is why when cn=config was introduced in 2.3 only the cn=config rootDN was 
allowed access to the tree.

In this particular case, there's a simpler solution - add schema definitions 
for the missing RDN attributes directly to the cn=config entry. In your case, 
move the "ou" definition from the cn=core schema entry.

There's nothing dirty about this solution - it has always been valid to define 
schema elements in the top-level slapd.conf file as well as in the top 
cn=config global config entry. The feature doesn't get used much because most 
3rd party schemas are distributed as their own files, so it's simpler to just 
use the include directive to reference them. But for your current situation, 
you need to define these schema elements as early as possible, so that they 
can be processed as valid later on.

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/

Comment 4 Christian Kratzer 2013-05-27 10:10:28 UTC
Hi Howard,

On Mon, 27 May 2013, hyc@symas.com wrote:
> ck@cksoft.de wrote:
>> Hi,
>>
>> Summary: it seems having a modifiersdn outside of cn=config in cn=config breaks replication once slapd is restarted.
>
> Yeah, using DNs other than the cn=config rootDN is frequently a problem. This
> is why when cn=config was introduced in 2.3 only the cn=config rootDN was
> allowed access to the tree.
>
> In this particular case, there's a simpler solution - add schema definitions
> for the missing RDN attributes directly to the cn=config entry. In your case,
> move the "ou" definition from the cn=core schema entry.
>
> There's nothing dirty about this solution - it has always been valid to define
> schema elements in the top-level slapd.conf file as well as in the top
> cn=config global config entry. The feature doesn't get used much because most
> 3rd party schemas are distributed as their own files, so it's simpler to just
> use the include directive to reference them. But for your current situation,
> you need to define these schema elements as early as possible, so that they
> can be processed as valid later on.

Thanks for the feedback.

As my sample had modifiersName: cn=Alice,ou=People,dc=test I added definitions for 'ou' and 'dc' to cn=config.

It seems this helps for modifiersNames of entries below cn=config but not for cn=config itself.

I have uploaded following three configs that illustrate the remaining problem:

     http://www.cksoft.de/paste/374f18f905d53f8e6e158702e686b563/config-1-fail.ldif
     http://www.cksoft.de/paste/374f18f905d53f8e6e158702e686b563/config-2-ok.ldif
     http://www.cksoft.de/paste/374f18f905d53f8e6e158702e686b563/config-3-fail.ldif

The original failure with config-1 because of a modifiersName on cn=module{0},cn=config:

     [root@test-centos64 test]# slapadd -v -n0 -F config-1 -l config-1-fail.ldif
     added: "cn=config" (00000001)
     51a32d4b str2entry: invalid value for attributeType modifiersName #0 (syntax 1.3.6.1.4.1.1466.115.121.1.12)
     slapadd: could not parse entry (line=42)
     _#                      7.41% eta   none elapsed            none spd   1.5 M/s
     Closing DB...
     [root@test-centos64 test]#

Workaround applied in config-2 with attribute definitions in cn=config

     [root@test-centos64 test]# diff -u config-1-fail.ldif config-2-ok.ldif
     --- config-1-fail.ldif  2013-05-27 11:50:35.368253951 +0200
     +++ config-2-ok.ldif    2013-05-27 11:49:17.691253291 +0200
     @@ -28,6 +28,12 @@
      olcTLSVerifyClient: never
      olcToolThreads: 1
      olcWriteTimeout: 0
     +olcAttributeTypes: ( 2.5.4.11 NAME ( 'ou' 'organizationalUnitName' ) DESC '
     + RFC2256: organizational unit this object belongs to' SUP name )
     +olcAttributeTypes: ( 0.9.2342.19200300.100.1.25 NAME ( 'dc' 'domainCompone
     + nt' ) DESC 'RFC1274/2247: domain component' EQUALITY caseIgnoreIA5Match SUBST
     + R caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 SINGLE-VA
     + LUE )
      structuralObjectClass: olcGlobal
      entryUUID: 3b1e9034-58d9-1032-8161-d3a3b8e342e7
      creatorsName: cn=config
     @@ -86,8 +92,6 @@
       ubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{128} )
      olcAttributeTypes: {7}( 2.5.4.10 NAME ( 'o' 'organizationName' ) DESC 'RFC2256
       : organization this object belongs to' SUP name )
     -olcAttributeTypes: {8}( 2.5.4.11 NAME ( 'ou' 'organizationalUnitName' ) DESC '
     - RFC2256: organizational unit this object belongs to' SUP name )
      olcAttributeTypes: {9}( 2.5.4.12 NAME 'title' DESC 'RFC2256: title associated
       with the entity' SUP name )
      olcAttributeTypes: {10}( 2.5.4.14 NAME 'searchGuide' DESC 'RFC2256: search gui
     @@ -193,10 +197,6 @@
      olcAttributeTypes: {48}( 0.9.2342.19200300.100.1.3 NAME ( 'mail' 'rfc822Mailbo
       x' ) DESC 'RFC1274: RFC822 Mailbox'   EQUALITY caseIgnoreIA5Match   SUBSTR ca
       seIgnoreIA5SubstringsMatch   SYNTAX 1.3.6.1.4.1.1466.115.121.1.26{256} )
     -olcAttributeTypes: {49}( 0.9.2342.19200300.100.1.25 NAME ( 'dc' 'domainCompone
     - nt' ) DESC 'RFC1274/2247: domain component' EQUALITY caseIgnoreIA5Match SUBST
     - R caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 SINGLE-VA
     - LUE )
      olcAttributeTypes: {50}( 0.9.2342.19200300.100.1.37 NAME 'associatedDomain' DE
       SC 'RFC1274: domain associated with object' EQUALITY caseIgnoreIA5Match SUBST
       R caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 )
     [root@test-centos64 test]#

     [root@test-centos64 test]# slapadd -v -n0 -F config-2 -l config-2-ok.ldif
     added: "cn=config" (00000001)
     added: "cn=module{0},cn=config" (00000001)
     added: "cn=schema,cn=config" (00000001)
     added: "cn={0}core,cn=schema,cn=config" (00000001)
     added: "olcDatabase={-1}frontend,cn=config" (00000001)
     added: "olcDatabase={0}config,cn=config" (00000001)
     added: "olcDatabase={1}mdb,cn=config" (00000001)
     _#################### 100.00% eta   none elapsed            none fast!
     Closing DB...
     [root@test-centos64 test]#

Breaks again after a modifiersname is added to cn=config

     [root@test-centos64 test]# diff -u config-2-ok.ldif config-3-fail.ldif
     --- config-2-ok.ldif    2013-05-27 11:49:17.691253291 +0200
     +++ config-3-fail.ldif  2013-05-27 11:52:57.346255334 +0200
     @@ -42,7 +42,7 @@
      olcLogLevel: Stats
      olcLogLevel: Stats2
      entryCSN: 20130524161850.764209Z#000000#000#000000
     -modifiersName: cn=config
     +modifiersName: cn=Alice,ou=People,dc=test
      modifyTimestamp: 20130524161850Z

      dn: cn=module{0},cn=config
     [root@test-centos64 test]#

     [root@test-centos64 test]# slapadd -v -n0 -F config-3 -l config-3-fail.ldif
     51a32daf str2entry: invalid value for attributeType modifiersName #0 (syntax 1.3.6.1.4.1.1466.115.121.1.12)
     slapadd: could not parse entry (line=1)
     _#                      7.35% eta   none elapsed            none spd   3.0 M/s
     Closing DB...
     [root@test-centos64 test]#

Sorry if I do not see the obvious.  Is there any possibility to get this to work for cn=config as well as entries below cn=config.

How much freedom would we have to rearrange the entries und cn=config so we could have the schema defintions read before olcGlobal ?

Greetings
Christian

-- 
Christian Kratzer                      CK Software GmbH
Email:   ck@cksoft.de                  Wildberger Weg 24/2
Phone:   +49 7032 893 997 - 0          D-71126 Gaeufelden
Fax:     +49 7032 893 997 - 9          HRB 245288, Amtsgericht Stuttgart
Web:     http://www.cksoft.de/         Geschaeftsfuehrer: Christian Kratzer

Comment 5 Howard Chu 2013-05-27 15:58:48 UTC
Christian Kratzer wrote:
> Hi Howard,

Never mind, this was actually a bug in the handling of proxied attributes. 
Fixed now in master, your test case should work fine there.
>
> On Mon, 27 May 2013, hyc@symas.com wrote:
>> ck@cksoft.de wrote:
>>> Hi,
>>>
>>> Summary: it seems having a modifiersdn outside of cn=config in cn=config breaks replication once slapd is restarted.
>>
>> Yeah, using DNs other than the cn=config rootDN is frequently a problem. This
>> is why when cn=config was introduced in 2.3 only the cn=config rootDN was
>> allowed access to the tree.
>>
>> In this particular case, there's a simpler solution - add schema definitions
>> for the missing RDN attributes directly to the cn=config entry. In your case,
>> move the "ou" definition from the cn=core schema entry.
>>
>> There's nothing dirty about this solution - it has always been valid to define
>> schema elements in the top-level slapd.conf file as well as in the top
>> cn=config global config entry. The feature doesn't get used much because most
>> 3rd party schemas are distributed as their own files, so it's simpler to just
>> use the include directive to reference them. But for your current situation,
>> you need to define these schema elements as early as possible, so that they
>> can be processed as valid later on.
>
> Thanks for the feedback.
>
> As my sample had modifiersName: cn=Alice,ou=People,dc=test I added definitions for 'ou' and 'dc' to cn=config.
>
> It seems this helps for modifiersNames of entries below cn=config but not for cn=config itself.
>
> I have uploaded following three configs that illustrate the remaining problem:
>
>       http://www.cksoft.de/paste/374f18f905d53f8e6e158702e686b563/config-1-fail.ldif
>       http://www.cksoft.de/paste/374f18f905d53f8e6e158702e686b563/config-2-ok.ldif
>       http://www.cksoft.de/paste/374f18f905d53f8e6e158702e686b563/config-3-fail.ldif
>
> The original failure with config-1 because of a modifiersName on cn=module{0},cn=config:
>
>       [root@test-centos64 test]# slapadd -v -n0 -F config-1 -l config-1-fail.ldif
>       added: "cn=config" (00000001)
>       51a32d4b str2entry: invalid value for attributeType modifiersName #0 (syntax 1.3.6.1.4.1.1466.115.121.1.12)
>       slapadd: could not parse entry (line=42)
>       _#                      7.41% eta   none elapsed            none spd   1.5 M/s
>       Closing DB...
>       [root@test-centos64 test]#
>
> Workaround applied in config-2 with attribute definitions in cn=config
>
>       [root@test-centos64 test]# diff -u config-1-fail.ldif config-2-ok.ldif
>       --- config-1-fail.ldif  2013-05-27 11:50:35.368253951 +0200
>       +++ config-2-ok.ldif    2013-05-27 11:49:17.691253291 +0200
>       @@ -28,6 +28,12 @@
>        olcTLSVerifyClient: never
>        olcToolThreads: 1
>        olcWriteTimeout: 0
>       +olcAttributeTypes: ( 2.5.4.11 NAME ( 'ou' 'organizationalUnitName' ) DESC '
>       + RFC2256: organizational unit this object belongs to' SUP name )
>       +olcAttributeTypes: ( 0.9.2342.19200300.100.1.25 NAME ( 'dc' 'domainCompone
>       + nt' ) DESC 'RFC1274/2247: domain component' EQUALITY caseIgnoreIA5Match SUBST
>       + R caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 SINGLE-VA
>       + LUE )
>        structuralObjectClass: olcGlobal
>        entryUUID: 3b1e9034-58d9-1032-8161-d3a3b8e342e7
>        creatorsName: cn=config
>       @@ -86,8 +92,6 @@
>         ubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{128} )
>        olcAttributeTypes: {7}( 2.5.4.10 NAME ( 'o' 'organizationName' ) DESC 'RFC2256
>         : organization this object belongs to' SUP name )
>       -olcAttributeTypes: {8}( 2.5.4.11 NAME ( 'ou' 'organizationalUnitName' ) DESC '
>       - RFC2256: organizational unit this object belongs to' SUP name )
>        olcAttributeTypes: {9}( 2.5.4.12 NAME 'title' DESC 'RFC2256: title associated
>         with the entity' SUP name )
>        olcAttributeTypes: {10}( 2.5.4.14 NAME 'searchGuide' DESC 'RFC2256: search gui
>       @@ -193,10 +197,6 @@
>        olcAttributeTypes: {48}( 0.9.2342.19200300.100.1.3 NAME ( 'mail' 'rfc822Mailbo
>         x' ) DESC 'RFC1274: RFC822 Mailbox'   EQUALITY caseIgnoreIA5Match   SUBSTR ca
>         seIgnoreIA5SubstringsMatch   SYNTAX 1.3.6.1.4.1.1466.115.121.1.26{256} )
>       -olcAttributeTypes: {49}( 0.9.2342.19200300.100.1.25 NAME ( 'dc' 'domainCompone
>       - nt' ) DESC 'RFC1274/2247: domain component' EQUALITY caseIgnoreIA5Match SUBST
>       - R caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 SINGLE-VA
>       - LUE )
>        olcAttributeTypes: {50}( 0.9.2342.19200300.100.1.37 NAME 'associatedDomain' DE
>         SC 'RFC1274: domain associated with object' EQUALITY caseIgnoreIA5Match SUBST
>         R caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 )
>       [root@test-centos64 test]#
>
>       [root@test-centos64 test]# slapadd -v -n0 -F config-2 -l config-2-ok.ldif
>       added: "cn=config" (00000001)
>       added: "cn=module{0},cn=config" (00000001)
>       added: "cn=schema,cn=config" (00000001)
>       added: "cn={0}core,cn=schema,cn=config" (00000001)
>       added: "olcDatabase={-1}frontend,cn=config" (00000001)
>       added: "olcDatabase={0}config,cn=config" (00000001)
>       added: "olcDatabase={1}mdb,cn=config" (00000001)
>       _#################### 100.00% eta   none elapsed            none fast!
>       Closing DB...
>       [root@test-centos64 test]#
>
> Breaks again after a modifiersname is added to cn=config
>
>       [root@test-centos64 test]# diff -u config-2-ok.ldif config-3-fail.ldif
>       --- config-2-ok.ldif    2013-05-27 11:49:17.691253291 +0200
>       +++ config-3-fail.ldif  2013-05-27 11:52:57.346255334 +0200
>       @@ -42,7 +42,7 @@
>        olcLogLevel: Stats
>        olcLogLevel: Stats2
>        entryCSN: 20130524161850.764209Z#000000#000#000000
>       -modifiersName: cn=config
>       +modifiersName: cn=Alice,ou=People,dc=test
>        modifyTimestamp: 20130524161850Z
>
>        dn: cn=module{0},cn=config
>       [root@test-centos64 test]#
>
>       [root@test-centos64 test]# slapadd -v -n0 -F config-3 -l config-3-fail.ldif
>       51a32daf str2entry: invalid value for attributeType modifiersName #0 (syntax 1.3.6.1.4.1.1466.115.121.1.12)
>       slapadd: could not parse entry (line=1)
>       _#                      7.35% eta   none elapsed            none spd   3.0 M/s
>       Closing DB...
>       [root@test-centos64 test]#
>
> Sorry if I do not see the obvious.  Is there any possibility to get this to work for cn=config as well as entries below cn=config.
>
> How much freedom would we have to rearrange the entries und cn=config so we could have the schema defintions read before olcGlobal ?
>
> Greetings
> Christian
>


-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/

Comment 6 Howard Chu 2013-05-27 16:40:34 UTC
changed notes
changed state Open to Test
moved from Incoming to Software Bugs
Comment 7 Christian Kratzer 2013-05-27 18:28:38 UTC
Hi Howard,

On Mon, 27 May 2013, Howard Chu wrote:

> Christian Kratzer wrote:
>> Hi Howard,
>
> Never mind, this was actually a bug in the handling of proxied attributes. 
> Fixed now in master, your test case should work fine there.

great.  Will the changes be automatically included in 2.4.36 when its released ?

I'll test against master and deploy a workaround with an acl to the customer site in the meantime.

Greetings
Christian

-- 
Christian Kratzer                      CK Software GmbH
Email:   ck@cksoft.de                  Wildberger Weg 24/2
Phone:   +49 7032 893 997 - 0          D-71126 Gaeufelden
Fax:     +49 7032 893 997 - 9          HRB 245288, Amtsgericht Stuttgart
Web:     http://www.cksoft.de/         Geschaeftsfuehrer: Christian Kratzer

Comment 8 Howard Chu 2013-05-27 18:38:40 UTC
Christian Kratzer wrote:
> Hi Howard,
>
> On Mon, 27 May 2013, Howard Chu wrote:
>
>> Christian Kratzer wrote:
>>> Hi Howard,
>>
>> Never mind, this was actually a bug in the handling of proxied attributes.
>> Fixed now in master, your test case should work fine there.
>
> great.  Will the changes be automatically included in 2.4.36 when its released ?

Assuming you test and send positive feedback, yes.
>
> I'll test against master and deploy a workaround with an acl to the customer site in the meantime.
>
> Greetings
> Christian
>


-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/

Comment 9 Christian Kratzer 2013-05-27 19:50:22 UTC
Hi,

On Mon, 27 May 2013, Howard Chu wrote:
>> great.  Will the changes be automatically included in 2.4.36 when its 
>> released ?
>
> Assuming you test and send positive feedback, yes.

I checked out master using git://git.openldap.org/openldap.git and built my rpm from it.

My config-3-fail.ldif sample still fails to import into cn=config on centos-64

   [root@test-centos64 test]# slapadd -v -n0 -F config-3 -l config-3-fail.ldif
   51a3b6c1 str2entry: invalid value for attributeType modifiersName #0 (syntax 1.3.6.1.4.1.1466.115.121.1.12)
   slapadd: could not parse entry (line=1)
   _#                      7.35% eta   none elapsed            none spd   8.1 M/s
   Closing DB...
   [root@test-centos64 test]#

It fails on the first entry. Do you see anything obviously fishy in it ?

   dn: cn=config
   objectClass: olcGlobal
   cn: config
   olcConfigFile: slapd.conf.test
   olcConfigDir: slapd.d/
   olcArgsFile: /var/run/openldap/slapd.args
   olcAttributeOptions: lang-
   olcAuthzPolicy: none
   olcConcurrency: 0
   olcConnMaxPending: 100
   olcConnMaxPendingAuth: 1000
   olcGentleHUP: FALSE
   olcIdleTimeout: 0
   olcIndexSubstrIfMaxLen: 4
   olcIndexSubstrIfMinLen: 2
   olcIndexSubstrAnyLen: 4
   olcIndexSubstrAnyStep: 2
   olcIndexIntLen: 4
   olcLocalSSF: 71
   olcPidFile: /var/run/openldap/slapd.pid
   olcReadOnly: FALSE
   olcReverseLookup: FALSE
   olcSaslSecProps: noplain,noanonymous
   olcSockbufMaxIncoming: 262143
   olcSockbufMaxIncomingAuth: 16777215
   olcThreads: 16
   olcTLSCRLCheck: none
   olcTLSVerifyClient: never
   olcToolThreads: 1
   olcWriteTimeout: 0
   olcAttributeTypes: ( 2.5.4.11 NAME ( 'ou' 'organizationalUnitName' ) DESC '
    RFC2256: organizational unit this object belongs to' SUP name )
   olcAttributeTypes: ( 0.9.2342.19200300.100.1.25 NAME ( 'dc' 'domainCompone
    nt' ) DESC 'RFC1274/2247: domain component' EQUALITY caseIgnoreIA5Match SUBST
    R caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 SINGLE-VA
    LUE )
   structuralObjectClass: olcGlobal
   entryUUID: 3b1e9034-58d9-1032-8161-d3a3b8e342e7
   creatorsName: cn=config
   createTimestamp: 20130524161753Z
   olcLogLevel: Conns
   olcLogLevel: Stats
   olcLogLevel: Stats2
   entryCSN: 20130524161850.764209Z#000000#000#000000
   modifiersName: cn=Alice,ou=People,dc=test
   modifyTimestamp: 20130524161850Z

Can you verify above error on your setup.

In the meantime I will build manually with default options without any rpm magic just to be sure.

Greetings
Christian

-- 
Christian Kratzer                      CK Software GmbH
Email:   ck@cksoft.de                  Wildberger Weg 24/2
Phone:   +49 7032 893 997 - 0          D-71126 Gaeufelden
Fax:     +49 7032 893 997 - 9          HRB 245288, Amtsgericht Stuttgart
Web:     http://www.cksoft.de/         Geschaeftsfuehrer: Christian Kratzer

Comment 10 Howard Chu 2013-05-28 01:53:14 UTC
Christian Kratzer wrote:
> Hi,
>
> On Mon, 27 May 2013, Howard Chu wrote:
>>> great.  Will the changes be automatically included in 2.4.36 when its
>>> released ?
>>
>> Assuming you test and send positive feedback, yes.
>
> I checked out master using git://git.openldap.org/openldap.git and built my rpm from it.
>
> My config-3-fail.ldif sample still fails to import into cn=config on centos-64
>
>     [root@test-centos64 test]# slapadd -v -n0 -F config-3 -l config-3-fail.ldif
>     51a3b6c1 str2entry: invalid value for attributeType modifiersName #0 (syntax 1.3.6.1.4.1.1466.115.121.1.12)
>     slapadd: could not parse entry (line=1)
>     _#                      7.35% eta   none elapsed            none spd   8.1 M/s
>     Closing DB...
>     [root@test-centos64 test]#
>
> It fails on the first entry. Do you see anything obviously fishy in it ?

This is now fixed in git master. Nothing fishy about the entry besides the 
modifiersName. I've now tweaked slapadd to allow it.

>
>     dn: cn=config
>     objectClass: olcGlobal
>     cn: config
>     olcConfigFile: slapd.conf.test
>     olcConfigDir: slapd.d/
>     olcArgsFile: /var/run/openldap/slapd.args
>     olcAttributeOptions: lang-
>     olcAuthzPolicy: none
>     olcConcurrency: 0
>     olcConnMaxPending: 100
>     olcConnMaxPendingAuth: 1000
>     olcGentleHUP: FALSE
>     olcIdleTimeout: 0
>     olcIndexSubstrIfMaxLen: 4
>     olcIndexSubstrIfMinLen: 2
>     olcIndexSubstrAnyLen: 4
>     olcIndexSubstrAnyStep: 2
>     olcIndexIntLen: 4
>     olcLocalSSF: 71
>     olcPidFile: /var/run/openldap/slapd.pid
>     olcReadOnly: FALSE
>     olcReverseLookup: FALSE
>     olcSaslSecProps: noplain,noanonymous
>     olcSockbufMaxIncoming: 262143
>     olcSockbufMaxIncomingAuth: 16777215
>     olcThreads: 16
>     olcTLSCRLCheck: none
>     olcTLSVerifyClient: never
>     olcToolThreads: 1
>     olcWriteTimeout: 0
>     olcAttributeTypes: ( 2.5.4.11 NAME ( 'ou' 'organizationalUnitName' ) DESC '
>      RFC2256: organizational unit this object belongs to' SUP name )
>     olcAttributeTypes: ( 0.9.2342.19200300.100.1.25 NAME ( 'dc' 'domainCompone
>      nt' ) DESC 'RFC1274/2247: domain component' EQUALITY caseIgnoreIA5Match SUBST
>      R caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 SINGLE-VA
>      LUE )
>     structuralObjectClass: olcGlobal
>     entryUUID: 3b1e9034-58d9-1032-8161-d3a3b8e342e7
>     creatorsName: cn=config
>     createTimestamp: 20130524161753Z
>     olcLogLevel: Conns
>     olcLogLevel: Stats
>     olcLogLevel: Stats2
>     entryCSN: 20130524161850.764209Z#000000#000#000000
>     modifiersName: cn=Alice,ou=People,dc=test
>     modifyTimestamp: 20130524161850Z
>
> Can you verify above error on your setup.
>
> In the meantime I will build manually with default options without any rpm magic just to be sure.
>
> Greetings
> Christian
>


-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/

Comment 11 Christian Kratzer 2013-05-28 11:40:47 UTC
Hi Howard,

On Mon, 27 May 2013, Howard Chu wrote:
<snipp/>
> This is now fixed in git master. Nothing fishy about the entry besides the 
> modifiersName. I've now tweaked slapadd to allow it.

I can confirm my test cases all work now on openlap master on CentOS 6.4:

I also applied following two patches to openldap-2.4.35 and tested with the customers full configuration:

 	From 6dab36e97afb680452a13efd41e8653a8949e2aa Mon Sep 17 00:00:00 2001
 	From: Howard Chu <hyc@openldap.org>
 	Date: Mon, 27 May 2013 08:57:15 -0700
 	Subject: [PATCH] ITS#7608 promoted attrs must have valid ad_index

 	From b7df586674c65f1637ab5a59bd0a386f2031e95f Mon Sep 17 00:00:00 2001
 	From: Howard Chu <hyc@openldap.org>
 	Date: Mon, 27 May 2013 18:51:34 -0700
 	Subject: [PATCH] ITS#7608 allow slapadd w/unknown RDNs for config DB

Thanks a lot for the prompt fix.

Greetings
Christian

-- 
Christian Kratzer                      CK Software GmbH
Email:   ck@cksoft.de                  Wildberger Weg 24/2
Phone:   +49 7032 893 997 - 0          D-71126 Gaeufelden
Fax:     +49 7032 893 997 - 9          HRB 245288, Amtsgericht Stuttgart
Web:     http://www.cksoft.de/         Geschaeftsfuehrer: Christian Kratzer

Comment 12 Howard Chu 2013-05-28 13:06:25 UTC
Christian Kratzer wrote:
> Hi Howard,
>
> On Mon, 27 May 2013, Howard Chu wrote:
> <snipp/>
>> This is now fixed in git master. Nothing fishy about the entry besides the
>> modifiersName. I've now tweaked slapadd to allow it.
>
> I can confirm my test cases all work now on openlap master on CentOS 6.4:
>
> I also applied following two patches to openldap-2.4.35 and tested with the customers full configuration:
>
>   	From 6dab36e97afb680452a13efd41e8653a8949e2aa Mon Sep 17 00:00:00 2001
>   	From: Howard Chu <hyc@openldap.org>
>   	Date: Mon, 27 May 2013 08:57:15 -0700
>   	Subject: [PATCH] ITS#7608 promoted attrs must have valid ad_index
>
>   	From b7df586674c65f1637ab5a59bd0a386f2031e95f Mon Sep 17 00:00:00 2001
>   	From: Howard Chu <hyc@openldap.org>
>   	Date: Mon, 27 May 2013 18:51:34 -0700
>   	Subject: [PATCH] ITS#7608 allow slapadd w/unknown RDNs for config DB
>
> Thanks a lot for the prompt fix.

You're welcome. Thanks for confirming.
>
> Greetings
> Christian
>


-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/

Comment 13 Christian Kratzer 2013-05-28 14:19:54 UTC
Hi Howard,
On Tue, 28 May 2013, Howard Chu wrote:
>> Thanks a lot for the prompt fix.
>
> You're welcome. Thanks for confirming.

could you commit both patches to RE24 so they would end up in 2.4.36 when it gets released.

Greetings
Christian

-- 
Christian Kratzer                      CK Software GmbH
Email:   ck@cksoft.de                  Wildberger Weg 24/2
Phone:   +49 7032 893 997 - 0          D-71126 Gaeufelden
Fax:     +49 7032 893 997 - 9          HRB 245288, Amtsgericht Stuttgart
Web:     http://www.cksoft.de/         Geschaeftsfuehrer: Christian Kratzer

Comment 14 Quanah Gibson-Mount 2013-05-28 17:06:37 UTC
changed notes
changed state Test to Release
Comment 15 Quanah Gibson-Mount 2013-09-09 15:21:28 UTC
changed notes
changed state Release to Closed
Comment 16 OpenLDAP project 2014-08-01 21:04:47 UTC
fixed in master
fixed in RE24