Full_Name: Maxime Besson Version: 2.4.48 OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (77.193.139.162) I am attempting to implement the following disaster recovery process: * rm all previous data * run a configuration script (puppet) to recreate a bare-bones LDAP server and DIT * restore a backed-up slapcat dump on top of the freshly installed OpenLDAP server, ignoring duplicates already inserted by Puppet But it seems that when skipping over the existing objects, something goes wrong and causes attributes to get mixed up. But only when certain attributes are present in the existing objects. Here is how to reproduce: slapd.conf: === include /etc/ldap/schema/core.schema include /etc/ldap/schema/cosine.schema include /etc/ldap/schema/inetorgperson.schema moduleload back_mdb database mdb maxsize 1073741824 suffix "dc=example,dc=com" directory /tmp === puppet_init.ldif === dn: dc=example,dc=com objectClass: domain dc: example === backup.ldif === dn: dc=example,dc=com objectClass: domain dc: example contextCSN: 20190909094705.796552Z#000000#001#000000 dn: uid=ttully,dc=example,dc=com objectClass: inetOrgPerson uid: ttully userPassword:: c2Nob29uZXI= facsimileTelephoneNumber: +1 408 555 0111 givenName: Torrey cn: Torrey Tully telephoneNumber: +1 408 555 2274 sn: Tully roomNumber: 3924 mail: ttully@example.com l: Sunnyvale ou: Human Resources ou: People === When running the following commands: === rm -f /tmp/*.mdb slapadd -f slapd.conf puppet_init.ldif slapadd -f slapd.conf -c backup.ldif === The redundant root object in backup.ldif gets skipped as -c should do, but the attributes from my "ttully" user (and every following object in the ldif dump) end up all mixed up: === slapcat -f slapd.conf ... dn: uid=ttully,dc=example,dc=com objectClass: inetOrgPerson userPassword:: dHR1bGx5 facsimileTelephoneNumber: schooner givenName: +1 408 555 0111 cn: Torrey telephoneNumber: Torrey Tully sn: +1 408 555 2274 roomNumber: Tully mail: 3924 l: ttully@example.com ou: Sunnyvale ou: Human Resources ou: People ... === This mixup does NOT happen if: * I import backup.ldif into an empty database * OR I remove "contextCSN" from the backup ldif * OR I use BDB instead of MDB So it seems that my issue is caused by a combination of MDB, skipping existing entries, and having special attributes (contextCSN) in the skipped objects. I was able to reproduce this behavior: * On Debian Buster (OpenLDAP 2.4.47) * On RHEL7 + LTB project RPMs 2.4.48 * Using a git snapshot (3be82f40d5cd4ca050e10859ecb961f28c807c41) with no particular config options * Using cn=config rather than slapd.conf * On a real production system, rather than the simplified version presented here. * Using another "special" attribute such as pwdAccountLockedTime instead of "contextCSN"
maxime.besson@worteks.com wrote: > Full_Name: Maxime Besson > Version: 2.4.48 > OS: Linux > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (77.193.139.162) > > > I am attempting to implement the following disaster recovery process: > > * rm all previous data > * run a configuration script (puppet) to recreate a bare-bones LDAP server and > DIT > * restore a backed-up slapcat dump on top of the freshly installed OpenLDAP > server, ignoring duplicates already inserted by Puppet > > But it seems that when skipping over the existing objects, something goes wrong > and causes attributes to get mixed up. But only when certain attributes are > present in the existing objects. Here is how to reproduce: Thanks for the report and simple test case. This is now fixed in git master. > > > slapd.conf: > === > include /etc/ldap/schema/core.schema > include /etc/ldap/schema/cosine.schema > include /etc/ldap/schema/inetorgperson.schema > moduleload back_mdb > database mdb > maxsize 1073741824 > suffix "dc=example,dc=com" > directory /tmp > === > > > puppet_init.ldif > === > dn: dc=example,dc=com > objectClass: domain > dc: example > === > > backup.ldif > === > dn: dc=example,dc=com > objectClass: domain > dc: example > contextCSN: 20190909094705.796552Z#000000#001#000000 > > dn: uid=ttully,dc=example,dc=com > objectClass: inetOrgPerson > uid: ttully > userPassword:: c2Nob29uZXI= > facsimileTelephoneNumber: +1 408 555 0111 > givenName: Torrey > cn: Torrey Tully > telephoneNumber: +1 408 555 2274 > sn: Tully > roomNumber: 3924 > mail: ttully@example.com > l: Sunnyvale > ou: Human Resources > ou: People > === > > When running the following commands: > > === > rm -f /tmp/*.mdb > slapadd -f slapd.conf puppet_init.ldif > slapadd -f slapd.conf -c backup.ldif > === > > The redundant root object in backup.ldif gets skipped as -c should do, but the > attributes from my "ttully" user (and every following object in the ldif dump) > end up all mixed up: > > > === > slapcat -f slapd.conf > ... > dn: uid=ttully,dc=example,dc=com > objectClass: inetOrgPerson > userPassword:: dHR1bGx5 > facsimileTelephoneNumber: schooner > givenName: +1 408 555 0111 > cn: Torrey > telephoneNumber: Torrey Tully > sn: +1 408 555 2274 > roomNumber: Tully > mail: 3924 > l: ttully@example.com > ou: Sunnyvale > ou: Human Resources > ou: People > ... > === > > > This mixup does NOT happen if: > > * I import backup.ldif into an empty database > * OR I remove "contextCSN" from the backup ldif > * OR I use BDB instead of MDB > > So it seems that my issue is caused by a combination of MDB, skipping existing > entries, and having special attributes (contextCSN) in the skipped objects. > > I was able to reproduce this behavior: > * On Debian Buster (OpenLDAP 2.4.47) > * On RHEL7 + LTB project RPMs 2.4.48 > * Using a git snapshot (3be82f40d5cd4ca050e10859ecb961f28c807c41) with no > particular config options > * Using cn=config rather than slapd.conf > * On a real production system, rather than the simplified version presented > here. > * Using another "special" attribute such as pwdAccountLockedTime instead of > "contextCSN" > > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
fixed in master
changed notes changed state Open to Test moved from Incoming to Software Bugs
Howard Chu wrote: > maxime.besson@worteks.com wrote: >> Full_Name: Maxime Besson >> Version: 2.4.48 >> OS: Linux >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (77.193.139.162) >> >> >> I am attempting to implement the following disaster recovery process: >> >> * rm all previous data >> * run a configuration script (puppet) to recreate a bare-bones LDAP server and >> DIT >> * restore a backed-up slapcat dump on top of the freshly installed OpenLDAP >> server, ignoring duplicates already inserted by Puppet >> >> But it seems that when skipping over the existing objects, something goes wrong >> and causes attributes to get mixed up. But only when certain attributes are >> present in the existing objects. Here is how to reproduce: > > Thanks for the report and simple test case. This is now fixed in git master. I'd also note that using -c has always been pretty risky. You would be better off using -j in situations like this, as show below: >> >> puppet_init.ldif >> === >> dn: dc=example,dc=com >> objectClass: domain >> dc: example >> === >> >> backup.ldif >> === >> dn: dc=example,dc=com >> objectClass: domain >> dc: example >> contextCSN: 20190909094705.796552Z#000000#001#000000 >> >> dn: uid=ttully,dc=example,dc=com >> objectClass: inetOrgPerson >> uid: ttully >> userPassword:: c2Nob29uZXI= >> facsimileTelephoneNumber: +1 408 555 0111 >> givenName: Torrey >> cn: Torrey Tully >> telephoneNumber: +1 408 555 2274 >> sn: Tully >> roomNumber: 3924 >> mail: ttully@example.com >> l: Sunnyvale >> ou: Human Resources >> ou: People >> === >> >> When running the following commands: >> >> === >> rm -f /tmp/*.mdb >> slapadd -f slapd.conf puppet_init.ldif >> slapadd -f slapd.conf -c backup.ldif >> === slapadd -f slapd.conf -l puppet_init.ldif slapadd -f slapd.conf -l backup.ldif -j 5 -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
On 10/14/19 8:47 PM, Howard Chu wrote: > > I'd also note that using -c has always been pretty risky. You would be better off using -j > in situations like this, as show below: > Thanks for the fix and tip!
Fixed in 2.4.49