Full_Name: Frank Dopheide Version: slapd 2.4.44 (Mar 3 2016 10:27:05) OS: Ubuntu 14.04.3 LTS URL: http://52.48.8.167/idlcache_error.pdf Submission from: (NULL) (87.139.29.59) Good morning Gentlemen, we've found a quite serious bug in slapd 2.4.44 on an Ubuntu 14.04.3 LTS server. As soon as we enable the idlcache the ldapsearch results will be wrong after moddn and modrdn commands. First result after idlcachesize activation is still correct as the cache is still empty and the daemon reads from db, but as soon as it starts reading from idlcache the latest modifications are missing until slapd will be restarted! We've created a small howto to replicatehe e error. The howto is available via http: http://52.48.8.167/idlcache_error.pdf Please let us now if you require any additional information! Thx, Frank
Hi, I am also adding the guide to replicate the error as plain text: Step-by-step guide The following guide was created on an Ubuntu Linux V. 14.04.3 LTS, but it should be executeable on an Debian based Linux operating systems. You should be logged in as root user or adopt all commands with sudo. 1. Add hostname echo 'ldap-debug.example.com' > /etc/hostname && hostname `cat /etc/hostname` && hostname echo '52.49.174.211 ldap-debug.example.com ldap-debug' >> /etc/hosts 2. Download latest Sources wget http://www.openldap.org/software/download/OpenLDAP/openldap-release/openldap-2.4.44.tgz 3. Validate Sources(compare md5 provided on website with generated md5 hash) md5sum openldap-2.4.44.tgz 4. Make archive executable and extract sources and change into extracted directory chmod +x openldap-2.4.44.tgz && tar xzf openldap-2.4.44.tgz && cd openldap-2.4.44 5. Fullfil most important software requirements (Dependencies) apt-get update apt-get install libdb-dev apt-get install openssl apt-get install gnutls-bin apt-get install gcc apt-get install make 6. Run configure with default settings and add a target dir only ./configure --prefix=/usr/local/openldap 7. Run make and install sources to target directory make depend make make test make install 8. Stop the daemon, because slapd was already started by make. pkill slapd 9. Edit main config file in /usr/local/openldap/etc/openldap/slapd.conf and add the following lines: include /usr/local/openldap/etc/openldap/schema/core.schema pidfile /usr/local/openldap/var/run/slapd.pid argsfile /usr/local/openldap/var/run/slapd.args modulepath /usr/lib/ldap moduleload back_hdb backend hdb database hdb suffix "dc=example,dc=com" rootdn "cn=Manager,dc=example,dc=com" rootpw secret directory /var/lib/ldap index objectClass eq 10. Call slappasswd and set rootpw to secret slappasswd 11. Create data directory for hdb files mkdir /var/lib/ldap 12. We will add a symbolic link to ease slapd commands ln -s /usr/local/openldap/libexec/slapd /usr/local/bin/slapd 13. Add the path to OpenLDAP bin and sbin dirs and activate the new path echo "export PATH=\$PATH:/usr/local/openldap/bin/:/usr/local/openldap/sbin/" >> /root/.bashrc && source /root/.bashrc 14. Start the slapd daemon with -f flag for location of the config file slapd -f /usr/local/openldap/etc/openldap/slapd.conf 15. Create initial DIT file with an editor of your choice, i.e. vi /root/create_dit.ldif and copy paste the following lines into this file. dn: dc=example,dc=com changetype: add objectClass: dcObject objectClass: organization objectClass: top dc: example o: example.com dn: ou=Projects,dc=example,dc=com changetype: add ou: Projects objectClass: organizationalUnit dn: ou=Trash,dc=example,dc=com changetype: add ou: Trash objectClass: organizationalUnit dn: ou=test1,ou=Projects,dc=example,dc=com changetype: add ou: test1 objectClass: organizationalUnit dn: ou=test2,ou=test1,ou=Projects,dc=example,dc=com changetype: add ou: test2 objectClass: organizationalUnit 16. Add the initial DIT ldapadd -H 'ldap://localhost' -D 'cn=Manager,dc=example,dc=com' -w secret -f /root/create_dit.ldif 17. Execute modrdn and assign directory Trash as a new superior of test1 directory ldapmodrdn -x -w secret -D "cn=Manager,dc=example,dc=com" -s "ou=Trash,dc=example,dc=com" -r "ou=test1,ou=Projects,dc=example,dc=com" "ou=test1" 18. Execute following ldapsearch and save the result somewhere for i in {,{,{,ou=test2\,}ou=test1\,}ou=Trash\,}dc=example,dc=com; do echo -e "\n=====> basedn: $i <=====\n"; ldapsearch -LLL -s 'sub' -H 'ldap://localhost:389' -D 'cn=Manager,dc=example,dc=com' -w secret -b "$i" '(ou=test*)' dn; done Result: =====> basedn: dc=example,dc=com <===== dn: ou=test1,ou=Trash,dc=example,dc=com dn: ou=test2,ou=test1,ou=Trash,dc=example,dc=com =====> basedn: ou=Trash,dc=example,dc=com <===== dn: ou=test1,ou=Trash,dc=example,dc=com dn: ou=test2,ou=test1,ou=Trash,dc=example,dc=com =====> basedn: ou=test1,ou=Trash,dc=example,dc=com <===== dn: ou=test1,ou=Trash,dc=example,dc=com dn: ou=test2,ou=test1,ou=Trash,dc=example,dc=com =====> basedn: ou=test2,ou=test1,ou=Trash,dc=example,dc=com <===== dn: ou=test2,ou=test1,ou=Trash,dc=example,dc=com 19. We see until here everything is working as expected, but now we will enable cache and idlcache as recommended in OpenLDAP docs and Performance Guides and problems will appear soon. echo -e "cachesize\t10000\nidlcachesize\t30000" >> /usr/local/openldap/etc/openldap/slapd.conf 20. Remove existing DIT ldapdelete -H 'ldap://localhost' -D 'cn=Manager,dc=example,dc=com' -w secret -r "dc=example,dc=com" -v 21. Restart daemon and recreate initial DIT pkill slapd slapd -f /usr/local/openldap/etc/openldap/slapd.conf ldapadd -H 'ldap://localhost' -D 'cn=Manager,dc=example,dc=com' -w secret -f /root/create_dit.ldif 22. Execute modrdn ldapmodrdn -x -w secret -D "cn=Manager,dc=example,dc=com" -s "ou=Trash,dc=example,dc=com" -r "ou=test1,ou=Projects,dc=example,dc=com" "ou=test1" 23. Check result again with ldapsearch for i in {,{,{,ou=test2\,}ou=test1\,}ou=Trash\,}dc=example,dc=com; do echo -e "\n=====> basedn: $i <=====\n"; ldapsearch -LLL -s 'sub' -H 'ldap://localhost:389' -D 'cn=Manager,dc=example,dc=com' -w secret -b "$i" '(ou=test*)' dn; done Result: =====> basedn: dc=example,dc=com <===== dn: ou=test1,ou=Trash,dc=example,dc=com dn: ou=test2,ou=test1,ou=Trash,dc=example,dc=com =====> basedn: ou=Trash,dc=example,dc=com <===== dn: ou=test1,ou=Trash,dc=example,dc=com dn: ou=test2,ou=test1,ou=Trash,dc=example,dc=com =====> basedn: ou=test1,ou=Trash,dc=example,dc=com <===== dn: ou=test1,ou=Trash,dc=example,dc=com dn: ou=test2,ou=test1,ou=Trash,dc=example,dc=com =====> basedn: ou=test2,ou=test1,ou=Trash,dc=example,dc=com <===== dn: ou=test2,ou=test1,ou=Trash,dc=example,dc=com 24. Once again the result was fine and as expected. Keep in mind cache and idlcache were empty as we've restarted the daemon. Now we are performing same actions again while we have cached results. 25. Remove directories in ou „Trash“ ldapdelete -w secret -D "cn=Manager,dc=example,dc=com" -r "ou=test1,ou=Trash,dc=example,dc=com" 26. Add directories to ou „Projects“ again ldapmodify -D "cn=Manager,dc=example,dc=com" -W Type in password Enter LDAP Password: Copy paste following ldif and be sure emtpy lines are copied, too! dn: ou=test1,ou=Projects,dc=example,dc=com changetype: add objectClass: organizationalUnit dn: ou=test2,ou=test1,ou=Projects,dc=example,dc=com changetype: add objectClass: organizationalUnit 27. Now we have same DIT structure again and can rerun modrdn command ldapmodrdn -x -w secret -D "cn=Manager,dc=example,dc=com" -s "ou=Trash,dc=example,dc=com" -r "ou=test1,ou=Projects,dc=example,dc=com" "ou=test1" 28. If we now check result with ldapsearch again the idlcache error will appear for i in {,{,{,ou=test2\,}ou=test1\,}ou=Trash\,}dc=example,dc=com; do echo -e "\n=====> basedn: $i <=====\n"; ldapsearch -LLL -s 'sub' -H 'ldap://localhost:389' -D 'cn=Manager,dc=example,dc=com' -w secret -b "$i" '(ou=test*)' dn; done Result: =====> basedn: dc=example,dc=com <===== dn: ou=test1,ou=Trash,dc=example,dc=com dn: ou=test2,ou=test1,ou=Trash,dc=example,dc=com =====> basedn: ou=Trash,dc=example,dc=com <===== dn: ou=test1,ou=Trash,dc=example,dc=com =====> basedn: ou=test1,ou=Trash,dc=example,dc=com <===== dn: ou=test1,ou=Trash,dc=example,dc=com dn: ou=test2,ou=test1,ou=Trash,dc=example,dc=com =====> basedn: ou=test2,ou=test1,ou=Trash,dc=example,dc=com <===== dn: ou=test2,ou=test1,ou=Trash,dc=example,dc=com 29. The search result of basedn: ou=Trash,dc=example,dc=com is wrong! The line of dn: ou=test2,ou=test1,ou=Trash,dc=example,dc=com is missing within the resultset. 30. After a restart of the daemon the missing line will appear again. Actually the one and only solution is to switch of idlcachesize and set it explicitely to "0" > Am 07.03.2016 um 11:45 schrieb fdopheide@iponweb.net: > > Full_Name: Frank Dopheide > Version: slapd 2.4.44 (Mar 3 2016 10:27:05) > OS: Ubuntu 14.04.3 LTS > URL: http://52.48.8.167/idlcache_error.pdf > Submission from: (NULL) (87.139.29.59) > > > Good morning Gentlemen, > > we've found a quite serious bug in slapd 2.4.44 on an Ubuntu 14.04.3 LTS server. > As soon as we enable the idlcache the ldapsearch results will be wrong after > moddn and modrdn commands. First result after idlcachesize activation is still > correct as the cache is still empty and the daemon reads from db, but as soon as > it starts reading from idlcache the latest modifications are missing until slapd > will be restarted! > > We've created a small howto to replicatehe e error. The howto is available via > http: http://52.48.8.167/idlcache_error.pdf > > Please let us now if you require any additional information! > > Thx, > Frank > > >
moved from Incoming to Software Bugs
*** This issue has been marked as a duplicate of issue 8378 ***