Re: (ITS#7432) slapd crashes when removing members from a large group

[hyc@symas.com]

marvin.mundry@uni-hamburg.de wrote:
> Full_Name: Marvin Mundry
> Version: 2.4.33
> OS: Ubuntu 12.10
> URL: https://idmswiki.rrz.uni-hamburg.de:8005/debug.tar.bz
> Submission from: (NULL) (134.100.2.183)
>

Thanks for the report. The crash has been fixed in git, but your test runs 
into another (known) issue in MDB.

You're working with a very large entry, which libmdb stores in overflow pages. 
In the current version of libmdb, freespace management for overflow pages is 
not fully implemented, so every time you update the entry libmdb will always 
use new pages (instead of reusing old pages). Thus, after a few hundred 
operations, your 1GB map will be exhausted.

It looks like you won't be able to use back-mdb until this feature is fully 
implemented in libmdb.

> i have slapcat'ed my production environment and slapadded it into an back_mdb
> database.
>
> when deleting members one by one from a large group (48428 Members), after some
> transactions slapd crashes.
> when i slapadd my dit to an mdb without having an eq index configured for
> "member" the removal of members works fine.
>
> when doing the same thing (member indexed, slapadding members, starting slapd,
> removing group members one by one) on a hdb, slapd does not crash.
>
> sortvals is set for members as otherwise deletion operations would take a very
> long time
>
> i failed to reproduce the problem without initially slapadding the data from my
> production environment into the mdb.
>
> i have removed all configuration settings not neccessary to reproduce the bug
>
> the debug.tar.bz should contain everything that is neccessary (including a
> description) to reproduce the problem. additionally the file contains a coredump
> and the full trace output
>
> slapd.conf:
> include		/usr/local/etc/openldap/schema/core.schema
> include		/usr/local/etc/openldap/schema/cosine.schema
> include		/usr/local/etc/openldap/schema/inetorgperson.schema
> include		/usr/local/etc/openldap/schema/rfc2307bis.schema
> pidfile		/usr/local/var/run/slapd.pid
> argsfile	/usr/local/var/run/slapd.args
> modulepath	/usr/local/libexec/openldap
> moduleload	back_mdb.la
> sortvals	member
> database	mdb
> suffix		"dc=test,dc=com"
> rootdn		"cn=admin,dc=test,dc=com"
> rootpw		secret
> directory	/usr/local/etc/openldap/test123db/AL
> index member eq
> maxsize	1000000000
>
>
> last lines of slapd -d Trace:
> 5096b5ae oc_check_allowed type "modifyTimestamp"
> 5096b5ae sl_malloc 8: ch_malloc #this line repeats 28450 times
> 5096b5ae => mdb_entry_encode(0x00010170): cn=gr97503,ou=groups,dc=test,dc=com
> 5096b5ae <= mdb_entry_encode(0x00010170): cn=gr97503,ou=groups,dc=test,dc=com
>
> build options:
> CFLAGS=-O0 -d
> ./configure --enable-dynamic=yes --enable-syslog=yes --enable-slapd=yes
> --enable-dynacl=yes --enable-crypt=yes --enable-modules=yes --enable-rewrite=ye
> s --enable-bdb=mod --enable-hdb=mod --enable-ldap=mod --enable-mdb=mod
> --enable-meta=mod --enable-monitor=mod --enable-overlays=mod --with-tls=openssl
>
> --enable-debug
>
> (gdb) bt:
> #0  0x000002fc000002fa in ?? ()
> #1  0x000002ff000002fd in ?? ()
> #2  0x0000030200000300 in ?? ()
> #3  0x0000030500000303 in ?? ()
> #4  0x0000030800000306 in ?? ()
> #5  0x0000030b00000309 in ?? ()
> #6  0x00007f1400000312 in ?? ()
> #7  0x0000009cffffffbf in ?? ()
> #8  0x00007f1400000011 in ?? ()
> #9  0x00000000000102a3 in ?? ()
> #10 0x00007f140c101880 in ?? ()
> #11 0x00007f140883c200 in ?? ()
> #12 0x0000000000000891 in ?? ()
> #13 0x00007f14155ef060 in ?? ()
> #14 0x00007f14523dd5a6 in ?? ()
> #15 0x00007f1451fa1010 in ?? ()
> #16 0x00000000523dd2a7 in ?? ()
> #17 0x00007f14155ef390 in ?? ()
> #18 0x0000000000000000 in ?? ()
>
>


-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/