Re: (ITS#7537) ppolicy hangs slapd on 64bit version debian

[ryan@nardis.ca]

I reproduced this with current RE24. It seems to be caused by setting 
pwdMaxAge to a very large value.

In Marek's previous debug output you can see his policy object as 
returned from a search. Among other attrs it has:

pwdMaxAge: 2592000000

I reproduced the report with the following setup:

### slapd.conf

include ../openldap/servers/slapd/schema/core.schema
include ../openldap/servers/slapd/schema/ppolicy.schema

moduleload back_hdb
moduleload ppolicy

database hdb
directory db
suffix dc=example,dc=com
rootdn cn=root,dc=example,dc=com
rootpw secret

access to *
	by dn="cn=admin,dc=example,dc=com" manage
	by * read

index objectClass eq

overlay ppolicy
ppolicy_default cn=default,ou=policies,dc=example,dc=com

### init.ldif

dn: dc=example,dc=com
objectClass: domain

dn: cn=admin,dc=example,dc=com
objectClass: organizationalRole
objectClass: simpleSecurityObject
userPassword: secret

dn: ou=policies,dc=example,dc=com
objectClass: organizationalUnit

dn: cn=default,ou=policies,dc=example,dc=com
objectClass: organizationalRole
objectClass: pwdPolicy
pwdAttribute: userPassword
pwdMaxAge: 2592000000

### mod.ldif

dn: cn=default,ou=policies,dc=example,dc=com
add: pwdMaxFailure
pwdMaxFailure: 7

Launched slapd and triggered via:

$ ldapmodify -x -D cn=admin,dc=example,dc=com -W -f mod.ldif

slapd hangs, ldapmodify is still waiting for it, and gdb says:

Program received signal SIGINT, Interrupt.
0x00007ffff75f24db in pthread_join (threadid=140737286698752, thread_return=0x0) at pthread_join.c:92
92	pthread_join.c: No such file or directory.
(gdb) thread apply all bt

Thread 3 (Thread 0x7ffff3faf700 (LWP 25350)):
#0  0x00007ffff7326653 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81
#1  0x00000000004257b9 in slapd_daemon_task (ptr=0xa83a10) at daemon.c:2539
#2  0x00007ffff75f10a4 in start_thread (arg=0x7ffff3faf700) at pthread_create.c:309
#3  0x00007ffff732607d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Thread 2 (Thread 0x7ffff37ae700 (LWP 25353)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00007ffff7a4e79d in __db_pthread_mutex_lock () from /usr/lib/x86_64-linux-gnu/libdb-5.3.so
#2  0x00007ffff7af74e0 in __lock_get_internal () from /usr/lib/x86_64-linux-gnu/libdb-5.3.so
#3  0x00007ffff7af8719 in __lock_vec () from /usr/lib/x86_64-linux-gnu/libdb-5.3.so
#4  0x00007ffff7af8e9f in __lock_vec_pp () from /usr/lib/x86_64-linux-gnu/libdb-5.3.so
#5  0x000000000052f7ed in hdb_cache_entry_db_relock (bdb=0x934920, txn=0x7fffe4102810, ei=0x7fffe4101e80, rw=1,
    tryOnly=0, lock=0x7ffff37ac340) at cache.c:198
#6  0x0000000000531a43 in hdb_cache_modify (bdb=0x934920, e=0x9aca18, newAttrs=0x9c0690, txn=0x7fffe4102810,
    lock=0x7ffff37ac340) at cache.c:1231
#7  0x00000000004d85f6 in hdb_modify (op=0x7fffe4000aa0, rs=0x7ffff37adae0) at modify.c:764
#8  0x00000000004b740c in overlay_op_walk (op=0x7fffe4000aa0, rs=0x7ffff37adae0, which=op_modify, oi=0x936420, on=0x0)
    at backover.c:677
#9  0x00000000004b7637 in over_op_func (op=0x7fffe4000aa0, rs=0x7ffff37adae0, which=op_modify) at backover.c:730
#10 0x00000000004b776b in over_op_modify (op=0x7fffe4000aa0, rs=0x7ffff37adae0) at backover.c:769
#11 0x0000000000449541 in fe_op_modify (op=0x7fffe4000aa0, rs=0x7ffff37adae0) at modify.c:303
#12 0x0000000000448e13 in do_modify (op=0x7fffe4000aa0, rs=0x7ffff37adae0) at modify.c:177
#13 0x0000000000429a3f in connection_operation (ctx=0x7ffff37adc10, arg_v=0x7fffe4000aa0) at connection.c:1155
#14 0x0000000000429fdb in connection_read_thread (ctx=0x7ffff37adc10, argv=0xe) at connection.c:1291
#15 0x000000000058415d in ldap_int_thread_pool_wrapper (xpool=0x909fd0) at tpool.c:696
#16 0x00007ffff75f10a4 in start_thread (arg=0x7ffff37ae700) at pthread_create.c:309
#17 0x00007ffff732607d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Thread 1 (Thread 0x7ffff7fec700 (LWP 25346)):
#0  0x00007ffff75f24db in pthread_join (threadid=140737286698752, thread_return=0x0) at pthread_join.c:92
#1  0x00000000005855f3 in ldap_pvt_thread_join (thread=140737286698752, thread_return=0x0) at thr_posix.c:197
#2  0x0000000000426986 in slapd_daemon () at daemon.c:2932
#3  0x000000000040602d in main (argc=8, argv=0x7fffffffe598) at main.c:1017

If I change the config to use back-mdb instead, I get a crash:

55dce2bf conn=1000 op=1 MOD dn="cn=default,ou=policies,dc=example,dc=com"
55dce2bf conn=1000 op=1 MOD attr=pwdMaxFailure
slapd: id2entry.c:520: mdb_opinfo_get: Assertion `!rc' failed.
[New Thread 0x7ffff2caf700 (LWP 25374)]
[New Thread 0x7ffff34b0700 (LWP 25371)]

Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7ffff2caf700 (LWP 25374)]
0x00007ffff7275107 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56	../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  0x00007ffff7275107 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007ffff72764e8 in __GI_abort () at abort.c:89
#2  0x00007ffff726e226 in __assert_fail_base (fmt=0x7ffff73a4d08 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
    assertion=assertion@entry=0x5f29e4 "!rc", file=file@entry=0x5f2975 "id2entry.c", line=line@entry=520,
    function=function@entry=0x5f2b30 <__PRETTY_FUNCTION__.12200> "mdb_opinfo_get") at assert.c:92
#3  0x00007ffff726e2d2 in __GI___assert_fail (assertion=0x5f29e4 "!rc", file=0x5f2975 "id2entry.c", line=520,
    function=0x5f2b30 <__PRETTY_FUNCTION__.12200> "mdb_opinfo_get") at assert.c:101
#4  0x0000000000553a71 in mdb_opinfo_get (op=0x7fffe4000aa0, mdb=0x7ffff7f2a010, rdonly=1, moip=0x7ffff2cacf18)
    at id2entry.c:520
#5  0x000000000055306b in mdb_entry_get (op=0x7fffe4000aa0, ndn=0x7fffe4000ad8, oc=0x0, at=0x0, rw=0,
    ent=0x7ffff2cad2c8) at id2entry.c:327
#6  0x00000000004b6a41 in overlay_entry_get_ov (op=0x7fffe4000aa0, dn=0x7fffe4000ad8, oc=0x0, ad=0x0, rw=0,
    e=0x7ffff2cad2c8, on=0x0) at backover.c:364
#7  0x00000000004b6b11 in over_entry_get_rw (op=0x7fffe4000aa0, dn=0x7fffe4000ad8, oc=0x0, ad=0x0, rw=0,
    e=0x7ffff2cad2c8) at backover.c:396
#8  0x000000000043ca86 in be_entry_get_rw (op=0x7fffe4000aa0, ndn=0x7fffe4000ad8, oc=0x0, at=0x0, rw=0,
    e=0x7ffff2cad2c8) at backend.c:1366
#9  0x0000000000561e6c in ppolicy_modify (op=0x7fffe4000aa0, rs=0x7ffff2caeae0) at ppolicy.c:1626
#10 0x00000000004b7362 in overlay_op_walk (op=0x7fffe4000aa0, rs=0x7ffff2caeae0, which=op_modify, oi=0x935700,
    on=0x9358e0) at backover.c:661
#11 0x00000000004b7637 in over_op_func (op=0x7fffe4000aa0, rs=0x7ffff2caeae0, which=op_modify) at backover.c:730
#12 0x00000000004b776b in over_op_modify (op=0x7fffe4000aa0, rs=0x7ffff2caeae0) at backover.c:769
#13 0x0000000000449541 in fe_op_modify (op=0x7fffe4000aa0, rs=0x7ffff2caeae0) at modify.c:303
#14 0x0000000000448e13 in do_modify (op=0x7fffe4000aa0, rs=0x7ffff2caeae0) at modify.c:177
#15 0x0000000000429a3f in connection_operation (ctx=0x7ffff2caec10, arg_v=0x7fffe4000aa0) at connection.c:1155
#16 0x0000000000429fdb in connection_read_thread (ctx=0x7ffff2caec10, argv=0xb) at connection.c:1291
#17 0x000000000058415d in ldap_int_thread_pool_wrapper (xpool=0x909fd0) at tpool.c:696
#18 0x00007ffff75f10a4 in start_thread (arg=0x7ffff2caf700) at pthread_create.c:309
#19 0x00007ffff732607d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

It seems like the crash does not happen if I bind as the rootdn, only 
when using the admin DN. The hang with back-hdb happens with both.

Like I mentioned above, the problem seems to be related to the large 
pwdMaxAge value. If I drop a few 0s off the end of it, everything works 
properly. That might be related to Marek's statement that he only 
experienced the problem on 64-bit...