[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: OpenLDAP freezes and doesn't respond





Here're the traces you required, for the 1st 4 OpenLDAP threads at least:

(gdb) attach 20884
Attaching to program: /opt/openldap-2.4.26/libexec/slapd, process 20884
Reading symbols from /opt/openldap-2.4.26/lib/libldap_r-2.4.so.2...done.
Loaded symbols for /opt/openldap-2.4.26/lib/libldap_r-2.4.so.2
Reading symbols from /opt/openldap-2.4.26/lib/liblber-2.4.so.2...done.
Loaded symbols for /opt/openldap-2.4.26/lib/liblber-2.4.so.2
Reading symbols from /lib64/libuuid.so.1...done.
Loaded symbols for /lib64/libuuid.so.1
Reading symbols from /usr/local/BerkeleyDB.4.8/lib/libdb-4.8.so...done.
Loaded symbols for /usr/local/BerkeleyDB.4.8/lib/libdb-4.8.so
Reading symbols from /lib64/libpthread.so.0...done.
[Thread debugging using libthread_db enabled]
[New Thread 0x2b1c9a53dbd0 (LWP 20884)]
[New Thread 0x48a36940 (LWP 21499)]
[New Thread 0x40930940 (LWP 21498)]
[New Thread 0x48235940 (LWP 21203)]
[New Thread 0x47a34940 (LWP 21202)]
[New Thread 0x47233940 (LWP 21201)]
[New Thread 0x46a32940 (LWP 21200)]
[New Thread 0x46231940 (LWP 21087)]
[New Thread 0x45a30940 (LWP 21086)]
[New Thread 0x4522f940 (LWP 21021)]
[New Thread 0x44a2e940 (LWP 21020)]
[New Thread 0x4422d940 (LWP 20891)]
[New Thread 0x43a2c940 (LWP 20890)]
[New Thread 0x4322b940 (LWP 20889)]
[New Thread 0x42a2a940 (LWP 20888)]
[New Thread 0x42229940 (LWP 20887)]
[New Thread 0x41a28940 (LWP 20886)]
[New Thread 0x41227940 (LWP 20885)]
Loaded symbols for /lib64/libpthread.so.0
Reading symbols from /usr/lib64/libsasl2.so.2...done.
Loaded symbols for /usr/lib64/libsasl2.so.2
Reading symbols from /lib64/libssl.so.6...done.
Loaded symbols for /lib64/libssl.so.6
Reading symbols from /lib64/libcrypto.so.6...done.
Loaded symbols for /lib64/libcrypto.so.6
Reading symbols from /lib64/libresolv.so.2...done.
Loaded symbols for /lib64/libresolv.so.2
Reading symbols from /lib64/libc.so.6...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib64/libdl.so.2...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/libcrypt.so.1...done.
Loaded symbols for /lib64/libcrypt.so.1
Reading symbols from /usr/lib64/libgssapi_krb5.so.2...done.
Loaded symbols for /usr/lib64/libgssapi_krb5.so.2
Reading symbols from /usr/lib64/libkrb5.so.3...done.
Loaded symbols for /usr/lib64/libkrb5.so.3
Reading symbols from /lib64/libcom_err.so.2...done.
Loaded symbols for /lib64/libcom_err.so.2
Reading symbols from /usr/lib64/libk5crypto.so.3...done.
Loaded symbols for /usr/lib64/libk5crypto.so.3
Reading symbols from /usr/lib64/libz.so.1...done.
Loaded symbols for /usr/lib64/libz.so.1
Reading symbols from /usr/lib64/libkrb5support.so.0...done.
Loaded symbols for /usr/lib64/libkrb5support.so.0
Reading symbols from /lib64/libkeyutils.so.1...done.
Loaded symbols for /lib64/libkeyutils.so.1
Reading symbols from /lib64/libselinux.so.1...done.
Loaded symbols for /lib64/libselinux.so.1
Reading symbols from /lib64/libsepol.so.1...done.
Loaded symbols for /lib64/libsepol.so.1
Reading symbols from /lib64/libnss_files.so.2...done.
Loaded symbols for /lib64/libnss_files.so.2
Reading symbols from /lib64/libnss_dns.so.2...done.
Loaded symbols for /lib64/libnss_dns.so.2
Reading symbols from /usr/lib64/sasl2/liblogin.so.2...done.
Loaded symbols for /usr/lib64/sasl2/liblogin.so.2
Reading symbols from /usr/lib64/sasl2/libanonymous.so.2...done.
Loaded symbols for /usr/lib64/sasl2/libanonymous.so.2
Reading symbols from /usr/lib64/sasl2/libplain.so.2...done.
Loaded symbols for /usr/lib64/sasl2/libplain.so.2
Reading symbols from /usr/lib64/sasl2/libsasldb.so.2...done.
Loaded symbols for /usr/lib64/sasl2/libsasldb.so.2
0x00000031a86077e5 in pthread_join () from /lib64/libpthread.so.0
(gdb) bt
#0  0x00000031a86077e5 in pthread_join () from /lib64/libpthread.so.0
#1  0x000000000042aca4 in slapd_daemon () at daemon.c:2922
#2  0x0000000000409f34 in main (argc=7, argv=0x7ffff62c9618) at main.c:983
(gdb) detach
Detaching from program: /opt/openldap-2.4.26/libexec/slapd, process 20884
(gdb) attach 21499
Attaching to program: /opt/openldap-2.4.26/libexec/slapd, process 21499
warning: process 21499 is a cloned process
Reading symbols from /opt/openldap-2.4.26/lib/libldap_r-2.4.so.2...done.
Loaded symbols for /opt/openldap-2.4.26/lib/libldap_r-2.4.so.2
Reading symbols from /opt/openldap-2.4.26/lib/liblber-2.4.so.2...done.
Loaded symbols for /opt/openldap-2.4.26/lib/liblber-2.4.so.2
Reading symbols from /lib64/libuuid.so.1...done.
Loaded symbols for /lib64/libuuid.so.1
Reading symbols from /usr/local/BerkeleyDB.4.8/lib/libdb-4.8.so...done.
Loaded symbols for /usr/local/BerkeleyDB.4.8/lib/libdb-4.8.so
Reading symbols from /lib64/libpthread.so.0...done.
[Thread debugging using libthread_db enabled]
[New Thread 0x2b1c9a53dbd0 (LWP 20884)]
[New Thread 0x48a36940 (LWP 21499)]
[New Thread 0x40930940 (LWP 21498)]
[New Thread 0x48235940 (LWP 21203)]
[New Thread 0x47a34940 (LWP 21202)]
[New Thread 0x47233940 (LWP 21201)]
[New Thread 0x46a32940 (LWP 21200)]
[New Thread 0x46231940 (LWP 21087)]
[New Thread 0x45a30940 (LWP 21086)]
[New Thread 0x4522f940 (LWP 21021)]
[New Thread 0x44a2e940 (LWP 21020)]
[New Thread 0x4422d940 (LWP 20891)]
[New Thread 0x43a2c940 (LWP 20890)]
[New Thread 0x4322b940 (LWP 20889)]
[New Thread 0x42a2a940 (LWP 20888)]
[New Thread 0x42229940 (LWP 20887)]
[New Thread 0x41a28940 (LWP 20886)]
[New Thread 0x41227940 (LWP 20885)]
Loaded symbols for /lib64/libpthread.so.0
Reading symbols from /usr/lib64/libsasl2.so.2...done.
Loaded symbols for /usr/lib64/libsasl2.so.2
Reading symbols from /lib64/libssl.so.6...done.
Loaded symbols for /lib64/libssl.so.6
Reading symbols from /lib64/libcrypto.so.6...done.
Loaded symbols for /lib64/libcrypto.so.6
Reading symbols from /lib64/libresolv.so.2...done.
Loaded symbols for /lib64/libresolv.so.2
Reading symbols from /lib64/libc.so.6...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib64/libdl.so.2...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/libcrypt.so.1...done.
Loaded symbols for /lib64/libcrypt.so.1
Reading symbols from /usr/lib64/libgssapi_krb5.so.2...done.
Loaded symbols for /usr/lib64/libgssapi_krb5.so.2
Reading symbols from /usr/lib64/libkrb5.so.3...done.
Loaded symbols for /usr/lib64/libkrb5.so.3
Reading symbols from /lib64/libcom_err.so.2...done.
Loaded symbols for /lib64/libcom_err.so.2
Reading symbols from /usr/lib64/libk5crypto.so.3...done.
Loaded symbols for /usr/lib64/libk5crypto.so.3
Reading symbols from /usr/lib64/libz.so.1...done.
Loaded symbols for /usr/lib64/libz.so.1
Reading symbols from /usr/lib64/libkrb5support.so.0...done.
Loaded symbols for /usr/lib64/libkrb5support.so.0
Reading symbols from /lib64/libkeyutils.so.1...done.
Loaded symbols for /lib64/libkeyutils.so.1
Reading symbols from /lib64/libselinux.so.1...done.
Loaded symbols for /lib64/libselinux.so.1
Reading symbols from /lib64/libsepol.so.1...done.
Loaded symbols for /lib64/libsepol.so.1
Reading symbols from /lib64/libnss_files.so.2...done.
Loaded symbols for /lib64/libnss_files.so.2
Reading symbols from /lib64/libnss_dns.so.2...done.
Loaded symbols for /lib64/libnss_dns.so.2
Reading symbols from /usr/lib64/sasl2/liblogin.so.2...done.
Loaded symbols for /usr/lib64/sasl2/liblogin.so.2
Reading symbols from /usr/lib64/sasl2/libanonymous.so.2...done.
Loaded symbols for /usr/lib64/sasl2/libanonymous.so.2
Reading symbols from /usr/lib64/sasl2/libplain.so.2...done.
Loaded symbols for /usr/lib64/sasl2/libplain.so.2
Reading symbols from /usr/lib64/sasl2/libsasldb.so.2...done.
Loaded symbols for /usr/lib64/sasl2/libsasldb.so.2
0x00000031a860ab99 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
(gdb) bt
#0 0x00000031a860ab99 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002b1c99d65f1b in ldap_pvt_thread_rmutex_lock (rmutex=<value optimized out>, owner=1218668864) at rmutex.c:129 #2 0x000000000052efb7 in accesslog_op_mod (op=0x48a34fe0, rs=0x48a35150) at accesslog.c:1889 #3 0x00000000004bbb2b in overlay_op_walk (op=0x48a34fe0, rs=0x48a35150, which=op_modify, oi=0x1ed31f10, on=0x1ed32d60) at backover.c:661 #4 0x00000000004bbdbd in over_op_func (op=0x48a34fe0, rs=0x48a35150, which=op_modify) at backover.c:723 #5 0x00000000004bbed3 in over_op_modify (op=0x48a34fe0, rs=0x48a35150) at backover.c:762 #6 0x000000000053c7af in ppolicy_bind_response (op=0x1fbf8d90, rs=0x48a35c30) at ppolicy.c:1173 #7 0x0000000000443a08 in slap_response_play (op=0x1fbf8d90, rs=0x48a35c30) at result.c:505 #8 0x0000000000443c2f in send_ldap_response (op=0x1fbf8d90, rs=0x48a35c30) at result.c:579 #9 0x0000000000444b14 in slap_send_ldap_result (op=0x1fbf8d90, rs=0x48a35c30) at result.c:851 #10 0x000000000050f2f3 in hdb_bind (op=0x1fbf8d90, rs=0x48a35c30) at bind.c:158 #11 0x00000000004bbbba in overlay_op_walk (op=0x1fbf8d90, rs=0x48a35c30, which=op_bind, oi=0x1ed31f10, on=0x0) at backover.c:671 #12 0x00000000004bbdbd in over_op_func (op=0x1fbf8d90, rs=0x48a35c30, which=op_bind) at backover.c:723 #13 0x00000000004bbe43 in over_op_bind (op=0x1fbf8d90, rs=0x48a35c30) at backover.c:738 #14 0x0000000000456411 in fe_op_bind (op=0x1fbf8d90, rs=0x48a35c30) at bind.c:383 #15 0x00000000004bbbba in overlay_op_walk (op=0x1fbf8d90, rs=0x48a35c30, which=op_bind, oi=0x1ed02760, on=0x0) at backover.c:671 #16 0x00000000004bbdbd in over_op_func (op=0x1fbf8d90, rs=0x48a35c30, which=op_bind) at backover.c:723 #17 0x00000000004bbe43 in over_op_bind (op=0x1fbf8d90, rs=0x48a35c30) at backover.c:738 #18 0x0000000000455b40 in do_bind (op=0x1fbf8d90, rs=0x48a35c30) at bind.c:205 #19 0x000000000042dbce in connection_operation (ctx=0x48a35d70, arg_v=0x1fbf8d90) at connection.c:1138 #20 0x000000000042e153 in connection_read_thread (ctx=0x48a35d70, argv=0x30) at connection.c:1274 #21 0x00002b1c99d66fc8 in ldap_int_thread_pool_wrapper (xpool=0x1ec9acb0) at tpool.c:685
#22 0x00000031a86064a7 in start_thread () from /lib64/libpthread.so.0
#23 0x00000031a7ad3c2d in clone () from /lib64/libc.so.6
(gdb) detach
Detaching from program: /opt/openldap-2.4.26/libexec/slapd, process 21499
(gdb) attach 21498
Attaching to program: /opt/openldap-2.4.26/libexec/slapd, process 21498
warning: process 21498 is a cloned process
Reading symbols from /opt/openldap-2.4.26/lib/libldap_r-2.4.so.2...done.
Loaded symbols for /opt/openldap-2.4.26/lib/libldap_r-2.4.so.2
Reading symbols from /opt/openldap-2.4.26/lib/liblber-2.4.so.2...done.
Loaded symbols for /opt/openldap-2.4.26/lib/liblber-2.4.so.2
Reading symbols from /lib64/libuuid.so.1...done.
Loaded symbols for /lib64/libuuid.so.1
Reading symbols from /usr/local/BerkeleyDB.4.8/lib/libdb-4.8.so...done.
Loaded symbols for /usr/local/BerkeleyDB.4.8/lib/libdb-4.8.so
Reading symbols from /lib64/libpthread.so.0...done.
[Thread debugging using libthread_db enabled]
[New Thread 0x2b1c9a53dbd0 (LWP 20884)]
[New Thread 0x48a36940 (LWP 21499)]
[New Thread 0x40930940 (LWP 21498)]
[New Thread 0x48235940 (LWP 21203)]
[New Thread 0x47a34940 (LWP 21202)]
[New Thread 0x47233940 (LWP 21201)]
[New Thread 0x46a32940 (LWP 21200)]
[New Thread 0x46231940 (LWP 21087)]
[New Thread 0x45a30940 (LWP 21086)]
[New Thread 0x4522f940 (LWP 21021)]
[New Thread 0x44a2e940 (LWP 21020)]
[New Thread 0x4422d940 (LWP 20891)]
[New Thread 0x43a2c940 (LWP 20890)]
[New Thread 0x4322b940 (LWP 20889)]
[New Thread 0x42a2a940 (LWP 20888)]
[New Thread 0x42229940 (LWP 20887)]
[New Thread 0x41a28940 (LWP 20886)]
[New Thread 0x41227940 (LWP 20885)]
Loaded symbols for /lib64/libpthread.so.0
Reading symbols from /usr/lib64/libsasl2.so.2...done.
Loaded symbols for /usr/lib64/libsasl2.so.2
Reading symbols from /lib64/libssl.so.6...done.
Loaded symbols for /lib64/libssl.so.6
Reading symbols from /lib64/libcrypto.so.6...done.
Loaded symbols for /lib64/libcrypto.so.6
Reading symbols from /lib64/libresolv.so.2...done.
Loaded symbols for /lib64/libresolv.so.2
Reading symbols from /lib64/libc.so.6...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib64/libdl.so.2...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/libcrypt.so.1...done.
Loaded symbols for /lib64/libcrypt.so.1
Reading symbols from /usr/lib64/libgssapi_krb5.so.2...done.
Loaded symbols for /usr/lib64/libgssapi_krb5.so.2
Reading symbols from /usr/lib64/libkrb5.so.3...done.
Loaded symbols for /usr/lib64/libkrb5.so.3
Reading symbols from /lib64/libcom_err.so.2...done.
Loaded symbols for /lib64/libcom_err.so.2
Reading symbols from /usr/lib64/libk5crypto.so.3...done.
Loaded symbols for /usr/lib64/libk5crypto.so.3
Reading symbols from /usr/lib64/libz.so.1...done.
Loaded symbols for /usr/lib64/libz.so.1
Reading symbols from /usr/lib64/libkrb5support.so.0...done.
Loaded symbols for /usr/lib64/libkrb5support.so.0
Reading symbols from /lib64/libkeyutils.so.1...done.
Loaded symbols for /lib64/libkeyutils.so.1
Reading symbols from /lib64/libselinux.so.1...done.
Loaded symbols for /lib64/libselinux.so.1
Reading symbols from /lib64/libsepol.so.1...done.
Loaded symbols for /lib64/libsepol.so.1
Reading symbols from /lib64/libnss_files.so.2...done.
Loaded symbols for /lib64/libnss_files.so.2
Reading symbols from /lib64/libnss_dns.so.2...done.
Loaded symbols for /lib64/libnss_dns.so.2
Reading symbols from /usr/lib64/sasl2/liblogin.so.2...done.
Loaded symbols for /usr/lib64/sasl2/liblogin.so.2
Reading symbols from /usr/lib64/sasl2/libanonymous.so.2...done.
Loaded symbols for /usr/lib64/sasl2/libanonymous.so.2
Reading symbols from /usr/lib64/sasl2/libplain.so.2...done.
Loaded symbols for /usr/lib64/sasl2/libplain.so.2
Reading symbols from /usr/lib64/sasl2/libsasldb.so.2...done.
Loaded symbols for /usr/lib64/sasl2/libsasldb.so.2
0x00000031a860ab99 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
(gdb) bt
#0 0x00000031a860ab99 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002b1c99d65f1b in ldap_pvt_thread_rmutex_lock (rmutex=<value optimized out>, owner=1083377984) at rmutex.c:129 #2 0x000000000052efb7 in accesslog_op_mod (op=0x4092efe0, rs=0x4092f150) at accesslog.c:1889 #3 0x00000000004bbb2b in overlay_op_walk (op=0x4092efe0, rs=0x4092f150, which=op_modify, oi=0x1ed31f10, on=0x1ed32d60) at backover.c:661 #4 0x00000000004bbdbd in over_op_func (op=0x4092efe0, rs=0x4092f150, which=op_modify) at backover.c:723 #5 0x00000000004bbed3 in over_op_modify (op=0x4092efe0, rs=0x4092f150) at backover.c:762 #6 0x000000000053c7af in ppolicy_bind_response (op=0x1f5bf360, rs=0x4092fc30) at ppolicy.c:1173 #7 0x0000000000443a08 in slap_response_play (op=0x1f5bf360, rs=0x4092fc30) at result.c:505 #8 0x0000000000443c2f in send_ldap_response (op=0x1f5bf360, rs=0x4092fc30) at result.c:579 #9 0x0000000000444b14 in slap_send_ldap_result (op=0x1f5bf360, rs=0x4092fc30) at result.c:851 #10 0x000000000050f2f3 in hdb_bind (op=0x1f5bf360, rs=0x4092fc30) at bind.c:158 #11 0x00000000004bbbba in overlay_op_walk (op=0x1f5bf360, rs=0x4092fc30, which=op_bind, oi=0x1ed31f10, on=0x0) at backover.c:671 #12 0x00000000004bbdbd in over_op_func (op=0x1f5bf360, rs=0x4092fc30, which=op_bind) at backover.c:723 #13 0x00000000004bbe43 in over_op_bind (op=0x1f5bf360, rs=0x4092fc30) at backover.c:738 #14 0x0000000000456411 in fe_op_bind (op=0x1f5bf360, rs=0x4092fc30) at bind.c:383 #15 0x00000000004bbbba in overlay_op_walk (op=0x1f5bf360, rs=0x4092fc30, which=op_bind, oi=0x1ed02760, on=0x0) at backover.c:671 #16 0x00000000004bbdbd in over_op_func (op=0x1f5bf360, rs=0x4092fc30, which=op_bind) at backover.c:723 #17 0x00000000004bbe43 in over_op_bind (op=0x1f5bf360, rs=0x4092fc30) at backover.c:738 #18 0x0000000000455b40 in do_bind (op=0x1f5bf360, rs=0x4092fc30) at bind.c:205 #19 0x000000000042dbce in connection_operation (ctx=0x4092fd70, arg_v=0x1f5bf360) at connection.c:1138 #20 0x000000000042e153 in connection_read_thread (ctx=0x4092fd70, argv=0x25) at connection.c:1274 #21 0x00002b1c99d66fc8 in ldap_int_thread_pool_wrapper (xpool=0x1ec9acb0) at tpool.c:685
#22 0x00000031a86064a7 in start_thread () from /lib64/libpthread.so.0
#23 0x00000031a7ad3c2d in clone () from /lib64/libc.so.6
(gdb) detach
Detaching from program: /opt/openldap-2.4.26/libexec/slapd, process 21498
(gdb) attach 21203
Attaching to program: /opt/openldap-2.4.26/libexec/slapd, process 21203
warning: process 21203 is a cloned process
Reading symbols from /opt/openldap-2.4.26/lib/libldap_r-2.4.so.2...done.
Loaded symbols for /opt/openldap-2.4.26/lib/libldap_r-2.4.so.2
Reading symbols from /opt/openldap-2.4.26/lib/liblber-2.4.so.2...done.
Loaded symbols for /opt/openldap-2.4.26/lib/liblber-2.4.so.2
Reading symbols from /lib64/libuuid.so.1...done.
Loaded symbols for /lib64/libuuid.so.1
Reading symbols from /usr/local/BerkeleyDB.4.8/lib/libdb-4.8.so...done.
Loaded symbols for /usr/local/BerkeleyDB.4.8/lib/libdb-4.8.so
Reading symbols from /lib64/libpthread.so.0...done.
[Thread debugging using libthread_db enabled]
[New Thread 0x2b1c9a53dbd0 (LWP 20884)]
[New Thread 0x48a36940 (LWP 21499)]
[New Thread 0x40930940 (LWP 21498)]
[New Thread 0x48235940 (LWP 21203)]
[New Thread 0x47a34940 (LWP 21202)]
[New Thread 0x47233940 (LWP 21201)]
[New Thread 0x46a32940 (LWP 21200)]
[New Thread 0x46231940 (LWP 21087)]
[New Thread 0x45a30940 (LWP 21086)]
[New Thread 0x4522f940 (LWP 21021)]
[New Thread 0x44a2e940 (LWP 21020)]
[New Thread 0x4422d940 (LWP 20891)]
[New Thread 0x43a2c940 (LWP 20890)]
[New Thread 0x4322b940 (LWP 20889)]
[New Thread 0x42a2a940 (LWP 20888)]
[New Thread 0x42229940 (LWP 20887)]
[New Thread 0x41a28940 (LWP 20886)]
[New Thread 0x41227940 (LWP 20885)]
Loaded symbols for /lib64/libpthread.so.0
Reading symbols from /usr/lib64/libsasl2.so.2...done.
Loaded symbols for /usr/lib64/libsasl2.so.2
Reading symbols from /lib64/libssl.so.6...done.
Loaded symbols for /lib64/libssl.so.6
Reading symbols from /lib64/libcrypto.so.6...done.
Loaded symbols for /lib64/libcrypto.so.6
Reading symbols from /lib64/libresolv.so.2...done.
Loaded symbols for /lib64/libresolv.so.2
Reading symbols from /lib64/libc.so.6...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib64/libdl.so.2...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/libcrypt.so.1...done.
Loaded symbols for /lib64/libcrypt.so.1
Reading symbols from /usr/lib64/libgssapi_krb5.so.2...done.
Loaded symbols for /usr/lib64/libgssapi_krb5.so.2
Reading symbols from /usr/lib64/libkrb5.so.3...done.
Loaded symbols for /usr/lib64/libkrb5.so.3
Reading symbols from /lib64/libcom_err.so.2...done.
Loaded symbols for /lib64/libcom_err.so.2
Reading symbols from /usr/lib64/libk5crypto.so.3...done.
Loaded symbols for /usr/lib64/libk5crypto.so.3
Reading symbols from /usr/lib64/libz.so.1...done.
Loaded symbols for /usr/lib64/libz.so.1
Reading symbols from /usr/lib64/libkrb5support.so.0...done.
Loaded symbols for /usr/lib64/libkrb5support.so.0
Reading symbols from /lib64/libkeyutils.so.1...done.
Loaded symbols for /lib64/libkeyutils.so.1
Reading symbols from /lib64/libselinux.so.1...done.
Loaded symbols for /lib64/libselinux.so.1
Reading symbols from /lib64/libsepol.so.1...done.
Loaded symbols for /lib64/libsepol.so.1
Reading symbols from /lib64/libnss_files.so.2...done.
Loaded symbols for /lib64/libnss_files.so.2
Reading symbols from /lib64/libnss_dns.so.2...done.
Loaded symbols for /lib64/libnss_dns.so.2
Reading symbols from /usr/lib64/sasl2/liblogin.so.2...done.
Loaded symbols for /usr/lib64/sasl2/liblogin.so.2
Reading symbols from /usr/lib64/sasl2/libanonymous.so.2...done.
Loaded symbols for /usr/lib64/sasl2/libanonymous.so.2
Reading symbols from /usr/lib64/sasl2/libplain.so.2...done.
Loaded symbols for /usr/lib64/sasl2/libplain.so.2
Reading symbols from /usr/lib64/sasl2/libsasldb.so.2...done.
Loaded symbols for /usr/lib64/sasl2/libsasldb.so.2
0x00000031a860ab99 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
(gdb) bt
#0 0x00000031a860ab99 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002b1c99d65f1b in ldap_pvt_thread_rmutex_lock (rmutex=<value optimized out>, owner=1210276160) at rmutex.c:129 #2 0x000000000052efb7 in accesslog_op_mod (op=0x48233fe0, rs=0x48234150) at accesslog.c:1889 #3 0x00000000004bbb2b in overlay_op_walk (op=0x48233fe0, rs=0x48234150, which=op_modify, oi=0x1ed31f10, on=0x1ed32d60) at backover.c:661 #4 0x00000000004bbdbd in over_op_func (op=0x48233fe0, rs=0x48234150, which=op_modify) at backover.c:723 #5 0x00000000004bbed3 in over_op_modify (op=0x48233fe0, rs=0x48234150) at backover.c:762 #6 0x000000000053c7af in ppolicy_bind_response (op=0x1fe91140, rs=0x48234c30) at ppolicy.c:1173 #7 0x0000000000443a08 in slap_response_play (op=0x1fe91140, rs=0x48234c30) at result.c:505 #8 0x0000000000443c2f in send_ldap_response (op=0x1fe91140, rs=0x48234c30) at result.c:579 #9 0x0000000000444b14 in slap_send_ldap_result (op=0x1fe91140, rs=0x48234c30) at result.c:851 #10 0x000000000050f2f3 in hdb_bind (op=0x1fe91140, rs=0x48234c30) at bind.c:158 #11 0x00000000004bbbba in overlay_op_walk (op=0x1fe91140, rs=0x48234c30, which=op_bind, oi=0x1ed31f10, on=0x0) at backover.c:671 #12 0x00000000004bbdbd in over_op_func (op=0x1fe91140, rs=0x48234c30, which=op_bind) at backover.c:723 #13 0x00000000004bbe43 in over_op_bind (op=0x1fe91140, rs=0x48234c30) at backover.c:738 #14 0x0000000000456411 in fe_op_bind (op=0x1fe91140, rs=0x48234c30) at bind.c:383 #15 0x00000000004bbbba in overlay_op_walk (op=0x1fe91140, rs=0x48234c30, which=op_bind, oi=0x1ed02760, on=0x0) at backover.c:671 #16 0x00000000004bbdbd in over_op_func (op=0x1fe91140, rs=0x48234c30, which=op_bind) at backover.c:723 #17 0x00000000004bbe43 in over_op_bind (op=0x1fe91140, rs=0x48234c30) at backover.c:738 #18 0x0000000000455b40 in do_bind (op=0x1fe91140, rs=0x48234c30) at bind.c:205 #19 0x000000000042dbce in connection_operation (ctx=0x48234d70, arg_v=0x1fe91140) at connection.c:1138 #20 0x000000000042e153 in connection_read_thread (ctx=0x48234d70, argv=0x64) at connection.c:1274 #21 0x00002b1c99d66fc8 in ldap_int_thread_pool_wrapper (xpool=0x1ec9acb0) at tpool.c:685
#22 0x00000031a86064a7 in start_thread () from /lib64/libpthread.so.0
#23 0x00000031a7ad3c2d in clone () from /lib64/libc.so.6
(gdb) detach
Detaching from program: /opt/openldap-2.4.26/libexec/slapd, process 21203
(gdb) attach 21202
Attaching to program: /opt/openldap-2.4.26/libexec/slapd, process 21202
warning: process 21202 is a cloned process
Reading symbols from /opt/openldap-2.4.26/lib/libldap_r-2.4.so.2...done.
Loaded symbols for /opt/openldap-2.4.26/lib/libldap_r-2.4.so.2
Reading symbols from /opt/openldap-2.4.26/lib/liblber-2.4.so.2...done.
Loaded symbols for /opt/openldap-2.4.26/lib/liblber-2.4.so.2
Reading symbols from /lib64/libuuid.so.1...done.
Loaded symbols for /lib64/libuuid.so.1
Reading symbols from /usr/local/BerkeleyDB.4.8/lib/libdb-4.8.so...done.
Loaded symbols for /usr/local/BerkeleyDB.4.8/lib/libdb-4.8.so
Reading symbols from /lib64/libpthread.so.0...done.
[Thread debugging using libthread_db enabled]
[New Thread 0x2b1c9a53dbd0 (LWP 20884)]
[New Thread 0x48a36940 (LWP 21499)]
[New Thread 0x40930940 (LWP 21498)]
[New Thread 0x48235940 (LWP 21203)]
[New Thread 0x47a34940 (LWP 21202)]
[New Thread 0x47233940 (LWP 21201)]
[New Thread 0x46a32940 (LWP 21200)]
[New Thread 0x46231940 (LWP 21087)]
[New Thread 0x45a30940 (LWP 21086)]
[New Thread 0x4522f940 (LWP 21021)]
[New Thread 0x44a2e940 (LWP 21020)]
[New Thread 0x4422d940 (LWP 20891)]
[New Thread 0x43a2c940 (LWP 20890)]
[New Thread 0x4322b940 (LWP 20889)]
[New Thread 0x42a2a940 (LWP 20888)]
[New Thread 0x42229940 (LWP 20887)]
[New Thread 0x41a28940 (LWP 20886)]
[New Thread 0x41227940 (LWP 20885)]
Loaded symbols for /lib64/libpthread.so.0
Reading symbols from /usr/lib64/libsasl2.so.2...done.
Loaded symbols for /usr/lib64/libsasl2.so.2
Reading symbols from /lib64/libssl.so.6...done.
Loaded symbols for /lib64/libssl.so.6
Reading symbols from /lib64/libcrypto.so.6...done.
Loaded symbols for /lib64/libcrypto.so.6
Reading symbols from /lib64/libresolv.so.2...done.
Loaded symbols for /lib64/libresolv.so.2
Reading symbols from /lib64/libc.so.6...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib64/libdl.so.2...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/libcrypt.so.1...done.
Loaded symbols for /lib64/libcrypt.so.1
Reading symbols from /usr/lib64/libgssapi_krb5.so.2...done.
Loaded symbols for /usr/lib64/libgssapi_krb5.so.2
Reading symbols from /usr/lib64/libkrb5.so.3...done.
Loaded symbols for /usr/lib64/libkrb5.so.3
Reading symbols from /lib64/libcom_err.so.2...done.
Loaded symbols for /lib64/libcom_err.so.2
Reading symbols from /usr/lib64/libk5crypto.so.3...done.
Loaded symbols for /usr/lib64/libk5crypto.so.3
Reading symbols from /usr/lib64/libz.so.1...done.
Loaded symbols for /usr/lib64/libz.so.1
Reading symbols from /usr/lib64/libkrb5support.so.0...done.
Loaded symbols for /usr/lib64/libkrb5support.so.0
Reading symbols from /lib64/libkeyutils.so.1...done.
Loaded symbols for /lib64/libkeyutils.so.1
Reading symbols from /lib64/libselinux.so.1...done.
Loaded symbols for /lib64/libselinux.so.1
Reading symbols from /lib64/libsepol.so.1...done.
Loaded symbols for /lib64/libsepol.so.1
Reading symbols from /lib64/libnss_files.so.2...done.
Loaded symbols for /lib64/libnss_files.so.2
Reading symbols from /lib64/libnss_dns.so.2...done.
Loaded symbols for /lib64/libnss_dns.so.2
Reading symbols from /usr/lib64/sasl2/liblogin.so.2...done.
Loaded symbols for /usr/lib64/sasl2/liblogin.so.2
Reading symbols from /usr/lib64/sasl2/libanonymous.so.2...done.
Loaded symbols for /usr/lib64/sasl2/libanonymous.so.2
Reading symbols from /usr/lib64/sasl2/libplain.so.2...done.
Loaded symbols for /usr/lib64/sasl2/libplain.so.2
Reading symbols from /usr/lib64/sasl2/libsasldb.so.2...done.
Loaded symbols for /usr/lib64/sasl2/libsasldb.so.2
0x00000031a860ab99 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
(gdb) bt
#0 0x00000031a860ab99 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002b1c99d65f1b in ldap_pvt_thread_rmutex_lock (rmutex=<value optimized out>, owner=1201883456) at rmutex.c:129 #2 0x000000000052efb7 in accesslog_op_mod (op=0x47a32fe0, rs=0x47a33150) at accesslog.c:1889 #3 0x00000000004bbb2b in overlay_op_walk (op=0x47a32fe0, rs=0x47a33150, which=op_modify, oi=0x1ed31f10, on=0x1ed32d60) at backover.c:661 #4 0x00000000004bbdbd in over_op_func (op=0x47a32fe0, rs=0x47a33150, which=op_modify) at backover.c:723 #5 0x00000000004bbed3 in over_op_modify (op=0x47a32fe0, rs=0x47a33150) at backover.c:762 #6 0x000000000053c7af in ppolicy_bind_response (op=0x1fe6e000, rs=0x47a33c30) at ppolicy.c:1173 #7 0x0000000000443a08 in slap_response_play (op=0x1fe6e000, rs=0x47a33c30) at result.c:505 #8 0x0000000000443c2f in send_ldap_response (op=0x1fe6e000, rs=0x47a33c30) at result.c:579 #9 0x0000000000444b14 in slap_send_ldap_result (op=0x1fe6e000, rs=0x47a33c30) at result.c:851 #10 0x000000000050f2f3 in hdb_bind (op=0x1fe6e000, rs=0x47a33c30) at bind.c:158 #11 0x00000000004bbbba in overlay_op_walk (op=0x1fe6e000, rs=0x47a33c30, which=op_bind, oi=0x1ed31f10, on=0x0) at backover.c:671 #12 0x00000000004bbdbd in over_op_func (op=0x1fe6e000, rs=0x47a33c30, which=op_bind) at backover.c:723 #13 0x00000000004bbe43 in over_op_bind (op=0x1fe6e000, rs=0x47a33c30) at backover.c:738 #14 0x0000000000456411 in fe_op_bind (op=0x1fe6e000, rs=0x47a33c30) at bind.c:383 #15 0x00000000004bbbba in overlay_op_walk (op=0x1fe6e000, rs=0x47a33c30, which=op_bind, oi=0x1ed02760, on=0x0) at backover.c:671 #16 0x00000000004bbdbd in over_op_func (op=0x1fe6e000, rs=0x47a33c30, which=op_bind) at backover.c:723 #17 0x00000000004bbe43 in over_op_bind (op=0x1fe6e000, rs=0x47a33c30) at backover.c:738 #18 0x0000000000455b40 in do_bind (op=0x1fe6e000, rs=0x47a33c30) at bind.c:205 #19 0x000000000042dbce in connection_operation (ctx=0x47a33d70, arg_v=0x1fe6e000) at connection.c:1138 #20 0x000000000042e153 in connection_read_thread (ctx=0x47a33d70, argv=0x6a) at connection.c:1274 #21 0x00002b1c99d66fc8 in ldap_int_thread_pool_wrapper (xpool=0x1ec9acb0) at tpool.c:685
#22 0x00000031a86064a7 in start_thread () from /lib64/libpthread.so.0
#23 0x00000031a7ad3c2d in clone () from /lib64/libc.so.6

I can provide you with the other threads backtrace if needed.




Quanah Gibson-Mount a écrit :
--On Thursday, August 04, 2011 2:23 PM +0200 Cyril Grosjean <cgrosjean@janua.fr> wrote:


I've just tested with OpenLDAP 2.4.26, as you suggested, but the problem
remains.

I've no load at all,  I use mirror mode replication, and this time, the
freeze occurred when trying to modify a user's password:
The modification is done from a usual Web interface from a Java
application. The Java application is set to connect to OpenLDAP
with a unique dedicated account, but in the case of a password change, we
use the PROXYAUTH operation to do the modification
with the user's rights, so that his pwdreset attribute is removed if
previously set to TRUE

I've also opened a case with Red Hat, they suggested me to test under
RHEL 6, with their OpenLDAP rpm package (2.4.23-15.el6).
So I'll do it right now but if you need more details on the RHEL 5.4 hung
slapd process, I can send them since it's still frozen.

Please build with debugging symbols and no optimization (-O0 and -g). Make sure when you install, you set STRIP="" so it doesn't strip the symbols out. Then get a full backtrace.

--Quanah

--

Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra ::  the leader in open source messaging and collaboration