Full_Name: John Morrissey Version: 2.4.16 OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (2001:4978:194:0:21f:5bff:fee9:da92) After about 7 days of uptime, slapd catches a SIGABRT and dies. The failed assertion is in connection_close(): assert( c->c_writers == 0); Backtrace is below. I also have a ~1.7GB core if it's deemed useful; I'll keep it around for a week or two. This is with BDB 4.7+all three patches. GNU gdb 6.8-debian Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i486-linux-gnu"... Attaching to program: /usr/sbin/slapd, process 9365 Reading symbols from /usr/lib/libtcmalloc.so.0...done. Loaded symbols for /usr/lib/libtcmalloc.so.0 Reading symbols from /usr/lib/libprofiler.so.0...done. Loaded symbols for /usr/lib/libprofiler.so.0 Reading symbols from /usr/lib/libldap_r-2.4.so.2...Reading symbols from /usr/lib/debug/usr/lib/libldap_r-2.4.so.2.4.2...done. done. Loaded symbols for /usr/lib/libldap_r-2.4.so.2 Reading symbols from /usr/lib/liblber-2.4.so.2...Reading symbols from /usr/lib/debug/usr/lib/liblber-2.4.so.2.4.2...done. done. Loaded symbols for /usr/lib/liblber-2.4.so.2 Reading symbols from /usr/lib/libdb-4.7.so...done. Loaded symbols for /usr/lib/libdb-4.7.so Reading symbols from /usr/lib/libodbc.so.1...done. Loaded symbols for /usr/lib/libodbc.so.1 Reading symbols from /usr/lib/libslp.so.1...done. Loaded symbols for /usr/lib/libslp.so.1 Reading symbols from /usr/lib/libsasl2.so.2...done. Loaded symbols for /usr/lib/libsasl2.so.2 Reading symbols from /usr/lib/libgnutls.so.26...Reading symbols from /usr/lib/debug/usr/lib/libgnutls.so.26.4.6...done. done. Loaded symbols for /usr/lib/libgnutls.so.26 Reading symbols from /usr/lib/debug/libcrypt.so.1...done. Loaded symbols for /usr/lib/debug/libcrypt.so.1 Reading symbols from /usr/lib/debug/libresolv.so.2...done. Loaded symbols for /usr/lib/debug/libresolv.so.2 Reading symbols from /usr/lib/libltdl.so.3...done. Loaded symbols for /usr/lib/libltdl.so.3 Reading symbols from /usr/lib/debug/libpthread.so.0...done. [Thread debugging using libthread_db enabled] [New Thread 0xb78648e0 (LWP 9365)] [New Thread 0x60e2eb90 (LWP 9371)] [New Thread 0x6122eb90 (LWP 9370)] [New Thread 0x61644b90 (LWP 9369)] [New Thread 0x61a54b90 (LWP 9368)] [New Thread 0x61e54b90 (LWP 9367)] Loaded symbols for /usr/lib/debug/libpthread.so.0 Reading symbols from /usr/lib/debug/libc.so.6...done. Loaded symbols for /usr/lib/debug/libc.so.6 Reading symbols from /usr/lib/libstdc++.so.6...done. Loaded symbols for /usr/lib/libstdc++.so.6 Reading symbols from /usr/lib/debug/libm.so.6...done. Loaded symbols for /usr/lib/debug/libm.so.6 Reading symbols from /lib/libgcc_s.so.1...done. Loaded symbols for /lib/libgcc_s.so.1 Reading symbols from /usr/lib/debug/libnsl.so.1...done. Loaded symbols for /usr/lib/debug/libnsl.so.1 Reading symbols from /usr/lib/debug/libdl.so.2...done. Loaded symbols for /usr/lib/debug/libdl.so.2 Reading symbols from /usr/lib/libtasn1.so.3...done. Loaded symbols for /usr/lib/libtasn1.so.3 Reading symbols from /usr/lib/libgpg-error.so.0...done. Loaded symbols for /usr/lib/libgpg-error.so.0 Reading symbols from /usr/lib/libz.so.1...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /usr/lib/libgcrypt.so.11...done. Loaded symbols for /usr/lib/libgcrypt.so.11 Reading symbols from /lib/ld-linux.so.2...Reading symbols from /usr/lib/debug/lib/ld-2.7.so...done. done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /usr/lib/debug/libnss_files.so.2...done. Loaded symbols for /usr/lib/debug/libnss_files.so.2 Reading symbols from /usr/lib/debug/libnss_compat.so.2...done. Loaded symbols for /usr/lib/debug/libnss_compat.so.2 Reading symbols from /usr/lib/debug/libnss_nis.so.2...done. Loaded symbols for /usr/lib/debug/libnss_nis.so.2 Reading symbols from /lib/libnss_ldap.so.2...done. Loaded symbols for /lib/libnss_ldap.so.2 Reading symbols from /usr/lib/libkrb5.so.3...done. Loaded symbols for /usr/lib/libkrb5.so.3 Reading symbols from /lib/libcom_err.so.2...done. Loaded symbols for /lib/libcom_err.so.2 Reading symbols from /usr/lib/libgssapi_krb5.so.2...done. Loaded symbols for /usr/lib/libgssapi_krb5.so.2 Reading symbols from /usr/lib/libk5crypto.so.3...done. Loaded symbols for /usr/lib/libk5crypto.so.3 Reading symbols from /usr/lib/libkrb5support.so.0...done. Loaded symbols for /usr/lib/libkrb5support.so.0 Reading symbols from /lib/libkeyutils.so.1...done. Loaded symbols for /lib/libkeyutils.so.1 Reading symbols from /usr/lib/sasl2/libsasldb.so.2...done. Loaded symbols for /usr/lib/sasl2/libsasldb.so.2 Reading symbols from /usr/lib/libdb-4.6.so...done. Loaded symbols for /usr/lib/libdb-4.6.so Reading symbols from /usr/lib/ldap/back_monitor-2.4.so.2...Reading symbols from /usr/lib/debug/usr/lib/ldap/back_monitor-2.4.so.2.4.2...done. done. Loaded symbols for /usr/lib/ldap/back_monitor-2.4.so.2 Reading symbols from /usr/lib/ldap/back_bdb-2.4.so.2...Reading symbols from /usr/lib/debug/usr/lib/ldap/back_bdb-2.4.so.2.4.2...done. done. Loaded symbols for /usr/lib/ldap/back_bdb-2.4.so.2 Reading symbols from /usr/lib/ldap/accesslog-2.4.so.2...Reading symbols from /usr/lib/debug/usr/lib/ldap/accesslog-2.4.so.2.4.2...done. done. Loaded symbols for /usr/lib/ldap/accesslog-2.4.so.2 Reading symbols from /usr/lib/ldap/pw-netscape.so...Reading symbols from /usr/lib/debug/usr/lib/ldap/pw-netscape.so...done. done. Loaded symbols for /usr/lib/ldap/pw-netscape.so Reading symbols from /usr/lib/debug/libnss_dns.so.2...done. Loaded symbols for /usr/lib/debug/libnss_dns.so.2 0xb7b79492 in pthread_join (threadid=1642417040, thread_return=0x0) at pthread_join.c:89 89 pthread_join.c: No such file or directory. in pthread_join.c [New Thread 0x609beb90 (LWP 9538)] [New Thread 0x605beb90 (LWP 9539)] [New Thread 0x601beb90 (LWP 9540)] [New Thread 0x5fdbeb90 (LWP 9541)] [New Thread 0x5f99eb90 (LWP 9542)] [New Thread 0x5f58eb90 (LWP 9543)] [New Thread 0x5f16eb90 (LWP 9544)] [New Thread 0x5ed2eb90 (LWP 9545)] Program received signal SIGABRT, Aborted. [Switching to Thread 0x601beb90 (LWP 9540)] 0xb7a5f556 in *__GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 64 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory. in ../nptl/sysdeps/unix/sysv/linux/raise.c Thread 14 (Thread 0x5ed2eb90 (LWP 9545)): #0 0xb7b80038 in __libc_pread64 (fd=18, buf=0xa11727a8, count=4096, offset=213258240) at ../sysdeps/unix/sysv/linux/pread64.c:54 oldtype = 0 result = <value optimized out> #1 0xb7e298cf in __os_io () from /usr/lib/libdb-4.7.so No symbol table info available. #2 0xb7e167b8 in __memp_pgread () from /usr/lib/libdb-4.7.so No symbol table info available. #3 0xb7e188f0 in __memp_fget () from /usr/lib/libdb-4.7.so No symbol table info available. #4 0xb7d31379 in __bam_search () from /usr/lib/libdb-4.7.so No symbol table info available. #5 0xb7d206a7 in ?? () from /usr/lib/libdb-4.7.so No symbol table info available. #6 0x0e717dc0 in ?? () No symbol table info available. #7 0x00000001 in ?? () No symbol table info available. #8 0x5ec6ccd8 in ?? () No symbol table info available. #9 0x00000581 in ?? () No symbol table info available. #10 0x00000001 in ?? () No symbol table info available. #11 0x00000000 in ?? () No symbol table info available. Thread 13 (Thread 0x5f16eb90 (LWP 9544)): #0 0xb7b7c8f0 in pthread_cond_wait@@GLIBC_2.3.2 () from /usr/lib/debug/libpthread.so.0 No locals. #1 0xb7e6f264 in ldap_pvt_thread_cond_wait (cond=0x9bb722c, mutex=0x9bb7214) at /tmp/buildd/openldap-2.4.16/libraries/libldap_r/thr_posix.c:277 No locals. #2 0xb7e6e63a in ldap_int_thread_pool_wrapper (xpool=0x9bb7210) at /tmp/buildd/openldap-2.4.16/libraries/libldap_r/tpool.c:654 task = (ldap_int_thread_task_t *) 0x0 work_list = <value optimized out> ctx = {ltu_id = 1595337616, ltu_key = {{ltk_key = 0x8076090, ltk_data = 0xa10bb40, ltk_free = 0x8076160 <conn_counter_destroy>}, {ltk_key = 0x80ced40, ltk_data = 0xcef69e0, ltk_free = 0x80cec20 <slap_sl_mem_destroy>}, {ltk_key = 0xa171c00, ltk_data = 0xcefc630, ltk_free = 0xb738aee0 <bdb_reader_free>}, {ltk_key = 0x808b890, ltk_data = 0x14aeb600, ltk_free = 0x808b680 <slap_op_q_destroy>}, {ltk_key = 0xb737d100, ltk_data = 0xe71c000, ltk_free = 0xb737d1f0 <search_stack_free>}, {ltk_key = 0x9b3e400, ltk_data = 0x12b67e10, ltk_free = 0xb738aee0 <bdb_reader_free>}, {ltk_key = 0x0, ltk_data = 0x1a739200, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0} <repeats 25 times>}} kctx = <value optimized out> keyslot = 368 hash = <value optimized out> __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper" #3 0xb7b78f3b in start_thread (arg=0x5f16eb90) at pthread_create.c:297 __res = <value optimized out> __ignore1 = <value optimized out> __ignore2 = <value optimized out> pd = (struct pthread *) 0x5f16eb90 unwind_buf = {cancel_jmp_buf = {{jmp_buf = {-1212645388, 0, 4001536, 1595335864, 969392448, 2065569425}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0xb7b78e9b}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = <value optimized out> robust = <value optimized out> #4 0xb7affbee in clone () from /usr/lib/debug/libc.so.6 fstab_state = {fs_fp = 0x0, fs_buffer = 0x0, fs_mntres = {mnt_fsname = 0x0, mnt_dir = 0x0, mnt_type = 0x0, mnt_opts = 0x0, mnt_freq = 0, mnt_passno = 0}, fs_ret = {fs_spec = 0x0, fs_file = 0x0, fs_vfstype = 0x0, fs_mntops = 0x0, fs_type = 0x0, fs_freq = 0, fs_passno = 0}} __elf_set___libc_subfreeres_element_fstab_free__ = (const void *) 0xb7b3d820 Thread 12 (Thread 0x5f58eb90 (LWP 9543)): #0 0xb7b7c8f0 in pthread_cond_wait@@GLIBC_2.3.2 () from /usr/lib/debug/libpthread.so.0 No locals. #1 0xb7e6f264 in ldap_pvt_thread_cond_wait (cond=0x9bb722c, mutex=0x9bb7214) at /tmp/buildd/openldap-2.4.16/libraries/libldap_r/thr_posix.c:277 No locals. #2 0xb7e6e63a in ldap_int_thread_pool_wrapper (xpool=0x9bb7210) at /tmp/buildd/openldap-2.4.16/libraries/libldap_r/tpool.c:654 task = (ldap_int_thread_task_t *) 0x0 work_list = <value optimized out> ctx = {ltu_id = 1599662992, ltu_key = {{ltk_key = 0x8076090, ltk_data = 0xa10b960, ltk_free = 0x8076160 <conn_counter_destroy>}, {ltk_key = 0x80ced40, ltk_data = 0xa104c80, ltk_free = 0x80cec20 <slap_sl_mem_destroy>}, {ltk_key = 0x808b890, ltk_data = 0x1396ddc0, ltk_free = 0x808b680 <slap_op_q_destroy>}, {ltk_key = 0xa171c00, ltk_data = 0xcefc7e0, ltk_free = 0xb738aee0 <bdb_reader_free>}, {ltk_key = 0xb737d100, ltk_data = 0xf76b000, ltk_free = 0xb737d1f0 <search_stack_free>}, {ltk_key = 0x9b3e400, ltk_data = 0x12ba9cf0, ltk_free = 0xb738aee0 <bdb_reader_free>}, {ltk_key = 0x0, ltk_data = 0x1276e200, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0} <repeats 25 times>}} kctx = <value optimized out> keyslot = 242 hash = <value optimized out> __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper" #3 0xb7b78f3b in start_thread (arg=0x5f58eb90) at pthread_create.c:297 __res = <value optimized out> __ignore1 = <value optimized out> __ignore2 = <value optimized out> pd = (struct pthread *) 0x5f58eb90 unwind_buf = {cancel_jmp_buf = {{jmp_buf = {-1212645388, 0, 4001536, 1599661240, -1513635520, 2065569425}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0xb7b78e9b}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = <value optimized out> robust = <value optimized out> #4 0xb7affbee in clone () from /usr/lib/debug/libc.so.6 fstab_state = {fs_fp = 0x0, fs_buffer = 0x0, fs_mntres = {mnt_fsname = 0x0, mnt_dir = 0x0, mnt_type = 0x0, mnt_opts = 0x0, mnt_freq = 0, mnt_passno = 0}, fs_ret = {fs_spec = 0x0, fs_file = 0x0, fs_vfstype = 0x0, fs_mntops = 0x0, fs_type = 0x0, fs_freq = 0, fs_passno = 0}} __elf_set___libc_subfreeres_element_fstab_free__ = (const void *) 0xb7b3d820 Current language: auto; currently asm Thread 11 (Thread 0x5f99eb90 (LWP 9542)): #0 0xb7b7c8f0 in pthread_cond_wait@@GLIBC_2.3.2 () from /usr/lib/debug/libpthread.so.0 No locals. #1 0xb7e6f264 in ldap_pvt_thread_cond_wait (cond=0x9bb722c, mutex=0x9bb7214) at /tmp/buildd/openldap-2.4.16/libraries/libldap_r/thr_posix.c:277 No locals. #2 0xb7e6e63a in ldap_int_thread_pool_wrapper (xpool=0x9bb7210) at /tmp/buildd/openldap-2.4.16/libraries/libldap_r/tpool.c:654 task = (ldap_int_thread_task_t *) 0x0 work_list = <value optimized out> ctx = {ltu_id = 1603922832, ltu_key = {{ltk_key = 0x8076090, ltk_data = 0xa10ba50, ltk_free = 0x8076160 <conn_counter_destroy>}, {ltk_key = 0x80ced40, ltk_data = 0xcef6960, ltk_free = 0x80cec20 <slap_sl_mem_destroy>}, {ltk_key = 0xa171c00, ltk_data = 0xcefc900, ltk_free = 0xb738aee0 <bdb_reader_free>}, {ltk_key = 0x808b890, ltk_data = 0x174842c0, ltk_free = 0x808b680 <slap_op_q_destroy>}, {ltk_key = 0xb737d100, ltk_data = 0x11007000, ltk_free = 0xb737d1f0 <search_stack_free>}, {ltk_key = 0x9b3e400, ltk_data = 0x12578ab0, ltk_free = 0xb738aee0 <bdb_reader_free>}, {ltk_key = 0x0, ltk_data = 0x1a1b2400, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0} <repeats 25 times>}} kctx = <value optimized out> keyslot = 595 hash = <value optimized out> __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper" #3 0xb7b78f3b in start_thread (arg=0x5f99eb90) at pthread_create.c:297 __res = <value optimized out> __ignore1 = <value optimized out> __ignore2 = <value optimized out> pd = (struct pthread *) 0x5f99eb90 unwind_buf = {cancel_jmp_buf = {{jmp_buf = {-1212645388, 0, 4001536, 1603921080, 667402561, 2065569425}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0xb7b78e9b}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = <value optimized out> robust = <value optimized out> #4 0xb7affbee in clone () from /usr/lib/debug/libc.so.6 fstab_state = {fs_fp = 0x0, fs_buffer = 0x0, fs_mntres = {mnt_fsname = 0x0, mnt_dir = 0x0, mnt_type = 0x0, mnt_opts = 0x0, mnt_freq = 0, mnt_passno = 0}, fs_ret = {fs_spec = 0x0, fs_file = 0x0, fs_vfstype = 0x0, fs_mntops = 0x0, fs_type = 0x0, fs_freq = 0, fs_passno = 0}} __elf_set___libc_subfreeres_element_fstab_free__ = (const void *) 0xb7b3d820 Thread 10 (Thread 0x5fdbeb90 (LWP 9541)): #0 0xb7b7c8f0 in pthread_cond_wait@@GLIBC_2.3.2 () from /usr/lib/debug/libpthread.so.0 No locals. #1 0xb7e6f264 in ldap_pvt_thread_cond_wait (cond=0x9bb722c, mutex=0x9bb7214) at /tmp/buildd/openldap-2.4.16/libraries/libldap_r/thr_posix.c:277 No locals. #2 0xb7e6e63a in ldap_int_thread_pool_wrapper (xpool=0x9bb7210) at /tmp/buildd/openldap-2.4.16/libraries/libldap_r/tpool.c:654 task = (ldap_int_thread_task_t *) 0x0 work_list = <value optimized out> ctx = {ltu_id = 1608248208, ltu_key = {{ltk_key = 0x8076090, ltk_data = 0xa10b870, ltk_free = 0x8076160 <conn_counter_destroy>}, {ltk_key = 0x80ced40, ltk_data = 0xcef6600, ltk_free = 0x80cec20 <slap_sl_mem_destroy>}, {ltk_key = 0x808b890, ltk_data = 0x13f752c0, ltk_free = 0x808b680 <slap_op_q_destroy>}, {ltk_key = 0xa171c00, ltk_data = 0xacc7c60, ltk_free = 0xb738aee0 <bdb_reader_free>}, {ltk_key = 0xb737d100, ltk_data = 0xef24000, ltk_free = 0xb737d1f0 <search_stack_free>}, {ltk_key = 0x9b3e400, ltk_data = 0x12644a20, ltk_free = 0xb738aee0 <bdb_reader_free>}, {ltk_key = 0x0, ltk_data = 0x16eb3200, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0} <repeats 25 times>}} kctx = <value optimized out> keyslot = 469 hash = <value optimized out> __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper" #3 0xb7b78f3b in start_thread (arg=0x5fdbeb90) at pthread_create.c:297 __res = <value optimized out> __ignore1 = <value optimized out> __ignore2 = <value optimized out> pd = (struct pthread *) 0x5fdbeb90 unwind_buf = {cancel_jmp_buf = {{jmp_buf = {-1212645388, 0, 4001536, 1608246456, -1547189951, 2065569425}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0xb7b78e9b}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = <value optimized out> robust = <value optimized out> #4 0xb7affbee in clone () from /usr/lib/debug/libc.so.6 fstab_state = {fs_fp = 0x0, fs_buffer = 0x0, fs_mntres = {mnt_fsname = 0x0, mnt_dir = 0x0, mnt_type = 0x0, mnt_opts = 0x0, mnt_freq = 0, mnt_passno = 0}, fs_ret = {fs_spec = 0x0, fs_file = 0x0, fs_vfstype = 0x0, fs_mntops = 0x0, fs_type = 0x0, fs_freq = 0, fs_passno = 0}} __elf_set___libc_subfreeres_element_fstab_free__ = (const void *) 0xb7b3d820 Thread 9 (Thread 0x601beb90 (LWP 9540)): #0 0xb7a5f556 in *__GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 resultvar = <value optimized out> pid = -1212747788 selftid = 9540 #1 0xb7a60d78 in *__GI_abort () at abort.c:88 act = {__sigaction_handler = {sa_handler = 0xb7844020, sa_sigaction = 0xb7844020}, sa_mask = {__val = {3104663397, 269, 229510049, 0, 1612439568, 329789888, 329789988, 3085884411, 3082219508, 126, 329789888, 1612439740, 3081340890, 295243776, 295243776, 125, 329789888, 0, 125, 4222451712, 295243776, 295243877, 295243776, 295243776, 295243901, 295244076, 295243776, 295244076, 0, 0, 0, 0}}, sa_flags = 0, sa_restorer = 0xb7b5593c} sigs = {__val = {32, 0 <repeats 31 times>}} #2 0xb7a58590 in *__GI___assert_fail (assertion=0x8118e89 "c->c_writers == 0", file=0x81194a8 "/tmp/buildd/openldap-2.4.16/servers/slapd/connection.c", line=601, function=0x81193a5 "connection_destroy") at assert.c:78 buf = 0x17439900 "\200\221C\027d: /tmp/buildd/openldap-2.4.16/servers/slapd/connection.c:601: connection_destroy: Assertion `c->c_writers == 0' failed.\n" errstr = "Unexpected error.\n" #3 0x080767bc in connection_close (c=0x9d7de78) at /tmp/buildd/openldap-2.4.16/servers/slapd/connection.c:601 __PRETTY_FUNCTION__ = "connection_close" #4 0x080773e2 in connection_operation (ctx=0x601be238, arg_v=0x11a9f600) at /tmp/buildd/openldap-2.4.16/servers/slapd/connection.c:1661 rc = <value optimized out> rs = {sr_type = REP_RESULT, sr_tag = 0, sr_msgid = 0, sr_err = 0, sr_matched = 0x0, sr_text = 0x0, sr_ref = 0x0, sr_ctrls = 0x0, sr_un = {sru_sasl = {r_sasldata = 0x0}, sru_extended = {r_rspoid = 0x0, r_rspdata = 0x0}, sru_search = {r_entry = 0x0, r_attr_flags = 0, r_operational_attrs = 0x0, r_attrs = 0x0, r_nentries = 0, r_v2ref = 0x0}}, sr_flags = 0} tag = 80 opidx = SLAP_OP_ABANDON conn = (Connection *) 0x9d7de78 memctx = (void *) 0xcef6560 memctx_null = (void *) 0x0 __PRETTY_FUNCTION__ = "connection_operation" #5 0xb7e6e5c8 in ldap_int_thread_pool_wrapper (xpool=0x9bb7210) at /tmp/buildd/openldap-2.4.16/libraries/libldap_r/tpool.c:663 task = (ldap_int_thread_task_t *) 0x1471f7b0 work_list = <value optimized out> ctx = {ltu_id = 1612442512, ltu_key = {{ltk_key = 0x8076090, ltk_data = 0xa10b780, ltk_free = 0x8076160 <conn_counter_destroy>}, {ltk_key = 0x80ced40, ltk_data = 0xcef6560, ltk_free = 0x80cec20 <slap_sl_mem_destroy>}, {ltk_key = 0xa171c00, ltk_data = 0xcefc2d0, ltk_free = 0xb738aee0 <bdb_reader_free>}, {ltk_key = 0x808b890, ltk_data = 0x12482080, ltk_free = 0x808b680 <slap_op_q_destroy>}, {ltk_key = 0xb737d100, ltk_data = 0xdd10000, ltk_free = 0xb737d1f0 <search_stack_free>}, {ltk_key = 0x9b3e400, ltk_data = 0x11b173f0, ltk_free = 0xb738aee0 <bdb_reader_free>}, {ltk_key = 0x0, ltk_data = 0x14836000, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0} <repeats 25 times>}} kctx = <value optimized out> keyslot = 406 hash = <value optimized out> __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper" #6 0xb7b78f3b in start_thread (arg=0x601beb90) at pthread_create.c:297 __res = <value optimized out> __ignore1 = <value optimized out> __ignore2 = <value optimized out> pd = (struct pthread *) 0x601beb90 unwind_buf = {cancel_jmp_buf = {{jmp_buf = {-1212645388, 0, 4001536, 1612440760, 600293694, 2065569425}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0xb7b78e9b}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = <value optimized out> robust = <value optimized out> #7 0xb7affbee in clone () from /usr/lib/debug/libc.so.6 fstab_state = {fs_fp = 0x0, fs_buffer = 0x0, fs_mntres = {mnt_fsname = 0x0, mnt_dir = 0x0, mnt_type = 0x0, mnt_opts = 0x0, mnt_freq = 0, mnt_passno = 0}, fs_ret = {fs_spec = 0x0, fs_file = 0x0, fs_vfstype = 0x0, fs_mntops = 0x0, fs_type = 0x0, fs_freq = 0, fs_passno = 0}} __elf_set___libc_subfreeres_element_fstab_free__ = (const void *) 0xb7b3d820 Thread 8 (Thread 0x605beb90 (LWP 9539)): #0 0xb7b7c8f0 in pthread_cond_wait@@GLIBC_2.3.2 () from /usr/lib/debug/libpthread.so.0 No locals. #1 0xb7e6f264 in ldap_pvt_thread_cond_wait (cond=0x9bb722c, mutex=0x9bb7214) at /tmp/buildd/openldap-2.4.16/libraries/libldap_r/thr_posix.c:277 No locals. #2 0xb7e6e63a in ldap_int_thread_pool_wrapper (xpool=0x9bb7210) at /tmp/buildd/openldap-2.4.16/libraries/libldap_r/tpool.c:654 task = (ldap_int_thread_task_t *) 0x0 work_list = <value optimized out> ctx = {ltu_id = 1616636816, ltu_key = {{ltk_key = 0x8076090, ltk_data = 0xa10b690, ltk_free = 0x8076160 <conn_counter_destroy>}, {ltk_key = 0x80ced40, ltk_data = 0xcef6740, ltk_free = 0x80cec20 <slap_sl_mem_destroy>}, {ltk_key = 0xa171c00, ltk_data = 0xcefc510, ltk_free = 0xb738aee0 <bdb_reader_free>}, {ltk_key = 0xb737d100, ltk_data = 0xd108000, ltk_free = 0xb737d1f0 <search_stack_free>}, {ltk_key = 0x808b890, ltk_data = 0x17485340, ltk_free = 0x808b680 <slap_op_q_destroy>}, {ltk_key = 0x9b3e400, ltk_data = 0x13138ea0, ltk_free = 0xb738aee0 <bdb_reader_free>}, {ltk_key = 0x0, ltk_data = 0x1a39f200, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0} <repeats 25 times>}} kctx = <value optimized out> keyslot = 342 hash = <value optimized out> __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper" #3 0xb7b78f3b in start_thread (arg=0x605beb90) at pthread_create.c:297 __res = <value optimized out> __ignore1 = <value optimized out> __ignore2 = <value optimized out> pd = (struct pthread *) 0x605beb90 unwind_buf = {cancel_jmp_buf = {{jmp_buf = {-1212645388, 0, 4001536, 1616635064, -1547189954, 2065569425}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0xb7b78e9b}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = <value optimized out> robust = <value optimized out> #4 0xb7affbee in clone () from /usr/lib/debug/libc.so.6 fstab_state = {fs_fp = 0x0, fs_buffer = 0x0, fs_mntres = {mnt_fsname = 0x0, mnt_dir = 0x0, mnt_type = 0x0, mnt_opts = 0x0, mnt_freq = 0, mnt_passno = 0}, fs_ret = {fs_spec = 0x0, fs_file = 0x0, fs_vfstype = 0x0, fs_mntops = 0x0, fs_type = 0x0, fs_freq = 0, fs_passno = 0}} __elf_set___libc_subfreeres_element_fstab_free__ = (const void *) 0xb7b3d820 Current language: auto; currently c Thread 7 (Thread 0x609beb90 (LWP 9538)): #0 0xb7b7c8f0 in pthread_cond_wait@@GLIBC_2.3.2 () from /usr/lib/debug/libpthread.so.0 No locals. #1 0xb7e6f264 in ldap_pvt_thread_cond_wait (cond=0x9bb722c, mutex=0x9bb7214) at /tmp/buildd/openldap-2.4.16/libraries/libldap_r/thr_posix.c:277 No locals. #2 0xb7e6e63a in ldap_int_thread_pool_wrapper (xpool=0x9bb7210) at /tmp/buildd/openldap-2.4.16/libraries/libldap_r/tpool.c:654 task = (ldap_int_thread_task_t *) 0x0 work_list = <value optimized out> ctx = {ltu_id = 1620831120, ltu_key = {{ltk_key = 0x8076090, ltk_data = 0xa10b3c0, ltk_free = 0x8076160 <conn_counter_destroy>}, {ltk_key = 0x80ced40, ltk_data = 0xcef6120, ltk_free = 0x80cec20 <slap_sl_mem_destroy>}, {ltk_key = 0xa171c00, ltk_data = 0xcefc120, ltk_free = 0xb738aee0 <bdb_reader_free>}, {ltk_key = 0x808b890, ltk_data = 0x124028c0, ltk_free = 0x808b680 <slap_op_q_destroy>}, {ltk_key = 0xb737d100, ltk_data = 0x10807000, ltk_free = 0xb737d1f0 <search_stack_free>}, {ltk_key = 0x9b3e400, ltk_data = 0x1393c480, ltk_free = 0xb738aee0 <bdb_reader_free>}, {ltk_key = 0x0, ltk_data = 0x1a739200, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0} <repeats 25 times>}} kctx = <value optimized out> keyslot = 534 hash = <value optimized out> __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper" #3 0xb7b78f3b in start_thread (arg=0x609beb90) at pthread_create.c:297 __res = <value optimized out> __ignore1 = <value optimized out> __ignore2 = <value optimized out> pd = (struct pthread *) 0x609beb90 unwind_buf = {cancel_jmp_buf = {{jmp_buf = {-1212645388, 0, 4001536, 1620829368, 600293695, 2065569425}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0xb7b78e9b}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = <value optimized out> robust = <value optimized out> #4 0xb7affbee in clone () from /usr/lib/debug/libc.so.6 fstab_state = {fs_fp = 0x0, fs_buffer = 0x0, fs_mntres = {mnt_fsname = 0x0, mnt_dir = 0x0, mnt_type = 0x0, mnt_opts = 0x0, mnt_freq = 0, mnt_passno = 0}, fs_ret = {fs_spec = 0x0, fs_file = 0x0, fs_vfstype = 0x0, fs_mntops = 0x0, fs_type = 0x0, fs_freq = 0, fs_passno = 0}} __elf_set___libc_subfreeres_element_fstab_free__ = (const void *) 0xb7b3d820 Current language: auto; currently asm Thread 6 (Thread 0x61e54b90 (LWP 9367)): #0 0xb7b0039c in epoll_wait () from /usr/lib/debug/libc.so.6 fstab_state = {fs_fp = 0x0, fs_buffer = 0x0, fs_mntres = {mnt_fsname = 0x0, mnt_dir = 0x0, mnt_type = 0x0, mnt_opts = 0x0, mnt_freq = 0, mnt_passno = 0}, fs_ret = {fs_spec = 0x0, fs_file = 0x0, fs_vfstype = 0x0, fs_mntops = 0x0, fs_type = 0x0, fs_freq = 0, fs_passno = 0}} __elf_set___libc_subfreeres_element_fstab_free__ = (const void *) 0xb7b3d820 #1 0x08073c3f in slapd_daemon_task (ptr=0x0) at /tmp/buildd/openldap-2.4.16/servers/slapd/daemon.c:2291 i = 1 revents = (struct epoll_event *) 0x9b5f000 tv = {tv_sec = 289, tv_usec = 0} rtask = <value optimized out> ns = 1 at = 0 now = 1240942264 tvp = (struct timeval *) 0x61e543b4 cat = {tv_sec = 1240942553, tv_usec = 0} l = 4 last_idle_check = 0 ebadf = 0 #2 0xb7b78f3b in start_thread (arg=0x61e54b90) at pthread_create.c:297 __res = <value optimized out> __ignore1 = <value optimized out> __ignore2 = <value optimized out> pd = (struct pthread *) 0x61e54b90 unwind_buf = {cancel_jmp_buf = {{jmp_buf = {-1212645388, 0, 4001536, 1642415288, -561528515, 2065569425}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0xb7b78e9b}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = <value optimized out> robust = <value optimized out> #3 0xb7affbee in clone () from /usr/lib/debug/libc.so.6 fstab_state = {fs_fp = 0x0, fs_buffer = 0x0, fs_mntres = {mnt_fsname = 0x0, mnt_dir = 0x0, mnt_type = 0x0, mnt_opts = 0x0, mnt_freq = 0, mnt_passno = 0}, fs_ret = {fs_spec = 0x0, fs_file = 0x0, fs_vfstype = 0x0, fs_mntops = 0x0, fs_type = 0x0, fs_freq = 0, fs_passno = 0}} __elf_set___libc_subfreeres_element_fstab_free__ = (const void *) 0xb7b3d820 Thread 5 (Thread 0x61a54b90 (LWP 9368)): #0 0xb7b7c8f0 in pthread_cond_wait@@GLIBC_2.3.2 () from /usr/lib/debug/libpthread.so.0 No locals. #1 0xb7e6f264 in ldap_pvt_thread_cond_wait (cond=0x9bb722c, mutex=0x9bb7214) at /tmp/buildd/openldap-2.4.16/libraries/libldap_r/thr_posix.c:277 No locals. #2 0xb7e6e63a in ldap_int_thread_pool_wrapper (xpool=0x9bb7210) at /tmp/buildd/openldap-2.4.16/libraries/libldap_r/tpool.c:654 task = (ldap_int_thread_task_t *) 0x0 work_list = <value optimized out> ctx = {ltu_id = 1638222736, ltu_key = {{ltk_key = 0x80ced40, ltk_data = 0xa16af20, ltk_free = 0x80cec20 <slap_sl_mem_destroy>}, {ltk_key = 0x9b3e400, ltk_data = 0xacc7120, ltk_free = 0xb738aee0 <bdb_reader_free>}, {ltk_key = 0xb737d100, ltk_data = 0xadd2000, ltk_free = 0xb737d1f0 <search_stack_free>}, {ltk_key = 0x8076090, ltk_data = 0xa10b1e0, ltk_free = 0x8076160 <conn_counter_destroy>}, {ltk_key = 0xa171c00, ltk_data = 0xacc71b0, ltk_free = 0xb738aee0 <bdb_reader_free>}, {ltk_key = 0x808b890, ltk_data = 0x15244600, ltk_free = 0x808b680 <slap_op_q_destroy>}, {ltk_key = 0x0, ltk_data = 0x186c2000, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0} <repeats 25 times>}} kctx = <value optimized out> keyslot = 769 hash = <value optimized out> __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper" #3 0xb7b78f3b in start_thread (arg=0x61a54b90) at pthread_create.c:297 __res = <value optimized out> __ignore1 = <value optimized out> __ignore2 = <value optimized out> pd = (struct pthread *) 0x61a54b90 unwind_buf = {cancel_jmp_buf = {{jmp_buf = {-1212645388, 0, 4001536, 1638220984, 1585955133, 2065569425}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0xb7b78e9b}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = <value optimized out> robust = <value optimized out> #4 0xb7affbee in clone () from /usr/lib/debug/libc.so.6 fstab_state = {fs_fp = 0x0, fs_buffer = 0x0, fs_mntres = {mnt_fsname = 0x0, mnt_dir = 0x0, mnt_type = 0x0, mnt_opts = 0x0, mnt_freq = 0, mnt_passno = 0}, fs_ret = {fs_spec = 0x0, fs_file = 0x0, fs_vfstype = 0x0, fs_mntops = 0x0, fs_type = 0x0, fs_freq = 0, fs_passno = 0}} __elf_set___libc_subfreeres_element_fstab_free__ = (const void *) 0xb7b3d820 Current language: auto; currently c Thread 4 (Thread 0x61644b90 (LWP 9369)): #0 0xb7b7c8f0 in pthread_cond_wait@@GLIBC_2.3.2 () from /usr/lib/debug/libpthread.so.0 No locals. #1 0xb7e6f264 in ldap_pvt_thread_cond_wait (cond=0x9bb722c, mutex=0x9bb7214) at /tmp/buildd/openldap-2.4.16/libraries/libldap_r/thr_posix.c:277 No locals. #2 0xb7e6e63a in ldap_int_thread_pool_wrapper (xpool=0x9bb7210) at /tmp/buildd/openldap-2.4.16/libraries/libldap_r/tpool.c:654 task = (ldap_int_thread_task_t *) 0x0 work_list = <value optimized out> ctx = {ltu_id = 1633962896, ltu_key = {{ltk_key = 0x80ced40, ltk_data = 0xa16ac00, ltk_free = 0x80cec20 <slap_sl_mem_destroy>}, {ltk_key = 0xa171c00, ltk_data = 0xacc7090, ltk_free = 0xb738aee0 <bdb_reader_free>}, {ltk_key = 0x9b3e400, ltk_data = 0xacc76c0, ltk_free = 0xb738aee0 <bdb_reader_free>}, {ltk_key = 0xb737d100, ltk_data = 0xbee7000, ltk_free = 0xb737d1f0 <search_stack_free>}, {ltk_key = 0x8076090, ltk_data = 0xa10b0f0, ltk_free = 0x8076160 <conn_counter_destroy>}, {ltk_key = 0x808b890, ltk_data = 0x11992580, ltk_free = 0x808b680 <slap_op_q_destroy>}, {ltk_key = 0x0, ltk_data = 0x159ff000, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0} <repeats 25 times>}} kctx = <value optimized out> keyslot = 352 hash = <value optimized out> __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper" #3 0xb7b78f3b in start_thread (arg=0x61644b90) at pthread_create.c:297 __res = <value optimized out> __ignore1 = <value optimized out> __ignore2 = <value optimized out> pd = (struct pthread *) 0x61644b90 unwind_buf = {cancel_jmp_buf = {{jmp_buf = {-1212645388, 0, 4001536, 1633961144, -595082948, 2065569425}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0xb7b78e9b}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = <value optimized out> robust = <value optimized out> #4 0xb7affbee in clone () from /usr/lib/debug/libc.so.6 fstab_state = {fs_fp = 0x0, fs_buffer = 0x0, fs_mntres = {mnt_fsname = 0x0, mnt_dir = 0x0, mnt_type = 0x0, mnt_opts = 0x0, mnt_freq = 0, mnt_passno = 0}, fs_ret = {fs_spec = 0x0, fs_file = 0x0, fs_vfstype = 0x0, fs_mntops = 0x0, fs_type = 0x0, fs_freq = 0, fs_passno = 0}} __elf_set___libc_subfreeres_element_fstab_free__ = (const void *) 0xb7b3d820 Current language: auto; currently asm Thread 3 (Thread 0x6122eb90 (LWP 9370)): #0 0xb7b7c8f0 in pthread_cond_wait@@GLIBC_2.3.2 () from /usr/lib/debug/libpthread.so.0 No locals. #1 0xb7e6f264 in ldap_pvt_thread_cond_wait (cond=0x9bb722c, mutex=0x9bb7214) at /tmp/buildd/openldap-2.4.16/libraries/libldap_r/thr_posix.c:277 No locals. #2 0xb7e6e63a in ldap_int_thread_pool_wrapper (xpool=0x9bb7210) at /tmp/buildd/openldap-2.4.16/libraries/libldap_r/tpool.c:654 task = (ldap_int_thread_task_t *) 0x0 work_list = <value optimized out> ctx = {ltu_id = 1629678480, ltu_key = {{ltk_key = 0x80ced40, ltk_data = 0xadd0620, ltk_free = 0x80cec20 <slap_sl_mem_destroy>}, {ltk_key = 0x8076090, ltk_data = 0xa10b000, ltk_free = 0x8076160 <conn_counter_destroy>}, {ltk_key = 0xa171c00, ltk_data = 0xacc72d0, ltk_free = 0xb738aee0 <bdb_reader_free>}, {ltk_key = 0x808b890, ltk_data = 0x17559b00, ltk_free = 0x808b680 <slap_op_q_destroy>}, {ltk_key = 0x9b3e400, ltk_data = 0xacc75a0, ltk_free = 0xb738aee0 <bdb_reader_free>}, {ltk_key = 0xb737d100, ltk_data = 0xc6e7000, ltk_free = 0xb737d1f0 <search_stack_free>}, {ltk_key = 0x0, ltk_data = 0x1a1b2400, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0} <repeats 25 times>}} kctx = <value optimized out> keyslot = 638 hash = <value optimized out> __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper" #3 0xb7b78f3b in start_thread (arg=0x6122eb90) at pthread_create.c:297 __res = <value optimized out> __ignore1 = <value optimized out> __ignore2 = <value optimized out> pd = (struct pthread *) 0x6122eb90 unwind_buf = {cancel_jmp_buf = {{jmp_buf = {-1212645388, 0, 4001536, 1629676728, 1372045628, 2065569425}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0xb7b78e9b}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = <value optimized out> robust = <value optimized out> #4 0xb7affbee in clone () from /usr/lib/debug/libc.so.6 fstab_state = {fs_fp = 0x0, fs_buffer = 0x0, fs_mntres = {mnt_fsname = 0x0, mnt_dir = 0x0, mnt_type = 0x0, mnt_opts = 0x0, mnt_freq = 0, mnt_passno = 0}, fs_ret = {fs_spec = 0x0, fs_file = 0x0, fs_vfstype = 0x0, fs_mntops = 0x0, fs_type = 0x0, fs_freq = 0, fs_passno = 0}} __elf_set___libc_subfreeres_element_fstab_free__ = (const void *) 0xb7b3d820 Thread 2 (Thread 0x60e2eb90 (LWP 9371)): #0 0xb7b80038 in __libc_pread64 (fd=17, buf=0x8309a608, count=16384, offset=1147437056) at ../sysdeps/unix/sysv/linux/pread64.c:54 oldtype = 0 result = <value optimized out> #1 0xb7e298cf in __os_io () from /usr/lib/libdb-4.7.so No symbol table info available. #2 0xb7e167b8 in __memp_pgread () from /usr/lib/libdb-4.7.so No symbol table info available. #3 0xb7e188f0 in __memp_fget () from /usr/lib/libdb-4.7.so No symbol table info available. #4 0xb7d31379 in __bam_search () from /usr/lib/libdb-4.7.so No symbol table info available. #5 0xb7d206a7 in ?? () from /usr/lib/libdb-4.7.so No symbol table info available. #6 0x0cef5dc0 in ?? () No symbol table info available. #7 0x00000001 in ?? () No symbol table info available. #8 0x60d6cd64 in ?? () No symbol table info available. #9 0x00000581 in ?? () No symbol table info available. #10 0x00000001 in ?? () No symbol table info available. #11 0x00000000 in ?? () No symbol table info available. Thread 1 (Thread 0xb78648e0 (LWP 9365)): #0 0xb7b79492 in pthread_join (threadid=1642417040, thread_return=0x0) at pthread_join.c:89 __ignore = <value optimized out> _tid = 9367 _buffer = {__routine = 0xb7b79360 <cleanup>, __arg = 0x61e54d8c, __canceltype = -1208843552, __prev = 0x0} oldtype = 0 result = <value optimized out> #1 0xb7e6f3a4 in ldap_pvt_thread_join (thread=1642417040, thread_return=0x0) at /tmp/buildd/openldap-2.4.16/libraries/libldap_r/thr_posix.c:197 No locals. #2 0x08070c43 in slapd_daemon () at /tmp/buildd/openldap-2.4.16/servers/slapd/daemon.c:2665 listener_tid = 1642417040 rc = 0 #3 0x0805e0d5 in main (argc=9, argv=0xbf82df14) at /tmp/buildd/openldap-2.4.16/servers/slapd/main.c:948 i = 162787680 no_detach = 0 rc = 0 urls = 0x9b37380 "ldap:/// ldaps:///" username = 0x9b3d020 "127.0.0.1" groupname = 0x9b3d000 "H\220≥\t" sandbox = 0x0 syslogUser = 160 configfile = 0x9b37360 "/etc/ldap/slapd.conf" configdir = 0x0 serverName = 0xbf82fdcb "slapd" scp = <value optimized out> scp_entry = <value optimized out> debug_unknowns = (char **) 0x0 syslog_unknowns = (char **) 0x0 slapd_pid_file_unlink = 1 slapd_args_file_unlink = 1 firstopt = <value optimized out> __PRETTY_FUNCTION__ = "main" Current language: auto; currently c 0xb7a5f556 64 in ../nptl/sysdeps/unix/sysv/linux/raise.c Saved corefile /var/lib/slapd-debug/core
Poked around a bit in the core: (gdb) print c->c_writers $1 = -1 (gdb) print c->c_pending_ops->stqh_last[0] $5 = (struct Operation *) 0x0 (gdb) print c->c_n_ops_pending $6 = 0 So there are no pending ops on this connection, but c_writers == -1 indicates that one blocked writer remains to notice that the connection has been closed. I looked at send_ldap_ber() and can't immediately find fault with its manipulation of c_writers. connection_closing() wakes up all blocked writers, so by the time connection_close() calls connection_destroy() and this assertion is checked, c_writers should be 0. Furthermore, c->c_conn_state is SLAP_C_CLOSING (0x4), which can only happen in connection_closing(). This is about as far as I can get; any other ideas? john (gdb) print *c $2 = {c_struct_state = 2, c_conn_state = 4, c_conn_idx = 1878, c_sd = 1878, c_close_reason = 0x0, c_mutex = {__data = {__lock = 1, __count = 0, __owner = 9540, __kind = 0, __nusers = 1, {__spins = 0, __list = { __next = 0x0}}}, __size = "\001\000\000\000\000\000\000\000D%\000\000\000\000\000\000\001\000\000\000\000\000\000", __align = 1}, c_sb = 0x175c7240, c_starttime = 1240942246, c_activitytime = 1240942264, c_connid = 73097614, c_peer_domain = {bv_len = 7, bv_val = 0x137e02c8 "unknown"}, c_peer_name = { bv_len = 23, bv_val = 0x191c0be0 "IP=172.16.182.103:50023"}, c_listener = 0x9bb7370, c_sasl_bind_mech = {bv_len = 0, bv_val = 0x0}, c_sasl_dn = {bv_len = 0, bv_val = 0x0}, c_sasl_authz_dn = {bv_len = 0, bv_val = 0x0}, c_authz_backend = 0x9bba500, c_authz_cookie = 0x0, c_authz = {sai_method = 128, sai_mech = {bv_len = 0, bv_val = 0x0}, sai_dn = {bv_len = 27, bv_val = 0x1876bd20 "cn=nsspluspasswd,o=example2"}, sai_ndn = {bv_len = 27, bv_val = 0x189f5780 "cn=nsspluspasswd,o=example2"}, sai_ssf = 128, sai_transport_ssf = 0, sai_tls_ssf = 128, sai_sasl_ssf = 0}, c_protocol = 3, c_ops = {stqh_first = 0x0, stqh_last = 0x9d7df18}, c_pending_ops = {stqh_first = 0x0, stqh_last = 0x9d7df20}, c_write1_mutex = { __data = {__lock = 0, __count = 0, __owner = 0, __kind = 0, __nusers = 0, { __spins = 0, __list = {__next = 0x0}}}, __size = '\0' <repeats 23 times>, __align = 0}, c_write1_cv = {__data = { __lock = 0, __futex = 2, __total_seq = 1, __wakeup_seq = 1, __woken_seq = 1, __mutex = 0x9d7df28, __nwaiters = 0, __broadcast_seq = 1}, __size = "\000\000\000\000\002\000\000\000\001\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000(##\t\000\000\000\000\001\000\000\000\000\000\000", __align = 8589934592}, c_write2_mutex = { __data = {__lock = 0, __count = 0, __owner = 0, __kind = 0, __nusers = 0, { __spins = 0, __list = {__next = 0x0}}}, __size = '\0' <repeats 23 times>, __align = 0}, c_write2_cv = {__data = { __lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\0' <repeats 47 times>, __align = 0}, c_currentber = 0x0, c_writers = -1, c_sasl_bind_in_progress = 0 '\0', c_writewaiter = 0 '\0', c_is_tls = 1 '\001', c_needs_tls_accept = 0 '\0', c_sasl_layers = 0 '\0', c_sasl_done = 0 '\0', c_sasl_authctx = 0x13497200, c_sasl_sockctx = 0x0, c_sasl_extra = 0x14d956c0, c_sasl_bindop = 0x0, c_pagedresults_state = { ps_be = 0x0, ps_size = 0, ps_count = 0, ps_cookie = 0, ps_cookieval = { bv_len = 0, bv_val = 0x0}}, c_n_ops_received = 6, c_n_ops_executing = 0, c_n_ops_pending = 0, c_n_ops_completed = 6, c_n_get = 4, c_n_read = 4, c_n_write = 0, c_extensions = 0x0, c_clientfunc = 0, c_clientarg = 0x0, c_send_ldap_result = 0x808aea0 <slap_send_ldap_result>, c_send_search_entry = 0x80885e0 <slap_send_search_entry>, c_send_search_reference = 0x8087da0 <slap_send_search_reference>, c_send_ldap_extended = 0x808a910 <slap_send_ldap_extended>, c_send_ldap_intermediate = 0x808a760 <slap_send_ldap_intermediate>} -- John Morrissey _o /\ ---- __o jwm@horde.net _-< \_ / \ ---- < \, www.horde.net/ __(_)/_(_)________/ \_______(_) /_(_)__
jwm@horde.net wrote: > Poked around a bit in the core: > > (gdb) print c->c_writers > $1 = -1 > (gdb) print c->c_pending_ops->stqh_last[0] > $5 = (struct Operation *) 0x0 > (gdb) print c->c_n_ops_pending > $6 = 0 > > So there are no pending ops on this connection, but c_writers == -1 > indicates that one blocked writer remains to notice that the connection has > been closed. > > I looked at send_ldap_ber() and can't immediately find fault with its > manipulation of c_writers. connection_closing() wakes up all blocked > writers, so by the time connection_close() calls connection_destroy() and > this assertion is checked, c_writers should be 0. Furthermore, > c->c_conn_state is SLAP_C_CLOSING (0x4), which can only happen in > connection_closing(). > > This is about as far as I can get; any other ideas? I'm stumped. The only way out of connection_closing() is for c_writers to go to zero. And from your core file, there are no other threads blocked on any of the write mutexes or condition variables. You can reproduce this easily? > john > > > (gdb) print *c > $2 = {c_struct_state = 2, c_conn_state = 4, c_conn_idx = 1878, c_sd = 1878, > c_close_reason = 0x0, c_mutex = {__data = {__lock = 1, __count = 0, > __owner = 9540, __kind = 0, __nusers = 1, {__spins = 0, __list = { > __next = 0x0}}}, > __size = "\001\000\000\000\000\000\000\000D%\000\000\000\000\000\000\001\000\000\000\000\000\000", __align = 1}, c_sb = 0x175c7240, > c_starttime = 1240942246, c_activitytime = 1240942264, c_connid = 73097614, > c_peer_domain = {bv_len = 7, bv_val = 0x137e02c8 "unknown"}, c_peer_name = { > bv_len = 23, bv_val = 0x191c0be0 "IP=172.16.182.103:50023"}, > c_listener = 0x9bb7370, c_sasl_bind_mech = {bv_len = 0, bv_val = 0x0}, > c_sasl_dn = {bv_len = 0, bv_val = 0x0}, c_sasl_authz_dn = {bv_len = 0, > bv_val = 0x0}, c_authz_backend = 0x9bba500, c_authz_cookie = 0x0, > c_authz = {sai_method = 128, sai_mech = {bv_len = 0, bv_val = 0x0}, > sai_dn = {bv_len = 27, bv_val = 0x1876bd20 "cn=nsspluspasswd,o=example2"}, > sai_ndn = {bv_len = 27, > bv_val = 0x189f5780 "cn=nsspluspasswd,o=example2"}, sai_ssf = 128, > sai_transport_ssf = 0, sai_tls_ssf = 128, sai_sasl_ssf = 0}, > c_protocol = 3, c_ops = {stqh_first = 0x0, stqh_last = 0x9d7df18}, > c_pending_ops = {stqh_first = 0x0, stqh_last = 0x9d7df20}, c_write1_mutex = { > __data = {__lock = 0, __count = 0, __owner = 0, __kind = 0, __nusers = 0, { > __spins = 0, __list = {__next = 0x0}}}, > __size = '\0'<repeats 23 times>, __align = 0}, c_write1_cv = {__data = { > __lock = 0, __futex = 2, __total_seq = 1, __wakeup_seq = 1, > __woken_seq = 1, __mutex = 0x9d7df28, __nwaiters = 0, > __broadcast_seq = 1}, > __size = "\000\000\000\000\002\000\000\000\001\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000(##\t\000\000\000\000\001\000\000\000\000\000\000", __align = 8589934592}, c_write2_mutex = { > __data = {__lock = 0, __count = 0, __owner = 0, __kind = 0, __nusers = 0, { > __spins = 0, __list = {__next = 0x0}}}, > __size = '\0'<repeats 23 times>, __align = 0}, c_write2_cv = {__data = { > __lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, > __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, > __size = '\0'<repeats 47 times>, __align = 0}, c_currentber = 0x0, > c_writers = -1, c_sasl_bind_in_progress = 0 '\0', c_writewaiter = 0 '\0', > c_is_tls = 1 '\001', c_needs_tls_accept = 0 '\0', c_sasl_layers = 0 '\0', > c_sasl_done = 0 '\0', c_sasl_authctx = 0x13497200, c_sasl_sockctx = 0x0, > c_sasl_extra = 0x14d956c0, c_sasl_bindop = 0x0, c_pagedresults_state = { > ps_be = 0x0, ps_size = 0, ps_count = 0, ps_cookie = 0, ps_cookieval = { > bv_len = 0, bv_val = 0x0}}, c_n_ops_received = 6, c_n_ops_executing = 0, > c_n_ops_pending = 0, c_n_ops_completed = 6, c_n_get = 4, c_n_read = 4, > c_n_write = 0, c_extensions = 0x0, c_clientfunc = 0, c_clientarg = 0x0, > c_send_ldap_result = 0x808aea0<slap_send_ldap_result>, > c_send_search_entry = 0x80885e0<slap_send_search_entry>, > c_send_search_reference = 0x8087da0<slap_send_search_reference>, > c_send_ldap_extended = 0x808a910<slap_send_ldap_extended>, > c_send_ldap_intermediate = 0x808a760<slap_send_ldap_intermediate>} > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
On Wed, May 06, 2009 at 02:18:30AM -0700, Howard Chu wrote: > jwm@horde.net wrote: >> Poked around a bit in the core: >> >> (gdb) print c->c_writers >> $1 = -1 >> (gdb) print c->c_pending_ops->stqh_last[0] >> $5 = (struct Operation *) 0x0 >> (gdb) print c->c_n_ops_pending >> $6 = 0 >> >> So there are no pending ops on this connection, but c_writers == -1 >> indicates that one blocked writer remains to notice that the connection has >> been closed. >> >> I looked at send_ldap_ber() and can't immediately find fault with its >> manipulation of c_writers. connection_closing() wakes up all blocked >> writers, so by the time connection_close() calls connection_destroy() and >> this assertion is checked, c_writers should be 0. Furthermore, >> c->c_conn_state is SLAP_C_CLOSING (0x4), which can only happen in >> connection_closing(). >> >> This is about as far as I can get; any other ideas? > > I'm stumped. The only way out of connection_closing() is for c_writers to > go to zero. And from your core file, there are no other threads blocked > on any of the write mutexes or condition variables. You can reproduce > this easily? It's happened once or twice, but it hasn't had much time since this is the same machine I just got done chasing the i386 heap fragmentation on. Our backtrace/core/BDB harness is still on this slapd, so we should have information if it happens again. john -- John Morrissey _o /\ ---- __o jwm@horde.net _-< \_ / \ ---- < \, www.horde.net/ __(_)/_(_)________/ \_______(_) /_(_)__
changed notes changed state Open to Test moved from Incoming to Software Bugs
John Morrissey wrote: > It's happened once or twice, but it hasn't had much time since this is the > same machine I just got done chasing the i386 heap fragmentation on. Our > backtrace/core/BDB harness is still on this slapd, so we should have > information if it happens again. I was able to reproduce this today, a fix is now in HEAD. Please test, thanks. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
On Tue, Jun 30, 2009 at 06:05:09PM -0700, Howard Chu wrote: > John Morrissey wrote: > > It's happened once or twice, but it hasn't had much time since this is > > the same machine I just got done chasing the i386 heap fragmentation on. > > Our backtrace/core/BDB harness is still on this slapd, so we should have > > information if it happens again. > > I was able to reproduce this today, a fix is now in HEAD. Please test, > thanks. Sorry Howard, we've retired our i386 machines and haven't seen this with RE24 as of 20090511 on amd64. john -- John Morrissey _o /\ ---- __o jwm@horde.net _-< \_ / \ ---- < \, www.horde.net/ __(_)/_(_)________/ \_______(_) /_(_)__
changed notes changed state Test to Release
changed notes changed state Release to Closed
fixed in HEAD Fixed in RE24