[Date Prev][Date Next] [Chronological] [Thread] [Top]

back-meta hangs (ITS#3058)



Full_Name: Aleksander Adamowski
Version: 2.1.29
OS: Fedora Core 1
URL: 
Submission from: (NULL) (157.25.5.68)


I have a setup where back-meta is used as a dynamic failover mechanism to
forward operations to 2 other OpenLDAP instances that are running back-bdb. It
listens on a port different than 389.

The relevant config options for the back-meta instance are (schema, access
control, TLS and SASL options omitted):

allow bind_v2 bind_anon_dn
pidfile /var/run/ldap/slapd_meta.pid
sizelimit 1024
threads 64
concurrency 36
database        meta
suffix          "dc=ourdomain,dc=pl,o=OurOrganization"
rootdn    "cn=Manager,dc=ourdomain,dc=pl,o=OurOrganization"
rootpw    {SSHA}CONFIDENTIALDATA
uri           "ldap://localhost:389/dc=ourdomain,dc=pl,o=OurOrganization 
ldaps://ldapreplica.ourdomain.pl"

# Legacy LDAP systems backward compatibility suffixes:
database        meta
suffix          "o=ourdomain.com.pl"
uri           "ldap://localhost:389/o=ourdomain.com.pl
ldaps://ldapreplica.ourdomain.pl"
suffixmassage  "o=ourdomain.com.pl" "dc=ourdomain,dc=pl,o=OurOrganization"

database        meta
suffix          ""
uri           "ldap://localhost:389/dc=ourdomain,dc=pl,o=OurOrganization 
ldaps://ldapreplica.ourdomain.pl"



About once in every 2-4 days back-meta hangs, and queries start to time out.

Usually only a few threads hang (it looks like they're deadlocked), so some
clients can receive results, and some are waiting for a reply until they
timeout.

I started running the back-meta instance in foreground mode on a screen session,
to have better control over it and more comfortable debugging. When back-meta
goes into the deadlock, and I press CTRL-C on its terminal, it starts exiting
cleanly, then ends up waiting for the deadlocked threads to terminate, e.g.:

<... multiple connections closing ...>
conn=148275 fd=89 closed
conn=149007 fd=98 closed
conn=148311 fd=102 closed
slapd shutdown: waiting for 4 threads to terminate


I have to kill the process manually to get rid of it and be able to start it
again.

I've attached with gdb to the slapd process, and extracted stacktraces for all
threads using gdb's command 'thread apply all bt'. Here are the results of 2
groups of stacktraces:

stacktrace group 1) When some back-meta threads are deadlocked, but before I
sent CTRL-C to slapd:

(no debugging symbols found)...Using host libthread_db library
"/lib/tls/libthread_db.so.1".
[Thread debugging using libthread_db enabled]
[New Thread -1085073760 (LWP 10746)]
[New Thread -1230042192 (LWP 15671)]
[New Thread -1225843792 (LWP 15661)]
[New Thread -1221645392 (LWP 15658)]
[New Thread -1217446992 (LWP 8958)]
[New Thread -1213248592 (LWP 12915)]
[New Thread -1209050192 (LWP 12716)]
[New Thread -1204851792 (LWP 12715)]
[New Thread -1200653392 (LWP 12714)]
[New Thread -1196454992 (LWP 12713)]
[New Thread -1192256592 (LWP 12626)]
[New Thread -1188058192 (LWP 12625)]
[New Thread -1183859792 (LWP 17675)]
[New Thread -1179661392 (LWP 17673)]
[New Thread -1175462992 (LWP 15964)]
[New Thread -1171264592 (LWP 15963)]
[New Thread -1167066192 (LWP 15960)]
[New Thread -1160795216 (LWP 23206)]
[New Thread -1156596816 (LWP 23203)]
[New Thread -1152398416 (LWP 23189)]
[New Thread -1148200016 (LWP 23188)]
[New Thread -1144001616 (LWP 23179)]
[New Thread -1139803216 (LWP 23178)]
[New Thread -1133548624 (LWP 23175)]
[New Thread -1129346128 (LWP 23170)]
[New Thread -1125147728 (LWP 23165)]
[New Thread -1120949328 (LWP 23164)]
[New Thread -1116750928 (LWP 23163)]
[New Thread -1112552528 (LWP 23162)]
[New Thread -1108354128 (LWP 23156)]
[New Thread -1104155728 (LWP 23146)]
[New Thread -1099957328 (LWP 23120)]
[New Thread -1093809232 (LWP 18990)]
[New Thread -1089610832 (LWP 11050)]
[New Thread -1085412432 (LWP 10761)]
0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
277	_dl_start (void *arg)
Thread 35 (Thread -1085412432 (LWP 10761)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00c5bb21 in ?? () at regex_internal.c:1295 from /lib/tls/libc.so.6
#2  0x0805e1a9 in slapd_daemon_destroy ()
#3  0x0011e7fc in start_thread (arg=0xfffffdfe) at pthread_create.c:264
#4  0x00c62aba in ?? () from /lib/tls/libc.so.6
Thread 34 (Thread -1089610832 (LWP 11050)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00c6fbe6 in ?? () at
../nptl/sysdeps/unix/sysv/linux/i386/i486/lowlevellock.S:64 from
/lib/tls/libc.so.6
#2  0xbf0dc5e4 in ?? ()
#3  0x00cbd238 in ?? () from /lib/tls/libc.so.6
#4  0x00000001 in ?? ()
#5  0x00c5eda3 in ?? () from /lib/tls/libc.so.6
#6  0xba767c40 in ?? ()
#7  0x0812ff00 in lt_preloaded_symbols ()
#8  0xbf0dc7dc in ?? ()
#9  0xbf0dc768 in ?? ()
#10 0x00cbad40 in ?? () from /lib/tls/libc.so.6
Thread 33 (Thread -1093809232 (LWP 18990)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00121424 in ?? () from /lib/tls/libpthread.so.0
#2  0x080f5346 in ldap_pvt_thread_cond_wait ()
#3  0x080f4f25 in ldap_pvt_thread_pool_destroy ()
#4  0x096931c4 in ?? ()
Thread 32 (Thread -1099957328 (LWP 23120)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00c5bb21 in ?? () at regex_internal.c:1295 from /lib/tls/libc.so.6
#2  0x081085b9 in ldap_int_select ()
#3  0x080f63bf in ldap_result ()
#4  0x080fcd16 in ldap_sasl_bind_s ()
#5  0x080fd55c in ldap_simple_bind_s ()
#6  0x080f5545 in ldap_bind_s ()
#7  0x080dd911 in meta_back_bind ()
#8  0x080dd5b9 in meta_back_bind ()
#9  0x0807e174 in do_bind ()
---Type <return> to continue, or q <return> to quit---
#10 0x08061ed9 in connection_done ()
#11 0x080f4dd4 in ldap_pvt_thread_pool_destroy ()
#12 0x09729be0 in ?? ()
Thread 31 (Thread -1104155728 (LWP 23146)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00121424 in ?? () from /lib/tls/libpthread.so.0
#2  0x080f5346 in ldap_pvt_thread_cond_wait ()
#3  0x080f4f25 in ldap_pvt_thread_pool_destroy ()
#4  0x096931c4 in ?? ()
Thread 30 (Thread -1108354128 (LWP 23156)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00121424 in ?? () from /lib/tls/libpthread.so.0
#2  0x080f5346 in ldap_pvt_thread_cond_wait ()
#3  0x080f4f25 in ldap_pvt_thread_pool_destroy ()
#4  0x096931c4 in ?? ()
Thread 29 (Thread -1112552528 (LWP 23162)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00c5bb21 in ?? () at regex_internal.c:1295 from /lib/tls/libc.so.6
#2  0x081085b9 in ldap_int_select ()
#3  0x080f63bf in ldap_result ()
#4  0x080fcd16 in ldap_sasl_bind_s ()
#5  0x080fd55c in ldap_simple_bind_s ()
#6  0x080f5545 in ldap_bind_s ()
#7  0x080dd911 in meta_back_bind ()
#8  0x080dd5b9 in meta_back_bind ()
#9  0x0807e174 in do_bind ()
#10 0x08061ed9 in connection_done ()
#11 0x080f4dd4 in ldap_pvt_thread_pool_destroy ()
#12 0xbe7ce988 in ?? ()
Thread 28 (Thread -1116750928 (LWP 23163)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00121424 in ?? () from /lib/tls/libpthread.so.0
#2  0x080f5346 in ldap_pvt_thread_cond_wait ()
#3  0x080f4f25 in ldap_pvt_thread_pool_destroy ()
#4  0x096931c4 in ?? ()
Thread 27 (Thread -1120949328 (LWP 23164)):
---Type <return> to continue, or q <return> to quit---
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00121424 in ?? () from /lib/tls/libpthread.so.0
#2  0x080f5346 in ldap_pvt_thread_cond_wait ()
#3  0x080f4f25 in ldap_pvt_thread_pool_destroy ()
#4  0x096931c4 in ?? ()
Thread 26 (Thread -1125147728 (LWP 23165)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00c5bb21 in ?? () at regex_internal.c:1295 from /lib/tls/libc.so.6
#2  0x081085b9 in ldap_int_select ()
#3  0x080f63bf in ldap_result ()
#4  0x080fcd16 in ldap_sasl_bind_s ()
#5  0x080fd55c in ldap_simple_bind_s ()
#6  0x080f5545 in ldap_bind_s ()
#7  0x080dd911 in meta_back_bind ()
#8  0x080dd5b9 in meta_back_bind ()
#9  0x0807e174 in do_bind ()
#10 0x08061ed9 in connection_done ()
#11 0x080f4dd4 in ldap_pvt_thread_pool_destroy ()
#12 0xbc1941a0 in ?? ()
Thread 25 (Thread -1129346128 (LWP 23170)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00c6fbe6 in ?? () at
../nptl/sysdeps/unix/sysv/linux/i386/i486/lowlevellock.S:64 from
/lib/tls/libc.so.6
#2  0xbcaf8604 in ?? ()
#3  0x00cbd238 in ?? () from /lib/tls/libc.so.6
#4  0x00000001 in ?? ()
#5  0x00c5eda3 in ?? () from /lib/tls/libc.so.6
#6  0xba767c40 in ?? ()
#7  0x08135660 in lt_preloaded_symbols ()
#8  0xbcaf87f8 in ?? ()
#9  0xbcaf8788 in ?? ()
#10 0x00cbad40 in ?? () from /lib/tls/libc.so.6
#11 0x00000001 in ?? ()
#12 0xbcaf8608 in ?? ()
#13 0x08118a4d in ber_free ()
#14 0x00c5e35f in syslog (pri=-4, fmt=0xfffffffc <Address 0xfffffffc out of
bounds>) at syslog.c:122
#15 0x0807e326 in do_bind ()
#16 0x08061ed9 in connection_done ()
#17 0x080f4dd4 in ldap_pvt_thread_pool_destroy ()
#18 0xba705608 in ?? ()
---Type <return> to continue, or q <return> to quit---
Thread 24 (Thread -1133548624 (LWP 23175)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00c5bb21 in ?? () at regex_internal.c:1295 from /lib/tls/libc.so.6
#2  0x081085b9 in ldap_int_select ()
#3  0x080f63bf in ldap_result ()
#4  0x080fcd16 in ldap_sasl_bind_s ()
#5  0x080fd55c in ldap_simple_bind_s ()
#6  0x080f5545 in ldap_bind_s ()
#7  0x080dd911 in meta_back_bind ()
#8  0x080dd5b9 in meta_back_bind ()
#9  0x0807e174 in do_bind ()
#10 0x08061ed9 in connection_done ()
#11 0x080f4dd4 in ldap_pvt_thread_pool_destroy ()
#12 0x0972ea90 in ?? ()
Thread 23 (Thread -1139803216 (LWP 23178)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00c5bb21 in ?? () at regex_internal.c:1295 from /lib/tls/libc.so.6
#2  0x081085b9 in ldap_int_select ()
#3  0x080f63bf in ldap_result ()
#4  0x080fcd16 in ldap_sasl_bind_s ()
#5  0x080fd55c in ldap_simple_bind_s ()
#6  0x080f5545 in ldap_bind_s ()
#7  0x080dd911 in meta_back_bind ()
#8  0x080dd5b9 in meta_back_bind ()
#9  0x0807e174 in do_bind ()
#10 0x08061ed9 in connection_done ()
#11 0x080f4dd4 in ldap_pvt_thread_pool_destroy ()
#12 0xbe78ebf8 in ?? ()
Thread 22 (Thread -1144001616 (LWP 23179)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00121424 in ?? () from /lib/tls/libpthread.so.0
#2  0x080f5346 in ldap_pvt_thread_cond_wait ()
#3  0x080f4f25 in ldap_pvt_thread_pool_destroy ()
#4  0x096931c4 in ?? ()
Thread 21 (Thread -1148200016 (LWP 23188)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00121424 in ?? () from /lib/tls/libpthread.so.0
---Type <return> to continue, or q <return> to quit---
#2  0x080f5346 in ldap_pvt_thread_cond_wait ()
#3  0x080f4f25 in ldap_pvt_thread_pool_destroy ()
#4  0x096931c4 in ?? ()
Thread 20 (Thread -1152398416 (LWP 23189)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00c5bb21 in ?? () at regex_internal.c:1295 from /lib/tls/libc.so.6
#2  0x081085b9 in ldap_int_select ()
#3  0x080f63bf in ldap_result ()
#4  0x080fcd16 in ldap_sasl_bind_s ()
#5  0x080fd55c in ldap_simple_bind_s ()
#6  0x080f5545 in ldap_bind_s ()
#7  0x080dd911 in meta_back_bind ()
#8  0x080dd5b9 in meta_back_bind ()
#9  0x0807e174 in do_bind ()
#10 0x08061ed9 in connection_done ()
#11 0x080f4dd4 in ldap_pvt_thread_pool_destroy ()
#12 0xbc104568 in ?? ()
Thread 19 (Thread -1156596816 (LWP 23203)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00c6fbe6 in ?? () at
../nptl/sysdeps/unix/sysv/linux/i386/i486/lowlevellock.S:64 from
/lib/tls/libc.so.6
#2  0xbb0fa5e4 in ?? ()
#3  0x00cbd238 in ?? () from /lib/tls/libc.so.6
#4  0x00000001 in ?? ()
#5  0x00c5eda3 in ?? () from /lib/tls/libc.so.6
#6  0xba767c40 in ?? ()
#7  0x0812fde9 in lt_preloaded_symbols ()
#8  0xbb0fa7d4 in ?? ()
#9  0xbb0fa768 in ?? ()
#10 0x00cbad40 in ?? () from /lib/tls/libc.so.6
Thread 18 (Thread -1160795216 (LWP 23206)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00c5bb21 in ?? () at regex_internal.c:1295 from /lib/tls/libc.so.6
#2  0x081085b9 in ldap_int_select ()
#3  0x080f63bf in ldap_result ()
#4  0x080fcd16 in ldap_sasl_bind_s ()
#5  0x080fd55c in ldap_simple_bind_s ()
#6  0x080f5545 in ldap_bind_s ()
#7  0x080dd911 in meta_back_bind ()
---Type <return> to continue, or q <return> to quit---
#8  0x080dd5b9 in meta_back_bind ()
#9  0x0807e174 in do_bind ()
#10 0x08061ed9 in connection_done ()
#11 0x080f4dd4 in ldap_pvt_thread_pool_destroy ()
#12 0xbc16d3b8 in ?? ()
Thread 17 (Thread -1167066192 (LWP 15960)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00c5bb21 in ?? () at regex_internal.c:1295 from /lib/tls/libc.so.6
#2  0x081085b9 in ldap_int_select ()
#3  0x080f63bf in ldap_result ()
#4  0x080b58b5 in meta_back_search ()
#5  0x08063b89 in do_search ()
#6  0xbf4e1744 in ?? ()
#7  0xbe708a00 in ?? ()
#8  0xba6fe850 in ?? ()
#9  0xba6fe858 in ?? ()
#10 0x00000002 in ?? ()
Thread 16 (Thread -1171264592 (LWP 15963)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00c5bb21 in ?? () at regex_internal.c:1295 from /lib/tls/libc.so.6
#2  0x081085b9 in ldap_int_select ()
#3  0x080f63bf in ldap_result ()
#4  0x080fcd16 in ldap_sasl_bind_s ()
#5  0x080fd55c in ldap_simple_bind_s ()
#6  0x080f5545 in ldap_bind_s ()
#7  0x080dd911 in meta_back_bind ()
#8  0x080dd5b9 in meta_back_bind ()
#9  0x0807e174 in do_bind ()
#10 0x08061ed9 in connection_done ()
#11 0x080f4dd4 in ldap_pvt_thread_pool_destroy ()
#12 0x0972c158 in ?? ()
Thread 15 (Thread -1175462992 (LWP 15964)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00c6fbe6 in ?? () at
../nptl/sysdeps/unix/sysv/linux/i386/i486/lowlevellock.S:64 from
/lib/tls/libc.so.6
#2  0xb9efd604 in ?? ()
#3  0x00cbd238 in ?? () from /lib/tls/libc.so.6
#4  0x00000001 in ?? ()
#5  0x00c5eda3 in ?? () from /lib/tls/libc.so.6
---Type <return> to continue, or q <return> to quit---
#6  0xbc13f2a8 in ?? ()
#7  0x08135660 in lt_preloaded_symbols ()
#8  0xb9efd7f8 in ?? ()
#9  0xb9efd788 in ?? ()
#10 0x00cbad40 in ?? () from /lib/tls/libc.so.6
#11 0x00000001 in ?? ()
#12 0xb9efd608 in ?? ()
#13 0x08118a4d in ber_free ()
#14 0x00c5e35f in syslog (pri=-4, fmt=0xfffffffc <Address 0xfffffffc out of
bounds>) at syslog.c:122
#15 0x0807e326 in do_bind ()
#16 0x08061ed9 in connection_done ()
#17 0x080f4dd4 in ldap_pvt_thread_pool_destroy ()
#18 0xbe706608 in ?? ()
Thread 14 (Thread -1179661392 (LWP 17673)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00c5bb21 in ?? () at regex_internal.c:1295 from /lib/tls/libc.so.6
#2  0x081085b9 in ldap_int_select ()
#3  0x080f63bf in ldap_result ()
#4  0x080fcd16 in ldap_sasl_bind_s ()
#5  0x080fd55c in ldap_simple_bind_s ()
#6  0x080f5545 in ldap_bind_s ()
#7  0x080dd911 in meta_back_bind ()
#8  0x080dd5b9 in meta_back_bind ()
#9  0x0807e174 in do_bind ()
#10 0x08061ed9 in connection_done ()
#11 0x080f4dd4 in ldap_pvt_thread_pool_destroy ()
#12 0xbc17faf8 in ?? ()
Thread 13 (Thread -1183859792 (LWP 17675)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00c5bb21 in ?? () at regex_internal.c:1295 from /lib/tls/libc.so.6
#2  0x081085b9 in ldap_int_select ()
#3  0x080f63bf in ldap_result ()
#4  0x080b58b5 in meta_back_search ()
#5  0x08063b89 in do_search ()
#6  0xbf4e1234 in ?? ()
#7  0xbe709ef0 in ?? ()
#8  0xb96fa850 in ?? ()
#9  0xb96fa858 in ?? ()
#10 0x00000002 in ?? ()
---Type <return> to continue, or q <return> to quit---
Thread 12 (Thread -1188058192 (LWP 12625)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00121424 in ?? () from /lib/tls/libpthread.so.0
#2  0x080f5346 in ldap_pvt_thread_cond_wait ()
#3  0x080f4f25 in ldap_pvt_thread_pool_destroy ()
#4  0x096931c4 in ?? ()
Thread 11 (Thread -1192256592 (LWP 12626)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00c63838 in ?? () from /lib/tls/libc.so.6
#2  0x00c5e6fb in vsyslog (pri=0, fmt=0xfffffe00 <Address 0xfffffe00 out of
bounds>, 
    ap=0xfffffe00 <Address 0xfffffe00 out of bounds>) at syslog.c:273
#3  0x00c5e35f in syslog (pri=-512, fmt=0xfffffe00 <Address 0xfffffe00 out of
bounds>) at syslog.c:122
#4  0x08063ece in do_search ()
#5  0x0812fde9 in lt_preloaded_symbols ()
#6  0x000245f3 in ?? ()
#7  0x00000035 in ?? ()
#8  0xb8ef8880 in ?? ()
Thread 10 (Thread -1196454992 (LWP 12713)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00121424 in ?? () from /lib/tls/libpthread.so.0
#2  0x080f5346 in ldap_pvt_thread_cond_wait ()
#3  0x080f4f25 in ldap_pvt_thread_pool_destroy ()
#4  0x096931c4 in ?? ()
Thread 9 (Thread -1200653392 (LWP 12714)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00121424 in ?? () from /lib/tls/libpthread.so.0
#2  0x080f5346 in ldap_pvt_thread_cond_wait ()
#3  0x080f4f25 in ldap_pvt_thread_pool_destroy ()
#4  0x096931c4 in ?? ()
Thread 8 (Thread -1204851792 (LWP 12715)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00c6fbe6 in ?? () at
../nptl/sysdeps/unix/sysv/linux/i386/i486/lowlevellock.S:64 from
/lib/tls/libc.so.6
#2  0xb82f55e4 in ?? ()
#3  0x00cbd238 in ?? () from /lib/tls/libc.so.6
#4  0x00000001 in ?? ()
#5  0x00c5eda3 in ?? () from /lib/tls/libc.so.6
---Type <return> to continue, or q <return> to quit---
#6  0xbe7cc840 in ?? ()
#7  0x0812ff00 in lt_preloaded_symbols ()
#8  0xb82f57dc in ?? ()
#9  0xb82f5768 in ?? ()
#10 0x00cbad40 in ?? () from /lib/tls/libc.so.6
Thread 7 (Thread -1209050192 (LWP 12716)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00121424 in ?? () from /lib/tls/libpthread.so.0
#2  0x080f5346 in ldap_pvt_thread_cond_wait ()
#3  0x080f4f25 in ldap_pvt_thread_pool_destroy ()
#4  0x096931c4 in ?? ()
Thread 6 (Thread -1213248592 (LWP 12915)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00121424 in ?? () from /lib/tls/libpthread.so.0
#2  0x080f5346 in ldap_pvt_thread_cond_wait ()
#3  0x080f4f25 in ldap_pvt_thread_pool_destroy ()
#4  0x096931c4 in ?? ()
Thread 5 (Thread -1217446992 (LWP 8958)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00121424 in ?? () from /lib/tls/libpthread.so.0
#2  0x080f5346 in ldap_pvt_thread_cond_wait ()
#3  0x080f4f25 in ldap_pvt_thread_pool_destroy ()
#4  0x096931c4 in ?? ()
Thread 4 (Thread -1221645392 (LWP 15658)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00121424 in ?? () from /lib/tls/libpthread.so.0
#2  0x080f5346 in ldap_pvt_thread_cond_wait ()
#3  0x080f4f25 in ldap_pvt_thread_pool_destroy ()
#4  0x096931c4 in ?? ()
Thread 3 (Thread -1225843792 (LWP 15661)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00c6fbe6 in ?? () at
../nptl/sysdeps/unix/sysv/linux/i386/i486/lowlevellock.S:64 from
/lib/tls/libc.so.6
#2  0xb6ef05e4 in ?? ()
#3  0x00cbd238 in ?? () from /lib/tls/libc.so.6
#4  0x00000001 in ?? ()
#5  0x00c5eda3 in ?? () from /lib/tls/libc.so.6
---Type <return> to continue, or q <return> to quit---
#6  0xb6506918 in ?? ()
#7  0x0812ff00 in lt_preloaded_symbols ()
#8  0xb6ef07dc in ?? ()
#9  0xb6ef0768 in ?? ()
#10 0x00cbad40 in ?? () from /lib/tls/libc.so.6
Thread 2 (Thread -1230042192 (LWP 15671)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00121424 in ?? () from /lib/tls/libpthread.so.0
#2  0x080f5346 in ldap_pvt_thread_cond_wait ()
#3  0x080f4f25 in ldap_pvt_thread_pool_destroy ()
#4  0x096931c4 in ?? ()
Thread 1 (Thread -1085073760 (LWP 10746)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x0011fe83 in pthread_join (threadid=3209554864, thread_return=0x0) at
pthread_join.c:86
#2  0x080f5206 in ldap_pvt_thread_join ()
#3  0x0805fc33 in slapd_daemon ()
#4  0x0805b89d in main ()
0x00b6fc32	277	_dl_start (void *arg)


stacktrace group 2) When only deadlocked back-meta threads are left, after I've
sent CTRL-C to slapd:

(no debugging symbols found)...Using host libthread_db library
"/lib/tls/libthread_db.so.1".
[Thread debugging using libthread_db enabled]
[New Thread -1085073760 (LWP 10746)]
[New Thread -1152398416 (LWP 23189)]
[New Thread -1139803216 (LWP 23178)]
[New Thread -1125147728 (LWP 23165)]
[New Thread -1099957328 (LWP 23120)]
[New Thread -1085412432 (LWP 10761)]
0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
277	_dl_start (void *arg)
Thread 6 (Thread -1085412432 (LWP 10761)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00c4a9dc in ?? () at regex_internal.c:1295 from /lib/tls/libc.so.6
#2  0x080f5259 in ldap_pvt_thread_yield ()
#3  0x080f4c25 in ldap_pvt_thread_pool_destroy ()
#4  0x08159de4 in ?? ()
#5  0x08159de4 in ?? ()
Thread 5 (Thread -1099957328 (LWP 23120)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00c5bb21 in ?? () at regex_internal.c:1295 from /lib/tls/libc.so.6
#2  0x081085b9 in ldap_int_select ()
#3  0x080f63bf in ldap_result ()
#4  0x080fcd16 in ldap_sasl_bind_s ()
#5  0x080fd55c in ldap_simple_bind_s ()
#6  0x080f5545 in ldap_bind_s ()
#7  0x080dd911 in meta_back_bind ()
#8  0x080dd5b9 in meta_back_bind ()
#9  0x0807e174 in do_bind ()
#10 0x08061ed9 in connection_done ()
#11 0x080f4dd4 in ldap_pvt_thread_pool_destroy ()
#12 0x09729be0 in ?? ()
Thread 4 (Thread -1125147728 (LWP 23165)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00c5bb21 in ?? () at regex_internal.c:1295 from /lib/tls/libc.so.6
#2  0x081085b9 in ldap_int_select ()
#3  0x080f63bf in ldap_result ()
#4  0x080fcd16 in ldap_sasl_bind_s ()
#5  0x080fd55c in ldap_simple_bind_s ()
#6  0x080f5545 in ldap_bind_s ()
#7  0x080dd911 in meta_back_bind ()
#8  0x080dd5b9 in meta_back_bind ()
#9  0x0807e174 in do_bind ()
#10 0x08061ed9 in connection_done ()
#11 0x080f4dd4 in ldap_pvt_thread_pool_destroy ()
#12 0xbc1941a0 in ?? ()
---Type <return> to continue, or q <return> to quit---
Thread 3 (Thread -1139803216 (LWP 23178)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00c5bb21 in ?? () at regex_internal.c:1295 from /lib/tls/libc.so.6
#2  0x081085b9 in ldap_int_select ()
#3  0x080f63bf in ldap_result ()
#4  0x080fcd16 in ldap_sasl_bind_s ()
#5  0x080fd55c in ldap_simple_bind_s ()
#6  0x080f5545 in ldap_bind_s ()
#7  0x080dd911 in meta_back_bind ()
#8  0x080dd5b9 in meta_back_bind ()
#9  0x0807e174 in do_bind ()
#10 0x08061ed9 in connection_done ()
#11 0x080f4dd4 in ldap_pvt_thread_pool_destroy ()
#12 0xbe78ebf8 in ?? ()
Thread 2 (Thread -1152398416 (LWP 23189)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x00c5bb21 in ?? () at regex_internal.c:1295 from /lib/tls/libc.so.6
#2  0x081085b9 in ldap_int_select ()
#3  0x080f63bf in ldap_result ()
#4  0x080fcd16 in ldap_sasl_bind_s ()
#5  0x080fd55c in ldap_simple_bind_s ()
#6  0x080f5545 in ldap_bind_s ()
#7  0x080dd911 in meta_back_bind ()
#8  0x080dd5b9 in meta_back_bind ()
#9  0x0807e174 in do_bind ()
#10 0x08061ed9 in connection_done ()
#11 0x080f4dd4 in ldap_pvt_thread_pool_destroy ()
#12 0xbc104568 in ?? ()
Thread 1 (Thread -1085073760 (LWP 10746)):
#0  0x00b6fc32 in ?? () at rtld.c:277 from /lib/ld-linux.so.2
#1  0x0011fe83 in pthread_join (threadid=3209554864, thread_return=0x0) at
pthread_join.c:86
#2  0x080f5206 in ldap_pvt_thread_join ()
#3  0x0805fc33 in slapd_daemon ()
#4  0x0805b89d in main ()
0x00b6fc32	277	_dl_start (void *arg)