Full_Name: gerard gobillard Version: 2.4.30 OS: RHEL 5.2 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (92.139.226.97) Hello all, Summary : on an Intel basic box (dual core) we create search base traffic with slamd in asynchronous mode (30 threads). When, after 1 or 2 seconds, Ctrl-C is sent to slamd (SIGINT), slapd gets stuck with main thread in epoll_wait() and all other threads in pthread_cond_wait() (in send_ldap_ber()). It may be necessary to repeat a few times before slapd gets stuck Notes : no change to slapd.conf except RootDN. We build OpenLDAP as follows : CPPFLAGS="-I/usr/local/BerkeleyDB.4.7/include -g" export CPPFLAGS LDFLAGS="-L/usr/local/lib -L/usr/local/BerkeleyDB.4.7/lib -R/usr/local/BerkeleyDB.4.7/lib" export LDFLAGS LD_LIBRARY_PATH=/usr/local/BerkeleyDB.4.7/lib ./configure --prefix=/usr/slapd.2.4.23.Dbg --enable-spasswd=yes --enable-crypt=yes --disable-syslog --enable-debug --enable-ppolicy make depend make make install gstack: [root@lalxsdmcomp db-5.2.36]# gstack 19444 Thread 18 (Thread 1099237696 (LWP 19445)): #0 0x0000003cf08d21d8 in epoll_wait () from /lib64/libc.so.6 #1 0x000000000042bf11 in slapd_daemon_task () #2 0x0000003cf1406307 in start_thread () from /lib64/libpthread.so.0 #3 0x0000003cf08d1ded in clone () from /lib64/libc.so.6 Thread 17 (Thread 1112529216 (LWP 19463)): #0 0x0000003cf140a4b6 in pthread_cond_wait@@GLIBC_2.3.2 () #1 0x0000000000444428 in send_ldap_ber () #2 0x00000000004477da in slap_send_search_entry () #3 0x00000000004cdbe6 in bdb_search () #4 0x00000000004334e4 in fe_op_search () #5 0x0000000000432d9b in do_search () #6 0x000000000042fec7 in connection_operation () #7 0x000000000056562c in ldap_int_thread_pool_wrapper () #8 0x0000003cf1406307 in start_thread () from /lib64/libpthread.so.0 #9 0x0000003cf08d1ded in clone () from /lib64/libc.so.6 Thread 16 (Thread 1120921920 (LWP 19464)): #0 0x0000003cf140a4b6 in pthread_cond_wait@@GLIBC_2.3.2 () #1 0x0000000000444428 in send_ldap_ber () #2 0x00000000004477da in slap_send_search_entry () #3 0x00000000004cdbe6 in bdb_search () #4 0x00000000004334e4 in fe_op_search () #5 0x0000000000432d9b in do_search () #6 0x000000000042fec7 in connection_operation () #7 0x000000000056562c in ldap_int_thread_pool_wrapper () #8 0x0000003cf1406307 in start_thread () from /lib64/libpthread.so.0 #9 0x0000003cf08d1ded in clone () from /lib64/libc.so.6 Thread 15 (Thread 1129314624 (LWP 19465)): #0 0x0000003cf140a4b6 in pthread_cond_wait@@GLIBC_2.3.2 () #1 0x0000000000444428 in send_ldap_ber () #2 0x00000000004477da in slap_send_search_entry () #3 0x00000000004cdbe6 in bdb_search () #4 0x00000000004334e4 in fe_op_search () #5 0x0000000000432d9b in do_search () #6 0x000000000042fec7 in connection_operation () #7 0x000000000056562c in ldap_int_thread_pool_wrapper () #8 0x0000003cf1406307 in start_thread () from /lib64/libpthread.so.0 #9 0x0000003cf08d1ded in clone () from /lib64/libc.so.6 Thread 14 (Thread 1137707328 (LWP 19466)): #0 0x0000003cf140a4b6 in pthread_cond_wait@@GLIBC_2.3.2 () #1 0x0000000000444428 in send_ldap_ber () #2 0x00000000004477da in slap_send_search_entry () #3 0x00000000004cdbe6 in bdb_search () #4 0x00000000004334e4 in fe_op_search () #5 0x0000000000432d9b in do_search () #6 0x000000000042fec7 in connection_operation () #7 0x000000000056562c in ldap_int_thread_pool_wrapper () #8 0x0000003cf1406307 in start_thread () from /lib64/libpthread.so.0 #9 0x0000003cf08d1ded in clone () from /lib64/libc.so.6 Thread 13 (Thread 1146100032 (LWP 19467)): #0 0x0000003cf140a4b6 in pthread_cond_wait@@GLIBC_2.3.2 () #1 0x0000000000444428 in send_ldap_ber () #2 0x00000000004477da in slap_send_search_entry () #3 0x00000000004cdbe6 in bdb_search () #4 0x00000000004334e4 in fe_op_search () #5 0x0000000000432d9b in do_search () #6 0x000000000042fec7 in connection_operation () #7 0x000000000056562c in ldap_int_thread_pool_wrapper () #8 0x0000003cf1406307 in start_thread () from /lib64/libpthread.so.0 #9 0x0000003cf08d1ded in clone () from /lib64/libc.so.6 Thread 12 (Thread 1154492736 (LWP 19468)): #0 0x0000003cf140a4b6 in pthread_cond_wait@@GLIBC_2.3.2 () #1 0x0000000000444428 in send_ldap_ber () #2 0x00000000004477da in slap_send_search_entry () #3 0x00000000004cdbe6 in bdb_search () #4 0x00000000004334e4 in fe_op_search () #5 0x0000000000432d9b in do_search () #6 0x000000000042fec7 in connection_operation () #7 0x000000000056562c in ldap_int_thread_pool_wrapper () #8 0x0000003cf1406307 in start_thread () from /lib64/libpthread.so.0 #9 0x0000003cf08d1ded in clone () from /lib64/libc.so.6 Thread 11 (Thread 1162885440 (LWP 19469)): #0 0x0000003cf140a4b6 in pthread_cond_wait@@GLIBC_2.3.2 () #1 0x00000000004446a0 in send_ldap_ber () #2 0x00000000004477da in slap_send_search_entry () #3 0x00000000004cdbe6 in bdb_search () #4 0x00000000004334e4 in fe_op_search () #5 0x0000000000432d9b in do_search () #6 0x000000000042fec7 in connection_operation () #7 0x000000000056562c in ldap_int_thread_pool_wrapper () #8 0x0000003cf1406307 in start_thread () from /lib64/libpthread.so.0 #9 0x0000003cf08d1ded in clone () from /lib64/libc.so.6 Thread 10 (Thread 1171278144 (LWP 19470)): #0 0x0000003cf140a4b6 in pthread_cond_wait@@GLIBC_2.3.2 () #1 0x0000000000444428 in send_ldap_ber () #2 0x00000000004477da in slap_send_search_entry () #3 0x00000000004cdbe6 in bdb_search () #4 0x00000000004334e4 in fe_op_search () #5 0x0000000000432d9b in do_search () #6 0x000000000042fec7 in connection_operation () #7 0x000000000056562c in ldap_int_thread_pool_wrapper () #8 0x0000003cf1406307 in start_thread () from /lib64/libpthread.so.0 #9 0x0000003cf08d1ded in clone () from /lib64/libc.so.6 Thread 9 (Thread 1087490368 (LWP 19471)): #0 0x0000003cf140a4b6 in pthread_cond_wait@@GLIBC_2.3.2 () #1 0x0000000000444428 in send_ldap_ber () #2 0x00000000004477da in slap_send_search_entry () #3 0x00000000004cdbe6 in bdb_search () #4 0x00000000004334e4 in fe_op_search () #5 0x0000000000432d9b in do_search () #6 0x000000000042fec7 in connection_operation () #7 0x000000000056562c in ldap_int_thread_pool_wrapper () #8 0x0000003cf1406307 in start_thread () from /lib64/libpthread.so.0 #9 0x0000003cf08d1ded in clone () from /lib64/libc.so.6 Thread 8 (Thread 1179670848 (LWP 19472)): #0 0x0000003cf140a4b6 in pthread_cond_wait@@GLIBC_2.3.2 () #1 0x0000000000444428 in send_ldap_ber () #2 0x00000000004477da in slap_send_search_entry () #3 0x00000000004cdbe6 in bdb_search () #4 0x00000000004334e4 in fe_op_search () #5 0x0000000000432d9b in do_search () #6 0x000000000042fec7 in connection_operation () #7 0x000000000056562c in ldap_int_thread_pool_wrapper () #8 0x0000003cf1406307 in start_thread () from /lib64/libpthread.so.0 #9 0x0000003cf08d1ded in clone () from /lib64/libc.so.6 Thread 7 (Thread 1188063552 (LWP 19473)): #0 0x0000003cf140a4b6 in pthread_cond_wait@@GLIBC_2.3.2 () #1 0x0000000000444428 in send_ldap_ber () #2 0x00000000004477da in slap_send_search_entry () #3 0x00000000004cdbe6 in bdb_search () #4 0x00000000004334e4 in fe_op_search () #5 0x0000000000432d9b in do_search () #6 0x000000000042fec7 in connection_operation () #7 0x000000000056562c in ldap_int_thread_pool_wrapper () #8 0x0000003cf1406307 in start_thread () from /lib64/libpthread.so.0 #9 0x0000003cf08d1ded in clone () from /lib64/libc.so.6 Thread 6 (Thread 1196456256 (LWP 19474)): #0 0x0000003cf140a4b6 in pthread_cond_wait@@GLIBC_2.3.2 () #1 0x00000000004446a0 in send_ldap_ber () #2 0x00000000004477da in slap_send_search_entry () #3 0x00000000004cdbe6 in bdb_search () #4 0x00000000004334e4 in fe_op_search () #5 0x0000000000432d9b in do_search () #6 0x000000000042fec7 in connection_operation () #7 0x000000000056562c in ldap_int_thread_pool_wrapper () #8 0x0000003cf1406307 in start_thread () from /lib64/libpthread.so.0 #9 0x0000003cf08d1ded in clone () from /lib64/libc.so.6 Thread 5 (Thread 1204848960 (LWP 19475)): #0 0x0000003cf140a4b6 in pthread_cond_wait@@GLIBC_2.3.2 () #1 0x0000000000444428 in send_ldap_ber () #2 0x00000000004477da in slap_send_search_entry () #3 0x00000000004cdbe6 in bdb_search () #4 0x00000000004334e4 in fe_op_search () #5 0x0000000000432d9b in do_search () #6 0x000000000042fec7 in connection_operation () #7 0x000000000056562c in ldap_int_thread_pool_wrapper () #8 0x0000003cf1406307 in start_thread () from /lib64/libpthread.so.0 #9 0x0000003cf08d1ded in clone () from /lib64/libc.so.6 Thread 4 (Thread 1213241664 (LWP 19476)): #0 0x0000003cf140a4b6 in pthread_cond_wait@@GLIBC_2.3.2 () #1 0x0000000000444428 in send_ldap_ber () #2 0x00000000004477da in slap_send_search_entry () #3 0x00000000004cdbe6 in bdb_search () #4 0x00000000004334e4 in fe_op_search () #5 0x0000000000432d9b in do_search () #6 0x000000000042fec7 in connection_operation () #7 0x000000000056562c in ldap_int_thread_pool_wrapper () #8 0x0000003cf1406307 in start_thread () from /lib64/libpthread.so.0 #9 0x0000003cf08d1ded in clone () from /lib64/libc.so.6 Thread 3 (Thread 1221634368 (LWP 19477)): #0 0x0000003cf140a4b6 in pthread_cond_wait@@GLIBC_2.3.2 () #1 0x0000000000444428 in send_ldap_ber () #2 0x00000000004477da in slap_send_search_entry () #3 0x00000000004cdbe6 in bdb_search () #4 0x00000000004334e4 in fe_op_search () #5 0x0000000000432d9b in do_search () #6 0x000000000042fec7 in connection_operation () #7 0x000000000056562c in ldap_int_thread_pool_wrapper () #8 0x0000003cf1406307 in start_thread () from /lib64/libpthread.so.0 #9 0x0000003cf08d1ded in clone () from /lib64/libc.so.6 Thread 2 (Thread 1230027072 (LWP 19478)): #0 0x0000003cf140a4b6 in pthread_cond_wait@@GLIBC_2.3.2 () #1 0x00000000004446a0 in send_ldap_ber () #2 0x00000000004477da in slap_send_search_entry () #3 0x00000000004cdbe6 in bdb_search () #4 0x00000000004334e4 in fe_op_search () #5 0x0000000000432d9b in do_search () #6 0x000000000042fec7 in connection_operation () #7 0x000000000056562c in ldap_int_thread_pool_wrapper () #8 0x0000003cf1406307 in start_thread () from /lib64/libpthread.so.0 #9 0x0000003cf08d1ded in clone () from /lib64/libc.so.6 Thread 1 (Thread 47813429243440 (LWP 19444)): #0 0x0000003cf14075c5 in pthread_join () from /lib64/libpthread.so.0 #1 0x000000000042cf32 in slapd_daemon () #2 0x000000000040d2a0 in main () #0 0x0000003cf14075c5 in pthread_join () from /lib64/libpthread.so.0 netstat: [root@lalxsdmcomp db-5.2.36]# netstat -pan | grep :389 tcp 0 0 0.0.0.0:389 0.0.0.0:* LISTEN 19444/slapd tcp 28071 0 172.25.198.71:389 172.25.195.198:38743 CLOSE_WAIT 19444/slapd tcp 14026 0 172.25.198.71:389 172.25.195.198:38749 CLOSE_WAIT 19444/slapd tcp 57221 0 172.25.198.71:389 172.25.195.198:38751 CLOSE_WAIT 19444/slapd tcp 31304 0 172.25.198.71:389 172.25.195.198:38750 CLOSE_WAIT 19444/slapd tcp 38300 0 172.25.198.71:389 172.25.195.198:38745 CLOSE_WAIT 19444/slapd tcp 83048 0 172.25.198.71:389 172.25.195.198:38744 ESTABLISHED 19444/slapd tcp 3433 0 172.25.198.71:389 172.25.195.198:38747 CLOSE_WAIT 19444/slapd tcp 11058 0 172.25.198.71:389 172.25.195.198:38746 CLOSE_WAIT 19444/slapd tcp 47204 0 172.25.198.71:389 172.25.195.198:38757 CLOSE_WAIT 19444/slapd tcp 36392 0 172.25.198.71:389 172.25.195.198:38756 CLOSE_WAIT 19444/slapd tcp 2341 0 172.25.198.71:389 172.25.195.198:38759 CLOSE_WAIT 19444/slapd tcp 73183 0 172.25.198.71:389 172.25.195.198:38758 ESTABLISHED 19444/slapd tcp 71796 0 172.25.198.71:389 172.25.195.198:38753 CLOSE_WAIT 19444/slapd tcp 78209 0 172.25.198.71:389 172.25.195.198:38752 CLOSE_WAIT 19444/slapd tcp 35915 0 172.25.198.71:389 172.25.195.198:38755 CLOSE_WAIT 19444/slapd tcp 12701 0 172.25.198.71:389 172.25.195.198:38754 CLOSE_WAIT 19444/slapd tcp 20333 0 172.25.198.71:389 172.25.195.198:38765 CLOSE_WAIT 19444/slapd tcp 74817 0 172.25.198.71:389 172.25.195.198:38764 CLOSE_WAIT 19444/slapd tcp 73757 0 172.25.198.71:389 172.25.195.198:38767 CLOSE_WAIT 19444/slapd tcp 2705 0 172.25.198.71:389 172.25.195.198:38766 CLOSE_WAIT 19444/slapd tcp 36180 0 172.25.198.71:389 172.25.195.198:38761 CLOSE_WAIT 19444/slapd tcp 75029 0 172.25.198.71:389 172.25.195.198:38760 CLOSE_WAIT 19444/slapd tcp 49112 0 172.25.198.71:389 172.25.195.198:38763 CLOSE_WAIT 19444/slapd tcp 8726 0 172.25.198.71:389 172.25.195.198:38762 CLOSE_WAIT 19444/slapd tcp 44 0 172.25.198.71:389 172.25.195.198:38772 ESTABLISHED - tcp 4005 0 172.25.198.71:389 172.25.195.198:38769 CLOSE_WAIT 19444/slapd tcp 17789 0 172.25.198.71:389 172.25.195.198:38768 CLOSE_WAIT 19444/slapd tcp 9945 0 172.25.198.71:389 172.25.195.198:38770 CLOSE_WAIT 19444/slapd tcp 0 0 :::389 :::* LISTEN 19444/slapd Provides attachments at https://online-storage.yuuwaa.com/s/HkSdr/94bc6ba666 slapd_stuck.log - openldap logs populBlits.ldif - ldif file to get a blitz population (that's what we used but it works with any other data) testWriteTO.pl - a Perl script to loop thru client disconnection
moved from Incoming to Software Bugs
Unfortunately the reproduction case was not uploaded to the OpenLDAP FTP site and the referenced website no longer exists. Not sure we can do anything with this.