Issue 5407 - slapd hang in multimaster test
Summary: slapd hang in multimaster test
Status: VERIFIED FIXED
Alias: None
Product: OpenLDAP
Classification: Unclassified
Component: slapd (show other issues)
Version: unspecified
Hardware: All All
: --- normal
Target Milestone: ---
Assignee: OpenLDAP project
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-03-08 15:56 UTC by ando@openldap.org
Modified: 2014-08-01 21:04 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description ando@openldap.org 2008-03-08 15:56:13 UTC
Full_Name: Pierangelo Masarati
Version: HEAD
OS: GNU/Linux 2.6 (CentOS 5.1)
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (82.63.140.131)
Submitted by: ando


I have the slapd starting on port 9013 in test050 (with back-hdb, if it matters
at all) that's hanging.  The test, so far, got to

>>>>> Starting test050-syncrepl-multimaster ...
running defines.sh
Initializing server configurations...
Starting producer slapd on TCP/IP port 9011...
Using ldapsearch to check that producer slapd is running...
Inserting syncprov overlay on producer...
Starting consumer slapd on TCP/IP port 9012...
Using ldapsearch to check that consumer slapd is running...
Configuring syncrepl on consumer...
Starting consumer2 slapd on TCP/IP port 9013...
Using ldapsearch to check that consumer2 slapd is running...
Configuring syncrepl on consumer2...
Adding schema and databases on producer...
Using ldapadd to populate producer...
Waiting 20 seconds for syncrepl to receive changes...
Using ldapsearch to check that syncrepl received database changes...
Using ldapsearch to check that syncrepl received database changes on
consumer2...

I could attach the process with gdb; it yields:

thread apply all bt

Thread 8 (Thread 1081371536 (LWP 19254)):
#0  0x40000402 in __kernel_vsyscall ()
#1  0x00d5b256 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x0821197c in ldap_pvt_thread_cond_wait (cond=0x94cc1ec, mutex=0x94cc1d4)
    at thr_posix.c:277
#3  0x08210598 in ldap_pvt_thread_pool_destroy (tpool=0x8345318, run_pending=1)
    at tpool.c:570
#4  0x08081b3e in slapd_daemon_task (ptr=0x0) at daemon.c:2578
#5  0x00d5745b in start_thread () from /lib/libpthread.so.0
#6  0x001cf24e in clone () from /lib/libc.so.6

Thread 7 (Thread 1085569936 (LWP 19257)):
#0  0x40000402 in __kernel_vsyscall ()
#1  0x00d5b256 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x0821197c in ldap_pvt_thread_cond_wait (cond=0x94cc1ec, mutex=0x94cc1d4)
    at thr_posix.c:277
#3  0x082107fe in ldap_int_thread_pool_wrapper (xpool=0x94cc1d0) at tpool.c:654
#4  0x00d5745b in start_thread () from /lib/libpthread.so.0
#5  0x001cf24e in clone () from /lib/libc.so.6

Thread 6 (Thread 1089768336 (LWP 19258)):
#0  0x40000402 in __kernel_vsyscall ()
#1  0x00d5b256 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x0821197c in ldap_pvt_thread_cond_wait (cond=0x94cc21c, mutex=0x94cc1d4)
    at thr_posix.c:277
#3  0x08210bb4 in ldap_pvt_thread_pool_pause (tpool=0x8345318) at tpool.c:736
#4  0x08072ddb in config_back_add (op=0x40f47db4, rs=0x40f47a44)
    at bconfig.c:4548
#5  0x080fe265 in overlay_op_walk (op=0x40f47db4, rs=0x40f47a44, which=op_add, 
    oi=0x9517a40, on=0x0) at backover.c:653
#6  0x080fe41a in over_op_func (op=0x40f47db4, rs=0x40f47a44, which=op_add)
    at backover.c:705
#7  0x080fe526 in over_op_add (op=0x40f47db4, rs=0x40f47a44) at backover.c:751
#8  0x080f3700 in syncrepl_entry (si=0x951af10, op=0x40f47db4, 
    entry=0x94f5db4, modlist=0x40f47bcc, syncstate=1, syncUUID=0x40f47c24, 
    syncCSN=0x95191f0) at syncrepl.c:2052
#9  0x080ef610 in do_syncrep2 (op=0x40f47db4, si=0x951af10) at syncrepl.c:844
#10 0x080f0c40 in do_syncrepl (ctx=0x40f48210, arg=0x951b170)
    at syncrepl.c:1233
#11 0x08210871 in ldap_int_thread_pool_wrapper (xpool=0x94cc1d0) at tpool.c:663
#12 0x00d5745b in start_thread () from /lib/libpthread.so.0
#13 0x001cf24e in clone () from /lib/libc.so.6

Thread 5 (Thread 1095019408 (LWP 19260)):
#0  0x40000402 in __kernel_vsyscall ()
#1  0x001b3f7c in sched_yield () from /lib/libc.so.6
#2  0x0821190e in ldap_pvt_thread_yield () at thr_posix.c:232
#3  0x081f04a9 in syncprov_op_mod (op=0x41449d64, rs=0x414499f4)
    at syncprov.c:1783
#4  0x080fe1e5 in overlay_op_walk (op=0x41449d64, rs=0x414499f4, which=op_add, 
    oi=0x9517a40, on=0x9519630) at backover.c:643
#5  0x080fe41a in over_op_func (op=0x41449d64, rs=0x414499f4, which=op_add)
    at backover.c:705
#6  0x080fe526 in over_op_add (op=0x41449d64, rs=0x414499f4) at backover.c:751
#7  0x080f3700 in syncrepl_entry (si=0x9518440, op=0x41449d64, 
    entry=0x94f5e54, modlist=0x41449b7c, syncstate=1, syncUUID=0x41449bd4, 
    syncCSN=0x0) at syncrepl.c:2052
#8  0x080ef610 in do_syncrep2 (op=0x41449d64, si=0x9518440) at syncrepl.c:844
#9  0x080f0c40 in do_syncrepl (ctx=0x4144a210, arg=0x9516a48)
    at syncrepl.c:1233
#10 0x08084f67 in connection_read_thread (ctx=0x4144a210, argv=0x9)
    at connection.c:1213
#11 0x08210871 in ldap_int_thread_pool_wrapper (xpool=0x94cc1d0) at tpool.c:663
#12 0x00d5745b in start_thread () from /lib/libpthread.so.0
#13 0x001cf24e in clone () from /lib/libc.so.6

Thread 4 (Thread 1099217808 (LWP 19261)):
#0  0x40000402 in __kernel_vsyscall ()
#1  0x00d5b256 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x0821197c in ldap_pvt_thread_cond_wait (cond=0x94cc1ec, mutex=0x94cc1d4)
    at thr_posix.c:277
#3  0x082107fe in ldap_int_thread_pool_wrapper (xpool=0x94cc1d0) at tpool.c:654
#4  0x00d5745b in start_thread () from /lib/libpthread.so.0
#5  0x001cf24e in clone () from /lib/libc.so.6

Thread 3 (Thread 1106574224 (LWP 19264)):
#0  0x40000402 in __kernel_vsyscall ()
#1  0x00d5b256 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x0821197c in ldap_pvt_thread_cond_wait (cond=0x94cc1ec, mutex=0x94cc1d4)
    at thr_posix.c:277
#3  0x0821073a in ldap_int_thread_pool_wrapper (xpool=0x94cc1d0) at tpool.c:615
#4  0x00d5745b in start_thread () from /lib/libpthread.so.0
#5  0x001cf24e in clone () from /lib/libc.so.6

Thread 2 (Thread 1110772624 (LWP 19266)):
#0  0x40000402 in __kernel_vsyscall ()
#1  0x00d5b256 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0x0821197c in ldap_pvt_thread_cond_wait (cond=0x94cc1ec, mutex=0x94cc1d4)
    at thr_posix.c:277
#3  0x082107fe in ldap_int_thread_pool_wrapper (xpool=0x94cc1d0) at tpool.c:654
#4  0x00d5745b in start_thread () from /lib/libpthread.so.0
#5  0x001cf24e in clone () from /lib/libc.so.6

Thread 1 (Thread 1074717776 (LWP 19253)):
#0  0x40000402 in __kernel_vsyscall ()
#1  0x00d58557 in pthread_join () from /lib/libpthread.so.0
#2  0x082118e7 in ldap_pvt_thread_join (thread=1081371536, thread_return=0x0)
    at thr_posix.c:197
#3  0x08081c36 in slapd_daemon () at daemon.c:2644
#4  0x0806708c in main (argc=8, argv=0xbfe2cd04) at main.c:946
#0  0x40000402 in __kernel_vsyscall ()

This was after the

	syncprov.c  1.215 -> 1.216

commit for ITS#5405, which required some cleanup to compile.  I'm posting this
as a separate ITS just to keep track, but I need to shut the hanging process
down.  If this is related to ITS#5405, it can be closed.

p.

Comment 1 Howard Chu 2008-03-08 21:37:58 UTC
ando@sys-net.it wrote:
> This was after the
>
> 	syncprov.c  1.215 ->  1.216
>
> commit for ITS#5405, which required some cleanup to compile.  I'm posting this
> as a separate ITS just to keep track, but I need to shut the hanging process
> down.  If this is related to ITS#5405, it can be closed.

It was unrelated. It seems to be similar to this
http://www.openldap.org/lists/openldap-devel/200802/msg00093.html

Should be OK with HEAD now, please test.

-- 
   -- Howard Chu
   Chief Architect, Symas Corp.  http://www.symas.com
   Director, Highland Sun        http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP     http://www.openldap.org/project/

Comment 2 Howard Chu 2008-03-09 07:17:36 UTC
changed notes
changed state Open to Test
moved from Incoming to Software Bugs
Comment 3 Howard Chu 2008-03-21 01:44:31 UTC
changed notes
changed state Test to Release
Comment 4 Quanah Gibson-Mount 2008-05-13 03:18:20 UTC
changed state Release to Closed
Comment 5 OpenLDAP project 2014-08-01 21:04:13 UTC
fixed in HEAD/RE24