[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
(ITS#7235) test058 hanging in slapd shutdown
Full_Name: Hallvard B Furuseth
Version: RE24, 0e03b1369f4da5db9d0bdd2445b3a7ccc0c5638f
OS: Linux x86_64
URL:
Submission from: (NULL) (195.1.106.125)
Submitted by: hallvard
>>>>> Starting test058-syncrepl-asymmetric for hdb...
running defines.sh
Initializing master configurations...
Initializing search configurations...
Starting central master slapd on TCP/IP port 9011...
Using ldapsearch to check that central master slapd is running...
Starting site1 master slapd on TCP/IP port 9012...
Using ldapsearch to check that site1 master is running...
Starting site2 master slapd on TCP/IP port 9013...
Using ldapsearch to check that site2 master is running...
Starting central search slapd on TCP/IP port 9014...
Using ldapsearch to check that central search slapd is running...
Starting site1 search slapd on TCP/IP port 9015...
Using ldapsearch to check that site1 search slapd is running...
Starting site2 search slapd on TCP/IP port 9016...
Using ldapsearch to check that site2 search slapd is running...
Adding schema on ldap://localhost:9011/...
Adding schema on ldap://localhost:9012/...
Adding schema on ldap://localhost:9013/...
Adding schema on ldap://localhost:9014/...
Adding schema on ldap://localhost:9015/...
Adding schema on ldap://localhost:9016/...
Adding database config on central master...
Adding database config on site1 master...
Adding database config on site2 master...
Adding access rules on central master...
Adding access rules on site1 master...
Adding access rules on site2 master...
Adding database config on central search...
Adding database config on site1 search...
Adding database config on site2 search...
Populating central master...
Adding syncrepl on site1 master...
Adding syncrepl on site2 master...
Using ldapsearch to check that site1 master received changes...
Using ldapsearch to check that site2 master received changes...
Populating site1 master...
Populating site2 master...
Stopping site1 master...
Adding syncrepl on central master...
Using ldapsearch to check that central master received site2 entries...
Using ldapmodify to modify central master...
Restarting site1 master slapd on TCP/IP port 9012...
Using ldapsearch to check that site1 master is running...
Using ldapsearch to check that central master received site1 entries...
Using ldapsearch to check that site1 master received central master update...
Using ldapsearch to check that site2 master received central master update...
Adding syncrepl consumer on central search...
Adding syncrepl consumer on site1 search...
Adding syncrepl consumer on site2 search...
Using ldapsearch to check that central search received changes...
Using ldapsearch to check that site1 search received changes...
Using ldapsearch to check that site2 search received changes...
Checking contextCSN after initial replication...
Using ldapmodify to modify first backend on central master...
Using ldapsearch to check replication to central search...
Using ldapsearch to check replication to site1 search...
Using ldapsearch to check replication to site2 search...
Checking contextCSN after modify of first backend on central master...
Using ldapmodify to modify second backend on central master...
Using ldapsearch to check replication to site2 search...
Using ldapsearch to check no replication to site1 master...
Using ldapsearch to check no replication to central search...
Checking contextCSN after modify of second backend on central master...
Using ldapmodify to modify first backend on site1 master...
Using ldapsearch to check replication to site1 search...
Using ldapsearch to check replication to site2 master...
Using ldapsearch to check no replication to site2 search...
Using ldapsearch to check no replication to central search...
Checking contextCSN after modify of first backend on site1 master...
Using ldapmodify to modify second backend on site1 master...
Using ldapsearch to check replication to site1 search...
Using ldapsearch to check no replication to central master...
Checking contextCSN after modify of second backend on site1 master...
Using ldapmodify to modify first backend on site2 master...
Using ldapsearch to check replication to central master...
Using ldapsearch to check replication to site2 search...
Using ldapsearch to check no replication to site1 master...
Using ldapsearch to check no replication to central search...
Checking contextCSN after modify of first backend on site2 master...
Using ldapmodify to modify second backend on site2 master...
Using ldapsearch to check replication to site2 search...
Using ldapsearch to check no replication to central master...
Checking contextCSN after modify of second backend on site2 master...
Stopping central master and site2 servers to test start with emtpy db...
Starting site2 master slapd on TCP/IP port 9013...
Using ldapsearch to check that site2 master slapd is running...
Starting site2 search slapd on TCP/IP port 9016...
Using ldapsearch to check that site2 search slapd is running...
Starting central master slapd on TCP/IP port 9011...
Using ldapsearch to check that central master slapd is running...
Using ldapsearch to check that site2 master received base...
Using ldapsearch to check that site2 search received base...
Waiting 1 seconds for syncrepl to receive changes...
Checking contextCSN after site2 servers repopulated...
Adding syncrepl of second site1 master backend on central master...
Using ldapsearch to check that central master received second site1 backend...
Waiting 1 seconds for syncrepl to receive changes...
Waiting 2 seconds for syncrepl to receive changes...
Waiting 3 seconds for syncrepl to receive changes...
Waiting 4 seconds for syncrepl to receive changes...
Waiting 5 seconds for syncrepl to receive changes...
ERROR: Second site1 backend not replicated to central master
Restarting central master slapd on TCP/IP port 9011...
Using ldapsearch to check that central master slapd is running...
Waiting 1 seconds for slapd to start...
Using ldapsearch to check that central master received second site1 backend...
Using ldapsearch to check that central search received second site1 backend...
Waiting 1 seconds for syncrepl to receive changes...
Waiting 2 seconds for syncrepl to receive changes...
Waiting 3 seconds for syncrepl to receive changes...
Waiting 4 seconds for syncrepl to receive changes...
Waiting 5 seconds for syncrepl to receive changes...
ERROR: Second site1 backend not replicated to central search
Restarting central search slapd on TCP/IP port 9014...
Using ldapsearch to check that central search slapd is running...
Waiting 1 seconds for slapd to start...
Using ldapsearch to check that central search received second site1 backend...
Running 1 of 10 syncrepl race tests...
Stopping central master...
Using ldapadd to add entry on site1 master...
Starting central master again...
Using ldapsearch to check that central master received entry...
Using ldapsearch to check that central search received entry...
Stopping central master...
Using ldapdelete to delete entry on site1 master...
Starting central master again...
Using ldapsearch to check that entry was deleted on central master...
Using ldapsearch to check that entry was deleted on central search...
Running 2 of 10 syncrepl race tests...
Stopping central master...
(gdb) p slapd_shutdown
$14 = 1
(gdb) thread apply all backtrace full
3 threads:
Thread 1 in pthread_join () in main()
Thread 3 in pthread_cond_wait, ldap_pvt_thread_pool_destroy
Thread 2 (Thread 0x7f037bfff700 (LWP 9735)):
#0 0x000000321d20dff4 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x000000321d209328 in _L_lock_854 () from /lib64/libpthread.so.0
#2 0x000000321d2091f7 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x00007f038875055d in ldap_int_thread_mutex_lock (mutex=0x7f037410e208) at
thr_posix.c:296
No locals.
#4 0x00007f0388751a31 in ldap_pvt_thread_mutex_lock (mutex=0x7f037410e208) at
thr_debug.c:1000
rc = 1946161144
#5 0x00000000005ea3eb in syncprov_op_search (op=0x7f037410e460,
rs=0x7f037bffe970) at syncprov.c:2731
fand = 0x7f0374000fb0
fava = 0x7f0374000fd0
on = 0x22c1ae0
si = 0x22c1ce0
cb = 0x7f037bffd650
gotstate = 1
changed = 2
do_present = 1
sop = 0x7f037410e1b0
ss = 0x7f037bffe970
srs = 0x7f0374000c20
ctxcsn = 0x7f0374000ea8
i = 3
sids = 0x7f0374000f98
numcsns = 3
mincsn = {
bv_len = 40,
bv_val = 0x7f0374000d58 "20120405104436.067185Z#000000#002#000000"
}
maxcsn = {
bv_len = 40,
bv_val = 0x7f0374000d58 "20120405104436.067185Z#000000#002#000000"
}
minsid = 2
maxsid = 2
dirty = 0
#6 0x00000000004c8d9a in overlay_op_walk (op=0x7f037410e460, rs=0x7f037bffe970,
which=op_search, oi=0x22c0f90,
on=0x22c1ae0) at backover.c:661
func = 0x22c1b38
rc = 32768
#7 0x00000000004c9049 in over_op_func (op=0x7f037410e460, rs=0x7f037bffe970,
which=op_search) at backover.c:723
oi = 0x22c0f90
on = 0x22c1ae0
be = 0x22bf2b0
db = {
bd_info = 0x22c1ae0,
bd_self = 0x22bf2b0,
be_ctrls =
"\000\000\000\001\001\001\000\001\000\000\001\000\000\001\001\000\001\000\000\000\000\001\000\000\000\000\000\000\000\000\000\000\001",
be_flags = 2328,
be_restrictops = 0,
be_requires = 0,
be_ssf_set = {
sss_ssf = 0,
sss_transport = 0,
sss_tls = 0,
sss_sasl = 0,
sss_update_ssf = 0,
sss_update_transport = 0,
sss_update_tls = 0,
sss_update_sasl = 0,
sss_simple_bind = 0
},
be_suffix = 0x22b7980,
be_nsuffix = 0x22ba3e0,
be_schemadn = {
bv_len = 0,
bv_val = 0x0
},
be_schemandn = {
bv_len = 0,
bv_val = 0x0
},
be_rootdn = {
bv_len = 28,
bv_val = 0x22bf800 "cn=Manager,dc=example,dc=com"
},
be_rootndn = {
bv_len = 28,
bv_val = 0x22bbee0 "cn=manager,dc=example,dc=com"
},
be_rootpw = {
bv_len = 6,
bv_val = 0x22bdfd0 "secret"
},
be_max_deref_depth = 15,
be_def_limit = {
lms_t_soft = 3600,
lms_t_hard = 0,
lms_s_soft = 500,
lms_s_hard = 0,
lms_s_unchecked = -1,
lms_s_pr = 0,
lms_s_pr_hide = 0,
lms_s_pr_total = 0
},
be_limits = 0x0,
be_acl = 0x0,
be_dfltaccess = ACL_READ,
be_extra_anlist = 0x0,
be_update_ndn = {
bv_len = 0,
bv_val = 0x0
},
be_update_refs = 0x0,
be_pending_csn_list = 0x28e5d90,
be_pcl_mutex = {
__data = {
__lock = 0,
__count = 0,
__owner = 0,
__nusers = 0,
__kind = 2,
__spins = 0,
__list = {
__prev = 0x0,
__next = 0x0
}
},
__size = '\000' <repeats 16 times>, "\002", '\000' <repeats 22
times>,
__align = 0
},
be_syncinfo = 0x0,
be_pb = 0x0,
be_cf_ocs = 0x8b27c0,
be_private = 0x22bf460,
be_next = {
stqe_next = 0x0
}
}
cb = {
sc_next = 0x0,
sc_response = 0x4c7e0f <over_back_response>,
sc_cleanup = 0,
sc_private = 0x22c0f90
}
sc = 0x7f037bffe970
rc = 32768
__PRETTY_FUNCTION__ = "over_op_func"
#8 0x00000000004c9131 in over_op_search (op=0x7f037410e460, rs=0x7f037bffe970)
at backover.c:750
No locals.
#9 0x000000000044553a in fe_op_search (op=0x7f037410e460, rs=0x7f037bffe970) at
search.c:402
bd = 0x8ba380
#10 0x0000000000444d7e in do_search (op=0x7f037410e460, rs=0x7f037bffe970) at
search.c:247
base = {
bv_len = 17,
bv_val = 0x7f037410dfd7 "dc=example,dc=com"
}
siz = 2
off = 0
i = 2
#11 0x0000000000441d6c in connection_operation (ctx=0x7f037bffeaa0,
arg_v=0x7f037410e460) at connection.c:1150
rc = 80
cancel = 33
op = 0x7f037410e460
rs = {
sr_type = REP_RESULT,
sr_tag = 0,
sr_msgid = 0,
sr_err = 0,
sr_matched = 0x0,
sr_text = 0x0,
sr_ref = 0x0,
sr_ctrls = 0x0,
sr_un = {
sru_search = {
r_entry = 0x0,
r_attr_flags = 0,
r_operational_attrs = 0x0,
r_attrs = 0x0,
r_nentries = 0,
r_v2ref = 0x0
},
sru_sasl = {
r_sasldata = 0x0
},
sru_extended = {
r_rspoid = 0x0,
r_rspdata = 0x0
}
},
sr_flags = 0
}
tag = 99
opidx = SLAP_OP_SEARCH
conn = 0x22f2960
memctx = 0x7f03740008d0
memctx_null = 0x0
memsiz = 1048576
__PRETTY_FUNCTION__ = "connection_operation"
#12 0x00000000004422db in connection_read_thread (ctx=0x7f037bffeaa0, argv=0x21)
at connection.c:1286
rc = 0
cri = {
op = 0x7f037410e460,
func = 0,
arg = 0x0,
ctx = 0x7f037bffeaa0,
nullop = 0
}
s = 33
#13 0x00007f038874ef12 in ldap_int_thread_pool_wrapper (xpool=0x2178a40) at
tpool.c:688
pool = 0x2178a40
task = 0x7f037c000bd0
work_list = 0x2178ad8
ctx = {
ltu_id = 139652942001920,
ltu_key = {{
ltk_key = 0x4ab5f3,
ltk_data = 0x7f03740008d0,
ltk_free = 0x4ab418 <slap_sl_mem_destroy>
}, {
ltk_key = 0x28e5dc0,
ltk_data = 0x7f0374108fd0,
ltk_free = 0x550307 <bdb_reader_free>
}, {
ltk_key = 0x441924,
ltk_data = 0x7f037410e840,
ltk_free = 0x441768 <conn_counter_destroy>
}, {
ltk_key = 0x45a695,
ltk_data = 0x0,
ltk_free = 0x45a5e8 <slap_op_q_destroy>
}, {
ltk_key = 0x4f3113,
ltk_data = 0x7f036afff020,
ltk_free = 0x4f30f0 <search_stack_free>
}, {
ltk_key = 0x22392a0,
ltk_data = 0x7f0374109630,
ltk_free = 0x550307 <bdb_reader_free>
}, {
ltk_key = 0x2298a30,
ltk_data = 0x7f0374109b00,
ltk_free = 0x550307 <bdb_reader_free>
}, {
ltk_key = 0x2240020,
ltk_data = 0x7f03741098a0,
ltk_free = 0x550307 <bdb_reader_free>
}, {
ltk_key = 0x2244460,
ltk_data = 0x7f037410e1b0,
ltk_free = 0x550307 <bdb_reader_free>
}, {
ltk_key = 0x280ab40,
ltk_data = 0x7f037410e2f0,
ltk_free = 0x550307 <bdb_reader_free>
}, {
ltk_key = 0x0,
ltk_data = 0x0,
ltk_free = 0
} <repeats 22 times>}
}
kctx = 0x0
i = 32
keyslot = 947
hash = 2196128691
__PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper"