Full_Name: Paul B. Henson Version: 2.4.44 + ITS 8432 patch OS: Linux (Gentoo) URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (96.251.22.157) As discussed in: https://www.openldap.org/lists/openldap-technical/201702/msg00077.html I'm getting random segfaults with openldap 2.4.44 (with the ITS 8432 patch applied) with the following backtrace: (gdb) bt full #0 0x0000000000485c61 in modify_add_values (e=e@entry=0x7f50727fb480, mod=mod@entry=0x7f50600019a0, permissive=0, text=text@entry=0x7f50727fb9a0, textbuf=textbuf@entry=0x7f50727fb4d0 "modify/delete: eduPersonAffiliation: n o such attribute", textlen=textlen@entry=256) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4 .44/servers/slapd/mods.c:61 rc = <optimized out> op = 0x4fe045 "add" a = <optimized out> pmod = {sm_desc = <optimized out>, sm_values = 0x0, sm_nvalues = 0x0, sm _numvals = <optimized out>, sm_op = <optimized out>, sm_flags = <optimized out>, sm_type = {bv_len = <optimized out>, bv_val = <optimized out>}} __PRETTY_FUNCTION__ = "modify_add_values" #1 0x00007f526dfccd0c in mdb_modify_internal (op=op@entry=0x7f50727fc600, tid=tid@entry=0x1088a90, modlist=0x7f5060001960, e=e@entry=0x7f50727fb480, text=text@entry=0x7f50727f b9a0, textbuf=textbuf@entry=0x7f50727fb4d0 "modify/delete: eduPersonAffiliation: n o such attribute", textlen=256) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4 .44/servers/slapd/back-mdb/mod ify.c:137 rc = <optimized out> err = <optimized out> mod = 0x7f50600019a0 ml = 0x7f50600019a0 save_attrs = 0x7f5060004270 ap = 0x7f5272f53010 glue_attr_delete = <optimized out> got_delete = 0 __PRETTY_FUNCTION__ = "mdb_modify_internal" #2 0x00007f526dfce134 in mdb_modify (op=0x7f50727fc600, rs=0x7f50727fb980) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4 .44/servers/slapd/back-mdb/mod ify.c:624 mdb = 0x7f5272f53010 e = 0x7f5060004220 manageDSAit = <optimized out> textbuf = "modify/delete: eduPersonAffiliation: no such attribute\000\37 7\037\000\000\000\000\000\000\000@\003\315\000\000\000\000\000\200\271\177rP\177 \000\000\021\000\000\000\000\000\000\000.,\020`P\177\000\000 \360o\315\000\000\000\000\000\360\002\315\000\000\000\000\000\037\000\000\000\00 0\000\000\000\000\306\177rP\17 7\000\000\300\064\000`P\177\000\000\000\000\000\000\000\000\000\000\360o\315\000 \000\000\000\000 T\315\000\000 \000\000\000\000\306\177rP\177\000\000\270\232N\000\000\000\000\000 l\315\000\000\000\000\000\350\064\000`P\17 7\000\000\000\306\177rP\177\000\000\000\000\000\000\000\000\000\000"... txn = 0x1088a90 opinfo = {moi_oe = {oe_next = {sle_next = 0x7f50727fb930}, oe_key = 0x7f 5272f53010}, moi_txn = 0x1088a90, moi_ref = 1, moi_flag = 0 '\000'} moi = 0x7f50727fb430 dummy = {e_id = 66196, e_name = {bv_len = 40, bv_val = 0x7f50600041e8 "uid=aapriest,ou=user,dc=csupomona,dc=edu"}, e_nname = {bv_len = 40, bv_val = 0x7f5060004d18 "uid=aapriest,ou=user,dc=csupomona,dc=edu"}, e_attrs = 0xd202e8, e_ocflags = 256, e_bv = {bv_len = 0, bv_val = 0x0}, e_private = 0x7f50 60004220} preread_ctrl = 0x0 postread_ctrl = 0x0 ctrls = {0x0, 0x0, 0x0, 0x0, 0x0, 0x7f50727fb250} num_ctrls = 0 numads = 55 #3 0x000000000049c2ea in overlay_op_walk (op=op@entry=0x7f50727fc600, rs=0x7f50727fb980, which=op_modify, oi=0xcd6a40, on=<optimized out>) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4 .44/servers/slapd/backover.c:6 77 func = <optimized out> rc = <optimized out> #4 0x000000000049c441 in over_op_func (op=0x7f50727fc600, rs=<optimized out>, which=<optimized out>) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4 .44/servers/slapd/backover.c:7 30 oi = <optimized out> on = <optimized out> be = 0xcd0ac0 db = {bd_info = 0x7f526e1e6d40 <bi>, bd_self = 0xcd0ac0, be_ctrls = "\000\001\001\001\000\001\000\000\001\000\000\001\001\000\0 01\000\000\001", '\000' <repea ts 14 times>, "\001", be_flags = 563464, be_restrictops = 0, be_requires = 0, be_ssf_set = {sss_ssf = 0, sss_transport = 0, sss_tls = 0, sss_sasl = 0, sss_update_ssf = 0, ss s_update_transport = 0, sss_update_tls = 0, sss_update_sasl = 0, sss_simple_bind = 64}, be_s uffix = 0xcd0880, be_nsuffix = 0xca5c90, be_schemadn = {bv_len = 0, bv_val = 0x0}, be_sc hemandn = {bv_len = 0, bv_val = 0x0}, be_rootdn = {bv_len = 31, bv_val = 0xcd02f0 "cn=ldapr oot,dc=csupomona,dc=edu"}, be_rootndn = {bv_len = 31, bv_val = 0xcd0340 "cn=ldaproot,dc=csupomona ,dc=edu"}, be_rootpw = { bv_len = 38, bv_val = 0xca49a0 "{SSHA}XXXXXXXXXXXXXXXXXXXXXXX"}, be_max_deref_depth = 15, be_def_limit = {lms_t_soft = 3600, lms_t_hard = 0, lms_s_soft = -1, lms_s_hard = 0, lms_s_unchecked = -1, lms_s_pr = 0, lms_s_pr_hide = 0, lms_s_pr_total = 0}, be_limits = 0xca5840, be_acl = 0xcd1c80, be_dfltaccess = ACL_READ, be_ extra_anlist = 0x0, be_update_ndn = {bv_len = 0, bv_val = 0x0}, be_update_refs = 0x0, be_p ending_csn_list = 0xf85dc0, be_pcl_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nuse rs = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}} , __size = '\000' <repeats 39 times>, __align = 0}, be_syncinfo = 0xcd 75c0, be_pb = 0x0, be_cf_ocs = 0x7f526e1e68c0 <mdbocs>, be_private = 0x7f5272f53010, be_n ext = {stqe_next = 0x0}} cb = {sc_next = 0x7f50727fb950, sc_response = 0x49b650 <over_back_respon se>, sc_cleanup = 0x0, sc_writewait = 0x0, sc_private = 0xcd6a40} sc = <optimized out> rc = 32768 __PRETTY_FUNCTION__ = "over_op_func" #5 0x000000000048d938 in syncrepl_message_to_op (si=si@entry=0xcd8520, op=op@entry=0x7f50727fc600, msg=0x7f5060102940) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4 .44/servers/slapd/syncrepl.c:2 394 oes = {oe = {oe_next = {sle_next = 0x0}, oe_key = 0x48cd60 <syncrepl_mes sage_to_op>}, oe_si = 0xcd8520} ber = 0x7f50604b5390 modlist = 0x7f5060496f60 ls = 0x778940 <accesslog_sc> rs = {sr_type = REP_RESULT, sr_tag = 0, sr_msgid = 0, sr_err = 0, sr_mat ched = 0x0, sr_text = 0x7f50727fb4d0 "modify/delete: eduPersonAffiliation: no such attribute", sr_ref = 0x0, sr_ctrls = 0x0, sr_un = {sru_search = {r_entry = 0x0, r_attr_flags = 0 , r_operational_attrs = 0x0, r_attrs = 0x0, r_nentries = 0, r_v2ref = 0x0}, sru_sasl = {r_sasld ata = 0x0}, sru_extended = { r_rspoid = 0x0, r_rspdata = 0x0}}, sr_flags = 0} cb = {sc_next = 0x7f5060003490, sc_response = 0x48c8d0 <null_callback>, sc_cleanup = 0x0, sc_writewait = 0x0, sc_private = 0x0} text = 0x0 txtbuf = "@\277\177rP\177\000\000\020\274\177rP\177\000\000X\371P\000\00 0\000\000\000\266P\246rR\177\0 00\000@\277\177rP\177\000\000X\371P\000\000\000\000\000\240\272\177rP\177\000\00 0\060\275\246rR\177\000\000X\3 71P\000\000\000\000\000@^\246rR\177\000\000\000\000\000\000\000\000\000\000\002\ 000\000\000P\177\000\000\020\0 00\000\000\000\000\000\000\260\272\177rP\177\000\000\377\377\377\377\377\377\377 \377&\000\000\017\000\000\000\ 000\020\t\020\230P\177\000\000x\215I`P\177\000\000\177\215I`P\177\000\000\000\27 5\177rP\177\000\000p\234\305\0 00\000\000\000\000@\275\177rP\177\000\000 \205\315\000\000\000\000\000\313\343I\000\000\000\000\000\020", '\00 0' <repeats 23 times>... bdn = {bv_len = 40, bv_val = 0x7f50600013ac "uid=aapriest,ou=user,dc=csu pomona,dc=edu"} dn = {bv_len = 40, bv_val = 0x7f5060001c48 "\360!"} ndn = {bv_len = 40, bv_val = 0x7f5060001cd8 "\340\034"} bv = {bv_len = 0, bv_val = 0x0} bv2 = {bv_len = 0, bv_val = 0x0} bvals = 0x7f506049fb80 rdn = {bv_len = 0, bv_val = 0x0} sup = {bv_len = 0, bv_val = 0x0} prdn = {bv_len = 0, bv_val = 0x0} nrdn = {bv_len = 0, bv_val = 0x0} psup = {bv_len = 0, bv_val = 0x0} nsup = {bv_len = 0, bv_val = 0x0} rc = 0 deleteOldRdn = 0 freeReqDn = 1 do_graduate = 1 #6 0x00000000004915d7 in do_syncrep2 (op=op@entry=0x7f50727fc600, si=si@entry=0xcd8520) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4 .44/servers/slapd/syncrepl.c:1 013 match = <optimized out> syncUUID = {{bv_len = 16, bv_val = 0x7f506049fb47 "~\373\337l\214\335IȻ\346\230\350\r\n", <inc omplete sequence \316>}, { bv_len = 0, bv_val = 0x0}} cookie = {bv_len = 15, bv_val = 0x7f506049fb59 "rid=003,sid=003"} rctrls = 0x7f506049abe0 rctrlp = <optimized out> bdn = {bv_len = 44, bv_val = 0x7f506000135a "reqStart=20161225113103.000 003Z,cn=accesslog"} si_tag = <optimized out> entry = <optimized out> punlock = -1 syncstate = 1 retdata = 0x0 retoid = 0x78 <error: Cannot access memory at address 0x78> syncUUIDs = 0x0 len = 15 berbuf = { buffer = "\002\000\001", '\000' <repeats 29 times>, "@\373I`P\177\000\ 000h\373I`P\177\000\000h\373I` P\177", '\000' <repeats 26 times>, "п\177rP\177\000\000\000\000\000\000R\177\000\000\001\000\000\000\000\000\0 00\000dZ\023qR\177\000\000Dh,\200\207\001\305\345\000\000\000\000\001\000\000\00 0\200\255J`P\177\000\000`\305\ 177rP\177\000\000\257X\244qR\177\000\000\031p-rR\177\000\000\000\000\000\000\000 \000\000\000\b\246\246rR\177\0 00\000\060", '\000' <repeats 15 times>, "\001\000\000\000\000\000\000\000d"..., ialign = 65538, lalign = 65538, falign = 9.18382988e-41, dalign = 3.2380074297143616e- 319, palign = 0x10002 <error: Cannot access memory at address 0x10002>} ber = 0x7f50727fbf40 msg = 0x7f5060102940 syncCookie = {ctxcsn = 0x0, sids = 0x0, numcsns = 0, rid = 3, octet_str = {bv_len = 15, bv_val = 0x7f5060498d70 "rid=003,sid=003"}, sid = 3, sc_next = {stqe _next = 0x0}} syncCookie_req = {ctxcsn = 0x7f50604a6c10, sids = 0x7f50604996c0, numcsn s = 4, rid = 3, octet_str = { bv_len = 183, bv_val = 0x7f50604984e0 "rid=003,sid=000,csn=20161225102351.582901Z# 000000#000#000000;201612130230 51.387118Z#000000#001#000000;20160721062752.989665Z#000000#002#000000;2016072106 2824.859874Z#000000#003#000000 "}, sid = 0, sc_next = {stqe_next = 0x0}} rc = 0 err = 0 modlist = 0x0 m = 11 tout_p = 0x7f50727fbc00 tout = {tv_sec = 0, tv_usec = 0} refreshDeletes = 0 empty = "empty" #7 0x0000000000495793 in do_syncrepl (ctx=ctx@entry=0x7f50727fcb90, arg=arg@entry=0xcd8a00) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4 .44/servers/slapd/syncrepl.c:1 564 rtask = 0xcd8a00 si = 0xcd8520 conn = {c_struct_state = SLAP_C_UNINITIALIZED, c_conn_state = SLAP_C_INV ALID, c_conn_idx = -1, c_sd = 0, c_close_reason = 0x0, c_mutex = {__data = {__lock = 0, __cou nt = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__ prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, c_sb = 0x0, c_star ttime = 0, c_activitytime = 0, c_connid = 18446744073709551615, c_peer_domain = { bv_len = 0, bv_val = 0x503e42 ""}, c_peer_name = {bv_len = 0, bv_val = 0x503e42 ""}, c_listener = 0x506e00 <dummy_list>, c_sasl_bind_mech = {bv_len = 0, bv _val = 0x0}, c_sasl_dn = { bv_len = 0, bv_val = 0x0}, c_sasl_authz_dn = {bv_len = 0, bv_val = 0 x0}, c_authz_backend = 0x0, c_authz_cookie = 0x0, c_authz = {sai_method = 0, sai_mech = {bv_len = 0, bv_val = 0x0}, sai_dn = { bv_len = 0, bv_val = 0x0}, sai_ndn = {bv_len = 0, bv_val = 0x0}, s ai_ssf = 0, sai_transport_ssf = 0, sai_tls_ssf = 0, sai_sasl_ssf = 0}, c_protoco l = 0, c_ops = { stqh_first = 0x0, stqh_last = 0x0}, c_pending_ops = {stqh_first = 0x 0, stqh_last = 0x0}, c_write1_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nu sers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}} , __size = '\000' <repeats 39 times>, __align = 0}, c_write1_cv = {__d ata = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, _ _mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}, c_write2_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nu sers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}} , __size = '\000' <repeats 39 times>, __align = 0}, c_write2_cv = {__d ata = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, _ _mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}, c_currentber = 0x0, c_writers = 0, c_writing = 0 '\000', c_sasl_bind_i n_progress = 0 '\000', c_writewaiter = 0 '\000', c_is_udp = 0 '\000', c_is_tls = 0 '\000', c_ needs_tls_accept = 0 '\000', c_sasl_layers = 0 '\000', c_sasl_done = 0 '\000', c_sasl_authctx = 0x0 , c_sasl_sockctx = 0x0, c_sasl_extra = 0x0, c_sasl_bindop = 0x0, c_pagedresults_state = {ps_be = 0x0, ps_size = 0, ps_count = 0, ps_cookie = 0, ps_cookieval = {bv_len = 0, bv_val = 0x 0}}, c_n_ops_received = 0, c_n_ops_executing = 0, c_n_ops_pending = 0, c_n_ops_completed = 0, c_n _get = 0, c_n_read = 0, c_n_write = 0, c_extensions = 0x0, c_clientfunc = 0x0, c_clientarg = 0 x0, c_send_ldap_result = 0x442d90 <slap_send_ldap_result>, c_send_search_entry = 0x4437d0 <slap_send_search_entry>, c_send_search_reference = 0x444d10 <slap_send_search_reference>, c_send_ldap_extended = 0x4434a0 <slap_send_ldap_extended>, c_send_ldap_intermediate = 0x443640 <slap_send_ldap_intermediate>} opbuf = {ob_op = {o_hdr = 0x7f50727fc770, o_tag = 102, o_time = 14826654 63, o_tincr = 13, o_bd = 0x7f50727fb690, o_req_dn = {bv_len = 40, bv_val = 0x7f506049f2c0 "uid=aapriest,ou=user,dc=csupomona,dc=edu" }, o_req_ndn = {bv_len = 40, bv_val = 0x7f506049f300 "uid=aapriest,ou=user,dc=csupomona,dc=edu" }, o_request = {oq_add = { rs_modlist = 0x7f5060001960, rs_e = 0x1}, oq_bind = {rb_method = 1610619232, rb_cred = { bv_len = 1, bv_val = 0x0}, rb_edn = {bv_len = 0, bv_val = 0x0} , rb_ssf = 0, rb_mech = { bv_len = 0, bv_val = 0x0}}, oq_compare = {rs_ava = 0x7f5060001 960}, oq_modify = { rs_mods = {rs_modlist = 0x7f5060001960, rs_no_opattrs = 1 '\001' }, rs_increment = 0}, oq_modrdn = {rs_mods = {rs_modlist = 0x7f5060001960, rs_no_opattrs = 1 '\001'}, rs_deleteoldrdn = 0, rs_newrdn = {bv_len = 0, bv_val = 0x0}, rs_ nnewrdn = {bv_len = 0, bv_val = 0x0}, rs_newSup = 0x0, rs_nnewSup = 0x0}, oq_search = {rs_scope = 1610619232, rs_deref = 32592, rs_slimit = 1, rs_tlimit = 0, rs_limit = 0x0, rs_attrsonly = 0, rs_attrs = 0x0, rs_filter = 0x0, rs_filterstr = {bv_len = 0, bv_ val = 0x0}}, oq_abandon = { rs_msgid = 1610619232}, oq_cancel = {rs_msgid = 1610619232}, oq_ extended = {rs_reqoid = { bv_len = 139983184730464, bv_val = 0x1 <error: Cannot access m emory at address 0x1>}, rs_flags = 0, rs_reqdata = 0x0}, oq_pwdexop = {rs_extended = {rs _reqoid = { bv_len = 139983184730464, bv_val = 0x1 <error: Cannot access memory at address 0x1>}, rs_flags = 0, rs_reqdata = 0x0}, rs_old = {bv_len = 0, bv_val = 0x0}, rs_new = { bv_len = 0, bv_val = 0x0}, rs_mods = 0x0, rs_modtail = 0x0}}, o_abandon = 0, o_cancel = 0, o_groups = 0x0, o_do_not_cache = 0 '\000', o_is_auth_check = 0 '\000 ', o_dont_replicate = 0 '\000', o_acl_priv = ACL_NONE, o_nocaching = 0 '\000', o_delete_glue_parent = 0 '\000', o_no_schema_check = 1 '\001', o_no_ subordinate_glue = 0 '\000', o_ctrlflag = '\000' <repeats 14 times>, "\002", '\000' <repeats 16 t imes>, o_controls = 0x7f50727fc8c0, o_authz = {sai_method = 0, sai_mech = { bv_len = 0, bv_val = 0x0}, sai_dn = {bv_len = 31, bv_val = 0xcd02f0 "cn=ldaproot,dc=csupomona ,dc=edu"}, sai_ndn = { sai_transport_ssf = 0, sai_tls_ssf = 0, sai_sasl_ssf = 0}, o_ber = 0x0, o_res_ber = 0x0, o_callback = 0x7f5060004198, o_ctrls = 0x0, o_csn = {bv_len = 40, bv_val = 0x7f5060102c10 "20161225113103.634923Z#000000#000#000000" }, o_private = 0x0, o_extra = {slh_first = 0x7f50727fb430}, o_next = {stqe_next = 0x0}}, ob_hdr = {oh_opid = 0, oh_connid = 3, oh_conn = 0x7f50727fc340, oh_msgid = 0, oh_protocol = 0, oh_tid = 139983495091968, oh_threadctx = 0x7f50727fcb90, oh_tmpmemct x = 0x7 oh_tmpmfuncs = 0x7787a0 <slap_sl_mfuncs>, oh_counters = 0x7ae000 <sl ap_counters>, oh_log_prefix = "conn=-1 op=0", '\000' <repeats 243 times>, oh_exten sions = 0x0}, ob_controls = { 0x0 <repeats 17 times>, 0x7f50727fbd00, 0x0 <repeats 14 times>}} op = 0x7f50727fc600 rc = 0 dostop = 0 s = 16 i = <optimized out> defer = 1 fail = 0 freeinfo = 0 be = 0xcd0ac0 #8 0x0000000000433037 in connection_read_thread (ctx=0x7f50727fcb90, argv=0x10) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4 .44/servers/slapd/connection.c :1296 rc = <optimized out> cri = {op = 0x0, func = 0x4954b0 <do_syncrepl>, arg = 0xcd8a00, ctx = <o ptimized out>, nullop = <optimized out>} s = <optimized out> #9 0x00007f5272c83a32 in ldap_int_thread_pool_wrapper (xpool=0xc63a40) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4 .44/libraries/libldap_r/tpool. c:696 pool = 0xc63a40 task = 0x7f50a8000a20 work_list = <optimized out> ctx = {ltu_id = 139983495091968, ltu_key = {{ltk_key = 0x430810 <conn_co unter_init>, ltk_data = 0x7f5060000d20, ltk_free = 0x4308f0 <conn_counter_destr oy>}, { ltk_key = 0x486ac0 <slap_sl_mem_init>, ltk_data = 0x7f5060000e30, ltk_free = 0x486990 <slap_sl_mem_destroy>}, {ltk_key = 0x445cd0 <s lap_op_free>, ltk_data = 0x7f50a0113e10, ltk_free = 0x445c30 <slap_op_q_destroy> }, {ltk_key = 0xf85de0, ltk_data = 0x7f5060102fd0, ltk_free = 0x7f526dfda950 <mdb_reader_f ree>}, { ltk_key = 0x7f526dfd07d0 <search_stack>, ltk_data = 0x7f5070ffc010 , ltk_free = 0x7f526dfd08e0 <search_stack_free>}, {ltk_key = 0x7f526 dfd0730 <scope_chunk_get>, ltk_data = 0x7f5060104e60, ltk_free = 0x7f526dfd08b0 <scope_chunk_ free>}, {ltk_key = 0xd7f560, ltk_data = 0x7f50604ab8f0, ltk_free = 0x7f526dfda950 <mdb_reader_f ree>}, {ltk_key = 0x0, ltk_data = 0x7f506c40fff0, ltk_free = 0x0}, {ltk_key = 0x0, ltk_da ta = 0x0, ltk_free = 0x0} <repeats 24 times>}} kctx = <optimized out> keyslot = <optimized out> hash = <optimized out> __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper"
The code making the update is in perl, using Net::LDAP, and looks like: $entry->replace(eduPersonAffiliation => \@eduPersonAffiliation); For the case of the crash, the array @eduPersonAffiliation is empty, effectively deleting the attribute. I'm not 100% sure what this generates on the wire, I'd have to debug it. I can say it's the same code that's been running for a decade or so with no issues. The schema that defines the attribute is available at: https://spaces.internet2.edu/display/macedir/OpenLDAP+eduPerson
Hi Paul, I've tried to reproduce the issue you're seeing, setting up deltasync replication and trying to send various combinations of modifies to the master. However, none of that has crashed either the server receiving the modification, nor the other servers it replicated to. No change either when the second server has to replicate directly off the accesslog database without the benefit of syncprov's sessionlog. Is that something that you can replicate reliably? Would you be able to share the configuration (especially overlays)? Based on the traces I've taken while testing, I haven't seen anything suspicious that the perl library would generate, but if you have the packets or slapd ber dumps, they might help confirm this for your case as well. Regards, Ondrej
> From: Ondřej Kuzník > Sent: Thursday, March 9, 2017 5:32 AM > > Is that something that you can replicate reliably? Would you be able to > share the configuration (especially overlays)? Unfortunately it is not deterministically reproducible, it just randomly happens occasionally. I haven't noticed anything in particular that seems to be a pattern or in common on each occurrence. I will shoot you a copy of my configuration directly. > Based on the traces I've taken while testing, I haven't seen anything > suspicious that the perl library would generate, but if you have the > packets or slapd ber dumps, they might help confirm this for your case > as well. I haven't been able to get a packet capture or leave debugging on long enough to catch it since the crash is so sporadic and it only occurs on a production system at widely interspersed intervals :(. Thanks for looking at it.
moved from Incoming to Software Bugs
Hi Paul, so far I've been unable to reproduce the issue, but going through the traceback again, there is something odd. In frame #5, syncrepl.c:2394 reads "rc = op->o_bd->be_add( op, &rs );" as per re24[0], but in your traceback, over_op_func calls overlay_op_walk with "which=op_modify" (see frame #3) which doesn't really make sense to me. [0]. http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=blob;f=servers/slapd/syncrepl.c;hb=refs/heads/OPENLDAP_REL_ENG_2_4#l2394 -- Ondřej Kuzník Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP
> From: Ondřej Kuzník > Sent: Thursday, March 23, 2017 5:29 AM > > so far I've been unable to reproduce the issue, but going through the > traceback again, there is something odd. In frame #5, syncrepl.c:2394 > reads "rc = op->o_bd->be_add( op, &rs );" as per re24[0], but in your > traceback, over_op_func calls overlay_op_walk with "which=op_modify" > (see frame #3) which doesn't really make sense to me. As I mentioned in the report, this is 2.4.44 with the ITS 8432 patch applied to prevent the infinite replication issue from occurring in our production environment. With that patch applied, line 2394 of syncrepl.c is: rc = op->o_bd->be_modify( op, &rs ); Sorry for any confusion that caused :(.
On Thu, Mar 23, 2017 at 05:51:46PM -0700, Paul B. Henson wrote: >> From: Ondřej Kuzník >> Sent: Thursday, March 23, 2017 5:29 AM >> >> so far I've been unable to reproduce the issue, but going through the >> traceback again, there is something odd. In frame #5, syncrepl.c:2394 >> reads "rc = op->o_bd->be_add( op, &rs );" as per re24[0], but in your >> traceback, over_op_func calls overlay_op_walk with "which=op_modify" >> (see frame #3) which doesn't really make sense to me. > > As I mentioned in the report, this is 2.4.44 with the ITS 8432 patch > applied to prevent the infinite replication issue from occurring in > our production environment. With that patch applied, line 2394 of > syncrepl.c is: > > rc = op->o_bd->be_modify( op, &rs ); > > Sorry for any confusion that caused :(. Looks like your patch differs from what was applied to re24 at least on that line (which calls be_add instead). Do you know why that is? Could you try with the current re24 branch (OPENLDAP_REL_ENG_2_4) instead? Thanks -- Ondřej Kuzník Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP
> From: Ondřej Kuzník > Sent: Friday, March 24, 2017 2:28 AM > > Looks like your patch differs from what was applied to re24 at least on > that line (which calls be_add instead). Do you know why that is? Could > you try with the current re24 branch (OPENLDAP_REL_ENG_2_4) instead? Ah, more confusion, sorry; I confirmed with Quanah the ITS 8432 patch I applied is identical to the one applied upstream, however, the line I referred to is in the patch applied to 2.4.44 release code, whereas I see you are referring to the current head of the 2.4 branch, which I'm sure has had numerous other changes applied to it that have probably shuffled things around a bit. If I could reliably reproduce this issue :(, I would definitely test it on our dev systems with whatever code you requested. Unfortunately, it only randomly occurs every 2-4 weeks in production, and I don't think I could get management approval to run the unreleased development branch of code in production for that long to see what happens. The 2.4.45 testing call came out last month, is there a timeline yet on getting that out? I should be able to push that into production fairly soon after it is released and see what happens there. Thanks much…
--On Friday, March 24, 2017 8:07 PM +0000 henson@acm.org wrote: >> From: Ond=C5=99ej Kuzn=C3=ADk >> Sent: Friday, March 24, 2017 2:28 AM >> =20 >> Looks like your patch differs from what was applied to re24 at least = > on >> that line (which calls be_add instead). Do you know why that is? Could >> you try with the current re24 branch (OPENLDAP_REL_ENG_2_4) instead? > > Ah, more confusion, sorry; I confirmed with Quanah the ITS 8432 patch I = > applied is identical to the one applied upstream, however, the line I = > referred to is in the patch applied to 2.4.44 release code, whereas I = > see you are referring to the current head of the 2.4 branch, which I'm = > sure has had numerous other changes applied to it that have probably = > shuffled things around a bit. > > If I could reliably reproduce this issue :(, I would definitely test it = > on our dev systems with whatever code you requested. Unfortunately, it = > only randomly occurs every 2-4 weeks in production, and I don't think I = > could get management approval to run the unreleased development branch = > of code in production for that long to see what happens. The 2.4.45 = > testing call came out last month, is there a timeline yet on getting = > that out? I should be able to push that into production fairly soon = > after it is released and see what happens there. There's too many outstanding issues to release 2.4.45, including this one. The remaining changes to syncrepl.c between 2.4.44 and current RE25 are: <http://www.openldap.org/its/index.cgi/?findid=8435> <http://www.openldap.org/its/index.cgi/?findid=8413> I would consider at least the patch for ITS#8413 as mandatory as well. Since ITS#8435 has been around since 2.4.40, it's unlikely to be related. <https://github.com/Zimbra/packages/blob/master/thirdparty/openldap/patches/ITS8413.patch> is an easy spot to grab it from. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>
On Fri, Mar 24, 2017 at 12:07:32PM -0700, Paul B. Henson wrote: > Ah, more confusion, sorry; I confirmed with Quanah the ITS 8432 patch > I applied is identical to the one applied upstream, however, the line > I referred to is in the patch applied to 2.4.44 release code, whereas > I see you are referring to the current head of the 2.4 branch, which > I'm sure has had numerous other changes applied to it that have > probably shuffled things around a bit. Hi Paul, yes it did, I should have checked for that and would have if the wrong line didn't line up so well. > If I could reliably reproduce this issue :(, I would definitely test > it on our dev systems with whatever code you requested. Unfortunately, > it only randomly occurs every 2-4 weeks in production, and I don't > think I could get management approval to run the unreleased > development branch of code in production for that long to see what > happens. The 2.4.45 testing call came out last month, is there a > timeline yet on getting that out? I should be able to push that into > production fairly soon after it is released and see what happens > there. I've had a look whether I could reproduce the issue somehow and there is a potential crasher if the accesslog entry contained "reqmod: eduPersonAffiliation:+". Can you confirm whether you have entries like this in your logdb? There shouldn't be a way to have this kind of entry generated that I could see but processing one can sometimes cause a crash like the one you've been seeing. The following patch should fix an issue where an invalid accesslog entry of this sort is encountered. It should help you if that is the issue. ftp://ftp.openldap.org/incoming/Ondrej-Kuznik-20170327-ITS8609-ignore-invalid-accesslog-ops.patch Regards, Ondrej
> From: Quanah Gibson-Mount > Sent: Monday, March 27, 2017 8:09 AM > > There's too many outstanding issues to release 2.4.45, including this one. Well, are those new issues that don't exist in 2.4.44, or simply existing bugs already in the wild that haven't been fixed? If the former, then certainly you don't want to release fresh bugs 8-/, but if the latter, it seems to me that pushing a release that improves the state of production by fixing some known bugs, even if other existing known bugs still remain to be quashed, is preferable than leaving a larger number of known bugs at large on production systems :(. It seems the circular replication bug is particularly bad, I'm surprised more people aren't having issues with it. > I would consider at least the patch for ITS#8413 as mandatory as well. Hmm, I haven't run into that one yet myself, but thanks for the pointer.
> From: Ondřej Kuzník > Sent: Monday, March 27, 2017 8:10 AM > > I've had a look whether I could reproduce the issue somehow and there is > a potential crasher if the accesslog entry contained "reqmod: > eduPersonAffiliation:+". Can you confirm whether you have entries like > this in your logdb? There aren't currently any entries like that in the log, although the last crash was a week or two ago. I will check again right after the next crash. However, here is one of the log entries that appeared to correlate with what was being processed in the backtrace of one of the crashes: dn: reqStart=20170214120013.000001Z,cn=accesslog objectClass: auditModify structuralObjectClass: auditModify reqStart: 20170214120013.000001Z reqEnd: 20170214120013.000002Z reqType: modify reqSession: 37859 reqAuthzID: cn=idmgmt,ou=user,ou=service,dc=csupomona,dc=edu reqDN: uid=vntruong,ou=user,dc=csupomona,dc=edu reqResult: 0 reqMod: eduPersonAffiliation:= reqMod: eduPersonPrimaryAffiliation:= reqMod: csupomonaEduPersonAffiliation:= reqMod: entryCSN:= 20170214120013.628665Z#000000#000#000000 reqMod: modifiersName:= cn=idmgmt,ou=user,ou=service,dc=csupomona,dc=edu reqMod: modifyTimestamp:= 20170214120013Z reqEntryUUID: bd5ba51c-7a1b-4bdb-8bf3-fe90552f5909 entryCSN: 20170214120013.628665Z#000000#000#000000 entryUUID: 4737c48c-e46e-45a4-ba4b-2eb61540b27b creatorsName: cn=accesslog createTimestamp: 20170214120013Z modifiersName: cn=accesslog modifyTimestamp: 20170214120013Z And it does not appear to have that signature. Thanks.
--On Thursday, March 30, 2017 8:20 PM +0000 henson@acm.org wrote: >> From: Quanah Gibson-Mount >> Sent: Monday, March 27, 2017 8:09 AM >> >> There's too many outstanding issues to release 2.4.45, including this >> one. > > Well, are those new issues that don't exist in 2.4.44, or simply existing > bugs already in the wild that haven't been fixed? If the former, then > certainly you don't want to release fresh bugs 8-/, but if the latter, it > seems to me that pushing a release that improves the state of production > by fixing some known bugs, even if other existing known bugs still remain > to be quashed, is preferable than leaving a larger number of known bugs > at large on production systems :(. It seems the circular replication bug > is particularly bad, I'm surprised more people aren't having issues with > it. 2.4.43 introduced new bugs and those were continued in 2.4.44. I think it's important that the regressions are addressed before 2.4.45 is released. In addition, this very bug report indicates a regression between 2.4.44 and what's currently scheduled for 2.4.45, which is further reasoning not to release 2.4.45 at this time. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>
> From: Quanah Gibson-Mount > Sent: Thursday, March 30, 2017 3:14 PM > > released. In addition, this very bug report indicates a regression between > 2.4.44 and what's currently scheduled for 2.4.45, which is further > reasoning not to release 2.4.45 at this time. Hmm, I started seeing this problem when I upgraded from 2.4.43 to 2.4.44+ITS 8432 patch. I've never run stock 2.4.44; so I can't say whether this issue would occur without the ITS 8432 patch...
--On Friday, March 31, 2017 1:05 AM +0000 henson@acm.org wrote: >> From: Quanah Gibson-Mount >> Sent: Thursday, March 30, 2017 3:14 PM >> >> released. In addition, this very bug report indicates a regression >> between 2.4.44 and what's currently scheduled for 2.4.45, which is >> further reasoning not to release 2.4.45 at this time. > > Hmm, I started seeing this problem when I upgraded from 2.4.43 to > 2.4.44+ITS 8432 patch. I've never run stock 2.4.44; so I can't say > whether this issue would occur without the ITS 8432 patch... Either way, it's a significant regression that needs fixing before I'd be comforable with a 2.4.45 release. Interestingly, I never saw this problem @ Zimbra, even though it is constantly modifying data and has 2.4.44+ITS8432. ;) So I think there's something specific to your use case (the hardest kinds to track down!). --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>
> From: Quanah Gibson-Mount > Sent: Thursday, March 30, 2017 5:16 PM > > 2.4.44+ITS8432. ;) So I think there's something specific to your use case > (the hardest kinds to track down!). * Specific to my use case - check * Not reproducible - check * Only occurs sporadically at randomly dispersed intervals - check Perhaps we've hit a heisenbug ;). I was kind of hoping it would get fixed by accident by other updates in the next release :).
On Thu, Mar 30, 2017 at 12:30:09PM -0700, Paul B. Henson wrote: >> From: Ondřej Kuzník >> Sent: Monday, March 27, 2017 8:10 AM >> >> I've had a look whether I could reproduce the issue somehow and there is >> a potential crasher if the accesslog entry contained "reqmod: >> eduPersonAffiliation:+". Can you confirm whether you have entries like >> this in your logdb? > > There aren't currently any entries like that in the log, although the last > crash was a week or two ago. I will check again right after the next crash. > However, here is one of the log entries that appeared to correlate with what > was being processed in the backtrace of one of the crashes: > > [...] > > And it does not appear to have that signature. Hi Paul, "modify: replace" entries should result in modify_replace_values being called instead, where modify_add_values is called when the mod is LDAP_MOD_ADD (set up from reqMod values starting "attr:+"). Not sure if there is a way to reset the LDAPResult *msg in syncrepl_message_to_op to retrieve the entry from there but that would be the place I'd look when you get a another crash. Regards, -- Ondřej Kuzník Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP
So slapd crashed two days in a row, yesterday morning and this morning :(. Same backtrace. What's worse whereas unlike today and every time before it was relatively harmless, something *weird* happened yesterday. I did my usual of just restarting slapd on the master and letting the load balancer fail back over to it and having it catch back up on replication from the changes made to the secondary, which has always been fine before other than maybe having to tune up whatever change was in the middle of being committed when the primary actually died. But somehow yesterday things that happened like a week and a half ago just mysteriously disappeared 8-/. 10 groups that were created on 3/22 were just *gone* without a trace. I had to go dig up ldif backups from then and manually readd them back to the directory to clean stuff up. It looks like there's some group membership corruption too, I'm going to look at it in more detail tomorrow, it should be identical to our AD environment so I can compare and clean up against that. It doesn't look like any users fell into limbo, I diff'd the user DN's from the backup the day before against the one after and they're all still there modulo 5 that were deleted intentionally. I haven't been overly concerned about this issue, while it's been annoying, it hasn't had much of a production impact. But it just turned ugly :(, assuming this was caused by the crash, and I can't see how it wouldn't have been... <sigh>. Is there any logging or something I could turn on that would help for the next occurance but wouldn't be an excessive impact in terms of load or disk utilization on a production box? Thanks...
henson@acm.org wrote: > So slapd crashed two days in a row, yesterday morning and this morning > :(. Same backtrace. You should probably switch to a non-optimized build. The backtrace you posted initially had most parameters optimized out, which also makes things harder to trace. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
On Mon, Apr 03, 2017 at 02:44:48PM +0100, Howard Chu wrote: > You should probably switch to a non-optimized build. The backtrace you > posted initially had most parameters optimized out, which also makes > things harder to trace. Ok, I installed binaries built with -O0 on the master server that's been failing; I'll post a new backtrace the next time it crashes, thanks.
On Mon, Apr 03, 2017 at 02:44:48PM +0100, Howard Chu wrote: > You should probably switch to a non-optimized build. The backtrace you > posted initially had most parameters optimized out, which also makes > things harder to trace. Well, as luck (8-/) would have it, it crashed both yesterday and today on the non-optimized build. Here are the full backtraces from both cores, I hope they provide some clue as to what's going on. Let me know if you want me to do any more post mortem debugging on the cores. Thanks much. Core was generated by `/usr/lib64/openldap/slapd -u ldap -g ldap -h ldaps:// ldap://'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x00000000004b8dbb in modify_add_values (e=0x7fac037fca30, mod=0x7fabfc000a90, permissive=0, text=0x7fac037fd000, textbuf=0x7fac037fca80 "modify/delete: eduPersonAffiliation: no such attribute", textlen=256) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/servers/slapd/mods.c:61 61 if ( !BER_BVISNULL( &mod->sm_values[mod->sm_numvals] )) { [Current thread is 1 (Thread 0x7fac037fe700 (LWP 1292))] (gdb) bt full #0 0x00000000004b8dbb in modify_add_values (e=0x7fac037fca30, mod=0x7fabfc000a90, permissive=0, text=0x7fac037fd000, textbuf=0x7fac037fca80 "modify/delete: eduPersonAffiliation: no such attribute", textlen=256) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/servers/slapd/mods.c:61 rc = -67095872 op = 0x5866e0 "add" a = 0x7fac037fc8a0 pmod = {sm_desc = 0x16d53c0, sm_values = 0x0, sm_nvalues = 0x0, sm_numvals = 0, sm_op = 0, sm_flags = 0, sm_type = {bv_len = 34, bv_val = 0x7fabfc000a78 "uid=bran\220\n"}} __PRETTY_FUNCTION__ = "modify_add_values" #1 0x00007fadf301c774 in mdb_modify_internal (op=0x7fac037fd730, tid=0x1a993c0, modlist=0x7fabfc000a50, e=0x7fac037fca30, text=0x7fac037fd000, textbuf=0x7fac037fca80 "modify/delete: eduPersonAffiliation: no such attribute", textlen=256) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/servers/slapd/back-mdb/modify.c:137 rc = 32684 err = 0 mod = 0x7fabfc000a90 ml = 0x7fabfc000a90 save_attrs = 0x7fabfc0032c0 ap = 0x7fac00000000 glue_attr_delete = 0 got_delete = 0 __PRETTY_FUNCTION__ = "mdb_modify_internal" #2 0x00007fadf301e5f2 in mdb_modify (op=0x7fac037fd730, rs=0x7fac037fcfe0) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/servers/slapd/back-mdb/modify.c:624 mdb = 0x7fadf7dbb010 e = 0x7fabfc003270 manageDSAit = 2 00\000\340\317\177\003\254\177\000\000\060\327\177\003\254\177\000\000 \313\177\003\254\177\000\000\000\000\000\000\260\000\000\000 \313\177\003\254\177\000\000\324\024\214\367\000\000\000\000(&\000\374\253\177\000\000\060\327\177\003\254\177\000\000p-n\001\000\000\000\000P/n\001\000\000\000\000\320%\000\374\253\177\000\000\370%\000\374\253\177\000\000`\322\177\003\254\177\000\000"... textlen = 256 txn = 0x1a993c0 opinfo = {moi_oe = {oe_next = {sle_next = 0x7fac037fcf90}, oe_key = 0x7fadf7dbb010}, moi_txn = 0x1a993c0, moi_ref = 1, moi_flag = 0 '\000'} moi = 0x7fac037fc9e0 dummy = {e_id = 171976, e_name = {bv_len = 34, bv_val = 0x7fabfc003240 "uid=brandonl,ou=user,dc=cpp,dc=edu"}, e_nname = {bv_len = 34, bv_val = 0x7fabfc003d20 "uid=brandonl,ou=user,dc=cpp,dc=edu"}, e_attrs = 0x7fac2442b588, e_ocflags = 256, e_bv = {bv_len = 0, bv_val = 0x0}, e_private = 0x7fabfc003270} preread_ctrl = 0x0 postread_ctrl = 0x0 ctrls = {0x0, 0x0, 0x7fabfc0009d0, 0x50, 0x0, 0x7fac037fc800} num_ctrls = 0 numads = 57 #3 0x00000000004dbc55 in overlay_op_walk (op=0x7fac037fd730, rs=0x7fac037fcfe0, which=op_modify, oi=0x16e2b90, on=0x0) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/servers/slapd/backover.c:677 func = 0x7fadf323ddd8 <bi+88> rc = 32768 #4 0x00000000004dbea0 in over_op_func (op=0x7fac037fd730, rs=0x7fac037fcfe0, which=op_modify) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/servers/slapd/backover.c:730 oi = 0x16e2b90 on = 0x16e3d20 be = 0x16d4b60 db = {bd_info = 0x7fadf323dd80 <bi>, bd_self = 0x16d4b60, ts 14 times>, "\001", be_flags = 563464, be_restrictops = 0, be_requires = 0, be_ssf_set = {sss_ssf = 0, sss_transport = 0, sss_tls = 0, sss_sasl = 0, sss_update_ssf = 0, sss_update_transport = 0, sss_update_tls = 0, sss_update_sasl = 0, sss_simple_bind = 64}, be_suffix = 0x16d3bf0, be_nsuffix = 0x16d4270, be_schemadn = {bv_len = 0, bv_val = 0x0}, be_schemandn = {bv_len = 0, bv_val = 0x0}, be_rootdn = {bv_len = 25, bv_val = 0x16d4490 "cn=ldaproot,dc=cpp,dc=edu"}, be_rootndn = {bv_len = 25, bv_val = 0x16d44e0 "cn=ldaproot,dc=cpp,dc=edu"}, be_rootpw = { bv_len = 14, bv_val = 0x16a7a30 "XXXXXXXXXXX"}, be_max_deref_depth = 15, be_def_limit = { lms_t_soft = 3600, lms_t_hard = 0, lms_s_soft = -1, lms_s_hard = 0, lms_s_unchecked = -1, lms_s_pr = 0, lms_s_pr_hide = 0, lms_s_pr_total = 0}, be_limits = 0x16d3b20, be_acl = 0x16d5eb0, be_dfltaccess = ACL_READ, be_extra_anlist = 0x0, be_update_ndn = {bv_len = 0, bv_val = 0x0}, be_update_refs = 0x0, be_pending_csn_list = 0x19966a0, be_pcl_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = { __prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, be_syncinfo = 0x16e3970, be_pb = 0x0, be_cf_ocs = 0x7fadf323dac0 <mdbocs>, be_private = 0x7fadf7dbb010, be_next = {stqe_next = 0x0}} cb = {sc_next = 0x7fac037fcfb0, sc_response = 0x4daa2c <over_back_response>, sc_cleanup = 0x0, sc_writewait = 0x0, sc_private = 0x16e2b90} sc = 0x0 rc = 32768 __PRETTY_FUNCTION__ = "over_op_func" #5 0x00000000004dc003 in over_op_modify (op=0x7fac037fd730, rs=0x7fac037fcfe0) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/servers/slapd/backover.c:769 No locals. #6 0x00000000004c84b4 in syncrepl_message_to_op (si=0x16e4630, op=0x7fac037fd730, msg=0x7fabfc4a1560) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/servers/slapd/syncrepl.c:2394 oes = {oe = {oe_next = {sle_next = 0x0}, oe_key = 0x4c7516 <syncrepl_message_to_op>}, oe_si = 0x16e4630} ber = 0x7fabfc4a76a0 modlist = 0x7fabfc498e20 ls = 0x7dfc40 <accesslog_sc> rs = {sr_type = REP_RESULT, sr_tag = 0, sr_msgid = 0, sr_err = 0, sr_matched = 0x0, sr_text = 0x7fac037fca80 "modify/delete: eduPersonAffiliation: no such attribute", sr_ref = 0x0, sr_ctrls = 0x0, sr_un = {sru_search = {r_entry = 0x0, r_attr_flags = 0, r_operational_attrs = 0x0, r_attrs = 0x0, r_nentries = 0, r_v2ref = 0x0}, sru_sasl = {r_sasldata = 0x0}, sru_extended = { r_rspoid = 0x0, r_rspdata = 0x0}}, sr_flags = 0} cb = {sc_next = 0x7fabfc0025a0, sc_response = 0x4cfda6 <null_callback>, sc_cleanup = 0x0, sc_writewait = 0x0, sc_private = 0x0} text = 0xa <error: Cannot access memory at address 0xa> txtbuf = "\220\320\177\003\254\177\000\000\000\321\177\003\254\177\000\000\300\320\177\003\254\177\000\000\030\000\000\000\060\000\000\000p\321\177\003\254\177\000\000\260\320\177\003\254\177\000\000\000\000\000\000\003\000\000\000 \000\000\374\253\177\000\000\377\377\377\377\377\377\377\377\000\000\000\017\000\000\000\000\257\350\377+\254\177\000\000\000\000\000\000\000\000\000\000`\321\177\003\254\177\000\000\000\000\000\000\000\000\000\000\257\350\377+\254\177", '\000' <repeats 11 times>, "\347\177\003\254\177\000\000P\347M", '\000' <repeats 13 times>, "`\322\177\003\254\177\000\000OOq\374\253\177\000\000@Oq\374\253\177\000\000`\325e\001\000\000\000\000OOq\374\253\177\000\000HOq\374\253\177\000\000\000\000\000\000\000\000\000\000"... textlen = 256 bdn = {bv_len = 34, bv_val = 0x7fabfc492c3c "uid=brandonl,ou=user,dc=cpp,dc=edu"} dn = {bv_len = 34, bv_val = 0x7fabfc000d30 "\220\022"} ndn = {bv_len = 34, bv_val = 0x7fabfc000db0 "p\023"} bv = {bv_len = 0, bv_val = 0x0} bv2 = {bv_len = 0, bv_val = 0x0} bvals = 0x7fabfc41f360 rdn = {bv_len = 0, bv_val = 0x0} sup = {bv_len = 0, bv_val = 0x0} prdn = {bv_len = 0, bv_val = 0x0} nrdn = {bv_len = 0, bv_val = 0x0} psup = {bv_len = 0, bv_val = 0x0} nsup = {bv_len = 0, bv_val = 0x0} rc = 0 deleteOldRdn = 0 freeReqDn = 1 do_graduate = 1 #7 0x00000000004c31cd in do_syncrep2 (op=0x7fac037fd730, si=0x16e4630) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/servers/slapd/syncrepl.c:1013 match = 0 syncUUID = {{bv_len = 16, bv_val = 0x7fabfc4c8cd7 "7\245|\231:\352AM\246\305\066\037i&\201\232"}, { bv_len = 0, bv_val = 0x3000000020 <error: Cannot access memory at address 0x3000000020>}} cookie = {bv_len = 15, bv_val = 0x7fabfc4c8ce9 "rid=003,sid=003"} rctrls = 0x7fabfc49f8f0 rctrlp = 0x7fabfc492de0 bdn = {bv_len = 44, bv_val = 0x7fabfc492bea "reqStart=20170415103641.000016Z,cn=accesslog"} si_tag = 140376648999836 entry = 0x7fabfc49f8f8 punlock = -1 syncstate = 1 retdata = 0x570e74 retoid = 0x7fadf6ac95c0 "" syncUUIDs = 0x0 len = 15 berbuf = { buffer = "\002\000\001", '\000' <repeats 29 times>, "ЌL\374\253\177\000\000\370\214L\374\253\177\000\000\370\214L\374\253\177", '\000' <repeats 26 times>, "\020\000\000\000\254\177\000\000\060\327\177\003\254\177\000\000\200\323\177\003\254\177\000\000\260Э\367\255\177\000\000\030\n\000\374\001\000\000\000\000\061~\000\000\000\000\000\200\323\177\003\254\177\000\000\260Э\367\255\177\000\000H\327\177\003\254\177\000\000 \v\020\024\254\177\000\000\000\324\177\003\254\177\000\000\377J\260\367\255\177\000\000\300\323\177\003\001\000\000\000,\324\177\003\254\177\000\000\060\327\177\003\001\000\000\000\020\t\020"..., ialign = 65538, lalign = 65538, falign = 9.18382988e-41, dalign = 3.2380074297143616e-319, palign = 0x10002 <error: Cannot access memory at address 0x10002>} ber = 0x7fac037fd2e0 msg = 0x7fabfc4a1560 syncCookie = {ctxcsn = 0x0, sids = 0x0, numcsns = 0, rid = 3, octet_str = {bv_len = 15, bv_val = 0x7fabfc714f40 "rid=003,sid=003"}, sid = 3, sc_next = {stqe_next = 0x0}} syncCookie_req = {ctxcsn = 0x7fabfc40cfe0, sids = 0x7fabfc4b99f0, numcsns = 4, rid = 3, octet_str = { bv_len = 183, bv_val = 0x7fabfc41b880 "rid=003,sid=000,csn=20170415103641.410645Z#000000#000#000000;20151030223206.483980Z#000000#001#000000;20160115030222.053144Z#000000#002#000000;20170406033805.626505Z#000000#003#000000"}, sid = 0, sc_next = {stqe_next = 0x0}} rc = 0 err = 0 modlist = 0x0 m = 1426881987 tout_p = 0x7fac037fd210 tout = {tv_sec = 0, tv_usec = 0} refreshDeletes = 0 empty = "empty" __PRETTY_FUNCTION__ = "do_syncrep2" #8 0x00000000004c54c8 in do_syncrepl (ctx=0x7fac037fdbc0, arg=0x16e4b10) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/servers/slapd/syncrepl.c:1564 rtask = 0x16e4b10 si = 0x16e4630 conn = {c_struct_state = SLAP_C_UNINITIALIZED, c_conn_state = SLAP_C_INVALID, c_conn_idx = -1, c_sd = 0, c_close_reason = 0x0, c_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, c_sb = 0x0, c_starttime = 0, c_activitytime = 0, c_connid = 18446744073709551615, c_peer_domain = {bv_len = 0, bv_val = 0x568650 ""}, c_peer_name = {bv_len = 0, bv_val = 0x568650 ""}, c_listener = 0x570900 <dummy_list>, c_sasl_bind_mech = {bv_len = 0, bv_val = 0x0}, c_sasl_dn = { bv_len = 0, bv_val = 0x0}, c_sasl_authz_dn = {bv_len = 0, bv_val = 0x0}, c_authz_backend = 0x0, c_authz_cookie = 0x0, c_authz = {sai_method = 0, sai_mech = {bv_len = 0, bv_val = 0x0}, sai_dn = { bv_len = 0, bv_val = 0x0}, sai_ndn = {bv_len = 0, bv_val = 0x0}, sai_ssf = 0, sai_transport_ssf = 0, sai_tls_ssf = 0, sai_sasl_ssf = 0}, c_protocol = 0, c_ops = { stqh_first = 0x0, stqh_last = 0x0}, c_pending_ops = {stqh_first = 0x0, stqh_last = 0x0}, c_write1_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, c_write1_cv = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}, c_write2_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, c_write2_cv = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}, c_currentber = 0x0, c_writers = 0, c_writing = 0 '\000', c_sasl_bind_in_progress = 0 '\000', c_writewaiter = 0 '\000', c_is_udp = 0 '\000', c_is_tls = 0 '\000', c_needs_tls_accept = 0 '\000', c_sasl_layers = 0 '\000', c_sasl_done = 0 '\000', c_sasl_authctx = 0x0, c_sasl_sockctx = 0x0, c_sasl_extra = 0x0, c_sasl_bindop = 0x0, c_pagedresults_state = {ps_be = 0x0, ps_size = 0, ps_count = 0, ps_cookie = 0, ps_cookieval = {bv_len = 0, bv_val = 0x0}}, c_n_ops_received = 0, c_n_ops_executing = 0, c_n_ops_pending = 0, c_n_ops_completed = 0, c_n_get = 0, c_n_read = 0, c_n_write = 0, c_extensions = 0x0, c_clientfunc = 0x0, c_clientarg = 0x0, c_send_ldap_result = 0x45801c <slap_send_ldap_result>, c_send_search_entry = 0x458cf0 <slap_send_search_entry>, c_send_search_reference = 0x45b0d8 <slap_send_search_reference>, c_send_ldap_extended = 0x45886d <slap_send_ldap_extended>, c_send_ldap_intermediate = 0x458ade <slap_send_ldap_intermediate>} opbuf = {ob_op = {o_hdr = 0x7fac037fd8a0, o_tag = 102, o_time = 1492252601, o_tincr = 74, o_bd = 0x7fac037fcc90, o_req_dn = {bv_len = 34, bv_val = 0x7fabfc492e70 "uid=brandonl,ou=user,dc=cpp,dc=edu"}, o_req_ndn = {bv_len = 34, bv_val = 0x7fabfc492ea0 "uid=brandonl,ou=user,dc=cpp,dc=edu"}, o_request = {oq_add = { rs_modlist = 0x7fabfc000a50, rs_e = 0x1}, oq_bind = {rb_method = -67106224, rb_cred = { bv_len = 1, bv_val = 0x0}, rb_edn = {bv_len = 0, bv_val = 0x0}, rb_ssf = 0, rb_mech = { bv_len = 0, bv_val = 0x0}}, oq_compare = {rs_ava = 0x7fabfc000a50}, oq_modify = { rs_mods = {rs_modlist = 0x7fabfc000a50, rs_no_opattrs = 1 '\001'}, rs_increment = 0}, oq_modrdn = {rs_mods = {rs_modlist = 0x7fabfc000a50, rs_no_opattrs = 1 '\001'}, rs_deleteoldrdn = 0, rs_newrdn = {bv_len = 0, bv_val = 0x0}, rs_nnewrdn = {bv_len = 0, bv_val = 0x0}, rs_newSup = 0x0, rs_nnewSup = 0x0}, oq_search = {rs_scope = -67106224, rs_deref = 32683, rs_slimit = 1, rs_tlimit = 0, rs_limit = 0x0, rs_attrsonly = 0, rs_attrs = 0x0, rs_filter = 0x0, rs_filterstr = {bv_len = 0, bv_val = 0x0}}, oq_abandon = { rs_msgid = -67106224}, oq_cancel = {rs_msgid = -67106224}, oq_extended = {rs_reqoid = { bv_len = 140376643996240, bv_val = 0x1 <error: Cannot access memory at address 0x1>}, rs_flags = 0, rs_reqdata = 0x0}, oq_pwdexop = {rs_extended = {rs_reqoid = { bv_len = 140376643996240, bv_val = 0x1 <error: Cannot access memory at address 0x1>}, rs_flags = 0, rs_reqdata = 0x0}, rs_old = {bv_len = 0, bv_val = 0x0}, rs_new = { bv_len = 0, bv_val = 0x0}, rs_mods = 0x0, rs_modtail = 0x0}}, o_abandon = 0, o_cancel = 0, o_groups = 0x0, o_do_not_cache = 0 '\000', o_is_auth_check = 0 '\000', o_dont_replicate = 0 '\000', o_acl_priv = ACL_NONE, o_nocaching = 0 '\000', o_delete_glue_parent = 0 '\000', o_no_schema_check = 1 '\001', o_no_subordinate_glue = 0 '\000', o_ctrlflag = '\000' <repeats 14 times>, "\002", '\000' <repeats 16 times>, o_controls = 0x7fac037fd9f0, o_authz = {sai_method = 0, sai_mech = {bv_len = 0, bv_val = 0x0}, sai_dn = {bv_len = 25, bv_val = 0x16d4490 "cn=ldaproot,dc=cpp,dc=edu"}, sai_ndn = { bv_len = 25, bv_val = 0x16d44e0 "cn=ldaproot,dc=cpp,dc=edu"}, sai_ssf = 0, sai_transport_ssf = 0, sai_tls_ssf = 0, sai_sasl_ssf = 0}, o_ber = 0x0, o_res_ber = 0x0, o_callback = 0x7fabfc0025d0, o_ctrls = 0x0, o_csn = {bv_len = 40, bv_val = 0x7fabfc492ed0 "20170415103641.599920Z#000000#000#000000"}, o_private = 0x0, o_extra = {slh_first = 0x7fac037fc9e0}, o_next = {stqe_next = 0x0}}, ob_hdr = {oh_opid = 0, oh_connid = 3, oh_conn = 0x7fac037fd470, oh_msgid = 0, oh_protocol = 0, oh_tid = 140376769816320, oh_threadctx = 0x7fac037fdbc0, oh_tmpmemctx = 0x7fabfc0009d0, oh_tmpmfuncs = 0x7dfb80 <slap_sl_mfuncs>, oh_counters = 0x8156c0 <slap_counters>, oh_log_prefix = "conn=-1 op=0", '\000' <repeats 243 times>, oh_extensions = 0x0}, ob_controls = { 0x0 <repeats 17 times>, 0x7fac037fd260, 0x0 <repeats 14 times>}} op = 0x7fac037fd730 rc = 0 dostop = 0 s = 13 i = -141804259 defer = 1 fail = 0 freeinfo = 0 be = 0x16d4b60 #9 0x0000000000440411 in connection_read_thread (ctx=0x7fac037fdbc0, argv=0xd) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/servers/slapd/connection.c:1296 rc = 0 cri = {op = 0x0, func = 0x4c4ec6 <do_syncrepl>, arg = 0x16e4b10, ctx = 0x7fac037fdbc0, nullop = 0} s = 13 #10 0x00007fadf7adba4b in ldap_int_thread_pool_wrapper (xpool=0x1667330) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/libraries/libldap_r/tpool.c:696 pool = 0x1667330 task = 0x7fac0c339570 work_list = 0x16673c8 ctx = {ltu_id = 140376769816320, ltu_key = {{ltk_key = 0x43f808 <conn_counter_init>, ltk_data = 0x7fabfc0008c0, ltk_free = 0x43f65a <conn_counter_destroy>}, { ltk_key = 0x4ba0e3 <slap_sl_mem_init>, ltk_data = 0x7fabfc0009d0, ltk_free = 0x4b9f08 <slap_sl_mem_destroy>}, {ltk_key = 0x1996710, ltk_data = 0x7fabfc100a20, ltk_free = 0x7fadf303113d <mdb_reader_free>}, {ltk_key = 0x7fadf3025bce <search_stack>, ltk_data = 0x7fac00ffc010, ltk_free = 0x7fadf3025bab <search_stack_free>}, { ltk_key = 0x7fadf30224aa <scope_chunk_get>, ltk_data = 0x7fabfc1028b0, ltk_free = 0x7fadf3022462 <scope_chunk_free>}, {ltk_key = 0x45c638 <slap_op_free>, ltk_data = 0x7fabf456a930, ltk_free = 0x45c58b <slap_op_q_destroy>}, {ltk_key = 0x178fe70, ltk_data = 0x7fabfc49f9a0, ltk_free = 0x7fadf303113d <mdb_reader_free>}, {ltk_key = 0x0, ltk_data = 0x7fabf44f5db0, ltk_free = 0x0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0x0} <repeats 24 times>}} kctx = 0x0 i = 32 keyslot = 948 hash = 3735031732 __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper" #11 0x00007fadf7124494 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #12 0x00007fadf6817edd in clone () from /lib64/libc.so.6 No symbol table info available. Core was generated by `/usr/lib64/openldap/slapd -u ldap -g ldap -h ldaps:// ldap://'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x00000000004b8dbb in modify_add_values (e=0x7fa0b59f0a30, mod=0x7fa0a0000980, permissive=0, text=0x7fa0b59f1000, textbuf=0x7fa0b59f0a80 "modify/delete: eduPersonAffiliation: no such attribute", textlen=256) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/servers/slapd/mods.c:61 61 if ( !BER_BVISNULL( &mod->sm_values[mod->sm_numvals] )) { [Current thread is 1 (Thread 0x7fa0b59f2700 (LWP 10464))] (gdb) bt full #0 0x00000000004b8dbb in modify_add_values (e=0x7fa0b59f0a30, mod=0x7fa0a0000980, permissive=0, text=0x7fa0b59f1000, textbuf=0x7fa0b59f0a80 "modify/delete: eduPersonAffiliation: no such attribute", textlen=256) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/servers/slapd/mods.c:61 rc = -1610599912 op = 0x5866e0 "add" a = 0x7fa0b59f08a0 pmod = {sm_desc = 0x1e9a3c0, sm_values = 0x0, sm_nvalues = 0x0, sm_numvals = 0, sm_op = 0, sm_flags = 0, sm_type = {bv_len = 34, bv_val = 0x7fa0a0000968 "uid=aaal\200\t"}} __PRETTY_FUNCTION__ = "modify_add_values" #1 0x00007fa277718774 in mdb_modify_internal (op=0x7fa0b59f1730, tid=0x225e3c0, modlist=0x7fa0a0000940, e=0x7fa0b59f0a30, text=0x7fa0b59f1000, textbuf=0x7fa0b59f0a80 "modify/delete: eduPersonAffiliation: no such attribute", textlen=256) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/servers/slapd/back-mdb/modify.c:137 rc = 32672 err = 0 mod = 0x7fa0a0000980 ml = 0x7fa0a0000980 save_attrs = 0x7fa0a0003218 ap = 0x7fa000000000 glue_attr_delete = 0 got_delete = 0 __PRETTY_FUNCTION__ = "mdb_modify_internal" #2 0x00007fa27771a5f2 in mdb_modify (op=0x7fa0b59f1730, rs=0x7fa0b59f0fe0) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/servers/slapd/back-mdb/modify.c:624 mdb = 0x7fa27c4b7010 e = 0x7fa0a00031c8 manageDSAit = 2 \000\340\017\237\265\240\177\000\000\060\027\237\265\240\177\000\000 \v\237\265\240\177\000\000\000\000\000\000\260\000\000\000 \v\237\265\240\177\000\000\324\324\373{\000\000\000\000\030%\000\240\240\177\000\000\060\027\237\265\240\177\000\000p}\352\001\000\000\000\000P\177\352\001\000\000\000\000\300$\000\240\240\177\000\000\350$\000\240\240\177\000\000`\022\237\265\240\177\000\000"... textlen = 256 txn = 0x225e3c0 opinfo = {moi_oe = {oe_next = {sle_next = 0x7fa0b59f0f90}, oe_key = 0x7fa27c4b7010}, moi_txn = 0x225e3c0, moi_ref = 1, moi_flag = 0 '\000'} moi = 0x7fa0b59f09e0 dummy = {e_id = 200434, e_name = {bv_len = 34, bv_val = 0x7fa0a0003198 "uid=aaalmora,ou=user,dc=cpp,dc=edu"}, e_nname = {bv_len = 34, bv_val = 0x7fa0a0003ce0 "uid=aaalmora,ou=user,dc=cpp,dc=edu"}, e_attrs = 0x7fa098ba4938, e_ocflags = 256, e_bv = {bv_len = 0, bv_val = 0x0}, e_private = 0x7fa0a00031c8} preread_ctrl = 0x0 postread_ctrl = 0x0 ctrls = {0x0, 0x0, 0x7fa0a00008c0, 0x50, 0x0, 0x7fa0b59f0800} num_ctrls = 0 numads = 57 #3 0x00000000004dbc55 in overlay_op_walk (op=0x7fa0b59f1730, rs=0x7fa0b59f0fe0, which=op_modify, oi=0x1ea7b90, on=0x0) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/servers/slapd/backover.c:677 func = 0x7fa277939dd8 <bi+88> rc = 32768 #4 0x00000000004dbea0 in over_op_func (op=0x7fa0b59f1730, rs=0x7fa0b59f0fe0, which=op_modify) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/servers/slapd/backover.c:730 oi = 0x1ea7b90 on = 0x1ea8d20 be = 0x1e99b60 db = {bd_info = 0x7fa277939d80 <bi>, bd_self = 0x1e99b60, ts 14 times>, "\001", be_flags = 563464, be_restrictops = 0, be_requires = 0, be_ssf_set = {sss_ssf = 0, sss_transport = 0, sss_tls = 0, sss_sasl = 0, sss_update_ssf = 0, sss_update_transport = 0, sss_update_tls = 0, sss_update_sasl = 0, sss_simple_bind = 64}, be_suffix = 0x1e98bf0, be_nsuffix = 0x1e99270, be_schemadn = {bv_len = 0, bv_val = 0x0}, be_schemandn = {bv_len = 0, bv_val = 0x0}, be_rootdn = {bv_len = 25, bv_val = 0x1e99490 "cn=ldaproot,dc=cpp,dc=edu"}, be_rootndn = {bv_len = 25, bv_val = 0x1e994e0 "cn=ldaproot,dc=cpp,dc=edu"}, be_rootpw = { bv_len = 14, bv_val = 0x1e6ca30 "XXXXXXXXXXXX"}, be_max_deref_depth = 15, be_def_limit = { lms_t_soft = 3600, lms_t_hard = 0, lms_s_soft = -1, lms_s_hard = 0, lms_s_unchecked = -1, lms_s_pr = 0, lms_s_pr_hide = 0, lms_s_pr_total = 0}, be_limits = 0x1e98b20, be_acl = 0x1e9aeb0, be_dfltaccess = ACL_READ, be_extra_anlist = 0x0, be_update_ndn = {bv_len = 0, bv_val = 0x0}, be_update_refs = 0x0, be_pending_csn_list = 0x215b6a0, be_pcl_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = { __prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, be_syncinfo = 0x1ea8970, be_pb = 0x0, be_cf_ocs = 0x7fa277939ac0 <mdbocs>, be_private = 0x7fa27c4b7010, be_next = {stqe_next = 0x0}} cb = {sc_next = 0x7fa0b59f0fb0, sc_response = 0x4daa2c <over_back_response>, sc_cleanup = 0x0, sc_writewait = 0x0, sc_private = 0x1ea7b90} sc = 0x0 rc = 32768 __PRETTY_FUNCTION__ = "over_op_func" #5 0x00000000004dc003 in over_op_modify (op=0x7fa0b59f1730, rs=0x7fa0b59f0fe0) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/servers/slapd/backover.c:769 No locals. #6 0x00000000004c84b4 in syncrepl_message_to_op (si=0x1ea8970, op=0x7fa0b59f1730, msg=0x7fa0a012df60) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/servers/slapd/syncrepl.c:2394 oes = {oe = {oe_next = {sle_next = 0x0}, oe_key = 0x4c7516 <syncrepl_message_to_op>}, oe_si = 0x1ea8970} ber = 0x7fa0a029eb70 modlist = 0x7fa0a0127460 ls = 0x7dfc40 <accesslog_sc> rs = {sr_type = REP_RESULT, sr_tag = 0, sr_msgid = 0, sr_err = 0, sr_matched = 0x0, sr_text = 0x7fa0b59f0a80 "modify/delete: eduPersonAffiliation: no such attribute", sr_ref = 0x0, sr_ctrls = 0x0, sr_un = {sru_search = {r_entry = 0x0, r_attr_flags = 0, r_operational_attrs = 0x0, r_attrs = 0x0, r_nentries = 0, r_v2ref = 0x0}, sru_sasl = {r_sasldata = 0x0}, sru_extended = { r_rspoid = 0x0, r_rspdata = 0x0}}, sr_flags = 0} cb = {sc_next = 0x7fa0a0002490, sc_response = 0x4cfda6 <null_callback>, sc_cleanup = 0x0, sc_writewait = 0x0, sc_private = 0x0} text = 0xa <error: Cannot access memory at address 0xa> txtbuf = "\220\020\237\265\240\177\000\000\000\021\237\265\240\177\000\000\300\020\237\265\240\177\000\000\030\000\000\000\060\000\000\000p\021\237\265\240\177\000\000\260\020\237\265\240\177\000\000\000\000\000\000\003\000\000\000 \000\000\240\240\177\000\000\377\377\377\377\377\377\377\377\000\000\000\017\000\000\000\000?\034\237\266\240\177\000\000\000\000\000\000\000\000\000\000`\021\237\265\240\177\000\000\000\000\000\000\000\000\000\000?\034\237\266\240\177", '\000' <repeats 11 times>, "'\237\265\240\177\000\000P\347M", '\000' <repeats 13 times>, "`\022\237\265\240\177\000\000\177\233\023\240\240\177\000\000p\233\023\240\240\177\000\000`%\342\001\000\000\000\000\177\233\023\240\240\177\000\000x\233\023\240\240\177\000\000\000\000\000\000\000\000\000\000"... textlen = 256 bdn = {bv_len = 34, bv_val = 0x7fa0a042265b "uid=aaalmora,ou=user,dc=cpp,dc=edu"} dn = {bv_len = 34, bv_val = 0x7fa0a0000c20 "\310\021"} ndn = {bv_len = 34, bv_val = 0x7fa0a0000ca0 "\250\022"} bv = {bv_len = 0, bv_val = 0x0} bv2 = {bv_len = 0, bv_val = 0x0} bvals = 0x7fa0a0106580 rdn = {bv_len = 0, bv_val = 0x0} sup = {bv_len = 0, bv_val = 0x0} prdn = {bv_len = 0, bv_val = 0x0} nrdn = {bv_len = 0, bv_val = 0x0} psup = {bv_len = 0, bv_val = 0x0} nsup = {bv_len = 0, bv_val = 0x0} rc = 0 deleteOldRdn = 0 freeReqDn = 1 do_graduate = 1 #7 0x00000000004c31cd in do_syncrep2 (op=0x7fa0b59f1730, si=0x1ea8970) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/servers/slapd/syncrepl.c:1013 match = 0 syncUUID = {{bv_len = 16, bv_val = 0x7fa0a0144147 "4w\232m2\230F>\273\200\032\350\227o\252i"}, { bv_len = 0, bv_val = 0x3000000020 <error: Cannot access memory at address 0x3000000020>}} cookie = {bv_len = 15, bv_val = 0x7fa0a0144159 "rid=001,sid=001"} rctrls = 0x7fa0a0111800 rctrlp = 0x7fa0a0157460 bdn = {bv_len = 44, bv_val = 0x7fa0a0422609 "reqStart=20170416103057.000005Z,cn=accesslog"} si_tag = 140327924125852 entry = 0x7fa0a0111808 punlock = -1 syncstate = 1 retdata = 0x570e74 retoid = 0x7fa27b1c55c0 "" syncUUIDs = 0x0 len = 15 berbuf = { buffer = "\002\000\001", '\000' <repeats 29 times>, "@A\024\240\240\177\000\000hA\024\240\240\177\000\000hA\024\240\240\177", '\000' <repeats 26 times>, "\017\000\000\000\240\177\000\000\060\027\237\265\240\177\000\000\200\023\237\265\240\177\000\000\260\220\035|\242\177\000\000\b\t\000\240\001\000\000\000\000\061~\000\000\000\000\000\200\023\237\265\240\177\000\000\260\220\035|\242\177\000\000H\027\237\265\240\177\000\000 \v\020\230\240\177\000\000\000\024\237\265\240\177\000\000\377\n |\242\177\000\000\300\023\237\265\001\000\000\000,\024\237\265\240\177\000\000\060\027\237\265\001\000\000\000"..., ialign = 65538, lalign = 65538, falign = 9.18382988e-41, dalign = 3.2380074297143616e-319, palign = 0x10002 <error: Cannot access memory at address 0x10002>} ber = 0x7fa0b59f12e0 msg = 0x7fa0a012df60 syncCookie = {ctxcsn = 0x0, sids = 0x0, numcsns = 0, rid = 1, octet_str = {bv_len = 15, bv_val = 0x7fa0a0139b70 "rid=001,sid=001"}, sid = 1, sc_next = {stqe_next = 0x0}} syncCookie_req = {ctxcsn = 0x7fa0a04230d0, sids = 0x7fa0a012bb00, numcsns = 4, rid = 1, octet_str = { bv_len = 183, bv_val = 0x7fa0a015e3f0 "rid=001,sid=000,csn=20170416081903.399068Z#000000#000#000000;20170416051743.054642Z#000000#001#000000;20160115030222.053144Z#000000#002#000000;20170416043935.843860Z#000000#003#000000"}, sid = 0, sc_next = {stqe_next = 0x0}} rc = 0 err = 0 modlist = 0x0 m = 1426881987 tout_p = 0x7fa0b59f1210 tout = {tv_sec = 0, tv_usec = 0} refreshDeletes = 0 empty = "empty" __PRETTY_FUNCTION__ = "do_syncrep2" #8 0x00000000004c54c8 in do_syncrepl (ctx=0x7fa0b59f1bc0, arg=0x1ea3e70) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/servers/slapd/syncrepl.c:1564 rtask = 0x1ea3e70 si = 0x1ea8970 conn = {c_struct_state = SLAP_C_UNINITIALIZED, c_conn_state = SLAP_C_INVALID, c_conn_idx = -1, c_sd = 0, c_close_reason = 0x0, c_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, c_sb = 0x0, c_starttime = 0, c_activitytime = 0, c_connid = 18446744073709551615, c_peer_domain = {bv_len = 0, bv_val = 0x568650 ""}, c_peer_name = {bv_len = 0, bv_val = 0x568650 ""}, c_listener = 0x570900 <dummy_list>, c_sasl_bind_mech = {bv_len = 0, bv_val = 0x0}, c_sasl_dn = { bv_len = 0, bv_val = 0x0}, c_sasl_authz_dn = {bv_len = 0, bv_val = 0x0}, c_authz_backend = 0x0, c_authz_cookie = 0x0, c_authz = {sai_method = 0, sai_mech = {bv_len = 0, bv_val = 0x0}, sai_dn = { bv_len = 0, bv_val = 0x0}, sai_ndn = {bv_len = 0, bv_val = 0x0}, sai_ssf = 0, sai_transport_ssf = 0, sai_tls_ssf = 0, sai_sasl_ssf = 0}, c_protocol = 0, c_ops = { stqh_first = 0x0, stqh_last = 0x0}, c_pending_ops = {stqh_first = 0x0, stqh_last = 0x0}, c_write1_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, c_write1_cv = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}, c_write2_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, c_write2_cv = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}, c_currentber = 0x0, c_writers = 0, c_writing = 0 '\000', c_sasl_bind_in_progress = 0 '\000', c_writewaiter = 0 '\000', c_is_udp = 0 '\000', c_is_tls = 0 '\000', c_needs_tls_accept = 0 '\000', c_sasl_layers = 0 '\000', c_sasl_done = 0 '\000', c_sasl_authctx = 0x0, c_sasl_sockctx = 0x0, c_sasl_extra = 0x0, c_sasl_bindop = 0x0, c_pagedresults_state = {ps_be = 0x0, ps_size = 0, ps_count = 0, ps_cookie = 0, ps_cookieval = {bv_len = 0, bv_val = 0x0}}, c_n_ops_received = 0, c_n_ops_executing = 0, c_n_ops_pending = 0, c_n_ops_completed = 0, c_n_get = 0, c_n_read = 0, c_n_write = 0, c_extensions = 0x0, c_clientfunc = 0x0, c_clientarg = 0x0, c_send_ldap_result = 0x45801c <slap_send_ldap_result>, c_send_search_entry = 0x458cf0 <slap_send_search_entry>, c_send_search_reference = 0x45b0d8 <slap_send_search_reference>, c_send_ldap_extended = 0x45886d <slap_send_ldap_extended>, c_send_ldap_intermediate = 0x458ade <slap_send_ldap_intermediate>} opbuf = {ob_op = {o_hdr = 0x7fa0b59f18a0, o_tag = 102, o_time = 1492338657, o_tincr = 65, o_bd = 0x7fa0b59f0c90, o_req_dn = {bv_len = 34, bv_val = 0x7fa0a0160390 "uid=aaalmora,ou=user,dc=cpp,dc=edu"}, o_req_ndn = {bv_len = 34, bv_val = 0x7fa0a0127430 "uid=aaalmora,ou=user,dc=cpp,dc=edu"}, o_request = {oq_add = { rs_modlist = 0x7fa0a0000940, rs_e = 0x1}, oq_bind = {rb_method = -1610610368, rb_cred = { bv_len = 1, bv_val = 0x0}, rb_edn = {bv_len = 0, bv_val = 0x0}, rb_ssf = 0, rb_mech = { bv_len = 0, bv_val = 0x0}}, oq_compare = {rs_ava = 0x7fa0a0000940}, oq_modify = { rs_mods = {rs_modlist = 0x7fa0a0000940, rs_no_opattrs = 1 '\001'}, rs_increment = 0}, oq_modrdn = {rs_mods = {rs_modlist = 0x7fa0a0000940, rs_no_opattrs = 1 '\001'}, rs_deleteoldrdn = 0, rs_newrdn = {bv_len = 0, bv_val = 0x0}, rs_nnewrdn = {bv_len = 0, bv_val = 0x0}, rs_newSup = 0x0, rs_nnewSup = 0x0}, oq_search = {rs_scope = -1610610368, rs_deref = 32672, rs_slimit = 1, rs_tlimit = 0, rs_limit = 0x0, rs_attrsonly = 0, rs_attrs = 0x0, rs_filter = 0x0, rs_filterstr = {bv_len = 0, bv_val = 0x0}}, oq_abandon = { rs_msgid = -1610610368}, oq_cancel = {rs_msgid = -1610610368}, oq_extended = {rs_reqoid = { bv_len = 140327855851840, bv_val = 0x1 <error: Cannot access memory at address 0x1>}, rs_flags = 0, rs_reqdata = 0x0}, oq_pwdexop = {rs_extended = {rs_reqoid = { bv_len = 140327855851840, bv_val = 0x1 <error: Cannot access memory at address 0x1>}, rs_flags = 0, rs_reqdata = 0x0}, rs_old = {bv_len = 0, bv_val = 0x0}, rs_new = { bv_len = 0, bv_val = 0x0}, rs_mods = 0x0, rs_modtail = 0x0}}, o_abandon = 0, o_cancel = 0, o_groups = 0x0, o_do_not_cache = 0 '\000', o_is_auth_check = 0 '\000', o_dont_replicate = 0 '\000', o_acl_priv = ACL_NONE, o_nocaching = 0 '\000', o_delete_glue_parent = 0 '\000', o_no_schema_check = 1 '\001', o_no_subordinate_glue = 0 '\000', o_ctrlflag = '\000' <repeats 14 times>, "\002", '\000' <repeats 16 times>, o_controls = 0x7fa0b59f19f0, o_authz = {sai_method = 0, sai_mech = {bv_len = 0, bv_val = 0x0}, sai_dn = {bv_len = 25, bv_val = 0x1e99490 "cn=ldaproot,dc=cpp,dc=edu"}, sai_ndn = { bv_len = 25, bv_val = 0x1e994e0 "cn=ldaproot,dc=cpp,dc=edu"}, sai_ssf = 0, sai_transport_ssf = 0, sai_tls_ssf = 0, sai_sasl_ssf = 0}, o_ber = 0x0, o_res_ber = 0x0, o_callback = 0x7fa0a00024c0, o_ctrls = 0x0, o_csn = {bv_len = 40, bv_val = 0x7fa0a0109a60 "20170416103057.530384Z#000000#000#000000"}, o_private = 0x0, o_extra = {slh_first = 0x7fa0b59f09e0}, o_next = {stqe_next = 0x0}}, ob_hdr = {oh_opid = 0, oh_connid = 1, oh_conn = 0x7fa0b59f1470, oh_msgid = 0, oh_protocol = 0, oh_tid = 140328218601216, oh_threadctx = 0x7fa0b59f1bc0, oh_tmpmemctx = 0x7fa0a00008c0, oh_tmpmfuncs = 0x7dfb80 <slap_sl_mfuncs>, oh_counters = 0x8156c0 <slap_counters>, oh_log_prefix = "conn=-1 op=0", '\000' <repeats 243 times>, oh_extensions = 0x0}, ob_controls = { 0x0 <repeats 17 times>, 0x7fa0b59f1260, 0x0 <repeats 14 times>}} op = 0x7fa0b59f1730 rc = 0 dostop = 0 s = 18 i = 2080111901 defer = 1 fail = 0 freeinfo = 0 be = 0x1e99b60 #9 0x0000000000440411 in connection_read_thread (ctx=0x7fa0b59f1bc0, argv=0x12) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/servers/slapd/connection.c:1296 rc = 0 cri = {op = 0x0, func = 0x4c4ec6 <do_syncrepl>, arg = 0x1ea3e70, ctx = 0x7fa0b59f1bc0, nullop = 0} s = 18 #10 0x00007fa27c1d7a4b in ldap_int_thread_pool_wrapper (xpool=0x1e2c330) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/libraries/libldap_r/tpool.c:696 pool = 0x1e2c330 task = 0x7fa0b0000e40 work_list = 0x1e2c3c8 ctx = {ltu_id = 140328218601216, ltu_key = {{ltk_key = 0x4ba0e3 <slap_sl_mem_init>, ltk_data = 0x7fa0a00008c0, ltk_free = 0x4b9f08 <slap_sl_mem_destroy>}, {ltk_key = 0x215b710, ltk_data = 0x7fa0a0ba3eb0, ltk_free = 0x7fa27772d13d <mdb_reader_free>}, { ltk_key = 0x7fa277721bce <search_stack>, ltk_data = 0x7fa096fff010, ltk_free = 0x7fa277721bab <search_stack_free>}, {ltk_key = 0x7fa27771e4aa <scope_chunk_get>, ltk_data = 0x7fa0a0ba5d40, ltk_free = 0x7fa27771e462 <scope_chunk_free>}, { ltk_key = 0x43f808 <conn_counter_init>, ltk_data = 0x7fa0a012d4f0, ltk_free = 0x43f65a <conn_counter_destroy>}, {ltk_key = 0x45c638 <slap_op_free>, ltk_data = 0x7fa0a825ab60, ltk_free = 0x45c58b <slap_op_q_destroy>}, {ltk_key = 0x1f54e70, ltk_data = 0x7fa0a01c3fa0, ltk_free = 0x7fa27772d13d <mdb_reader_free>}, {ltk_key = 0x0, ltk_data = 0x7fa09c04d880, ltk_free = 0x0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0x0} <repeats 24 times>}} kctx = 0x0 i = 32 keyslot = 634 hash = 3535879802 __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper" #11 0x00007fa27b820494 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #12 0x00007fa27af13edd in clone () from /lib64/libc.so.6 No symbol table info available.
On Sun, Apr 16, 2017 at 06:20:06PM +0000, henson@acm.org wrote: > On Mon, Apr 03, 2017 at 02:44:48PM +0100, Howard Chu wrote: > >> You should probably switch to a non-optimized build. The backtrace you >> posted initially had most parameters optimized out, which also makes >> things harder to trace. > > Well, as luck (8-/) would have it, it crashed both yesterday and today > on the non-optimized build. Here are the full backtraces from both > cores, I hope they provide some clue as to what's going on. Let me know > if you want me to do any more post mortem debugging on the cores. Thanks > much. > > > Core was generated by `/usr/lib64/openldap/slapd -u ldap -g ldap -h ldaps:// ldap://'. > Program terminated with signal SIGSEGV, Segmentation fault. > (gdb) bt full > #6 0x00000000004c84b4 in syncrepl_message_to_op (si=0x16e4630, op=0x7fac037fd730, msg=0x7fabfc4a1560) > at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/servers/slapd/syncrepl.c:2394 > #7 0x00000000004c31cd in do_syncrep2 (op=0x7fac037fd730, si=0x16e4630) > at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/servers/slapd/syncrepl.c:1013 > bdn = {bv_len = 44, bv_val = 0x7fabfc492bea "reqStart=20170415103641.000016Z,cn=accesslog"} > > > Core was generated by `/usr/lib64/openldap/slapd -u ldap -g ldap -h ldaps:// ldap://'. > Program terminated with signal SIGSEGV, Segmentation fault. > (gdb) bt full > #6 0x00000000004c84b4 in syncrepl_message_to_op (si=0x1ea8970, op=0x7fa0b59f1730, msg=0x7fa0a012df60) > at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/servers/slapd/syncrepl.c:2394 > #7 0x00000000004c31cd in do_syncrep2 (op=0x7fa0b59f1730, si=0x1ea8970) > at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.44-r1/work/openldap-2.4.44/servers/slapd/syncrepl.c:1013 > bdn = {bv_len = 44, bv_val = 0x7fa0a0422609 "reqStart=20170416103057.000005Z,cn=accesslog"} Hi Paul, could you post the two accesslog entries (or at least an outline) referenced in the backtrace? - reqStart=20170415103641.000016Z,cn=accesslog - reqStart=20170416103057.000005Z,cn=accesslog I assume you are still running the same code in production, correct? Thanks, Ondrej -- Ondřej Kuzník Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP
On Wed, Apr 19, 2017 at 05:37:35PM +0100, Ondřej Kuzník wrote: > could you post the two accesslog entries (or at least an outline) > referenced in the backtrace? That's weird. I can't find these exact entries. Based on the username, I found ones that are very close to the timestamp, but ones that match these exact timestamps do not exist. Last time Quanah asked me to look up the matching accesslog entry from the crash I believe I was able to find it. > - reqStart=20170415103641.000016Z,cn=accesslog dn: reqStart=20170415103641.000068Z,cn=accesslog objectClass: auditModify structuralObjectClass: auditModify reqStart: 20170415103641.000068Z reqEnd: 20170415103641.000069Z reqType: modify reqSession: 792228 reqAuthzID: cn=idmgmt,ou=user,ou=service,dc=cpp,dc=edu reqDN: uid=brandonl,ou=user,dc=cpp,dc=edu reqResult: 0 reqMod: eduPersonAffiliation:= reqMod: eduPersonPrimaryAffiliation:= reqMod: cppEduPersonAffiliation:= reqMod: entryCSN:= 20170415103641.599920Z#000000#000#000000 reqMod: modifiersName:= cn=idmgmt,ou=user,ou=service,dc=cpp,dc=edu reqMod: modifyTimestamp:= 20170415103641Z reqEntryUUID: 8b33981d-25ee-4f7b-b439-41c273e94b61 entryUUID: b132db96-9ebe-4ee5-9fd2-5257b0c352d2 creatorsName: cn=accesslog createTimestamp: 20170415103641Z entryCSN: 20170415103641.599920Z#000000#000#000000 modifiersName: cn=accesslog modifyTimestamp: 20170415103641Z > - reqStart=20170416103057.000005Z,cn=accesslog dn: reqStart=20170416103057.000045Z,cn=accesslog objectClass: auditModify structuralObjectClass: auditModify reqStart: 20170416103057.000045Z reqEnd: 20170416103057.000046Z reqType: modify reqSession: 54887 reqAuthzID: cn=idmgmt,ou=user,ou=service,dc=cpp,dc=edu reqDN: uid=aaalmora,ou=user,dc=cpp,dc=edu reqResult: 0 reqMod: eduPersonAffiliation:= reqMod: eduPersonPrimaryAffiliation:= reqMod: cppEduPersonAffiliation:= reqMod: entryCSN:= 20170416103057.530384Z#000000#000#000000 reqMod: modifiersName:= cn=idmgmt,ou=user,ou=service,dc=cpp,dc=edu reqMod: modifyTimestamp:= 20170416103057Z reqEntryUUID: 153aa3e7-cc27-48ce-b6f2-e14215348941 entryCSN: 20170416103057.530384Z#000000#000#000000 entryUUID: b0c18fef-134e-4c8e-b073-fcdff58d3d93 creatorsName: cn=accesslog createTimestamp: 20170416103057Z modifiersName: cn=accesslog modifyTimestamp: 20170416103057Z > I assume you are still running the same code in production, correct? Yes. It actually crashed 4 days in a row :(, the two days I provided backtraces for and then the following two days. Just for something to do I slapcat'd the db and accesslog and reloaded it after the fourth day in case there was some weird mdb corruption or something. It hasn't crashed since, but it crashes randomly so that could be meaningless <sigh>. I wouldn't think reloaded the db would have affected finding the matching accesslog entries but that is something different than last time too. And of course now this is the non-optimized build. Thanks...
On Wed, Apr 19, 2017 at 01:03:15PM -0700, Paul B. Henson wrote: > On Wed, Apr 19, 2017 at 05:37:35PM +0100, Ondřej Kuzník wrote: >> could you post the two accesslog entries (or at least an outline) >> referenced in the backtrace? > > That's weird. I can't find these exact entries. Based on the username, I > found ones that are very close to the timestamp, but ones that match > these exact timestamps do not exist. Last time Quanah asked me to look > up the matching accesslog entry from the crash I believe I was able to > find it. Hi Paul, those come from master with ServerID 0? Maybe you can find them in another server's accesslog? >> - reqStart=20170415103641.000016Z,cn=accesslog > > dn: reqStart=20170415103641.000068Z,cn=accesslog > reqDN: uid=brandonl,ou=user,dc=cpp,dc=edu > reqMod: entryCSN:= 20170415103641.599920Z#000000#000#000000 > entryCSN: 20170415103641.599920Z#000000#000#000000 > >> - reqStart=20170416103057.000005Z,cn=accesslog > > dn: reqStart=20170416103057.000045Z,cn=accesslog > reqDN: uid=aaalmora,ou=user,dc=cpp,dc=edu > reqMod: entryCSN:= 20170416103057.530384Z#000000#000#000000 > entryCSN: 20170416103057.530384Z#000000#000#000000 > >> I assume you are still running the same code in production, correct? > > Yes. It actually crashed 4 days in a row :(, the two days I provided > backtraces for and then the following two days. Just for something to do > I slapcat'd the db and accesslog and reloaded it after the fourth day in > case there was some weird mdb corruption or something. It hasn't crashed > since, but it crashes randomly so that could be meaningless <sigh>. I > wouldn't think reloaded the db would have affected finding the matching > accesslog entries but that is something different than last time too. > And of course now this is the non-optimized build. Would you be able to try the patch I provided earlier on one of your servers? Thanks, -- Ondřej Kuzník Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP
--On Thursday, April 20, 2017 9:34 AM +0000 okuznik@symas.com wrote: > On Wed, Apr 19, 2017 at 01:03:15PM -0700, Paul B. Henson wrote: >> On Wed, Apr 19, 2017 at 05:37:35PM +0100, Ond=C5=99ej Kuzn=C3=ADk wrote= > : >>> could you post the two accesslog entries (or at least an outline) >>> referenced in the backtrace? >> =20 >> That's weird. I can't find these exact entries. Based on the username, = > I >> found ones that are very close to the timestamp, but ones that match >> these exact timestamps do not exist. Last time Quanah asked me to look >> up the matching accesslog entry from the crash I believe I was able to >> find it. > > Hi Paul, > those come from master with ServerID 0? Maybe you can find them in > another server's accesslog? Hi Paul, You stated previously that you are in 4-way MMR. It's not valid to have a serverID of 0 in an MMR environment (See <http://www.openldap.org/its/index.cgi/?findid=8635>). --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>
> From: Ondřej Kuzník > Sent: Thursday, April 20, 2017 1:34 AM > > those come from master with ServerID 0? Maybe you can find them in > another server's accesslog? Hmm, no, I don't see them in any of the three other accesslogs either. > Would you be able to try the patch I provided earlier on one of your > servers? This one? ftp://ftp.openldap.org/incoming/Ondrej-Kuznik-20170327-ITS8609-ignore-invalid-accesslog-ops.patch Sure, I can drop this in on the master server that has been crashing and see what happens. Thanks…
> From: Quanah Gibson-Mount > Sent: Thursday, April 20, 2017 8:07 AM > > You stated previously that you are in 4-way MMR. It's not valid to have a > serverID of 0 in an MMR environment (See > <http://www.openldap.org/its/index.cgi/?findid=8635>). Well now, that's quite a fresh ITS 8-/. I'm guessing quite a few people have serverID's of 0 in their replicated environments, as I don't know that I've ever seen that information before. What are the ramifications of having a server with an ID of 0 in a replicated environment? What is the procedure for remediating the issue? Is it as simple as shutting down that server, updating the configuration to have a serverID of 4 and restarting it? Or does the database need to be stripped of all CSN's which have an ID of 0 via slapcat, grep -v, and then slapadd with the -w option on the master and then reloading from ldif on all the replicas?
--On Thursday, April 20, 2017 9:02 PM +0000 henson@acm.org wrote: >> From: Quanah Gibson-Mount >> Sent: Thursday, April 20, 2017 8:07 AM >> >> You stated previously that you are in 4-way MMR. It's not valid to have >> a serverID of 0 in an MMR environment (See >> <http://www.openldap.org/its/index.cgi/?findid=8635>). > > Well now, that's quite a fresh ITS 8-/. I'm guessing quite a few people > have serverID's of 0 in their replicated environments, as I don't know > that I've ever seen that information before. What are the ramifications > of having a server with an ID of 0 in a replicated environment? What is > the procedure for remediating the issue? Is it as simple as shutting down > that server, updating the configuration to have a serverID of 4 and > restarting it? Or does the database need to be stripped of all CSN's > which have an ID of 0 via slapcat, grep -v, and then slapadd with the -w > option on the master and then reloading from ldif on all the replicas? It's fine for there to be legacy entryCSNs and a contextCSN for serverID of 0. However, it is not fine for any master in an MMR setup to have a specific serverID of 0. If you're running with serverID's 0-3, I'd simply try an ldapmodify on the server with 0 to set its serverID to 4, and then do a modification against it, so it generates a new contextCSN that gets pushed out to all nodes. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>
quanah@symas.com wrote: > --On Thursday, April 20, 2017 9:02 PM +0000 henson@acm.org wrote: > >>> From: Quanah Gibson-Mount >>> Sent: Thursday, April 20, 2017 8:07 AM >>> >>> You stated previously that you are in 4-way MMR. It's not valid to have >>> a serverID of 0 in an MMR environment (See >>> <http://www.openldap.org/its/index.cgi/?findid=8635>). >> >> Well now, that's quite a fresh ITS 8-/. I'm guessing quite a few people >> have serverID's of 0 in their replicated environments, as I don't know >> that I've ever seen that information before. What are the ramifications >> of having a server with an ID of 0 in a replicated environment? What is >> the procedure for remediating the issue? Is it as simple as shutting down >> that server, updating the configuration to have a serverID of 4 and >> restarting it? Or does the database need to be stripped of all CSN's >> which have an ID of 0 via slapcat, grep -v, and then slapadd with the -w >> option on the master and then reloading from ldif on all the replicas? > > It's fine for there to be legacy entryCSNs and a contextCSN for serverID of > 0. However, it is not fine for any master in an MMR setup to have a > specific serverID of 0. If you're running with serverID's 0-3, I'd simply > try an ldapmodify on the server with 0 to set its serverID to 4, and then > do a modification against it, so it generates a new contextCSN that gets > pushed out to all nodes. Performance-wise, it would be better to delete the SID 0 contextCSN value too. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
--On Friday, April 21, 2017 2:26 AM +0100 Howard Chu <hyc@symas.com> wrote: >> It's fine for there to be legacy entryCSNs and a contextCSN for serverID >> of 0. However, it is not fine for any master in an MMR setup to have a >> specific serverID of 0. If you're running with serverID's 0-3, I'd >> simply try an ldapmodify on the server with 0 to set its serverID to 4, >> and then do a modification against it, so it generates a new contextCSN >> that gets pushed out to all nodes. > > Performance-wise, it would be better to delete the SID 0 contextCSN value > too. Yeah, definitely not optimal, but less of a worry at the moment. ;) --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>
> From: Ondřej Kuzník > Sent: Thursday, April 20, 2017 1:34 AM > > Would you be able to try the patch I provided earlier on one of your > servers? Sorry for the long delay on this. I had built an updated binary with the patch, but was waiting for the server to crash to drop it in. As luck would have it, it went almost a month without crashing 8-/. It crashed yesterday though, so it is now running with the patch (Ondrej-Kuznik-20170327-ITS8609-ignore-invalid-accesslog-ops.patch), although it might take some time to see if it makes any difference. Or it might crash again tomorrow. <sigh>. Thanks much…
> From: Quanah Gibson-Mount > Sent: Thursday, April 20, 2017 1:26 PM > > It's fine for there to be legacy entryCSNs and a contextCSN for serverID of > 0. However, it is not fine for any master in an MMR setup to have a > specific serverID of 0. If you're running with serverID's 0-3, I'd simply > try an ldapmodify on the server with 0 to set its serverID to 4, and then > do a modification against it, so it generates a new contextCSN that gets > pushed out to all nodes. I went ahead and updated the serverID from 0 to 4 when I dropped in the patched binary, it seems to have gone smoothly. I'm planning to migrate the memberOf configuration over the summer to the dynamic list implementation, which is going to require a dump to ldif and reload, I'll go ahead and strip out the legacy serverid 0 contextCSN's then. Thanks.
I've been running 2.4.44 with the test patch since May and haven't seen a crash since, for what that's worth. We upgraded to 2.4.45 last week, without that patch. I'm going to see if the crash shows up again, and if it does, apply the patch to 2.4.45 and see if it goes away again. Thanks...
changed notes
On Fri, Sep 15, 2017 at 07:54:22PM -0700, Paul B. Henson wrote: > I've been running 2.4.44 with the test patch since May and haven't seen > a crash since, for what that's worth. We upgraded to 2.4.45 last week, > without that patch. I'm going to see if the crash shows up again, and if > it does, apply the patch to 2.4.45 and see if it goes away again. Well, since upgrading, I've now seen two crashes; I don't have it compiled with debugging right now so the trace is limited: #0 0x0000000000483eb9 in modify_add_values () #1 0x00007f7b80d45bc3 in mdb_modify_internal () from /usr/lib64/openldap/openldap/back_mdb.so #2 0x00007f7b80d46f55 in mdb_modify () from /usr/lib64/openldap/openldap/back_mdb.so #3 0x000000000049a6d2 in overlay_op_walk () #4 0x000000000049a82a in ?? () #5 0x000000000048bbef in ?? () #6 0x000000000048f8cf in ?? () #7 0x0000000000493ae3 in ?? () #8 0x00007f7b858327f2 in ?? () from /usr/lib64/libldap_r-2.4.so.2 #9 0x00007f7b84bb03e4 in start_thread () from /lib64/libpthread.so.0 #10 0x00007f7b848f23ed in clone () from /lib64/libc.so.6 but it looks like the same issue to me. I'm going to go ahead and recompile 2.4.45 with the ignore-invalid-accesslog-ops.patch and run that for a while and see what happens. Thanks...
On Fri, Oct 06, 2017 at 06:45:41PM -0700, Paul B. Henson wrote: > but it looks like the same issue to me. I'm going to go ahead and > recompile 2.4.45 with the ignore-invalid-accesslog-ops.patch and run > that for a while and see what happens. Sadness :(, not one but two of my boxes crashed today, both running the patch, with what appears to be the same issue: Core was generated by `/usr/lib64/openldap/slapd -u ldap -g ldap -h ldaps:// ldap://'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x0000000000483eb9 in modify_add_values (e=e@entry=0x7f75994f9440, mod=mod@entry=0x7f758c0016c8, permissive=0, text=text@entry=0x7f75994f99a0, textbuf=textbuf@entry=0x7f75994f94c0 "modify/delete: eduPersonAffiliation: no such attribute", textlen=textlen@entry=256) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.45-r2/work/openldap-2.4.45/servers/slapd/mods.c:61 61 if ( !BER_BVISNULL( &mod->sm_values[mod->sm_numvals] )) { [Current thread is 1 (Thread 0x7f75994fb700 (LWP 44707))] (gdb) bt full #0 0x0000000000483eb9 in modify_add_values (e=e@entry=0x7f75994f9440, mod=mod@entry=0x7f758c0016c8, permissive=0, text=text@entry=0x7f75994f99a0, textbuf=textbuf@entry=0x7f75994f94c0 "modify/delete: eduPersonAffiliation: no such attribute", textlen=textlen@entry=256) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.45-r2/work/openldap-2.4.45/servers/slapd/mods.c:61 rc = <optimized out> op = 0x4c1265 "add" a = <optimized out> pmod = {sm_desc = <optimized out>, sm_values = 0x0, sm_nvalues = 0x0, sm_numvals = <optimized out>, sm_op = <optimized out>, sm_flags = <optimized out>, sm_type = {bv_len = <optimized out>, bv_val = <optimized out>}} __PRETTY_FUNCTION__ = "modify_add_values" #1 0x00007f779dbd9bc3 in mdb_modify_internal (op=op@entry=0x7f75994fa600, tid=tid@entry=0x16bbae0, modlist=0x7f758c001688, e=e@entry=0x7f75994f9440, text=text@entry=0x7f75994f99a0, textbuf=textbuf@entry=0x7f75994f94c0 "modify/delete: eduPersonAffiliation: no such attribute", textlen=256) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.45-r2/work/openldap-2.4.45/servers/slapd/back-mdb/modify.c:137 rc = <optimized out> err = <optimized out> mod = 0x7f758c0016c8 ml = 0x7f758c0016c8 save_attrs = 0x7f758c002708 ap = 0x7f77a2997010 glue_attr_delete = <optimized out> got_delete = 0 __PRETTY_FUNCTION__ = "mdb_modify_internal" #2 0x00007f779dbdaf55 in mdb_modify (op=0x7f75994fa600, rs=0x7f75994f9980) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.45-r2/work/openldap-2.4.45/servers/slapd/back-mdb/modify.c:624 mdb = 0x7f77a2997010 e = 0x7f758c0026b8 manageDSAit = <optimized out> ---Type <return> to continue, or q <return> to quit--- textbuf = "modify/delete: eduPersonAffiliation: no such attribute\000\000\276B2\214u\177\000\000\000\000\001\000\000\000\000\000\301B2\214u\177\000\000\276B2\214u\177\000\000\267B2\214u\177\000\000\220\031\000\214u\177\000\000\000\320\070\355\212˄\256\000\200", '\000' <repeats 15 times>, "\246O\231u\177\000\000pV0\001\000\000\000\000\220\031\000\214u\177\000\000PX0\001\000\000\000\000\000\200\000\000\000\000\000\000\236\313J\000\000\000\000\000\370[0\001\000\000\000\000\270\031\000\214u\177\000\000\000\246O\231u\177\000\000\260"... txn = 0x16bbae0 opinfo = {moi_oe = {oe_next = {sle_next = 0x7f75994f9930}, oe_key = 0x7f77a2997010}, moi_txn = 0x16bbae0, moi_ref = 1, moi_flag = 0 '\000'} moi = 0x7f75994f9420 dummy = {e_id = 84455, e_name = {bv_len = 34, bv_val = 0x7f758c002688 "uid=jredmond,ou=user,dc=cpp,dc=edu"}, e_nname = {bv_len = 34, bv_val = 0x7f758c0031f0 "uid=jredmond,ou=user,dc=cpp,dc=edu"}, e_attrs = 0x7f75c42d5ed8, e_ocflags = 256, e_bv = {bv_len = 0, bv_val = 0x0}, e_private = 0x7f758c0026b8} preread_ctrl = 0x0 postread_ctrl = 0x0 ctrls = {0x0, 0x2, 0xffffffffffffffff, 0x0, 0x0, 0x7f75994f9400} num_ctrls = 0 numads = 61 #3 0x000000000049a6c2 in overlay_op_walk (op=op@entry=0x7f75994fa600, rs=0x7f75994f9980, which=op_modify, oi=0x1305490, on=<optimized out>) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.45-r2/work/openldap-2.4.45/servers/slapd/backover.c:677 func = <optimized out> rc = <optimized out> #4 0x000000000049a81a in over_op_func (op=0x7f75994fa600, rs=<optimized out>, which=<optimized out>) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.45-r2/work/openldap-2.4.45/servers/slapd/backover.c:730 oi = <optimized out> on = <optimized out> be = 0x12f6f70 db = {bd_info = 0x7f779ddf3c60 <bi>, bd_self = 0x12f6f70, be_ctrls = "\000\001\001\001\000\001\000\000\001\000\000\001\001\000\001\000\000\001", '\000' <repeats 14 times>, "\001", be_flags = 563464, be_restrictops = 0, be_requires = 0, be_ssf_set = {sss_ssf = 0, ---Type <return> to continue, or q <return> to quit--- sss_transport = 0, sss_tls = 0, sss_sasl = 0, sss_update_ssf = 0, sss_update_transport = 0, sss_update_tls = 0, sss_update_sasl = 0, sss_simple_bind = 64}, be_suffix = 0x12f6000, be_nsuffix = 0x12f6680, be_schemadn = {bv_len = 0, bv_val = 0x0}, be_schemandn = {bv_len = 0, bv_val = 0x0}, be_rootdn = {bv_len = 25, bv_val = 0x12f68a0 "cn=ldaproot,dc=cpp,dc=edu"}, be_rootndn = {bv_len = 25, bv_val = 0x12f68f0 "cn=ldaproot,dc=cpp,dc=edu"}, be_rootpw = { bv_len = 14, bv_val = 0x12c72b0 ""}, be_max_deref_depth = 15, be_def_limit = { lms_t_soft = 3600, lms_t_hard = 0, lms_s_soft = -1, lms_s_hard = 0, lms_s_unchecked = -1, lms_s_pr = 0, lms_s_pr_hide = 0, lms_s_pr_total = 0}, be_limits = 0x12f5f30, be_acl = 0x12f82c0, be_dfltaccess = ACL_READ, be_extra_anlist = 0x0, be_update_ndn = {bv_len = 0, bv_val = 0x0}, be_update_refs = 0x0, be_pending_csn_list = 0x15b8d90, be_pcl_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = { __prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, be_syncinfo = 0x13060d0, be_pb = 0x0, be_cf_ocs = 0x7f779ddf3820 <mdbocs>, be_private = 0x7f77a2997010, be_next = {stqe_next = 0x0}} cb = {sc_next = 0x7f75994f9950, sc_response = 0x499a10 <over_back_response>, sc_cleanup = 0x0, sc_private = 0x1305490, sc_writewait = 0x0} sc = <optimized out> rc = 32768 __PRETTY_FUNCTION__ = "over_op_func" #5 0x000000000048bbdf in syncrepl_message_to_op (si=si@entry=0x1306800, op=op@entry=0x7f75994fa600, msg=0x7f758c112430) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.45-r2/work/openldap-2.4.45/servers/slapd/syncrepl.c:2423 oes = {oe = {oe_next = {sle_next = 0x0}, oe_key = 0x48b010 <syncrepl_message_to_op>}, oe_si = 0x1306800} ber = 0x7f758c1162e0 modlist = 0x7f758c129dc0 ls = 0x72e2a0 <accesslog_sc> rs = {sr_type = REP_RESULT, sr_tag = 0, sr_msgid = 0, sr_err = 0, sr_matched = 0x0, sr_text = 0x7f75994f94c0 "modify/delete: eduPersonAffiliation: no such attribute", sr_ref = 0x0, sr_ctrls = 0x0, sr_un = {sru_search = {r_entry = 0x0, r_attr_flags = 0, r_operational_attrs = 0x0, r_attrs = 0x0, r_nentries = 0, r_v2ref = 0x0}, sru_sasl = {r_sasldata = 0x0}, sru_extended = { r_rspoid = 0x0, r_rspdata = 0x0}}, sr_flags = 0} ---Type <return> to continue, or q <return> to quit--- cb = {sc_next = 0x7f758c001960, sc_response = 0x48ab80 <null_callback>, sc_cleanup = 0x0, sc_private = 0x0, sc_writewait = 0x0} text = 0x29 <error: Cannot access memory at address 0x29> txtbuf = "@\237O\231u\177\000\000\261\215J\242w\177\000\000@\237O\231u\177\000\000\020\234O\231u\177\000\000\020'M\000\000\000\000\000\026\220J\242w\177\000\000@\237O\231u\177\000\000\020'M\000\000\000\000\000\220\232O\231u\177\000\000`\374J\242w\177\000\000\020'M\000\000\000\000\000@\235J\242w\177\000\000\020\000\000\000\000\000\000\000\377\377\377\377\377\377\377\377\200\232O\017\000\000\000\000\f\\1\214u\177\000\000 \234O\231u\177\000\000\b\\1\214u\177\000\000\001\000\000\000\000\000\000\000\017\\1\214u\177\000\000\340\234O\231u\177\000\000@\231(\001\000\000\000\000\000h0\001\000\000\000\000\213\307I\000\000\000\000\000\020", '\000' <repeats 23 times>... bdn = {bv_len = 34, bv_val = 0x7f758c10fc4c "uid=jredmond,ou=user,dc=cpp,dc=edu"} dn = {bv_len = 34, bv_val = 0x7f758c001968 ""} ndn = {bv_len = 34, bv_val = 0x7f758c0019e8 "\""} bv = {bv_len = 0, bv_val = 0x0} bv2 = {bv_len = 0, bv_val = 0x0} bvals = 0x7f758c1144a0 rdn = {bv_len = 0, bv_val = 0x0} sup = {bv_len = 0, bv_val = 0x0} prdn = {bv_len = 0, bv_val = 0x0} nrdn = {bv_len = 0, bv_val = 0x0} psup = {bv_len = 0, bv_val = 0x0} nsup = {bv_len = 0, bv_val = 0x0} rc = 0 deleteOldRdn = 0 freeReqDn = 1 do_graduate = 1 #6 0x000000000048f8bf in do_syncrep2 (op=op@entry=0x7f75994fa600, si=si@entry=0x1306800) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.45-r2/work/openldap-2.4.45/servers/slapd/syncrepl.c:1014 match = <optimized out> syncUUID = {{bv_len = 16, bv_val = 0x7f758c11b257 "\006_\367\067\304\032H\273\273F \355\022\271\b\f"}, { bv_len = 140143060033504, ---Type <return> to continue, or q <return> to quit--- bv_val = 0x7f7500000001 <error: Cannot access memory at address 0x7f7500000001>}} cookie = {bv_len = 15, bv_val = 0x7f758c11b269 "rid=002,sid=001"} rctrls = 0x7f758c10fb80 rctrlp = <optimized out> bdn = {bv_len = 44, bv_val = 0x7f758c10fbfa "reqStart=20171101104541.000063Z,cn=accesslog"} si_tag = <optimized out> entry = <optimized out> punlock = -1 syncstate = 1 retdata = 0x0 retoid = 0x0 syncUUIDs = 0x7f758c11f0f0 len = 15 berbuf = { buffer = "\002\000\001", '\000' <repeats 29 times>, "P\262\021\214u\177\000\000x\262\021\214u\177\000\000x\262\021\214u\177", '\000' <repeats 26 times>, "\244q\021\234u\177\000\000m\252\365\240w\177\000\000\340\237O\231u\177\000\000b\232\356\240w\177\000\000\001\000\000\000\000\000\000\000ԏ\357\240w\177\000\000\364:\n\277\325\362^\203\250\336\vʶy\"\277=w3\340\204(\302\375P\262\021\214u\177\000\000\060\000\000\000\000\000\000\000\200\373\020\214u\177\000\000\060\000\000\000\000\000\000\000ТO\231u\177\000\000\000\242O\231u\177\000\000СO\231"..., ialign = 65538, lalign = 65538, falign = 9.18382988e-41, dalign = 3.2380074297143616e-319, palign = 0x10002 <error: Cannot access memory at address 0x10002>} ber = 0x7f75994f9f40 msg = 0x7f758c112430 syncCookie = {ctxcsn = 0x0, sids = 0x0, numcsns = 0, rid = 2, octet_str = {bv_len = 15, bv_val = 0x7f758c315c00 "rid=002,sid=001"}, sid = 1, sc_next = {stqe_next = 0x0}} syncCookie_req = {ctxcsn = 0x0, sids = 0x0, numcsns = 0, rid = 2, octet_str = {bv_len = 15, bv_val = 0x7f758c105fc0 "rid=002,sid=001"}, sid = 1, sc_next = {stqe_next = 0x0}} rc = 0 err = 0 modlist = 0x0 m = 11 tout_p = 0x7f75994f9c00 tout = {tv_sec = 0, tv_usec = 0} ---Type <return> to continue, or q <return> to quit--- refreshDeletes = 0 empty = "empty" #7 0x0000000000493ad3 in do_syncrepl (ctx=ctx@entry=0x7f75994fab90, arg=arg@entry=0x1306c20) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.45-r2/work/openldap-2.4.45/servers/slapd/syncrepl.c:1565 rtask = 0x1306c20 si = 0x1306800 conn = {c_struct_state = SLAP_C_UNINITIALIZED, c_conn_state = SLAP_C_INVALID, c_conn_idx = -1, c_sd = 0, c_close_reason = 0x0, c_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, c_sb = 0x0, c_starttime = 0, c_activitytime = 0, c_connid = 18446744073709551615, c_peer_domain = {bv_len = 0, bv_val = 0x4c7052 ""}, c_peer_name = {bv_len = 0, bv_val = 0x4c7052 ""}, c_listener = 0x4c9ee0 <dummy_list>, c_sasl_bind_mech = {bv_len = 0, bv_val = 0x0}, c_sasl_dn = { bv_len = 0, bv_val = 0x0}, c_sasl_authz_dn = {bv_len = 0, bv_val = 0x0}, c_authz_backend = 0x0, c_authz_cookie = 0x0, c_authz = {sai_method = 0, sai_mech = {bv_len = 0, bv_val = 0x0}, sai_dn = { bv_len = 0, bv_val = 0x0}, sai_ndn = {bv_len = 0, bv_val = 0x0}, sai_ssf = 0, sai_transport_ssf = 0, sai_tls_ssf = 0, sai_sasl_ssf = 0}, c_protocol = 0, c_ops = { stqh_first = 0x0, stqh_last = 0x0}, c_pending_ops = {stqh_first = 0x0, stqh_last = 0x0}, c_write1_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, c_write1_cv = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}, c_write2_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, c_write2_cv = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}, c_currentber = 0x0, c_writers = 0, c_writing = 0 '\000', c_sasl_bind_in_progress = 0 '\000', c_writewaiter = 0 '\000', c_is_udp = 0 '\000', c_is_tls = 0 '\000', c_needs_tls_accept = 0 '\000', c_sasl_layers = 0 '\000', c_sasl_done = 0 '\000', c_sasl_authctx = 0x0, c_sasl_sockctx = 0x0, c_sasl_extra = 0x0, c_sasl_bindop = 0x0, c_pagedresults_state = {ps_be = 0x0, ps_size = 0, ---Type <return> to continue, or q <return> to quit--- ps_count = 0, ps_cookie = 0, ps_cookieval = {bv_len = 0, bv_val = 0x0}}, c_n_ops_received = 0, c_n_ops_executing = 0, c_n_ops_pending = 0, c_n_ops_completed = 0, c_n_get = 0, c_n_read = 0, c_n_write = 0, c_extensions = 0x0, c_clientfunc = 0x0, c_clientarg = 0x0, c_send_ldap_result = 0x440a60 <slap_send_ldap_result>, c_send_search_entry = 0x4414c0 <slap_send_search_entry>, c_send_search_reference = 0x442a00 <slap_send_search_reference>, c_send_ldap_extended = 0x441190 <slap_send_ldap_extended>, c_send_ldap_intermediate = 0x441330 <slap_send_ldap_intermediate>} opbuf = {ob_op = {o_hdr = 0x7f75994fa770, o_tag = 102, o_time = 1509533141, o_tincr = 61, o_bd = 0x7f75994f9690, o_req_dn = {bv_len = 34, bv_val = 0x7f758c11f0f0 "uid=jredmond,ou=user,dc=cpp,dc=edu"}, o_req_ndn = {bv_len = 34, bv_val = 0x7f758c11d4c0 "uid=jredmond,ou=user,dc=cpp,dc=edu"}, o_request = {oq_add = { rs_modlist = 0x7f758c001688, rs_e = 0x1}, oq_bind = {rb_method = -1946151288, rb_cred = { bv_len = 1, bv_val = 0x0}, rb_edn = {bv_len = 0, bv_val = 0x0}, rb_ssf = 0, rb_mech = { bv_len = 0, bv_val = 0x0}}, oq_compare = {rs_ava = 0x7f758c001688}, oq_modify = { rs_mods = {rs_modlist = 0x7f758c001688, rs_no_opattrs = 1 '\001'}, rs_increment = 0}, oq_modrdn = {rs_mods = {rs_modlist = 0x7f758c001688, rs_no_opattrs = 1 '\001'}, rs_deleteoldrdn = 0, rs_newrdn = {bv_len = 0, bv_val = 0x0}, rs_nnewrdn = {bv_len = 0, bv_val = 0x0}, rs_newSup = 0x0, rs_nnewSup = 0x0}, oq_search = {rs_scope = -1946151288, rs_deref = 32629, rs_slimit = 1, rs_tlimit = 0, rs_limit = 0x0, rs_attrsonly = 0, rs_attrs = 0x0, rs_filter = 0x0, rs_filterstr = {bv_len = 0, bv_val = 0x0}}, oq_abandon = { rs_msgid = -1946151288}, oq_cancel = {rs_msgid = -1946151288}, oq_extended = {rs_reqoid = { bv_len = 140142836717192, bv_val = 0x1 <error: Cannot access memory at address 0x1>}, rs_flags = 0, rs_reqdata = 0x0}, oq_pwdexop = {rs_extended = {rs_reqoid = { bv_len = 140142836717192, bv_val = 0x1 <error: Cannot access memory at address 0x1>}, rs_flags = 0, rs_reqdata = 0x0}, rs_old = {bv_len = 0, bv_val = 0x0}, rs_new = { bv_len = 0, bv_val = 0x0}, rs_mods = 0x0, rs_modtail = 0x0}}, o_abandon = 0, o_cancel = 0, o_groups = 0x0, o_do_not_cache = 0 '\000', o_is_auth_check = 0 '\000', o_dont_replicate = 0 '\000', o_acl_priv = ACL_NONE, o_nocaching = 0 '\000', o_delete_glue_parent = 0 '\000', o_no_schema_check = 1 '\001', o_no_subordinate_glue = 0 '\000', o_ctrlflag = '\000' <repeats 14 times>, "\002", '\000' <repeats 16 times>, o_controls = 0x7f75994fa8c0, o_authz = {sai_method = 0, sai_mech = {bv_len = 0, bv_val = 0x0}, sai_dn = {bv_len = 25, bv_val = 0x12f68a0 "cn=ldaproot,dc=cpp,dc=edu"}, sai_ndn = { ---Type <return> to continue, or q <return> to quit--- bv_len = 25, bv_val = 0x12f68f0 "cn=ldaproot,dc=cpp,dc=edu"}, sai_ssf = 0, sai_transport_ssf = 0, sai_tls_ssf = 0, sai_sasl_ssf = 0}, o_ber = 0x0, o_res_ber = 0x0, o_callback = 0x7f758c001990, o_ctrls = 0x0, o_csn = {bv_len = 40, bv_val = 0x7f758c3242a0 "20171101104541.389271Z#000000#004#000000"}, o_private = 0x0, o_extra = {slh_first = 0x7f75994f9420}, o_next = {stqe_next = 0x0}}, ob_hdr = {oh_opid = 0, oh_connid = 2, oh_conn = 0x7f75994fa340, oh_msgid = 0, oh_protocol = 0, oh_tid = 140143060039424, oh_threadctx = 0x7f75994fab90, oh_tmpmemctx = 0x7f758c001150, oh_tmpmfuncs = 0x72e140 <slap_sl_mfuncs>, oh_counters = 0x731640 <slap_counters>, oh_log_prefix = "conn=-1 op=0", '\000' <repeats 243 times>, oh_extensions = 0x0}, ob_controls = { 0x0 <repeats 17 times>, 0x7f75994f9ce0, 0x0 <repeats 14 times>}} op = 0x7f75994fa600 rc = 0 dostop = 0 s = 60 i = <optimized out> defer = 1 fail = 0 freeinfo = 0 be = 0x12f6f70 #8 0x0000000000430bdf in connection_read_thread (ctx=0x7f75994fab90, argv=0x3c) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.45-r2/work/openldap-2.4.45/servers/slapd/connection.c:1296 rc = <optimized out> cri = {op = 0x0, func = 0x4937f0 <do_syncrepl>, arg = 0x1306c20, ctx = 0x7f75994fab90, nullop = <optimized out>} s = 60 #9 0x00007f77a26c67f2 in ldap_int_thread_pool_wrapper (xpool=0x1293710) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.45-r2/work/openldap-2.4.45/libraries/libldap_r/tpool.c:696 pool = 0x1293710 task = 0x7f75d8000b80 work_list = <optimized out> ctx = {ltu_id = 140143060039424, ltu_key = {{ltk_key = 0x42e370 <conn_counter_init>, ---Type <return> to continue, or q <return> to quit--- ltk_data = 0x7f758c001040, ltk_free = 0x42e450 <conn_counter_destroy>}, { ltk_key = 0x484d20 <slap_sl_mem_init>, ltk_data = 0x7f758c001150, ltk_free = 0x484bf0 <slap_sl_mem_destroy>}, {ltk_key = 0x15b8e00, ltk_data = 0x7f758c1011a0, ltk_free = 0x7f779dbe7730 <mdb_reader_free>}, {ltk_key = 0x7f779dbdd5f0 <search_stack>, ltk_data = 0x7f7586fff010, ltk_free = 0x7f779dbdd700 <search_stack_free>}, { ltk_key = 0x7f779dbdd550 <scope_chunk_get>, ltk_data = 0x7f75989fa010, ltk_free = 0x7f779dbdd6d0 <scope_chunk_free>}, {ltk_key = 0x443960 <slap_op_free>, ltk_data = 0x7f75cc315ce0, ltk_free = 0x443930 <slap_op_q_destroy>}, {ltk_key = 0x13b2430, ltk_data = 0x7f758c119050, ltk_free = 0x7f779dbe7730 <mdb_reader_free>}, {ltk_key = 0x0, ltk_data = 0x7f75c8130aa0, ltk_free = 0x0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0x0} <repeats 24 times>}} kctx = <optimized out> keyslot = <optimized out> hash = <optimized out> __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper" #10 0x00007f77a1a443e4 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #11 0x00007f77a17863ed in clone () from /lib64/libc.so.6 No symbol table info available. Core was generated by `/usr/lib64/openldap/slapd -u ldap -g ldap -h ldaps:// ldap://'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x0000000000483eb9 in modify_add_values (e=e@entry=0x7f306e4f6440, mod=mod@entry=0x7f3060000ad0, permissive=0, text=text@entry=0x7f306e4f69a0, textbuf=textbuf@entry=0x7f306e4f64c0 "modify/delete: eduPersonAffiliation: no such attribute", textlen=textlen@entry=256) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.45-r2/work/openldap-2.4.45/servers/slapd/mods.c:61 61 if ( !BER_BVISNULL( &mod->sm_values[mod->sm_numvals] )) { [Current thread is 1 (Thread 0x7f306e4f8700 (LWP 85701))] (gdb) bt full #0 0x0000000000483eb9 in modify_add_values (e=e@entry=0x7f306e4f6440, mod=mod@entry=0x7f3060000ad0, permissive=0, text=text@entry=0x7f306e4f69a0, textbuf=textbuf@entry=0x7f306e4f64c0 "modify/delete: eduPersonAffiliation: no such attribute", textlen=textlen@entry=256) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.45-r2/work/openldap-2.4.45/servers/slapd/mods.c:61 rc = <optimized out> op = 0x4c1265 "add" a = <optimized out> pmod = {sm_desc = <optimized out>, sm_values = 0x0, sm_nvalues = 0x0, sm_numvals = <optimized out>, sm_op = <optimized out>, sm_flags = <optimized out>, sm_type = {bv_len = <optimized out>, bv_val = <optimized out>}} __PRETTY_FUNCTION__ = "modify_add_values" #1 0x00007f3251ec4bc3 in mdb_modify_internal (op=op@entry=0x7f306e4f7600, tid=tid@entry=0xffc950, modlist=0x7f3060000a90, e=e@entry=0x7f306e4f6440, text=text@entry=0x7f306e4f69a0, textbuf=textbuf@entry=0x7f306e4f64c0 "modify/delete: eduPersonAffiliation: no such attribute", textlen=256) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.45-r2/work/openldap-2.4.45/servers/slapd/back-mdb/modify.c:137 rc = <optimized out> err = <optimized out> mod = 0x7f3060000ad0 ml = 0x7f3060000ad0 save_attrs = 0x7f3060001b50 ap = 0x7f3256c82010 glue_attr_delete = <optimized out> got_delete = 0 __PRETTY_FUNCTION__ = "mdb_modify_internal" #2 0x00007f3251ec5f55 in mdb_modify (op=0x7f306e4f7600, rs=0x7f306e4f6980) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.45-r2/work/openldap-2.4.45/servers/slapd/back-mdb/modify.c:624 mdb = 0x7f3256c82010 e = 0x7f3060001b00 manageDSAit = <optimized out> ---Type <return> to continue, or q <return> to quit--- textbuf = "modify/delete: eduPersonAffiliation: no such attribute\000\000\276\247\020`0\177\000\000\000\000\001\000\000\000\000\000\301\247\020`0\177\000\000\276\247\020`0\177\000\000\267\247\020`0\177\000\000\230\r\000`0\177\000\000\000_\246mQo\262.\000\200", '\000' <repeats 15 times>, "vOn0\177\000\000Pf\304\000\000\000\000\000\230\r\000`0\177\000\000\060h\304\000\000\000\000\000\000\200\000\000\000\000\000\000\236\313J\000\000\000\000\000\330k\304\000\000\000\000\000\300\r\000`0\177\000\000\000vOn0\177\000\000"... txn = 0xffc950 opinfo = {moi_oe = {oe_next = {sle_next = 0x7f306e4f6930}, oe_key = 0x7f3256c82010}, moi_txn = 0xffc950, moi_ref = 1, moi_flag = 0 '\000'} moi = 0x7f306e4f6420 dummy = {e_id = 94173, e_name = {bv_len = 31, bv_val = 0x7f3060001ad8 "uid=jplau,ou=user,dc=cpp,dc=edu"}, e_nname = {bv_len = 31, bv_val = 0x7f30600026a0 "uid=jplau,ou=user,dc=cpp,dc=edu"}, e_attrs = 0x7f3078126340, e_ocflags = 256, e_bv = {bv_len = 0, bv_val = 0x0}, e_private = 0x7f3060001b00} preread_ctrl = 0x0 postread_ctrl = 0x0 ctrls = {0x0, 0x2, 0xffffffffffffffff, 0x0, 0x0, 0x7f306e4f6400} num_ctrls = 0 numads = 61 #3 0x000000000049a6c2 in overlay_op_walk (op=op@entry=0x7f306e4f7600, rs=0x7f306e4f6980, which=op_modify, oi=0xc46470, on=<optimized out>) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.45-r2/work/openldap-2.4.45/servers/slapd/backover.c:677 func = <optimized out> rc = <optimized out> #4 0x000000000049a81a in over_op_func (op=0x7f306e4f7600, rs=<optimized out>, which=<optimized out>) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.45-r2/work/openldap-2.4.45/servers/slapd/backover.c:730 oi = <optimized out> on = <optimized out> be = 0xc37f50 db = {bd_info = 0x7f32520dec60 <bi>, bd_self = 0xc37f50, be_ctrls = "\000\001\001\001\000\001\000\000\001\000\000\001\001\000\001\000\000\001", '\000' <repeats 14 times>, "\001", be_flags = 563464, be_restrictops = 0, be_requires = 0, be_ssf_set = {sss_ssf = 0, ---Type <return> to continue, or q <return> to quit--- sss_transport = 0, sss_tls = 0, sss_sasl = 0, sss_update_ssf = 0, sss_update_transport = 0, sss_update_tls = 0, sss_update_sasl = 0, sss_simple_bind = 64}, be_suffix = 0xc36fe0, be_nsuffix = 0xc37660, be_schemadn = {bv_len = 0, bv_val = 0x0}, be_schemandn = {bv_len = 0, bv_val = 0x0}, be_rootdn = {bv_len = 25, bv_val = 0xc37880 "cn=ldaproot,dc=cpp,dc=edu"}, be_rootndn = {bv_len = 25, bv_val = 0xc378d0 "cn=ldaproot,dc=cpp,dc=edu"}, be_rootpw = { bv_len = 14, bv_val = 0xc08290 ""}, be_max_deref_depth = 15, be_def_limit = { lms_t_soft = 3600, lms_t_hard = 0, lms_s_soft = -1, lms_s_hard = 0, lms_s_unchecked = -1, lms_s_pr = 0, lms_s_pr_hide = 0, lms_s_pr_total = 0}, be_limits = 0xc36f10, be_acl = 0xc392a0, be_dfltaccess = ACL_READ, be_extra_anlist = 0x0, be_update_ndn = {bv_len = 0, bv_val = 0x0}, be_update_refs = 0x0, be_pending_csn_list = 0xdf8120, be_pcl_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = { __prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, be_syncinfo = 0xc470b0, be_pb = 0x0, be_cf_ocs = 0x7f32520de820 <mdbocs>, be_private = 0x7f3256c82010, be_next = {stqe_next = 0x0}} cb = {sc_next = 0x7f306e4f6950, sc_response = 0x499a10 <over_back_response>, sc_cleanup = 0x0, sc_private = 0xc46470, sc_writewait = 0x0} sc = <optimized out> rc = 32768 __PRETTY_FUNCTION__ = "over_op_func" #5 0x000000000048bbdf in syncrepl_message_to_op (si=si@entry=0xc477e0, op=op@entry=0x7f306e4f7600, msg=0x7f3060111950) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.45-r2/work/openldap-2.4.45/servers/slapd/syncrepl.c:2423 oes = {oe = {oe_next = {sle_next = 0x0}, oe_key = 0x48b010 <syncrepl_message_to_op>}, oe_si = 0xc477e0} ber = 0x7f3060115190 modlist = 0x7f3060108c00 ls = 0x72e2a0 <accesslog_sc> rs = {sr_type = REP_RESULT, sr_tag = 0, sr_msgid = 0, sr_err = 0, sr_matched = 0x0, sr_text = 0x7f306e4f64c0 "modify/delete: eduPersonAffiliation: no such attribute", sr_ref = 0x0, sr_ctrls = 0x0, sr_un = {sru_search = {r_entry = 0x0, r_attr_flags = 0, r_operational_attrs = 0x0, r_attrs = 0x0, r_nentries = 0, r_v2ref = 0x0}, sru_sasl = {r_sasldata = 0x0}, sru_extended = { r_rspoid = 0x0, r_rspdata = 0x0}}, sr_flags = 0} ---Type <return> to continue, or q <return> to quit--- cb = {sc_next = 0x7f3060000d68, sc_response = 0x48ab80 <null_callback>, sc_cleanup = 0x0, sc_private = 0x0, sc_writewait = 0x0} text = 0x29 <error: Cannot access memory at address 0x29> txtbuf = "@oOn0\177\000\000\261=yV2\177\000\000@oOn0\177\000\000\020lOn0\177\000\000\020'M\000\000\000\000\000\026@yV2\177\000\000@oOn0\177\000\000\020'M\000\000\000\000\000\220jOn0\177\000\000`\254yV2\177\000\000\020'M\000\000\000\000\000@MyV2\177\000\000\020\000\000\000\000\000\000\000\377\377\377\377\377\377\377\377\200jO\017\000\000\000\000\314\370\020`0\177\000\000 lOn0\177\000\000\310\370\020`0\177\000\000\001\000\000\000\000\000\000\000\317\370\020`0\177\000\000\340lOn0\177\000\000\060\251\274\000\000\000\000\000\340w\304\000\000\000\000\000\213\307I\000\000\000\000\000\020", '\000' <repeats 23 times>... bdn = {bv_len = 31, bv_val = 0x7f30601157bc "uid=jplau,ou=user,dc=cpp,dc=edu"} dn = {bv_len = 31, bv_val = 0x7f3060000d68 ""} ndn = {bv_len = 31, bv_val = 0x7f3060000de8 "h\032"} bv = {bv_len = 0, bv_val = 0x0} bv2 = {bv_len = 0, bv_val = 0x0} bvals = 0x7f3060103740 rdn = {bv_len = 0, bv_val = 0x0} sup = {bv_len = 0, bv_val = 0x0} prdn = {bv_len = 0, bv_val = 0x0} nrdn = {bv_len = 0, bv_val = 0x0} psup = {bv_len = 0, bv_val = 0x0} nsup = {bv_len = 0, bv_val = 0x0} rc = 0 deleteOldRdn = 0 freeReqDn = 1 do_graduate = 1 #6 0x000000000048f8bf in do_syncrep2 (op=op@entry=0x7f306e4f7600, si=si@entry=0xc477e0) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.45-r2/work/openldap-2.4.45/servers/slapd/syncrepl.c:1014 match = <optimized out> syncUUID = {{bv_len = 16, bv_val = 0x7f306010e577 ")\263\217MC\265F~\211\377\vB\020", <incomplete sequence \350\240>}, { bv_len = 139845985857504, bv_val = 0x7f3000000001 <error: Cannot access memory at address 0x7f3000000001>}} ---Type <return> to continue, or q <return> to quit--- cookie = {bv_len = 15, bv_val = 0x7f306010e589 "rid=002,sid=001"} rctrls = 0x7f306010ae50 rctrlp = <optimized out> bdn = {bv_len = 44, bv_val = 0x7f306011576a "reqStart=20171101104541.000020Z,cn=accesslog"} si_tag = <optimized out> entry = <optimized out> punlock = -1 syncstate = 1 retdata = 0x0 retoid = 0x0 syncUUIDs = 0x7f30601184b0 len = 15 berbuf = { buffer = "\002\000\001", '\000' <repeats 29 times>, "p\345\020`0\177\000\000\230\345\020`0\177\000\000\230\345\020`0\177", '\000' <repeats 26 times>, "$\243\021t0\177\000\000mZ$U2\177\000\000\340oOn0\177\000\000bJ\035U2\177\000\000\001\000\000\000\000\000\000\000\324?\036U2\177\000\000\215\005\217\022\023\366\017\311\035,D\303~M0\\\311\326\350\341e\204Xa0q\020`0\177\000\000\060\000\000\000\000\000\000\000 \246\020`0\177\000\000\060\000\000\000\000\000\000\000\320rOn0\177\000\000\000rOn0\177\000\000"..., ialign = 65538, lalign = 65538, falign = 9.18382988e-41, dalign = 3.2380074297143616e-319, palign = 0x10002 <error: Cannot access memory at address 0x10002>} ber = 0x7f306e4f6f40 msg = 0x7f3060111950 syncCookie = {ctxcsn = 0x0, sids = 0x0, numcsns = 0, rid = 2, octet_str = {bv_len = 15, bv_val = 0x7f306010f8c0 "rid=002,sid=001"}, sid = 1, sc_next = {stqe_next = 0x0}} syncCookie_req = {ctxcsn = 0x0, sids = 0x0, numcsns = 0, rid = 2, octet_str = {bv_len = 15, bv_val = 0x7f30601044f0 "rid=002,sid=001"}, sid = 1, sc_next = {stqe_next = 0x0}} rc = 0 err = 0 modlist = 0x0 m = 0 tout_p = 0x7f306e4f6c00 tout = {tv_sec = 0, tv_usec = 0} refreshDeletes = 0 ---Type <return> to continue, or q <return> to quit--- empty = "empty" #7 0x0000000000493ad3 in do_syncrepl (ctx=ctx@entry=0x7f306e4f7b90, arg=arg@entry=0xc47c00) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.45-r2/work/openldap-2.4.45/servers/slapd/syncrepl.c:1565 rtask = 0xc47c00 si = 0xc477e0 conn = {c_struct_state = SLAP_C_UNINITIALIZED, c_conn_state = SLAP_C_INVALID, c_conn_idx = -1, c_sd = 0, c_close_reason = 0x0, c_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, c_sb = 0x0, c_starttime = 0, c_activitytime = 0, c_connid = 18446744073709551615, c_peer_domain = {bv_len = 0, bv_val = 0x4c7052 ""}, c_peer_name = {bv_len = 0, bv_val = 0x4c7052 ""}, c_listener = 0x4c9ee0 <dummy_list>, c_sasl_bind_mech = {bv_len = 0, bv_val = 0x0}, c_sasl_dn = { bv_len = 0, bv_val = 0x0}, c_sasl_authz_dn = {bv_len = 0, bv_val = 0x0}, c_authz_backend = 0x0, c_authz_cookie = 0x0, c_authz = {sai_method = 0, sai_mech = {bv_len = 0, bv_val = 0x0}, sai_dn = { bv_len = 0, bv_val = 0x0}, sai_ndn = {bv_len = 0, bv_val = 0x0}, sai_ssf = 0, sai_transport_ssf = 0, sai_tls_ssf = 0, sai_sasl_ssf = 0}, c_protocol = 0, c_ops = { stqh_first = 0x0, stqh_last = 0x0}, c_pending_ops = {stqh_first = 0x0, stqh_last = 0x0}, c_write1_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, c_write1_cv = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}, c_write2_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, c_write2_cv = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}, c_currentber = 0x0, c_writers = 0, c_writing = 0 '\000', c_sasl_bind_in_progress = 0 '\000', c_writewaiter = 0 '\000', c_is_udp = 0 '\000', c_is_tls = 0 '\000', c_needs_tls_accept = 0 '\000', c_sasl_layers = 0 '\000', c_sasl_done = 0 '\000', c_sasl_authctx = 0x0, c_sasl_sockctx = 0x0, c_sasl_extra = 0x0, c_sasl_bindop = 0x0, c_pagedresults_state = {ps_be = 0x0, ps_size = 0, ps_count = 0, ps_cookie = 0, ps_cookieval = {bv_len = 0, bv_val = 0x0}}, c_n_ops_received = 0, ---Type <return> to continue, or q <return> to quit--- c_n_ops_executing = 0, c_n_ops_pending = 0, c_n_ops_completed = 0, c_n_get = 0, c_n_read = 0, c_n_write = 0, c_extensions = 0x0, c_clientfunc = 0x0, c_clientarg = 0x0, c_send_ldap_result = 0x440a60 <slap_send_ldap_result>, c_send_search_entry = 0x4414c0 <slap_send_search_entry>, c_send_search_reference = 0x442a00 <slap_send_search_reference>, c_send_ldap_extended = 0x441190 <slap_send_ldap_extended>, c_send_ldap_intermediate = 0x441330 <slap_send_ldap_intermediate>} opbuf = {ob_op = {o_hdr = 0x7f306e4f7770, o_tag = 102, o_time = 1509533141, o_tincr = 30, o_bd = 0x7f306e4f6690, o_req_dn = {bv_len = 31, bv_val = 0x7f3060109190 "uid=jplau,ou=user,dc=cpp,dc=edu"}, o_req_ndn = {bv_len = 31, bv_val = 0x7f306010d940 "uid=jplau,ou=user,dc=cpp,dc=edu"}, o_request = {oq_add = { rs_modlist = 0x7f3060000a90, rs_e = 0x1}, oq_bind = {rb_method = 1610615440, rb_cred = { bv_len = 1, bv_val = 0x0}, rb_edn = {bv_len = 0, bv_val = 0x0}, rb_ssf = 0, rb_mech = { bv_len = 0, bv_val = 0x0}}, oq_compare = {rs_ava = 0x7f3060000a90}, oq_modify = { rs_mods = {rs_modlist = 0x7f3060000a90, rs_no_opattrs = 1 '\001'}, rs_increment = 0}, oq_modrdn = {rs_mods = {rs_modlist = 0x7f3060000a90, rs_no_opattrs = 1 '\001'}, rs_deleteoldrdn = 0, rs_newrdn = {bv_len = 0, bv_val = 0x0}, rs_nnewrdn = {bv_len = 0, bv_val = 0x0}, rs_newSup = 0x0, rs_nnewSup = 0x0}, oq_search = {rs_scope = 1610615440, rs_deref = 32560, rs_slimit = 1, rs_tlimit = 0, rs_limit = 0x0, rs_attrsonly = 0, rs_attrs = 0x0, rs_filter = 0x0, rs_filterstr = {bv_len = 0, bv_val = 0x0}}, oq_abandon = { rs_msgid = 1610615440}, oq_cancel = {rs_msgid = 1610615440}, oq_extended = {rs_reqoid = { bv_len = 139845745773200, bv_val = 0x1 <error: Cannot access memory at address 0x1>}, rs_flags = 0, rs_reqdata = 0x0}, oq_pwdexop = {rs_extended = {rs_reqoid = { bv_len = 139845745773200, bv_val = 0x1 <error: Cannot access memory at address 0x1>}, rs_flags = 0, rs_reqdata = 0x0}, rs_old = {bv_len = 0, bv_val = 0x0}, rs_new = { bv_len = 0, bv_val = 0x0}, rs_mods = 0x0, rs_modtail = 0x0}}, o_abandon = 0, o_cancel = 0, o_groups = 0x0, o_do_not_cache = 0 '\000', o_is_auth_check = 0 '\000', o_dont_replicate = 0 '\000', o_acl_priv = ACL_NONE, o_nocaching = 0 '\000', o_delete_glue_parent = 0 '\000', o_no_schema_check = 1 '\001', o_no_subordinate_glue = 0 '\000', o_ctrlflag = '\000' <repeats 14 times>, "\002", '\000' <repeats 16 times>, o_controls = 0x7f306e4f78c0, o_authz = {sai_method = 0, sai_mech = {bv_len = 0, bv_val = 0x0}, sai_dn = {bv_len = 25, bv_val = 0xc37880 "cn=ldaproot,dc=cpp,dc=edu"}, sai_ndn = {bv_len = 25, bv_val = 0xc378d0 "cn=ldaproot,dc=cpp,dc=edu"}, sai_ssf = 0, sai_transport_ssf = 0, ---Type <return> to continue, or q <return> to quit--- sai_tls_ssf = 0, sai_sasl_ssf = 0}, o_ber = 0x0, o_res_ber = 0x0, o_callback = 0x7f3060000d98, o_ctrls = 0x0, o_csn = {bv_len = 40, bv_val = 0x7f306010a7a0 "20171101104537.144920Z#000000#004#000000"}, o_private = 0x0, o_extra = {slh_first = 0x7f306e4f6420}, o_next = {stqe_next = 0x0}}, ob_hdr = {oh_opid = 0, oh_connid = 2, oh_conn = 0x7f306e4f7340, oh_msgid = 0, oh_protocol = 0, oh_tid = 139845985863424, oh_threadctx = 0x7f306e4f7b90, oh_tmpmemctx = 0x7f30600008c0, oh_tmpmfuncs = 0x72e140 <slap_sl_mfuncs>, oh_counters = 0x731640 <slap_counters>, oh_log_prefix = "conn=-1 op=0", '\000' <repeats 243 times>, oh_extensions = 0x0}, ob_controls = { 0x0 <repeats 17 times>, 0x7f306e4f6ce0, 0x0 <repeats 14 times>}} op = 0x7f306e4f7600 rc = 0 dostop = 0 s = 19 i = <optimized out> defer = 1 fail = 0 freeinfo = 0 be = 0xc37f50 #8 0x0000000000430bdf in connection_read_thread (ctx=0x7f306e4f7b90, argv=0x13) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.45-r2/work/openldap-2.4.45/servers/slapd/connection.c:1296 rc = <optimized out> cri = {op = 0x0, func = 0x4937f0 <do_syncrepl>, arg = 0xc47c00, ctx = 0x7f306e4f7b90, nullop = <optimized out>} s = 19 #9 0x00007f32569b17f2 in ldap_int_thread_pool_wrapper (xpool=0xbd4700) at /var/lib/portage/tmp/portage/net-nds/openldap-2.4.45-r2/work/openldap-2.4.45/libraries/libldap_r/tpool.c:696 pool = 0xbd4700 task = 0x7f308c000ce0 work_list = <optimized out> ctx = {ltu_id = 139845985863424, ltu_key = {{ltk_key = 0x484d20 <slap_sl_mem_init>, ltk_data = 0x7f30600008c0, ltk_free = 0x484bf0 <slap_sl_mem_destroy>}, {ltk_key = 0xef9ca0, ---Type <return> to continue, or q <return> to quit--- ltk_data = 0x7f30601018f0, ltk_free = 0x7f3251ed2730 <mdb_reader_free>}, { ltk_key = 0x7f3251ec85f0 <search_stack>, ltk_data = 0x7f306c9f6010, ltk_free = 0x7f3251ec8700 <search_stack_free>}, {ltk_key = 0x7f3251ec8550 <scope_chunk_get>, ltk_data = 0x7f306d9f7010, ltk_free = 0x7f3251ec86d0 <scope_chunk_free>}, { ltk_key = 0x42e370 <conn_counter_init>, ltk_data = 0x7f30601091f0, ltk_free = 0x42e450 <conn_counter_destroy>}, {ltk_key = 0x443960 <slap_op_free>, ltk_data = 0x7f30681b0200, ltk_free = 0x443930 <slap_op_q_destroy>}, {ltk_key = 0xcf3410, ltk_data = 0x7f306010bef0, ltk_free = 0x7f3251ed2730 <mdb_reader_free>}, {ltk_key = 0x0, ltk_data = 0x7f308424c830, ltk_free = 0x0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0x0} <repeats 24 times>}} kctx = <optimized out> keyslot = <optimized out> hash = <optimized out> __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper" #10 0x00007f3255d2f3e4 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #11 0x00007f3255a713ed in clone () from /lib64/libc.so.6 No symbol table info available.
On Wed, Nov 01, 2017 at 08:22:15PM +0000, henson@acm.org wrote: > On Fri, Oct 06, 2017 at 06:45:41PM -0700, Paul B. Henson wrote: >> but it looks like the same issue to me. I'm going to go ahead and >> recompile 2.4.45 with the ignore-invalid-accesslog-ops.patch and run >> that for a while and see what happens. > > Sadness :(, not one but two of my boxes crashed today, both running the > patch, with what appears to be the same issue: Hi Paul, I've updated the patch slightly to make sure invalid operations get logged and rejected so some action can be taken. Based on the backtrace, I can't see how an add modification could have ended up without values even with the previous version, however. A patch against master is available here: https://github.com/mistotebe/openldap/tree/its8609 And you can get one that applies on RE24 here: ftp://ftp.openldap.org/incoming/ndrej-Kuznik-20170327-ITS8609-ignore-invalid-accesslog-ops.patch If you still see crashes with the above, I might need you to explore the core file and accesslog a bit further. Regards, -- Ondřej Kuzník Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP
On Thu, Apr 18, 2019 at 12:46:57PM +0200, Ondřej Kuzník wrote: > If you still see crashes with the above, I might need you to explore the > core file and accesslog a bit further. Hmm, actually, I haven't seen a crash for quite a while; I don't recall exactly when the last one was but it's been ages. Right now I'm running 2.4.46 with the patch for ITS#8927 applied, but without the last ITS#8609 patch you had given me. I think the last crash was on 2.4.45 with the ITS#8927 patch. Maybe something in 2.4.46 randomly fixed it?
--On Thursday, April 18, 2019 8:09 PM +0000 henson@acm.org wrote: > On Thu, Apr 18, 2019 at 12:46:57PM +0200, Ond??ej Kuzn??k wrote: > >> If you still see crashes with the above, I might need you to explore the >> core file and accesslog a bit further. > > Hmm, actually, I haven't seen a crash for quite a while; I don't recall > exactly when the last one was but it's been ages. Right now I'm running > 2.4.46 with the patch for ITS#8927 applied, but without the last > ITS#8609 patch you had given me. > > I think the last crash was on 2.4.45 with the ITS#8927 patch. Maybe > something in 2.4.46 randomly fixed it? Hi Paul, Thanks for the follow up. I'm going to go ahead and close this ITS. If you do happen to see a crash again in the future, please follow up and we'll address it at that time. Regards, Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>
Apparently fixed in 2.4.46 by other fixes.
changed notes changed state Open to Closed