[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: (ITS#8460) Invalid pointer free



quanah@zimbra.com wrote:
> --On Friday, July 08, 2016 12:01 AM +0000 quanah@openldap.org wrote:
>
>> Full_Name: Quanah Gibson-Mount
>> Version: 2.4.44+ITS8432
>> OS: Linux 3.13
>> URL: ftp://ftp.openldap.org/incoming/
>> Submission from: (NULL) (75.111.52.177)
>
> Also seeing this in 2.4.44 w/o ITS 8432, so not related to that fix.
> Hitting multiple customers.  Here's a backtrace from a different client.
> See Thread 1 Frame 11 or so.

The actual bug here is not in 2.4 at all, it's due to a 2.5 patch 
2d5996ac603391ddbd618425f88eb13e5e0e2cc0 that you backported into your 2.4 
build. Which explains why no other 2.4.44 users have hit it.

More comments inline below:

> Thread 1 (Thread 0x7f7939906700 (LWP 1946)):
> #0  0x000000344a2325e5 in raise () from /lib64/libc.so.6
> No symbol table info available.
> #1  0x000000344a233dc5 in abort () from /lib64/libc.so.6
> No symbol table info available.
> #2  0x00007f8cf3873f55 in tcmalloc::Log (mode=tcmalloc::kCrash,
> filename=<value optimized out>, line=<value optimized out>, a=..., b=...,
> c=..., d=...) at src/internal_logging.cc:120
>          state = {static kBufSize = -56, p_ = 0x7f7939903e75 "", end_ =
> 0x7f7939903ef8 "\017 \210\363\214\177",
>            buf_ = "src/tcmalloc.cc:278] Attempt to free invalid pointer
> 0x7f7aa5850ad0 \n\000\000\000\a\000\000\000\000\000\000\000\000
> \206\363\214\177\000\000\240\341\340I4\000\000\000\005\000\000\000y\177",
> '\000' <repeats 18 times>"\220,
> I\206\363\214\177\000\000\000\000\000\000\000\000\000\000
> \337\357\000\000\000\000\000\320\n\205\245z\177\000\000\210\271\252\363\214\177\000\000\300i\220\071y\177\000\000\325I\341I4\000\000\000\003\000\000\000y\177\000\000\000\000\000\000\000\000\000\000\026\001\000\000\000\000\000"}
>          msglen = 69
>          first_crash = true
> #3  0x00007f8cf386f3f3 in (anonymous namespace)::InvalidFree (ptr=<value
> optimized out>) at src/tcmalloc.cc:278
> No locals.
> #4  0x00007f8cf387fe25 in free_null_or_invalid (ptr=0x7f7aa5850ad0) at
> src/tcmalloc.cc:1141
> No locals.
> #5  do_free_helper (ptr=0x7f7aa5850ad0) at src/tcmalloc.cc:1185
>          span = <value optimized out>
>          p = <value optimized out>
>          cl = <value optimized out>
> ---Type <return> to continue, or q <return> to quit---
>          invalid_free_fn = 0x7f8cf386f370 <(anonymous
> namespace)::InvalidFree(void*)>
> #6  do_free_with_callback (ptr=0x7f7aa5850ad0) at src/tcmalloc.cc:1225
>          heap = 0xefdf20
>          invalid_free_fn = 0x7f8cf386f370 <(anonymous
> namespace)::InvalidFree(void*)>
> #7  do_free (ptr=0x7f7aa5850ad0) at src/tcmalloc.cc:1234
> No locals.
> #8  tc_free (ptr=0x7f7aa5850ad0) at src/tcmalloc.cc:1585
> No locals.
> #9  0x00007f8cf33f77d9 in ber_memfree_x (p=0x7f7aa5850ad0, ctx=0x0) at
> memory.c:152
>          __PRETTY_FUNCTION__ = "ber_memfree_x"
> #10 0x00000000004af21b in slap_sl_free (ptr=0x7f7aa5850ad0, ctx=0x3be91c0)
> at sl_malloc.c:503
>          sh = 0x3be91c0
>          size = 25450432
>          p = 0x7f7aa5850ad0
>          nextp = 0x44770f
>          tmpp = 0x7f79399040e0
>          __PRETTY_FUNCTION__ = "slap_sl_free"
> #11 0x00007f8cef5ded30 in accesslog_entry (op=0x7f79399053f0,
> rs=0x7f7939904f70, logop=2, op2=0x7f79399042a0) at accesslog.c:1332

accesslog.c:1332 is freeing a ntimestamp value that was just generated.

>          on = 0x1a03c20
>          li = 0x19ebb60
>          rdnbuf = "reqStart=20160722141557.1000000\000PD\220\071y\177"
>          nrdnbuf =
> "reqStart=V\313/\000\177\000\000\000\000\000\000\000\000\000\000lB\220\071y\177\000\000\000\000\205\245z\177"
>          rdn = {bv_len = 31, bv_val = 0x7f7939904150
> "reqStart=20160722141557.1000000"}
>          nrdn = {bv_len = 17, bv_val = 0x7f7939904120 "reqStart=V\313/"}
>          timestamp = {bv_len = 22, bv_val = 0x7f7939904159
> "20160722141557.1000000"}

This timestamp has a 7 digit microseconds portion and is missing its trailing 
'Z' timezone identifier. Since it's recording microseconds, it should never 
have more than 6 digits. There's a buffer overrun here due to this out of 
bounds value. The timestamp came from op->o_time and op->o_tincr.

>          ntimestamp = {bv_len = 8, bv_val = 0x7f7aa5850ad0 <Address
> 0x7f7aa5850ad0 out of bounds>}
>          bv = {bv_len = 140158633526384, bv_val = 0x7f7939904490 "\002"}
>          lo = 0x7f8cef7e5b50
>          e = 0x1973d68
> #12 0x00007f8cef5df684 in accesslog_response (op=0x7f79399053f0,
> rs=0x7f7939904f70) at accesslog.c:1528
>          on = 0x1a03c20
>          li = 0x19ebb60
>          a = 0x7f7aa5850810
>          last_attr = 0x8
>          m = 0x7f7939904488
>          b = 0x7f7aa1873ff8
>          uuid = {bv_len = 36, bv_val = 0x13638d30
> "7e6927a6-1cda-1030-907b-0f0bf0d58d6f"}
>          i = 0
>          logop = 2
>          do_graduate = 0
>          lo = 0x7f8cef7e5b50
>          e = 0x0
>          old = 0x0
>          e_uuid = 0x0
>          timebuf =
> "\300\210\244\001\000\000\000\000\000`\277\001\000\000\000\000\240D\220\071y\177\000\000U\313/\000\000"
>          bv = {bv_len = 64424509440, bv_val = 0x7f7939904520
> "pO\220\071y\177"}
>          ptr = 0x1bf6088 ""
>          vals = 0x1a48800
>          op2 = {o_hdr = 0x0, o_tag = 0, o_time = 0, o_tincr = 0, o_bd = 0x0,
> o_req_dn = {bv_len = 0, bv_val = 0x0}, o_req_ndn = {bv_len = 0, bv_val =
> 0x0}, o_request = {oq_add = {rs_modlist = 0x0, rs_e = 0x0}, oq_bind =
> {rb_method = 0, rb_cred = {
>                  bv_len = 0, bv_val = 0x0}, rb_edn = {bv_len = 0, bv_val =
> 0x0}, rb_ssf = 0, rb_mech = {bv_len = 0, bv_val = 0x0}}, oq_compare =
> {rs_ava = 0x0}, oq_modify = {rs_mods = {rs_modlist = 0x0, rs_no_opattrs = 0
> '\000'}, rs_increment = 0},
>              oq_modrdn = {rs_mods = {rs_modlist = 0x0, rs_no_opattrs = 0
> '\000'}, rs_deleteoldrdn = 0, rs_newrdn = {bv_len = 0, bv_val = 0x0},
> rs_nnewrdn = {bv_len = 0, bv_val = 0x0}, rs_newSup = 0x0, rs_nnewSup =
> 0x0}, oq_search = {rs_scope = 0,
>                rs_deref = 0, rs_slimit = 0, rs_tlimit = 0, rs_limit = 0x0,
> rs_attrsonly = 0, rs_attrs = 0x0, rs_filter = 0x0, rs_filterstr = {bv_len =
> 0, bv_val = 0x0}}, oq_abandon = {rs_msgid = 0}, oq_cancel = {rs_msgid = 0},
> oq_extended = {rs_reqoid = {
>                  bv_len = 0, bv_val = 0x0}, rs_flags = 0, rs_reqdata = 0x0},
> oq_pwdexop = {rs_extended = {rs_reqoid = {bv_len = 0, bv_val = 0x0},
> rs_flags = 0, rs_reqdata = 0x0}, rs_old = {bv_len = 0, bv_val = 0x0},
> rs_new = {bv_len = 0, bv_val = 0x0},
>                rs_mods = 0x0, rs_modtail = 0x0}}, o_abandon = 0, o_cancel =
> 0, o_groups = 0x0, o_do_not_cache = 0 '\000', o_is_auth_check = 0 '\000',
> o_dont_replicate = 0 '\000', o_acl_priv = ACL_NONE, o_nocaching = 0 '\000',
>            o_delete_glue_parent = 0 '\000', o_no_schema_check = 0 '\000',
> o_no_subordinate_glue = 0 '\000', o_ctrlflag = '\000' <repeats 31 times>,
> o_controls = 0x0, o_authz = {sai_method = 0, sai_mech = {bv_len = 0, bv_val
> = 0x0}, sai_dn = {bv_len = 0,
>                bv_val = 0x0}, sai_ndn = {bv_len = 0, bv_val = 0x0}, sai_ssf
> = 0, sai_transport_ssf = 0, sai_tls_ssf = 0, sai_sasl_ssf = 0}, o_ber =
> 0x0, o_res_ber = 0x0, o_callback = 0x0, o_ctrls = 0x0, o_csn = {bv_len = 0,
> bv_val = 0x0}, o_private = 0x0,
>            o_extra = {slh_first = 0x0}, o_next = {stqe_next = 0x0}}
>          rs2 = {sr_type = REP_RESULT, sr_tag = 0, sr_msgid = 0, sr_err = 0,
> sr_matched = 0x0, sr_text = 0x0, sr_ref = 0x0, sr_ctrls = 0x0, sr_un =
> {sru_search = {r_entry = 0x0, r_attr_flags = 0, r_operational_attrs = 0x0,
> r_attrs = 0x0, r_nentries = 0,
>                r_v2ref = 0x0}, sru_sasl = {r_sasldata = 0x0}, sru_extended =
> {r_rspoid = 0x0, r_rspdata = 0x0}}, sr_flags = 0}
> ---Type <return> to continue, or q <return> to quit---
> #13 0x00000000004cd56e in over_back_response (op=0x7f79399053f0,
> rs=0x7f7939904f70) at backover.c:237
>          oi = 0x1b72f00
>          on = 0x1a03c20
>          rc = 32768
>          be = 0x7f7939904c30
>          db = {bd_info = 0x1a03c20, bd_self = 0x1833d40, be_ctrls =
> "\000\001\001\001\000\001\000\000\001\000\000\001\001\000\001\000\000\001",
> '\000' <repeats 14 times>, "\001", be_flags = 563464, be_restrictops = 0,
> be_requires = 0, be_ssf_set = {
>              sss_ssf = 0, sss_transport = 0, sss_tls = 0, sss_sasl = 0,
> sss_update_ssf = 0, sss_update_transport = 0, sss_update_tls = 0,
> sss_update_sasl = 0, sss_simple_bind = 0}, be_suffix = 0x1b5e960,
> be_nsuffix = 0x1b5e920, be_schemadn = {bv_len = 0,
>              bv_val = 0x0}, be_schemandn = {bv_len = 0, bv_val = 0x0},
> be_rootdn = {bv_len = 9, bv_val = 0x1ba60d0 "cn=config"}, be_rootndn =
> {bv_len = 9, bv_val = 0x1ba60f0 "cn=config"}, be_rootpw = {bv_len = 0,
> bv_val = 0x0}, be_max_deref_depth = 15,
>            be_def_limit = {lms_t_soft = -1, lms_t_hard = 0, lms_s_soft = -1,
> lms_s_hard = 0, lms_s_unchecked = -1, lms_s_pr = 0, lms_s_pr_hide = 0,
> lms_s_pr_total = 0}, be_limits = 0x0, be_acl = 0x1ddb800, be_dfltaccess =
> ACL_READ, be_extra_anlist = 0x0,
>            be_update_ndn = {bv_len = 0, bv_val = 0x0}, be_update_refs = 0x0,
> be_pending_csn_list = 0x1fa3570, be_pcl_mutex = {__data = {__lock = 0,
> __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __list =
> {__prev = 0x0, __next = 0x0}},
>              __size = '\000' <repeats 39 times>, __align = 0}, be_syncinfo =
> 0x1999e40, be_pb = 0x0, be_cf_ocs = 0x7f8cefe67180, be_private = 0x1ede000,
> be_next = {stqe_next = 0x0}}
> #14 0x0000000000450592 in slap_response_play (op=0x7f79399053f0,
> rs=0x7f7939904f70) at result.c:537
>          sc_next = 0x7f7939904fe0
>          sc_nextp = 0x7f7939904c00
>          rc = 32768
>          sc = 0x1210c0a8
>          scp = 0x1210c0a8
> #15 0x00000000004507b7 in send_ldap_response (op=0x7f79399053f0,
> rs=0x7f7939904f70) at result.c:612
>          berbuf = {
>            buffer =
> "\000\000\000\000\000\000\000\000\260\301\020\022\000\000\000\000\200I\220\071y\177\000\000\360S\220\071y\177\000\000\n",
> '\000' <repeats 15 times>, "@\311b\001\000\000\000\000\060L\220\071y\177",
> '\000' <repeats 18 times>,
> "@H\220\071y\177\000\000\066\341\302\357\214\177\000\000\300\022(\245z\177\000\000\345\063\304\357\214\177\000\000\240H\220\071y\177\000\000\200H\220\071y\177\000\000\200I\220\071y\177\000\000\360S\220\071y\177\000\000\000\340\355\001\000\000\000\000\000\340\355\001\000\000\000\000\003\000\000\000\000\000\000\000\030\002",
> '\000' <repeats 14 times>"\351,
> \022(\245z\177\000\000t\021(\245z\177\000\000\000`\277\001\000\000\000\000pH\220\071y\177\000\000P\374\261?y\177\000\000\300i\220\071y\177\000\000\331w?\363\214\177",
> '\000' <repeats 17 times>, ialign = 0, lalign = 0, falign = 0, dalign = 0,
> palign = 0x0}
>          ber = 0x7f7939904770
>          rc = 0
>          bytes = 428045504
>          __PRETTY_FUNCTION__ = "send_ldap_response"
> #16 0x0000000000451701 in slap_send_ldap_result (op=0x7f79399053f0,
> rs=0x7f7939904f70) at result.c:891
>          tmp = 0x0
>          otext = 0x0
>          oref = 0x0
>          __PRETTY_FUNCTION__ = "slap_send_ldap_result"
> #17 0x00007f8cefc30b1e in mdb_modify (op=0x7f79399053f0, rs=0x7f7939904f70)
> at modify.c:708
>          mdb = 0x1ede000
>          e = 0x1210c160
>          manageDSAit = 2
>          textbuf =
> "\017\000\000\000\000\000\000\000\377\377\377\377\377\377\377\377\250\300\020\022\000\000\000\000\371\377\377\377\377\377\377\377\240J\220\071y\177\000\000P\374\261?y\177\000\000\300i\220\071y\177\000\000\004\000\000\000\000\000\000\000\a\000\000\000\000\000\000\000\335\bM\000\000\000\000\000\240J\220\071y\177\000\000\070U\220\071y\177\000\000\003\000\000\000\000\000\000\000^\300\020\022\000\000\000\000\320\300\020\022\000\000\000\000a\300\020\022\000\000\000\000^\300\020\022\000\000\000\000\001
> \000\000\000\001\000\000\000\250\300\020\022\000\000\000\000pO\220\071y\177\000\000pK\220\071y\177\000\000i\377~\357\214\177\000\000pO\220\071y\177\000\000\360S\220\071y\177\000\000\320\300\020\022\000\000\000\000\360S\220\071y\177\000\000\310\302\020\022\000\000\000\000(T\220\071y\177\000\000pK\220\071y\177\000\000l\021^\357\214\177\000\000pO\220\071y\177\000\000\360S\220\071y\177\000"
>          textlen = 256
>          txn = 0x0
>          opinfo = {moi_oe = {oe_next = {sle_next = 0x0}, oe_key = 0x0},
> moi_txn = 0x1bf6000, moi_ref = 1, moi_flag = 0 '\000'}
>          moi = 0x7f79399049e0
>          dummy = {e_id = 0, e_name = {bv_len = 0, bv_val = 0xb997b08 ""},
> e_nname = {bv_len = 0, bv_val = 0x1210c520 ""}, e_attrs = 0x1845a40,
> e_ocflags = 82208, e_bv = {bv_len = 0, bv_val = 0x0}, e_private =
> 0x1210c160}
>          preread_ctrl = 0x0
>          postread_ctrl = 0x0
>          ctrls = {0x0, 0x344a2a517e, 0x7f7939904f70, 0x7f79399053f0,
> 0x7f7939904a45, 0x0}
>          num_ctrls = 0
>          numads = 1063
> #18 0x00000000004ce4bb in overlay_op_walk (op=0x7f79399053f0,
> rs=0x7f7939904f70, which=op_modify, oi=0x1b72f00, on=0x0) at backover.c:677
>          func = 0x7f8cefe67478
>          rc = 32768
> #19 0x00000000004ce6e8 in over_op_func (op=0x7f79399053f0,
> rs=0x7f7939904f70, which=op_modify) at backover.c:730
>          oi = 0x1b72f00
>          on = 0x1a041c0
>          be = 0x1833d40
>          db = {bd_info = 0x7f8cefe67420, bd_self = 0x1833d40, be_ctrls =
> "\000\001\001\001\000\001\000\000\001\000\000\001\001\000\001\000\000\001",
> '\000' <repeats 14 times>, "\001", be_flags = 563464, be_restrictops = 0,
> be_requires = 0, be_ssf_set = {
>              sss_ssf = 0, sss_transport = 0, sss_tls = 0, sss_sasl = 0,
> sss_update_ssf = 0, sss_update_transport = 0, sss_update_tls = 0,
> sss_update_sasl = 0, sss_simple_bind = 0}, be_suffix = 0x1b5e960,
> be_nsuffix = 0x1b5e920, be_schemadn = {bv_len = 0,
> ---Type <return> to continue, or q <return> to quit---
>              bv_val = 0x0}, be_schemandn = {bv_len = 0, bv_val = 0x0},
> be_rootdn = {bv_len = 9, bv_val = 0x1ba60d0 "cn=config"}, be_rootndn =
> {bv_len = 9, bv_val = 0x1ba60f0 "cn=config"}, be_rootpw = {bv_len = 0,
> bv_val = 0x0}, be_max_deref_depth = 15,
>            be_def_limit = {lms_t_soft = -1, lms_t_hard = 0, lms_s_soft = -1,
> lms_s_hard = 0, lms_s_unchecked = -1, lms_s_pr = 0, lms_s_pr_hide = 0,
> lms_s_pr_total = 0}, be_limits = 0x0, be_acl = 0x1ddb800, be_dfltaccess =
> ACL_READ, be_extra_anlist = 0x0,
>            be_update_ndn = {bv_len = 0, bv_val = 0x0}, be_update_refs = 0x0,
> be_pending_csn_list = 0x1fa3570, be_pcl_mutex = {__data = {__lock = 0,
> __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __list =
> {__prev = 0x0, __next = 0x0}},
>              __size = '\000' <repeats 39 times>, __align = 0}, be_syncinfo =
> 0x1999e40, be_pb = 0x0, be_cf_ocs = 0x7f8cefe67180, be_private = 0x1ede000,
> be_next = {stqe_next = 0x0}}
>          cb = {sc_next = 0x7f7939904fe0, sc_response = 0x4cd492
> <over_back_response>, sc_cleanup = 0, sc_writewait = 0, sc_private =
> 0x1b72f00}
>          sc = 0x65fc800
>          rc = 32768
>          __PRETTY_FUNCTION__ = "over_op_func"
> #20 0x00000000004ce824 in over_op_modify (op=0x7f79399053f0,
> rs=0x7f7939904f70) at backover.c:769
> No locals.
> #21 0x00000000004c12a8 in syncrepl_updateCookie (si=0x1999e40,
> op=0x7f79399053f0, syncCookie=0x7f7939905230) at syncrepl.c:3885
>          be = 0x1833d40
>          mod = {sml_mod = {sm_desc = 0x162c940, sm_values = 0x65fadc0,
> sm_nvalues = 0x0, sm_numvals = 3, sm_op = 2, sm_flags = 1, sm_type =
> {bv_len = 10, bv_val = 0x1615330 "contextCSN"}}, sml_next = 0x0}
>          first = {bv_len = 40, bv_val = 0xba3a6f0
> "20160722141557.997975Z#000000#001#000000"}
>          sc = {ctxcsn = 0x65fadc0, sids = 0xb786cd0, numcsns = 3, rid = 0,
> octet_str = {bv_len = 0, bv_val = 0x0}, sid = 0, sc_next = {stqe_next =
> 0x0}}
>          syn = 0x1823980
>          rc = 0
>          i = 1
>          j = 1
>          changed = 1
>          len = 40
>          cb = {sc_next = 0x1210c078, sc_response = 0x4c2d92 <null_callback>,
> sc_cleanup = 0, sc_writewait = 0, sc_private = 0x1999e40}
>          rs_modify = {sr_type = REP_RESULT, sr_tag = 103, sr_msgid = 0,
> sr_err = 0, sr_matched = 0x0, sr_text = 0x0, sr_ref = 0x0, sr_ctrls = 0x0,
> sr_un = {sru_search = {r_entry = 0x0, r_attr_flags = 0, r_operational_attrs
> = 0x0, r_attrs = 0x0,
>                r_nentries = 0, r_v2ref = 0x0}, sru_sasl = {r_sasldata =
> 0x0}, sru_extended = {r_rspoid = 0x0, r_rspdata = 0x0}}, sr_flags = 0}
>          __PRETTY_FUNCTION__ = "syncrepl_updateCookie"
> #22 0x00000000004b7008 in do_syncrep2 (op=0x7f79399053f0, si=0x1999e40) at
> syncrepl.c:1012
>          match = 4443350
>          syncUUID = {{bv_len = 16, bv_val = 0x35c6287
> "\215\361\036\352\344b\020\065\236\334;\265\032\250!\025"}, {bv_len = 0,
> bv_val = 0xb39905620 <Address 0xb39905620 out of bounds>}}
>          cookie = {bv_len = 60, bv_val = 0x35c6299
> "rid=100,sid=001,csn=20160722141557.997975Z#000000#001#000000"}
>          rctrls = 0xcd631f0
>          rctrlp = 0x136188a0
>          bdn = {bv_len = 44, bv_val = 0xb8e5a09
> "reqStart=20160722141557.997904Z,cn=accesslog"}
>          si_tag = 140158633532208
>          entry = 0x344a58d440
>          punlock = 0
>          syncstate = 1
>          retdata = 0x1c
>          retoid = 0x7f7939905758 ""
>          syncUUIDs = 0x7f7939905720
>          len = 60
>          berbuf = {
>            buffer = "\002\000\001", '\000' <repeats 29 times>"\200,
> b\\\003\000\000\000\000\325b\\\003\000\000\000\000\325b\\\003", '\000'
> <repeats 28 times>, "
> S\220\071y\177\000\000\000\000\000\000\000\000\000\000\360R\220\071y\177\000\000\274\270\036\315\375\177\000\000\360S\220\071y\177\000\000\266\034a\363\214\177\000\000\060S\220\071y\177\000\000\000\226u\000\000\000\000\000\060S\220\071y\177\000\000QZE\000\000\000\000\000\bT\220\071y\177\000\000\000T\220\071y\177\000\000\235*\222W\000\000\000\000\266\034a\363\214\177\000\000\200S\220\071y\177\000\000\310a\357\003\000\000\000\000\300S\220\071y\177\000\000\026oc\363\214\177\000\000\360S\220\071y\17
> 7\000\000\354S\220\071y\177\000\000\000\000\000\000\001\000\000\000\360{\215\003\000\000\000",
> ialign = 65538,
>            lalign = 65538, falign = 9.18382988e-41, dalign =
> 3.2380074297143616e-319, palign = 0x10002 <Address 0x10002 out of bounds>}
>          ber = 0x7f7939905270
>          msg = 0x65fc640
>          syncCookie = {ctxcsn = 0x196d5640, sids = 0xb995e80, numcsns = 1,
> rid = 100, octet_str = {bv_len = 60, bv_val = 0x187b0d40
> "rid=100,sid=001,csn=20160722141557.997975Z#000000#001#000000"}, sid = 1,
> sc_next = {stqe_next = 0x0}}
>          syncCookie_req = {ctxcsn = 0xe359bc0, sids = 0xe2fd5b0, numcsns =
> 3, rid = 100, octet_str = {bv_len = 0, bv_val = 0x0}, sid = 2, sc_next =
> {stqe_next = 0x0}}
>          rc = 0
>          err = 0
>          modlist = 0x0
>          m = 32633
>          tout_p = 0x7f79399051c0
>          tout = {tv_sec = 0, tv_usec = 0}
>          refreshDeletes = 0
>          empty = "empty"
>          __PRETTY_FUNCTION__ = "do_syncrep2"
> #23 0x00000000004b9177 in do_syncrepl (ctx=0x7f7939905b30, arg=0x1638fa0)
> at syncrepl.c:1560
> ---Type <return> to continue, or q <return> to quit---
>          rtask = 0x1638fa0
>          si = 0x1999e40
>          conn = {c_struct_state = SLAP_C_UNINITIALIZED, c_conn_state =
> SLAP_C_INVALID, c_conn_idx = -1, c_sd = 0, c_close_reason = 0x0, c_mutex =
> {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0,
> __spins = 0, __list = {__prev = 0x0,
>                  __next = 0x0}}, __size = '\000' <repeats 39 times>, __align
> = 0}, c_sb = 0x0, c_starttime = 0, c_activitytime = 0, c_connid =
> 18446744073709551615, c_peer_domain = {bv_len = 0, bv_val = 0x4f2c70 ""},
> c_peer_name = {bv_len = 0,
>              bv_val = 0x4f2c70 ""}, c_listener = 0x4fad40, c_sasl_bind_mech
> = {bv_len = 0, bv_val = 0x0}, c_sasl_dn = {bv_len = 0, bv_val = 0x0},
> c_sasl_authz_dn = {bv_len = 0, bv_val = 0x0}, c_authz_backend = 0x0,
> c_authz_cookie = 0x0, c_authz = {
>              sai_method = 0, sai_mech = {bv_len = 0, bv_val = 0x0}, sai_dn =
> {bv_len = 0, bv_val = 0x0}, sai_ndn = {bv_len = 0, bv_val = 0x0}, sai_ssf =
> 0, sai_transport_ssf = 0, sai_tls_ssf = 0, sai_sasl_ssf = 0}, c_protocol =
> 0, c_ops = {stqh_first = 0x0,
>              stqh_last = 0x0}, c_pending_ops = {stqh_first = 0x0, stqh_last
> = 0x0}, c_write1_mutex = {__data = {__lock = 0, __count = 0, __owner = 0,
> __nusers = 0, __kind = 0, __spins = 0, __list = {__prev = 0x0, __next =
> 0x0}},
>              __size = '\000' <repeats 39 times>, __align = 0}, c_write1_cv =
> {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0,
> __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0},
> __size = '\000' <repeats 47 times>,
>              __align = 0}, c_write2_mutex = {__data = {__lock = 0, __count =
> 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __list = {__prev =
> 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0},
> c_write2_cv = {__data = {
>                __lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0,
> __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0},
> __size = '\000' <repeats 47 times>, __align = 0}, c_currentber = 0x0,
> c_writers = 0, c_writing = 0 '\000',
>            c_sasl_bind_in_progress = 0 '\000', c_writewaiter = 0 '\000',
> c_is_tls = 0 '\000', c_needs_tls_accept = 0 '\000', c_sasl_layers = 0
> '\000', c_sasl_done = 0 '\000', c_sasl_authctx = 0x0, c_sasl_sockctx = 0x0,
> c_sasl_extra = 0x0,
>            c_sasl_bindop = 0x0, c_pagedresults_state = {ps_be = 0x0, ps_size
> = 0, ps_count = 0, ps_cookie = 0, ps_cookieval = {bv_len = 0, bv_val =
> 0x0}}, c_n_ops_received = 0, c_n_ops_executing = 0, c_n_ops_pending = 0,
> c_n_ops_completed = 0, c_n_get = 0,
>            c_n_read = 0, c_n_write = 0, c_extensions = 0x0, c_clientfunc =
> 0, c_clientarg = 0x0, c_send_ldap_result = 0x4512ec
> <slap_send_ldap_result>, c_send_search_entry = 0x4521d0
> <slap_send_search_entry>,
>            c_send_search_reference = 0x454280 <slap_send_search_reference>,
> c_send_ldap_extended = 0x451c92 <slap_send_ldap_extended>,
> c_send_ldap_intermediate = 0x451fad <slap_send_ldap_intermediate>}
>          opbuf = {ob_op = {o_hdr = 0x7f7939905560, o_tag = 102, o_time =
> 1469196957, o_tincr = 1000000, o_bd = 0x7f7939904530, o_req_dn = {bv_len =
> 0, bv_val = 0x160d058 ""}, o_req_ndn = {bv_len = 0, bv_val = 0x160d058 ""},
> o_request = {oq_add = {

Here we see the offending o_tincr = 1000000

>                  rs_modlist = 0x7f7939905060, rs_e = 0x1}, oq_bind =
> {rb_method = 965759072, rb_cred = {bv_len = 1, bv_val = 0x0}, rb_edn =
> {bv_len = 0, bv_val = 0x0}, rb_ssf = 0, rb_mech = {bv_len = 0, bv_val =
> 0x0}}, oq_compare = {
>                  rs_ava = 0x7f7939905060}, oq_modify = {rs_mods =
> {rs_modlist = 0x7f7939905060, rs_no_opattrs = 1 '\001'}, rs_increment = 0},
> oq_modrdn = {rs_mods = {rs_modlist = 0x7f7939905060, rs_no_opattrs = 1
> '\001'}, rs_deleteoldrdn = 0, rs_newrdn = {
>                    bv_len = 0, bv_val = 0x0}, rs_nnewrdn = {bv_len = 0,
> bv_val = 0x0}, rs_newSup = 0x0, rs_nnewSup = 0x0}, oq_search = {rs_scope =
> 965759072, rs_deref = 32633, rs_slimit = 1, rs_tlimit = 0, rs_limit = 0x0,
> rs_attrsonly = 0, rs_attrs = 0x0,
>                  rs_filter = 0x0, rs_filterstr = {bv_len = 0, bv_val =
> 0x0}}, oq_abandon = {rs_msgid = 965759072}, oq_cancel = {rs_msgid =
> 965759072}, oq_extended = {rs_reqoid = {bv_len = 140158633529440, bv_val =
> 0x1 <Address 0x1 out of bounds>},
>                  rs_flags = 0, rs_reqdata = 0x0}, oq_pwdexop = {rs_extended
> = {rs_reqoid = {bv_len = 140158633529440, bv_val = 0x1 <Address 0x1 out of
> bounds>}, rs_flags = 0, rs_reqdata = 0x0}, rs_old = {bv_len = 0, bv_val =
> 0x0}, rs_new = {bv_len = 0,
>                    bv_val = 0x0}, rs_mods = 0x0, rs_modtail = 0x0}},
> o_abandon = 0, o_cancel = 0, o_groups = 0x0, o_do_not_cache = 0 '\000',
> o_is_auth_check = 0 '\000', o_dont_replicate = 1 '\001', o_acl_priv =
> ACL_NONE, o_nocaching = 0 '\000',
>              o_delete_glue_parent = 0 '\000', o_no_schema_check = 1 '\001',
> o_no_subordinate_glue = 0 '\000', o_ctrlflag = '\000' <repeats 14 times>,
> "\002", '\000' <repeats 16 times>, o_controls = 0x7f79399056a8, o_authz =
> {sai_method = 0, sai_mech = {
>                  bv_len = 0, bv_val = 0x0}, sai_dn = {bv_len = 9, bv_val =
> 0x1ba60d0 "cn=config"}, sai_ndn = {bv_len = 9, bv_val = 0x1ba60f0
> "cn=config"}, sai_ssf = 0, sai_transport_ssf = 0, sai_tls_ssf = 0,
> sai_sasl_ssf = 0}, o_ber = 0x0, o_res_ber = 0x0,
>              o_callback = 0x7f7939904c00, o_ctrls = 0x0, o_csn = {bv_len =
> 40, bv_val = 0x1210c040 "20160722141557.997975Z#000000#001#000000"},
> o_private = 0x0, o_extra = {slh_first = 0x0}, o_next = {stqe_next = 0x0}},
> ob_hdr = {oh_opid = 0,
>              oh_connid = 100, oh_conn = 0x7f79399057b0, oh_msgid = 0,
> oh_protocol = 0, oh_tid = 140158633535232, oh_threadctx = 0x7f7939905b30,
> oh_tmpmemctx = 0x3be91c0, oh_tmpmfuncs = 0x757640, oh_counters = 0x75ab80,
>              oh_log_prefix = "conn=-1 op=0", '\000' <repeats 243 times>},
> ob_controls = {0x0 <repeats 17 times>, 0x7f7939905230, 0x0 <repeats 14
> times>}}
>          op = 0x7f79399053f0
>          rc = 0
>          dostop = 0
>          s = 10
>          i = 1
>          defer = 1
>          fail = 0
>          freeinfo = 0
>          be = 0x1833d40
> #24 0x000000000043ae29 in connection_read_thread (ctx=0x7f7939905b30,
> argv=0xa) at connection.c:1273
>          rc = 0
>          cri = {op = 0x0, func = 0x4b8c4f <do_syncrepl>, arg = 0x1638fa0,
> ctx = 0x7f7939905b30, nullop = 0}
>          s = 10
> #25 0x00007f8cf3610552 in ldap_int_thread_pool_wrapper (xpool=0x1648000) at
> tpool.c:956
>          pq = 0x1648000
>          pool = 0x180c180
>          task = 0x65a78e0
>          work_list = 0x1648070
>          ctx = {ltu_pq = 0x1648000, ltu_id = 140158633535232, ltu_key =
> {{ltk_key = 0x43a3b7, ltk_data = 0x6570000, ltk_free = 0x43a1fb
> <conn_counter_destroy>}, {ltk_key = 0x4ae237, ltk_data = 0x3be91c0,
> ltk_free = 0x4ae05c <slap_sl_mem_destroy>}, {
>                ltk_key = 0x1810d00, ltk_data = 0x8ede200, ltk_free =
> 0x7f8cefc42783 <mdb_reader_free>}, {ltk_key = 0x7f8cefc375b4, ltk_data =
> 0x1250c000, ltk_free = 0x7f8cefc37591 <search_stack_free>}, {ltk_key =
> 0x7f8cefc34071, ltk_data = 0x1220c000,
>                ltk_free = 0x7f8cefc34029 <scope_chunk_free>}, {ltk_key =
> 0x455655, ltk_data = 0x1367d480, ltk_free = 0x4555a8 <slap_op_q_destroy>},
> {ltk_key = 0x1811400, ltk_data = 0x18d19400, ltk_free = 0x7f8cefc42783
> <mdb_reader_free>}, {ltk_key = 0x0,
>                ltk_data = 0xe393200, ltk_free = 0}, {ltk_key = 0x0, ltk_data
> = 0x0, ltk_free = 0} <repeats 24 times>}}
>          kctx = 0x0
>          i = 32
>          keyslot = 392
>          hash = 4080100744
>          pool_lock = 0
>          freeme = 0
>          __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper"
> #26 0x000000344a607aa1 in start_thread () from /lib64/libpthread.so.0
> No symbol table info available.
> #27 0x000000344a2e8aad in clone () from /lib64/libc.so.6
> No symbol table info available.
> (gdb)

Anyway, we know the bad patch was  2d5996ac603391ddbd618425f88eb13e5e0e2cc0 so 
this should be easy to fix.

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/