Issue 7723 - slapd crashes on multi core machines if a search request is *immediately* followed by an unbind
Summary: slapd crashes on multi core machines if a search request is *immediately* fol...
Status: VERIFIED FIXED
Alias: None
Product: OpenLDAP
Classification: Unclassified
Component: slapd (show other issues)
Version: 2.4.36
Hardware: All All
: --- normal
Target Milestone: ---
Assignee: OpenLDAP project
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-10-11 13:27 UTC by michael.vishchers@7p-group.com
Modified: 2014-10-23 07:31 UTC (History)
0 users

See Also:


Attachments
test_scenario.tgz (2.25 KB, application/x-compressed-tar)
2013-10-15 11:10 UTC, michael.vishchers@7p-group.com
Details

Note You need to log in before you can comment on or make changes to this issue.
Description Howard Chu 2013-10-11 09:54:58 UTC
changed state Open to Closed
Comment 1 Howard Chu 2013-10-11 10:05:11 UTC
changed state Closed to Feedback
Comment 2 michael.vishchers@7p-group.com 2013-10-11 13:27:04 UTC
Full_Name: Michael Vishchers
Version: 2.4.36
OS: RHEL 6.3
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (109.41.86.33)


The bug mentioned in ITS #7716 unfortunately still persists in 2.4.36.

We now found a way to reproduce it:

- make sure that threads can run freely (i.e., provide enough cpus)
- start slapd with the rwm overlay
- run a client loop that
-- binds as a valid user
-- sends a valid search request that is *immediately* (i.e., don't wait for any
answers) followed by an unbind request

after several iterations (with 2.4.23 it was between 2 and 7, with 2.4.36 it was
about 30) slapd crashes in malloc.c
Comment 3 Howard Chu 2013-10-11 16:53:19 UTC
michael.vishchers@7p-group.com wrote:
> Full_Name: Michael Vishchers
> Version: 2.4.36
> OS: RHEL 6.3
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (109.41.86.33)
>
>
> The bug mentioned in ITS #7716 unfortunately still persists in 2.4.36.
>
> We now found a way to reproduce it:
>
> - make sure that threads can run freely (i.e., provide enough cpus)

If your client loop is running as a single thread then the number of CPUs and 
running threads should be irrelevant. If the number of threads and CPUs matter 
then you're misusing the API. Closing this ITS.

> - start slapd with the rwm overlay
> - run a client loop that
> -- binds as a valid user
> -- sends a valid search request that is *immediately* (i.e., don't wait for any
> answers) followed by an unbind request
>
> after several iterations (with 2.4.23 it was between 2 and 7, with 2.4.36 it was
> about 30) slapd crashes in malloc.c
>
>


-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/

Comment 4 Howard Chu 2013-10-11 17:00:17 UTC
hyc@symas.com wrote:
> michael.vishchers@7p-group.com wrote:
>> Full_Name: Michael Vishchers
>> Version: 2.4.36
>> OS: RHEL 6.3
>> URL: ftp://ftp.openldap.org/incoming/
>> Submission from: (NULL) (109.41.86.33)
>>
>>
>> The bug mentioned in ITS #7716 unfortunately still persists in 2.4.36.
>>
>> We now found a way to reproduce it:

If you want anyone to spend time on this:

Send the stack trace from the crash. Also send your slapd config, sample data, 
and the actual query used in your client.
>>
>> - make sure that threads can run freely (i.e., provide enough cpus)

>
>> - start slapd with the rwm overlay
>> - run a client loop that
>> -- binds as a valid user
>> -- sends a valid search request that is *immediately* (i.e., don't wait for any
>> answers) followed by an unbind request
>>
>> after several iterations (with 2.4.23 it was between 2 and 7, with 2.4.36 it was
>> about 30) slapd crashes in malloc.c
>>
>>
>
>


-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/

Comment 5 michael.vishchers@7p-group.com 2013-10-15 11:10:23 UTC
It is not the client loop that is multithreading but the ldap server.

And it is not a misuse of the API but a problem that may be raised by day to day network problems.

I've boiled down the problem to a few simple configurations that work (or better, fail ;-) with both 2.4.23 and 2.4.36. A tgz file containing a setup with start script and testclient is attached. It should be sufficient to reproduce the fault.

The problem occurs only if we use session variable substitution in the rwm overlay, and only if a search is *immediately* (e.g. caused by network loss and client timeout) followed by an unbind.

Mit freundlichen Grüßen / Kind regards
Michael Vishchers
Senior System Engineer

7P Solutions & Consulting AG
Calor-Emag-Straße 1
40878 Ratingen

Phone  : +49-2102-5354-356
Fax      : +49-2102-5354-111
Email   : michael.vishchers@7p-group.com<mailto://michael.vishchers@7P-Group.com>
Home   : www.7p-group.com<http://www.7P-Group.com>

Sitz der Gesellschaft: Köln
Registriergericht: Amtsgericht Köln
Handelsregister: HRB 32361
Aufsichtsratsvorsitzender: Dr. Kai Höhmann
Vorstand: Jens Harig (Vorsitzender), Dr. Joachim Philippi
USt-ID-Nr.: DE197820124
Steuer-Nr.: 215/5917/1764

Eine Bitte: Denken Sie an Ihre Umwelt, bevor Sie diese E-Mail ausdrucken!

P Please consider the environment before printing this e-mail
Der Inhalt dieser e-Mail ist ausschließlich für den bezeichneten Adressaten bestimmt. Wenn Sie nicht der vorgesehene Adressat dieser e-Mail oder dessen Vertreter sein sollten, beachten Sie bitte, dass jede Form der Veröffentlichung, Vervielfältigung oder Weitergabe des Inhalts dieser e-Mail unzulässig ist. Wir bitten Sie sofort den Absender zu informieren und die E-mail zu löschen.
The information contained in this e-mail is intended solely for the addressee. Access to this e-mail by anyone else is unauthorized. If you are not the intended recipient, any form of disclosure, reproduction, distribution or any action taken or refrained from in reliance on it, is prohibited and may be unlawful. Please notify the sender immediately and destroy this e-mail.

Comment 6 Howard Chu 2013-10-15 12:03:41 UTC
michael.vishchers@7p-group.com wrote:
> --_004_96779DE2D69B3E4DB46CA3AEAA8C409D8B0DCCRTGMX02intranetlo_
> Content-Type: multipart/alternative;
> 	boundary="_000_96779DE2D69B3E4DB46CA3AEAA8C409D8B0DCCRTGMX02intranetlo_"
>
> --_000_96779DE2D69B3E4DB46CA3AEAA8C409D8B0DCCRTGMX02intranetlo_
> Content-Type: text/plain; charset="iso-8859-1"
> Content-Transfer-Encoding: quoted-printable
>
> It is not the client loop that is multithreading but the ldap server.

Thanks for clarifying.
>
> And it is not a misuse of the API but a problem that may be raised by day t=
> o day network problems.
>
> I've boiled down the problem to a few simple configurations that work (or b=
> etter, fail ;-) with both 2.4.23 and 2.4.36. A tgz file containing a setup =
> with start script and testclient is attached. It should be sufficient to re=
> produce the fault.
>
> The problem occurs only if we use session variable substitution in the rwm =
> overlay, and only if a search is *immediately* (e.g. caused by network loss=
>   and client timeout) followed by an unbind.

Yes, your test config shows that the problem is that rwm_conn_destroy frees 
the session context while rwm_op_search is using it. Seems like the rwm 
overlay is not doing reference counting properly.

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/

Comment 7 Michael Ströder 2013-10-18 22:29:14 UTC
hyc@symas.com wrote:
> Yes, your test config shows that the problem is that rwm_conn_destroy frees 
> the session context while rwm_op_search is using it. Seems like the rwm 
> overlay is not doing reference counting properly.

I had problems with a certain configuration of slapo-rwm where slapcat seg
faulted after writing all the LDIF data. Could this be related?

Never found the time to construct a simple config showing the issue though.

Ciao, Michael.

Comment 8 jsynacek@redhat.com 2013-10-23 12:43:37 UTC
On 10/15/2013 02:03 PM, hyc@symas.com wrote:
> Yes, your test config shows that the problem is that rwm_conn_destroy frees 
> the session context while rwm_op_search is using it. Seems like the rwm 
> overlay is not doing reference counting properly.
> 

With the current master (fe49824f83bb0f2dd2f543c26f186b516c1d75bd), this is how
the trace looks like:

(gdb) t apply all bt

Thread 5 (Thread 0x7f53f9009700 (LWP 28210)):
#0  0x000000309360dded in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x0000003093609ba1 in _L_lock_790 () from /lib64/libpthread.so.0
#2  0x0000003093609aa7 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x00007f5401b530a7 in ldap_pvt_thread_mutex_lock (mutex=0x7f53f980c768) at
thr_posix.c:296
#4  0x0000000000445287 in connection_operation (ctx=0x7f53f9008bb0,
arg_v=0x7f53e0001230) at connection.c:1147
#5  0x00007f5401b51993 in ldap_int_thread_pool_wrapper (xpool=0x937380) at
tpool.c:945
#6  0x0000003093607c53 in start_thread () from /lib64/libpthread.so.0
#7  0x0000003092ef5e1d in clone () from /lib64/libc.so.6

Thread 4 (Thread 0x7f53f3fff700 (LWP 28215)):
#0  0x000000309360b575 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1  0x00007f5401b53054 in ldap_pvt_thread_cond_wait (cond=0x9373b8,
mutex=0x937390) at thr_posix.c:277
#2  0x00007f5401b518be in ldap_int_thread_pool_wrapper (xpool=0x937380) at
tpool.c:927
#3  0x0000003093607c53 in start_thread () from /lib64/libpthread.so.0
#4  0x0000003092ef5e1d in clone () from /lib64/libc.so.6

Thread 3 (Thread 0x7f54016dd740 (LWP 28178)):
#0  0x0000003093608d97 in pthread_join () from /lib64/libpthread.so.0
#1  0x00007f5401b52f95 in ldap_pvt_thread_join (thread=139998644971264,
thread_return=0x0) at thr_posix.c:197
#2  0x0000000000441bb4 in slapd_daemon () at daemon.c:2907
#3  0x000000000041b28a in main (argc=10, argv=0x7fff84c32388) at main.c:1013

Thread 2 (Thread 0x7f53f980a700 (LWP 28197)):
#0  0x0000003092ef63f3 in epoll_wait () from /lib64/libc.so.6
#1  0x00000000004406b4 in slapd_daemon_task (ptr=0x9c02d0) at daemon.c:2514
#2  0x0000003093607c53 in start_thread () from /lib64/libpthread.so.0
#3  0x0000003092ef5e1d in clone () from /lib64/libc.so.6

Thread 1 (Thread 0x7f53f8808700 (LWP 28214)):
#0  0x00000000004c121a in slap_sl_free (ptr=0x7f53e8000b40, ctx=0x7f53e80009d0)
at sl_malloc.c:513
#1  0x0000000000449520 in do_search (op=0x7f53e0103c60, rs=0x7f53f8807ad0) at
search.c:257
#2  0x00000000004451d4 in connection_operation (ctx=0x7f53f8807bb0,
arg_v=0x7f53e0103c60) at connection.c:1134
#3  0x00007f5401b51993 in ldap_int_thread_pool_wrapper (xpool=0x937380) at
tpool.c:945
#4  0x0000003093607c53 in start_thread () from /lib64/libpthread.so.0
#5  0x0000003092ef5e1d in clone () from /lib64/libc.so.6


(gdb) bt full
#0  0x00000000004c121a in slap_sl_free (ptr=0x7f53e8000b40, ctx=0x7f53e80009d0)
at sl_malloc.c:513
        sh = 0x7f53e80009d0
        size = 7286935691776455520
        p = 0x7f53e8000b38
        nextp = 0x6520e4bb49737e98
        tmpp = 0x7f53f8807ad0
#1  0x0000000000449520 in do_search (op=0x7f53e0103c60, rs=0x7f53f8807ad0) at
search.c:257
        base = {bv_len = 6, bv_val = 0x7f53e00009c7 "o=test"}
        siz = 0
        off = 0
        i = 0
#2  0x00000000004451d4 in connection_operation (ctx=0x7f53f8807bb0,
arg_v=0x7f53e0103c60) at connection.c:1134
        rc = 80
        cancel = 0
        op = 0x7f53e0103c60
        rs = {sr_type = REP_RESULT, sr_tag = 101, sr_msgid = 2, sr_err = 80,
sr_matched = 0x0, sr_text = 0x7f53f98d4fe6 "searchDN massage error", sr_ref =
0x0, sr_ctrls = 0x0, sr_un = {sru_search = {r_entry = 0x0, r_attr_flags = 0,
              r_operational_attrs = 0x0, r_attrs = 0x0, r_nentries = 0, r_v2ref
= 0x0}, sru_sasl = {r_sasldata = 0x0}, sru_extended = {r_rspoid = 0x0, r_rspdata
= 0x0}}, sr_flags = 0}
        tag = 99
        opidx = SLAP_OP_SEARCH
        conn = 0x7f53f980c750
        memctx = 0x7f53e80009d0
        memctx_null = 0x0
        memsiz = 1048576
        __PRETTY_FUNCTION__ = "connection_operation"
#3  0x00007f5401b51993 in ldap_int_thread_pool_wrapper (xpool=0x937380) at
tpool.c:945
        pq = 0x937380
        pool = 0x937280
        task = 0x7f53ec113d60
        work_list = 0x9373f0
        ctx = {ltu_pq = 0x937380, ltu_id = 139998628185856, ltu_key = {{ltk_key
= 0x444c86 <conn_counter_init>, ltk_data = 0x7f53e80008c0, ltk_free = 0x444a7d
<conn_counter_destroy>}, {ltk_key = 0x4c0182 <slap_sl_mem_init>,
              ltk_data = 0x7f53e80009d0, ltk_free = 0x4bffa7
<slap_sl_mem_destroy>}, {ltk_key = 0x461f28 <slap_op_free>, ltk_data = 0x0,
ltk_free = 0x461e7b <slap_op_q_destroy>}, {ltk_key = 0x0, ltk_data = 0x0,
              ltk_free = 0x0} <repeats 29 times>}}
        kctx = 0x0
        i = 32
        keyslot = 817
        hash = 1596318513
        pool_lock = 0
        freeme = 0
        __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper"


I used the reproducer provided by Michael. The core was generated by the server
running with slapd1.conf.

-- 
Jan Synacek
Software Engineer, Red Hat

Comment 9 jsynacek@redhat.com 2013-11-04 13:54:41 UTC
On 10/15/2013 01:10 PM, michael.vishchers@7p-group.com wrote:
> It is not the client loop that is multithreading but the ldap server.
> 
> And it is not a misuse of the API but a problem that may be raised by day t=
> o day network problems.
> 
> I've boiled down the problem to a few simple configurations that work (or b=
> etter, fail ;-) with both 2.4.23 and 2.4.36. A tgz file containing a setup =
> with start script and testclient is attached. It should be sufficient to re=
> produce the fault.
> 
> The problem occurs only if we use session variable substitution in the rwm =
> overlay, and only if a search is *immediately* (e.g. caused by network loss=
>  and client timeout) followed by an unbind.
> 

I modified the reproducer a bit (the start script) and find out a few things.
You can find the reproducer I'm using at [1].

Valgrind's helgrind shows some lock problems in the rwm overlay and also in
back-ldap and connection.c. After correcting those the issue seems to be gone.

You can find helgrind logs at [2] (before the fix) and [3] (after).

Also, ElectricFence reveals some problems [4], which I didn't fix yet.

A fix attempt can be found at [5]. I'm not sure if that is a correct fix, or it
just masked the real issue. But I didn't to manage to reproduce the problem
after applying it.

[1] http://jsynacek.fedorapeople.org/openldap/its7723/reproducer/
[2]
http://jsynacek.fedorapeople.org/openldap/its7723/results/slapd1-helgrind-broken.log
[3]
http://jsynacek.fedorapeople.org/openldap/its7723/results/slapd1-helgrind-fixed.log
[4]
http://jsynacek.fedorapeople.org/openldap/its7723/results/slapd1-broken-efence-gdb.txt
[5]
http://jsynacek.fedorapeople.org/openldap/its7723/0001-fix-possible-race-conditions.patch

Cheers,
-- 
Jan Synacek
Software Engineer, Red Hat

Comment 10 Howard Chu 2013-11-04 14:11:36 UTC
jsynacek@redhat.com wrote:
> On 10/15/2013 01:10 PM, michael.vishchers@7p-group.com wrote:
>> It is not the client loop that is multithreading but the ldap server.
>>
>> And it is not a misuse of the API but a problem that may be raised by day t=
>> o day network problems.
>>
>> I've boiled down the problem to a few simple configurations that work (or b=
>> etter, fail ;-) with both 2.4.23 and 2.4.36. A tgz file containing a setup =
>> with start script and testclient is attached. It should be sufficient to re=
>> produce the fault.
>>
>> The problem occurs only if we use session variable substitution in the rwm =
>> overlay, and only if a search is *immediately* (e.g. caused by network loss=
>>   and client timeout) followed by an unbind.
>>
>
> I modified the reproducer a bit (the start script) and find out a few things.
> You can find the reproducer I'm using at [1].
>
> Valgrind's helgrind shows some lock problems in the rwm overlay and also in
> back-ldap and connection.c. After correcting those the issue seems to be gone.
>
> You can find helgrind logs at [2] (before the fix) and [3] (after).
>
> Also, ElectricFence reveals some problems [4], which I didn't fix yet.
>
> A fix attempt can be found at [5]. I'm not sure if that is a correct fix, or it
> just masked the real issue. But I didn't to manage to reproduce the problem
> after applying it.

I already explained the problem. The other issues you identified are not 
relevant, and your patch is not correct. Reread Followup #4 of this ITS.

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/

Comment 11 jsynacek@redhat.com 2013-11-13 11:53:58 UTC
On 11/04/2013 03:11 PM, hyc@symas.com wrote:
> jsynacek@redhat.com wrote:
>> On 10/15/2013 01:10 PM, michael.vishchers@7p-group.com wrote:
>>> It is not the client loop that is multithreading but the ldap server.
>>>
>>> And it is not a misuse of the API but a problem that may be raised by day t=
>>> o day network problems.
>>>
>>> I've boiled down the problem to a few simple configurations that work (or b=
>>> etter, fail ;-) with both 2.4.23 and 2.4.36. A tgz file containing a setup =
>>> with start script and testclient is attached. It should be sufficient to re=
>>> produce the fault.
>>>
>>> The problem occurs only if we use session variable substitution in the rwm =
>>> overlay, and only if a search is *immediately* (e.g. caused by network loss=
>>>   and client timeout) followed by an unbind.
>>>
>>
>> I modified the reproducer a bit (the start script) and find out a few things.
>> You can find the reproducer I'm using at [1].
>>
>> Valgrind's helgrind shows some lock problems in the rwm overlay and also in
>> back-ldap and connection.c. After correcting those the issue seems to be gone.
>>
>> You can find helgrind logs at [2] (before the fix) and [3] (after).
>>
>> Also, ElectricFence reveals some problems [4], which I didn't fix yet.
>>
>> A fix attempt can be found at [5]. I'm not sure if that is a correct fix, or it
>> just masked the real issue. But I didn't to manage to reproduce the problem
>> after applying it.
> 
> I already explained the problem. The other issues you identified are not 
> relevant, and your patch is not correct. Reread Followup #4 of this ITS.
> 

Another take on the fix:

http://jsynacek.fedorapeople.org/openldap/its7723/0001-ITS-7723-fix-reference-counting.patch

-- 
Jan Synacek
Software Engineer, Red Hat

Comment 12 Howard Chu 2014-03-18 03:39:42 UTC
changed notes
changed state Feedback to Test
moved from Incoming to Software Bugs
Comment 13 Michael Ströder 2014-03-18 08:22:51 UTC
Could anybody please review the patch Jan provided in his last follow-up?

http://www.openldap.org/its/index.cgi?findid=7723#followup9

Anythin wrong with his reference count fix?

We are getting occasional seg faults in slapo-rwm which are very hard to
reproduce and therefore are considering this patch.

Ciao, Michael.

Comment 14 Howard Chu 2014-03-18 10:38:56 UTC
michael@stroeder.com wrote:
> Could anybody please review the patch Jan provided in his last follow-up?
>
> http://www.openldap.org/its/index.cgi?findid=7723#followup9
>
> Anythin wrong with his reference count fix?
>
> We are getting occasional seg faults in slapo-rwm which are very hard to
> reproduce and therefore are considering this patch.

Looks OK to me, committed to master. Please report back any positive/negative 
results.


-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/

Comment 15 Quanah Gibson-Mount 2014-04-01 16:57:36 UTC
changed notes
changed state Test to Release
Comment 16 konrad.windszus@netcentric.biz 2014-04-29 14:41:46 UTC
I got a similar crash when issuing a Bind request. 
GDB has the following information about that:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f0898ff9700 (LWP 17201)]
0x00007f08d0a409f8 in hdb_reader_get (op=<value optimized out>, env=0x7f08d14c55a0, txn=0x7f0898ff80a8) at cache.c:1621
1621				rc = TXN_BEGIN( env, NULL, txn, DB_READ_COMMITTED );

The stack trace exposes the following:
(gdb) bt full
#0  0x00007f08d0a409f8 in hdb_reader_get (op=<value optimized out>, env=0x7f08d14c55a0, txn=0x7f0898ff80a8) at cache.c:1621
        i = <value optimized out>
        rc = <value optimized out>
        data = <value optimized out>
        ctx = 0x7f0898ff8b70
#1  0x00007f08d0a4bca7 in hdb_entry_get (op=0x7f0860000970, ndn=0x7f08600009a8, oc=0x0, at=0x0, rw=0, ent=0x7f0898ff8400) at id2entry.c:343
        bdb = 0x7f08d14cf5a0
        boi = 0x0
        txn = 0x0
        e = 0x0
        ei = <value optimized out>
        rc = <value optimized out>
        at_name = 0x7f08d0abdacd "(null)"
        lock = {off = 8, ndx = 4294967295, gen = 32520, mode = DB_LOCK_WWRITE}
#2  0x00007f08d09ef799 in overlay_entry_get_ov (op=0x7f0860000970, dn=0x7f08600009a8, oc=0x0, ad=0x0, rw=0, e=0x7f0898ff8400, on=0x0) at ../../../servers/slapd/backover.c:364
        oi = 0x7f08d14bd9d0
        be = 0x7f08d14cf410
        db = {bd_info = 0x2, bd_self = 0x7f0898ff81f0, be_ctrls = "\357\067\216\351\275q4\216\221\364\071\341\\nF\252\213K\022+\000\002\000\000\000\000\000\000ihdof", be_flags = 551504935474, 
          be_restrictops = 0, be_requires = 0, be_ssf_set = {sss_ssf = 0, sss_transport = 0, sss_tls = 2566881772, sss_sasl = 32520, sss_update_ssf = 0, sss_update_transport = 0, 
            sss_update_tls = 3081553768, sss_update_sasl = 32520, sss_simple_bind = 3513539536}, be_suffix = 0x7f08b7acc768, be_nsuffix = 0x7f08d16c5bd0, be_schemadn = {bv_len = 122, 
            bv_val = 0x7f08b7ace810 "z"}, be_schemandn = {bv_len = 139675828497508, bv_val = 0x7f086010f0b0 "."}, be_rootdn = {bv_len = 139675850001328, bv_val = 0x7f08d13bf590 ""}, be_rootndn = {
            bv_len = 139675417895224, bv_val = 0x7f08cc629670 ""}, be_rootpw = {bv_len = 7786477098, bv_val = 0x28 <Address 0x28 out of bounds>}, be_max_deref_depth = 2566882064, be_def_limit = {
            lms_t_soft = 32520, lms_t_hard = -781427760, lms_s_soft = 32520, lms_s_hard = -1213413528, lms_s_unchecked = 32520, lms_s_pr = -1728085380, lms_s_pr_hide = 32520, 
            lms_s_pr_total = -1213537992}, be_limits = 0x7f0898ff8310, be_acl = 0x7f08d0243101, be_dfltaccess = -1728085408, be_update_ndn = {bv_len = 139672336465920, 
            bv_val = 0x7f08d16c4bb0 "\220Ol\321\b\177"}, be_update_refs = 0x7f08d16c5bd0, be_pending_csn_list = 0x7f0898ff8310, be_pcl_mutex = {__data = {__lock = 0, __count = 0, __owner = -791385128, 
              __nusers = 32520, __kind = -802934423, __spins = 32520, __list = {__prev = 0x0, __next = 0x0}}, 
            __size = "\000\000\000\000\000\000\000\000\330k\324\320\b\177\000\000i1$\320\b\177", '\000' <repeats 17 times>, __align = 0}, be_syncinfo = 0x1, be_pb = 0x7f08d16c4bb0, 
          be_cf_ocs = 0x7f0898ff8310, be_private = 0x7f08d14c5730, be_next = {stqe_next = 0x7f08d0dc87a0}}
        bi = 0x7f08d14bd9d0
        rc = 32768
#3  0x00007f08d09f02e7 in over_entry_get_rw (op=<value optimized out>, dn=<value optimized out>, oc=<value optimized out>, ad=<value optimized out>, rw=<value optimized out>, e=<value optimized out>)
    at ../../../servers/slapd/backover.c:396
        oi = <value optimized out>
        on = <value optimized out>
        __PRETTY_FUNCTION__ = "over_entry_get_rw"
#4  0x00007f08cca73cb3 in ppolicy_bind_response (op=0x7f0860000970, rs=0x7f0898ff8920) at ../../../../servers/slapd/overlays/ppolicy.c:919
        ppb = 0x7f0860001630
        on = 0x7f08d14c7540
        mod = 0x0
        m = <value optimized out>
        pwExpired = 0
        ngut = -1
        warn = -1
        age = <value optimized out>
        rc = <value optimized out>
        a = <value optimized out>
        now = <value optimized out>
        pwtime = -1
        nowstr = "\240\000\000\000\000\000\000\000\325M=\316\b\177\000\000\200\216n\316\b\177"
        timestamp = {bv_len = 3978984383913624688, bv_val = 0x7f0898ff8486 ""}
        bi = 0x7f08d14d03e0
        e = <value optimized out>
#5  0x00007f08d0993f5e in slap_response_play (op=0x7f0860000970, rs=0x7f0898ff8920) at ../../../servers/slapd/result.c:402
        sc_next = 0x7f08600014c8
---Type <return> to continue, or q <return> to quit---
        sc_nextp = 0x7f0860001610
        rc = 32768
        sc = 0x7f0860001610
        scp = 0x7f0898ff85e8
#6  0x00007f08d0996cae in send_ldap_response (op=0x7f0860000970, rs=0x7f0898ff8920) at ../../../servers/slapd/result.c:476
        berbuf = {
          buffer = "\a\000\000\000\032\000\000\000\020\000\000\000\035\000\000\000\003\000\000\000r\000\000\000\002\000\000\000v\000\000\000\001\000\000\000\b\177\000\000 \034\000\000\000\000\000\000\220\343;\321\b\177\000\000`\r\000`\b\177\000\000\220\210\377\230\b\177\000\000\313\003\237\320\b\177\000\000\310\024\000`\b\177\000\000\240\365\236\320\b\177\000\000\000\000\000\000\000\000\000\000\340\003M\321\b\177\000\000 d\324\320\b\177\000\000\020\364L\321\b\177\000\000\200\v\021`\b\177\000\000\000\000\000\000\000\000\000\000\001\000\001\001\000\000\000\000\214\000\000\000\000\000\000\000\200\v\021`\b\177\000\000\177\266_S", '\000' <repeats 12 times>, "p\t\000`\b\177\000\000\330k\324\320\b\177\000\000 \375\324\320\b\177\000\000 \211\377\230\b\177\000\000`\r\000`\b\177\000\000\220\210\377\230\b\177\000\000c\342C\316\b\177\000\000\060\000\000\000"..., ialign = 7, lalign = 111669149703, falign = 9.80908925e-45, dalign = 5.517189056855554e-313, palign = 0x1a00000007 <Address 0x1a00000007 out of bounds>}
        ber = 0x7f0898ff8630
        rc = 0
        bytes = <value optimized out>
        __PRETTY_FUNCTION__ = "send_ldap_response"
#7  0x00007f08d0997c70 in slap_send_ldap_result (op=0x7f0860000970, rs=0x7f0898ff8920) at ../../../servers/slapd/result.c:750
        tmp = 0x0
        otext = 0x0
        oref = 0x0
        __PRETTY_FUNCTION__ = "slap_send_ldap_result"
#8  0x00007f08d09a1ba9 in fe_op_bind_success (op=0x7f0860000970, rs=0x7f0898ff8920) at ../../../servers/slapd/bind.c:440
No locals.
#9  0x00007f08d09a233f in fe_op_bind (op=0x7f0860000970, rs=0x7f0898ff8920) at ../../../servers/slapd/bind.c:386
        bd = 0x7f08d0d505e0
#10 0x00007f08d09a2b19 in do_bind (op=0x7f0860000970, rs=0x7f0898ff8920) at ../../../servers/slapd/bind.c:205
        ber = 0x7f0860000d60
        version = 3
        method = 128
        mech = {bv_len = 0, bv_val = 0x0}
        dn = {bv_len = 60, bv_val = 0x7f08600008ca "uid=konrad.windszus,ou=internal,ou=users,dc=radvision,dc=biz"}
        tag = <value optimized out>
        be = <value optimized out>
#11 0x00007f08d09839f9 in connection_operation (ctx=0x7f0898ff8b70, arg_v=0x7f0860000970) at ../../../servers/slapd/connection.c:1109
        rc = 80
        cancel = <value optimized out>
        op = 0x7f0860000970
        rs = {sr_type = REP_RESULT, sr_tag = 97, sr_msgid = 1, sr_err = 0, sr_matched = 0x0, sr_text = 0x0, sr_ref = 0x0, sr_ctrls = 0x0, sr_un = {sru_search = {r_entry = 0x0, r_attr_flags = 0, 
              r_operational_attrs = 0x0, r_attrs = 0x0, r_nentries = 0, r_v2ref = 0x0}, sru_sasl = {r_sasldata = 0x0}, sru_extended = {r_rspoid = 0x0, r_rspdata = 0x0}}, sr_flags = 0}
        tag = 96
        opidx = SLAP_OP_BIND
        conn = 0x7f08d14e3350
        memctx = 0x7f0860000ed0
        memctx_null = 0x0
        memsiz = 1048576
        __PRETTY_FUNCTION__ = "connection_operation"
#12 0x00007f08d098434d in connection_read_thread (ctx=0x7f0898ff8b70, argv=<value optimized out>) at ../../../servers/slapd/connection.c:1245
        rc = <value optimized out>
        cri = {op = 0x7f0860000970, func = 0, arg = 0x0, ctx = 0x7f0898ff8b70, nullop = <value optimized out>}
        s = <value optimized out>
#13 0x00007f08d0a83dd8 in ldap_int_thread_pool_wrapper (xpool=0x7f08d13ee170) at ../../../libraries/libldap_r/tpool.c:685
        pool = 0x7f08d13ee170
        task = 0x7f089c000f00
        work_list = <value optimized out>
        ctx = {ltu_id = 139674903353088, ltu_key = {{ltk_key = 0x7f08d0982540, ltk_data = 0x7f0860000dc0, ltk_free = 0x7f08d0982620 <conn_counter_destroy>}, {ltk_key = 0x7f08d09dc7d0, 
---Type <return> to continue, or q <return> to quit---
              ltk_data = 0x7f0860000ed0, ltk_free = 0x7f08d09dc6b0 <slap_sl_mem_destroy>}, {ltk_key = 0x7f08d16c99e0, ltk_data = 0x7f0860100f50, ltk_free = 0x7f08d0a40900 <bdb_reader_free>}, {
              ltk_key = 0x7f08d0998400, ltk_data = 0x0, ltk_free = 0x7f08d09981e0 <slap_op_q_destroy>}, {ltk_key = 0x7f08d16c4f90, ltk_data = 0x7f086010ead0, 
              ltk_free = 0x7f08d0a40900 <bdb_reader_free>}, {ltk_key = 0x0, ltk_data = 0x7f0854000ac0, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0} <repeats 26 times>}}
        kctx = <value optimized out>
        keyslot = <value optimized out>
        hash = <value optimized out>
        __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper"
#14 0x00007f08ce8ff9d1 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#15 0x00007f08ce441b6d in clone () from /lib64/libc.so.6
No symbol table info available.

I am using the binary 
@(#) $OpenLDAP: slapd 2.4.23 (Feb  3 2014 19:11:35) $
	mockbuild@c6b10.bsys.dev.centos.org:/builddir/build/BUILD/openldap-2.4.23/openldap-2.4.23/build-servers/servers/slapd

That should have the patch from Jan included, so I am wondering whether the original issue is still not solved or whether this is a separate issue.
Thanks,
Konrad

Comment 17 Quanah Gibson-Mount 2014-04-29 15:31:07 UTC

--On April 29, 2014 at 2:42:19 PM +0000 konrad.windszus@netcentric.biz 
wrote:

>
> --Apple-Mail=_78FDB261-EA91-4040-8A46-100A6608DDC6
> Content-Transfer-Encoding: quoted-printable
> Content-Type: text/plain;

/builddir/build/BUILD/openldap-2.4.23/openldap-2.4.23

You waste yours and everyone else's time using the RHEL/CentOS build of 
OpenLDAP.  Get a real build from Symas or the LTB project.

--Quanah



-- 
Quanah Gibson-Mount
Server Architect
Zimbra, Inc
--------------------
Zimbra ::  the leader in open source messaging and collaboration

Comment 18 OpenLDAP project 2014-10-23 07:31:00 UTC
patched in master
fixed in RE25
fixed in RE24
Comment 19 Quanah Gibson-Mount 2014-10-23 07:31:00 UTC
changed notes
changed state Release to Closed