Provide slapd Aborts on failed assertion

[Paul Turgyan <pturgyan@umich.edu>]

I have a consumer slapd in refreshandpersist mode and a provider slapd
that aborts on the first search operation issued by the consumer.
I've tracked it down to the cachesize option.  If I set the cachesize  
option to
any value,  the provider fails an assertion in pthread_mutex_lock.

here is the output from stdout when running the provider in gdb.

send_ldap_result: conn=0 op=1 p=3
slapd: pthread_mutex_lock.c:108: __pthread_mutex_lock: Assertion  
`mutex->__data.__owner == 0' failed.

Here is the contents of the mutex in pthread_mutex_lock
(gdb) up
#7  0xb7eb10c7 in __pthread_mutex_lock (mutex=0x8252b50)
    at pthread_mutex_lock.c:54
54      pthread_mutex_lock.c: No such file or directory.
        in pthread_mutex_lock.c
(gdb) p *mutex
$1 = {__data = {__lock = 1, __count = 0, __owner = 1000, __kind = 0,
    __nusers = 0, __spins = 0},
  __size = "\001\000\000\000\000\000\000\000?\003", '\0' <repeats 13  
times>,
  __align = 1}

Here is the rest of he stack Backtrace

Program received signal SIGABRT, Aborted.
[Switching to Thread -1860699216 (LWP 32141)]
0xb7f95410 in ?? ()
(gdb) bt
#0  0xb7f95410 in ?? ()
#1  0x9117dca4 in ?? ()
#2  0x00000006 in ?? ()
#3  0x00007d8d in ?? ()
#4  0xb7c29041 in *__GI_raise (sig=6)
    at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#5  0xb7c2a7d7 in *__GI_abort () at ../sysdeps/generic/abort.c:88
#6  0xb7c2279f in *__GI___assert_fail (assertion=0x0, file=0x0, line=0,
    function=0xb7eb6fc8 "__pthread_mutex_lock") at assert.c:83
#7  0xb7eb10c7 in __pthread_mutex_lock (mutex=0x8252b50)
    at pthread_mutex_lock.c:54
#8  0x0815a7d9 in ldap_pvt_thread_mutex_lock (mutex=0x8252b50)
    at thr_posix.c:333
#9  0x08142365 in syncprov_op_search (op=0x82b3f20, rs=0x9117f230)
    at syncprov.c:1993
#10 0x080ceb3c in overlay_op_walk (op=0x82b3f20, rs=0x9117f230,
    which=op_search, oi=0x8252c18, on=0x8252d08) at backover.c:498
#11 0x080ced6e in over_op_func (op=0x82b3f20, rs=0x9117f230,  
which=op_search)
    at backover.c:560
#12 0x080cedff in over_op_search (op=0x82b3f20, rs=0x9117f230)
    at backover.c:582
#13 0x08068fd7 in fe_op_search (op=0x82b3f20, rs=0x9117f230) at  
search.c:355
#14 0x08068aad in do_search (op=0x82b3f20, rs=0x9117f230) at search.c: 
217
#15 0x08065ebb in connection_operation (ctx=0x9117f2c0, arg_v=0x82b3f20)
    at connection.c:1133
#16 0x08159859 in ldap_int_thread_pool_wrapper (xpool=0x8232500) at  
tpool.c:478
#17 0xb7eafc6b in start_thread (arg=0x9117fbb0) at pthread_create.c:261
#18 0xb7cb8d9e in clone () from /lib/libc.so.6
(gdb)
=====================

I am running:
linux 2.6.18.3
glibc-2.3.6
OpenLDAP-2.3.35
BerkeleyDB.4.2 w/ patches

Here is my provider config file:
#
include         /etc/openldap/schema/core.schema
include         /etc/openldap/schema/cosine.schema
include         /etc/openldap/schema/inetorgperson.schema
include         /etc/openldap/schema/nis.schema
include         /etc/openldap/schema/local.schema
#
access to dn.base=""
       by * read

# default settings
defaultsearchbase       "dc=umich,dc=edu"
allow           bind_v2 bind_anon_cred bind_anon_dn
#
# file locations
pidfile         /home/pturgyan/openldap-2.3/slapd.prov.pid
argsfile        /home/pturgyan/openldap-2.3/slapd.prov.args
#
threads         32
#
# database settings
database        bdb
suffix          "dc=umich,dc=edu"
rootdn          "cn=root,dc=umich,dc=edu"
rootpw          BD=fast

overlay         syncprov
#syncprov-checkpoint 100 10
#syncprov-sessionlog 100

directory       /var/slapd/tmp/db/db.save
cachesize       1000
idlcachesize    25000
checkpoint      1024 5
lastmod         on
#
include         /home/pturgyan/openldap-2.3/slapd.index

=============================
here is the output from stdout when running the provider in gdb
and a stack backtrace after the abort

send_ldap_result: conn=0 op=1 p=3
slapd: pthread_mutex_lock.c:108: __pthread_mutex_lock: Assertion  
`mutex->__data.__owner == 0' failed.

Program received signal SIGABRT, Aborted.
[Switching to Thread -1860699216 (LWP 32141)]
0xb7f95410 in ?? ()
(gdb) bt
#0  0xb7f95410 in ?? ()
#1  0x9117dca4 in ?? ()
#2  0x00000006 in ?? ()
#3  0x00007d8d in ?? ()
#4  0xb7c29041 in *__GI_raise (sig=6)
    at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#5  0xb7c2a7d7 in *__GI_abort () at ../sysdeps/generic/abort.c:88
#6  0xb7c2279f in *__GI___assert_fail (assertion=0x0, file=0x0, line=0,
    function=0xb7eb6fc8 "__pthread_mutex_lock") at assert.c:83
#7  0xb7eb10c7 in __pthread_mutex_lock (mutex=0x8252b50)
    at pthread_mutex_lock.c:54
#8  0x0815a7d9 in ldap_pvt_thread_mutex_lock (mutex=0x8252b50)
    at thr_posix.c:333
#9  0x08142365 in syncprov_op_search (op=0x82b3f20, rs=0x9117f230)
    at syncprov.c:1993
#10 0x080ceb3c in overlay_op_walk (op=0x82b3f20, rs=0x9117f230,
    which=op_search, oi=0x8252c18, on=0x8252d08) at backover.c:498
#11 0x080ced6e in over_op_func (op=0x82b3f20, rs=0x9117f230,  
which=op_search)
    at backover.c:560
#12 0x080cedff in over_op_search (op=0x82b3f20, rs=0x9117f230)
    at backover.c:582
#13 0x08068fd7 in fe_op_search (op=0x82b3f20, rs=0x9117f230) at  
search.c:355
#14 0x08068aad in do_search (op=0x82b3f20, rs=0x9117f230) at search.c: 
217
#15 0x08065ebb in connection_operation (ctx=0x9117f2c0, arg_v=0x82b3f20)
    at connection.c:1133
#16 0x08159859 in ldap_int_thread_pool_wrapper (xpool=0x8232500) at  
tpool.c:478
#17 0xb7eafc6b in start_thread (arg=0x9117fbb0) at pthread_create.c:261
#18 0xb7cb8d9e in clone () from /lib/libc.so.6
(gdb)