[Date Prev][Date Next] [Chronological] [Thread] [Top]
bdb db / cache locking problem (ITS#3201)

To: openldap-its@OpenLDAP.org
Subject: bdb db / cache locking problem (ITS#3201)
From: jongchoi@OpenLDAP.org
Date: Wed, 23 Jun 2004 21:38:40 GMT
Full_Name: Jong-Hyuk
Version: HEAD
OS: RedHat 9
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (129.34.20.23)


I was met by a rather wierd threaing / locking problem while I tested concurrent
adding to a single slapd.

Server: single slapd server without any syncrepl setting
Clients: four add operations each of which is set to add 16K entries
         + two subsequent search operations initiated after the add was stuck
ACL : group acl
      two of the four add operations were done by cn=ldap3,o=ibm,c=us
      two of the four add operations were done by cn=ldap4,o=ibm,c=us
      both cn=ldap3,o=ibm,c=us and cn=ldap4,o=ibm,c=us are the members of
      the cn=ldapadmin,o=ibm,c=us group which has write permission to *

Four adds were stuck after a quite while. Interestingly, if I attach the
debugger to the slapd process and do "continue", it will continue for a while.
After some trials it seemed completely stuck. At this point I tried a subtree
search
to the base object but it returned "No such object". When I tried a second
search, now the search was stuck as well. Below is the stack trace obtained at
this point.

(gdb) thread apply all bt

Thread 7 (Thread 1907358272 (LWP 30127)):
#0  0xffffe002 in ?? ()
#1  0x40093932 in __lock_get_internal ()
   from /home/jongchoi/openldap/local/BerkeleyDB.4.2/lib/libdb-4.2.so
#2  0x40092c83 in __lock_vec ()
   from /home/jongchoi/openldap/local/BerkeleyDB.4.2/lib/libdb-4.2.so
#3  0x40092ac7 in __lock_vec_pp ()
   from /home/jongchoi/openldap/local/BerkeleyDB.4.2/lib/libdb-4.2.so
#4  0x080aef0c in bdb_cache_entry_db_relock (env=0x81959f0, locker=43,
    ei=0x8199810, rw=1, tryOnly=0, lock=0x71a3e15c)
    at ../../../../ldap/servers/slapd/back-bdb/cache.c:86
#5  0x080af6d7 in bdb_cache_find_id (op=0x6e2bef18, tid=0x0, id=1,
    eip=0x71a3e064, islocked=1, locker=43, lock=0x71a3e15c)
    at ../../../../ldap/servers/slapd/back-bdb/cache.c:664
#6  0x080b2b6f in bdb_dn2entry (op=0x6e2bef18, tid=0x0, dn=0x0, e=0x71a3e14c,
    matched=1, locker=43, lock=0x71a3e15c)
    at ../../../../ldap/servers/slapd/back-bdb/dn2entry.c:68
#7  0x08096fcf in bdb_do_search (op=0x6e2bef18, rs=0x71aff41c, sop=0x6e2bef18,
    ps_e=0x0, ps_type=0)
    at ../../../../ldap/servers/slapd/back-bdb/search.c:552
#8  0x08096d0b in bdb_search (op=0xfffffffc, rs=0x71aff41c)
    at ../../../../ldap/servers/slapd/back-bdb/search.c:361
#9  0x08059497 in do_search (op=0x6e2bef18, rs=0x71aff41c)
    at ../../../ldap/servers/slapd/search.c:417
#10 0x08057dd0 in connection_operation (ctx=0x71aff49c, arg_v=0x6e2bef18)
    at ../../../ldap/servers/slapd/connection.c:1073
#11 0x080dc717 in ldap_int_thread_pool_wrapper (xpool=0x817fc58)
    at ../../../ldap/libraries/libldap_r/tpool.c:467
#12 0x4020a2b6 in start_thread () from /lib/tls/libpthread.so.0

Thread 6 (Thread 1106672832 (LWP 30096)):
#0  0xffffe002 in ?? ()
#1  0x4020a2b6 in start_thread () from /lib/tls/libpthread.so.0

Thread 5 (Thread 1812753472 (LWP 30099)):
#0  0xffffe002 in ?? ()
#1  0x080af264 in bdb_cache_find_ndn (op=0x8197bb8, txn=0x819f8f8,
    ndn=0x6c0c55e4, res=0x6c0c5464)
    at ../../../../ldap/servers/slapd/back-bdb/cache.c:343
#2  0x080b2ad9 in bdb_dn2entry (op=0x8197bb8, tid=0x819f8f8, dn=0x6c0c55e4,
    e=0x6c0c54b4, matched=0, locker=2147557197, lock=0x6c0c54bc)
    at ../../../../ldap/servers/slapd/back-bdb/dn2entry.c:52
#3  0x080b5798 in bdb_entry_get (op=0x8197bb8, ndn=0x6c0c55e4, oc=0x81890e8,
    at=0x7, rw=0, ent=0xfffffffc)
    at ../../../../ldap/servers/slapd/back-bdb/id2entry.c:284
#4  0x08060f67 in be_entry_get_rw (op=0x8197bb8, ndn=0x6c0c55e4, oc=0x81890e8,
---Type <return> to continue, or q <return> to quit---
    at=0x818c328, rw=0, e=0x0) at ../../../ldap/servers/slapd/backend.c:1244
#5  0x0806142a in backend_group (op=0x8197bb8, target=0x7161a848,
    gr_ndn=0x6c0c55e4, op_ndn=0x8197cb8, group_oc=0x81890e8,
    group_at=0x818c328) at ../../../ldap/servers/slapd/backend.c:1289
#6  0x0806e53d in acl_mask (a=0x818c148, mask=0x6c0c5f08, op=0x8197bb8,
    e=0x7161a848, desc=0xfffffffc, val=0x0, matches=0x6c0c5f0c, count=3,
    state=0x0) at ../../../ldap/servers/slapd/acl.c:1282
#7  0x0806d2fe in access_allowed_mask (op=0x8197bb8, e=0x7161a848,
    desc=0x817dc38, val=0x0, access=ACL_WRITE, state=0x0, maskp=0x0)
    at ../../../ldap/servers/slapd/acl.c:403
#8  0x080ad82c in bdb_add (op=0x8197bb8, rs=0x6c0c661c)
    at ../../../../ldap/servers/slapd/back-bdb/add.c:208
#9  0x0805c979 in do_add (op=0x8197bb8, rs=0x6c0c661c)
    at ../../../ldap/servers/slapd/add.c:318
#10 0x08057e30 in connection_operation (ctx=0x6c0c669c, arg_v=0x8197bb8)
    at ../../../ldap/servers/slapd/connection.c:1048
#11 0x080dc717 in ldap_int_thread_pool_wrapper (xpool=0x817fc58)
    at ../../../ldap/libraries/libldap_r/tpool.c:467
#12 0x4020a2b6 in start_thread () from /lib/tls/libpthread.so.0

Thread 4 (Thread 1818000320 (LWP 30106)):
#0  0xffffe002 in ?? ()
#1  0x40093932 in __lock_get_internal ()
   from /home/jongchoi/openldap/local/BerkeleyDB.4.2/lib/libdb-4.2.so
#2  0x40092c83 in __lock_vec ()
   from /home/jongchoi/openldap/local/BerkeleyDB.4.2/lib/libdb-4.2.so
#3  0x40092ac7 in __lock_vec_pp ()
   from /home/jongchoi/openldap/local/BerkeleyDB.4.2/lib/libdb-4.2.so
#4  0x080aef0c in bdb_cache_entry_db_relock (env=0x81959f0, locker=26,
    ei=0x81a2178, rw=1, tryOnly=0, lock=0x6c5c738c)
    at ../../../../ldap/servers/slapd/back-bdb/cache.c:86
#5  0x080af6d7 in bdb_cache_find_id (op=0x70fd95d8, tid=0x0, id=12,
    eip=0x6c5c7344, islocked=1, locker=26, lock=0x6c5c738c)
    at ../../../../ldap/servers/slapd/back-bdb/cache.c:664
#6  0x080b2b4c in bdb_dn2entry (op=0x70fd95d8, tid=0x0, dn=0xffff86f2,
    e=0x6c5c7384, matched=1, locker=26, lock=0x6c5c738c)
    at ../../../../ldap/servers/slapd/back-bdb/dn2entry.c:60
#7  0x0809612b in bdb_referrals (op=0x70fd95d8, rs=0x6c5c759c)
    at ../../../../ldap/servers/slapd/back-bdb/referral.c:55
#8  0x08060efb in backend_check_referrals (op=0x70fd95d8, rs=0x6c5c759c)
    at ../../../ldap/servers/slapd/backend.c:1218
#9  0x0805c79e in do_add (op=0x70fd95d8, rs=0x6c5c759c)
    at ../../../ldap/servers/slapd/add.c:239
#10 0x08057e30 in connection_operation (ctx=0x6c5c761c, arg_v=0x70fd95d8)
    at ../../../ldap/servers/slapd/connection.c:1048
#11 0x080dc717 in ldap_int_thread_pool_wrapper (xpool=0x817fc58)
    at ../../../ldap/libraries/libldap_r/tpool.c:467
---Type <return> to continue, or q <return> to quit---
#12 0x4020a2b6 in start_thread () from /lib/tls/libpthread.so.0

Thread 3 (Thread 1824521024 (LWP 30109)):
#0  0xffffe002 in ?? ()
#1  0x40093932 in __lock_get_internal ()
   from /home/jongchoi/openldap/local/BerkeleyDB.4.2/lib/libdb-4.2.so
#2  0x40092c83 in __lock_vec ()
   from /home/jongchoi/openldap/local/BerkeleyDB.4.2/lib/libdb-4.2.so
#3  0x40092ac7 in __lock_vec_pp ()
   from /home/jongchoi/openldap/local/BerkeleyDB.4.2/lib/libdb-4.2.so
#4  0x080aef0c in bdb_cache_entry_db_relock (env=0x81959f0, locker=2147557196,
    ei=0x819b4e8, rw=1, tryOnly=0, lock=0x6cbfe3bc)
    at ../../../../ldap/servers/slapd/back-bdb/cache.c:86
#5  0x080af6d7 in bdb_cache_find_id (op=0x81a2360, tid=0x7161b1e0, id=6,
    eip=0x6cbfe364, islocked=1, locker=2147557196, lock=0x6cbfe3bc)
    at ../../../../ldap/servers/slapd/back-bdb/cache.c:664
#6  0x080b2b6f in bdb_dn2entry (op=0x81a2360, tid=0x7161b1e0, dn=0x0,
    e=0x6cbfe3b4, matched=0, locker=2147557196, lock=0x6cbfe3bc)
    at ../../../../ldap/servers/slapd/back-bdb/dn2entry.c:68
#7  0x080b5798 in bdb_entry_get (op=0x81a2360, ndn=0x6cbfe4e4, oc=0x81890e8,
    at=0x17c, rw=0, ent=0xfffffffc)
    at ../../../../ldap/servers/slapd/back-bdb/id2entry.c:284
#8  0x08060f67 in be_entry_get_rw (op=0x81a2360, ndn=0x6cbfe4e4, oc=0x81890e8,
    at=0x818c328, rw=0, e=0x0) at ../../../ldap/servers/slapd/backend.c:1244
#9  0x0806142a in backend_group (op=0x81a2360, target=0x7161a848,
    gr_ndn=0x6cbfe4e4, op_ndn=0x81a2460, group_oc=0x81890e8,
    group_at=0x818c328) at ../../../ldap/servers/slapd/backend.c:1289
#10 0x0806e53d in acl_mask (a=0x818c148, mask=0x6cbfee08, op=0x81a2360,
    e=0x7161a848, desc=0xfffffffc, val=0x0, matches=0x6cbfee0c, count=3,
    state=0x0) at ../../../ldap/servers/slapd/acl.c:1282
#11 0x0806d2fe in access_allowed_mask (op=0x81a2360, e=0x7161a848,
    desc=0x817dc38, val=0x0, access=ACL_WRITE, state=0x0, maskp=0x0)
    at ../../../ldap/servers/slapd/acl.c:403
#12 0x080ad82c in bdb_add (op=0x81a2360, rs=0x6cbff51c)
    at ../../../../ldap/servers/slapd/back-bdb/add.c:208
#13 0x0805c979 in do_add (op=0x81a2360, rs=0x6cbff51c)
    at ../../../ldap/servers/slapd/add.c:318
#14 0x08057e30 in connection_operation (ctx=0x6cbff59c, arg_v=0x81a2360)
    at ../../../ldap/servers/slapd/connection.c:1048
#15 0x080dc717 in ldap_int_thread_pool_wrapper (xpool=0x817fc58)
    at ../../../ldap/libraries/libldap_r/tpool.c:467
#16 0x4020a2b6 in start_thread () from /lib/tls/libpthread.so.0

Thread 2 (Thread 1854929600 (LWP 30110)):
#0  0xffffe002 in ?? ()
#1  0x080af106 in bdb_entryinfo_add_internal (bdb=0x81a2194, ei=0x6e8ff08c,
    res=0x6e8ff088) at ../../../../ldap/servers/slapd/back-bdb/cache.c:231
---Type <return> to continue, or q <return> to quit---
#2  0x080af872 in bdb_cache_add (bdb=0x818c780, eip=0x81a2178, e=0x81a2d90,
    nrdn=0xfffffffc, locker=2147557194)
    at ../../../../ldap/servers/slapd/back-bdb/cache.c:766
#3  0x080ade96 in bdb_add (op=0x890ac70, rs=0x6e8ff49c)
    at ../../../../ldap/servers/slapd/back-bdb/add.c:484
#4  0x0805c979 in do_add (op=0x890ac70, rs=0x6e8ff49c)
    at ../../../ldap/servers/slapd/add.c:318
#5  0x08057e30 in connection_operation (ctx=0x6e8ff51c, arg_v=0x890ac70)
    at ../../../ldap/servers/slapd/connection.c:1048
#6  0x080dc717 in ldap_int_thread_pool_wrapper (xpool=0x817fc58)
    at ../../../ldap/libraries/libldap_r/tpool.c:467
#7  0x4020a2b6 in start_thread () from /lib/tls/libpthread.so.0

Thread 1 (Thread 1076531840 (LWP 30087)):
#0  0xffffe002 in ?? ()
#1  0x08056059 in slapd_daemon () at ../../../ldap/servers/slapd/daemon.c:2037
#2  0x0804cb08 in main (argc=7, argv=0xbfffeb94)
    at ../../../ldap/servers/slapd/main.c:830
#3  0x42015574 in __libc_start_main () from /lib/tls/libc.so.6
(gdb)
Prev by Date: Re: ACL Evaluation bug (ITS#3173)
Next by Date: Re: commit: ldap/servers/slapd/back-bdb proto-bdb.h search.c (ITS#3172)
Index(es):
- Chronological
- Thread