[Date Prev][Date Next] [Chronological] [Thread] [Top]

(ITS#6798) Mutex starvation on two-level referral for SASL connection



Full_Name: Stephen Gallagher
Version: 2.4.21-0ubuntu5.2
OS: Ubuntu 10.04
URL: 
Submission from: (NULL) (98.110.239.235)


This was discovered by a user of SSSD. When referrals are enabled with
LDAP_OPT_REFERRALS on, SSSD has a rebind procedure set up to handle
authenticating to the new server. This seems to work fine when we're dealing
with a simple bind, but when we attempt to use SASL bind (for Kerberos-based
GSSAPI authentication), we discovered a problem.

Our rebind procedure calls ldap_sasl_interactive_bind_s() with LDAP_SASL_QUIET.

This arrangement works fine for a single referral, however if the server has
nested referrals (say, entry1 refers to entry2 which refers to entry3 on another
server) then we hit a deadlock condition.

Attaching gdb, we see the backtrace included at the bottom of this message. What
appears to be happening is that for the first ldap_sasl_interactive_bind_s(),
openldap is locking a mutex, and when it is called a second time it's attempting
to lock that same mutex that has not yet been released.


Backtrace:
#0  __lll_lock_wait () at
../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:136
No locals.
#1  0x00007f48285975d9 in _L_lock_953 () from /lib/libpthread.so.0
No symbol table info available.
#2  0x00007f48285973fb in __pthread_mutex_lock (mutex=0x7f4829c05980) at
pthread_mutex_lock.c:61
        ignore1 = <value optimized out>
        ignore2 = 700471680
        ignore3 = -512
        __PRETTY_FUNCTION__ = "__pthread_mutex_lock"
        type = <value optimized out>
#3  0x00007f48299d3e4e in ldap_sasl_interactive_bind_s () from
/usr/lib/libldap_r-2.4.so.2
No symbol table info available.
#4  0x00007f4825ae00fd in sdap_rebind_proc (ldap=0x2241b80, 
    url=0x2250220 "ldap://DomainDnsZones.org.example.com/DC=DomainDnsZones,DC=org,DC=example,DC=com";,

    request=<value optimized out>, msgid=<value optimized out>, params=<value
optimized out>)
    at src/providers/ldap/sdap_async_connection.c:1624
        p = <value optimized out>
        sasl_mech = <value optimized out>
        user_dn = <value optimized out>
        password = {bv_len = 0, bv_val = 0x0}
        ctrls = {0x0, 0x0}
        tmp_ctx = <value optimized out>
        ret = <value optimized out>
        __FUNCTION__ = "sdap_rebind_proc"
#5  0x00007f48299df6d1 in ldap_new_connection () from
/usr/lib/libldap_r-2.4.so.2
No symbol table info available.
#6  0x00007f48299e0523 in ldap_send_server_request () from
/usr/lib/libldap_r-2.4.so.2
No symbol table info available.
#7  0x00007f48299e11cd in ldap_chase_v3referrals () from
/usr/lib/libldap_r-2.4.so.2
No symbol table info available.
#8  0x00007f48299cbf95 in ?? () from /usr/lib/libldap_r-2.4.so.2
No symbol table info available.
#9  0x00007f48299ccc2d in ldap_result () from /usr/lib/libldap_r-2.4.so.2
No symbol table info available.
#10 0x00007f48299d4788 in ldap_sasl_bind_s () from /usr/lib/libldap_r-2.4.so.2
No symbol table info available.
#11 0x00007f48299d1751 in ldap_int_sasl_bind () from
/usr/lib/libldap_r-2.4.so.2
No symbol table info available.
#12 0x00007f48299d3ea8 in ldap_sasl_interactive_bind_s () from
/usr/lib/libldap_r-2.4.so.2
No symbol table info available.
#13 0x00007f4825ae00fd in sdap_rebind_proc (ldap=0x2241b80, 
    url=0x225bc20 "ldap://ForestDnsZones.org.example.com/DC=ForestDnsZones,DC=org,DC=example,DC=com";,

    request=<value optimized out>, msgid=<value optimized out>, params=<value
optimized out>)
    at src/providers/ldap/sdap_async_connection.c:1624
        p = <value optimized out>
        sasl_mech = <value optimized out>
        user_dn = <value optimized out>
        password = {bv_len = 0, bv_val = 0x0}
        ctrls = {0x0, 0x0}
        tmp_ctx = <value optimized out>
        ret = <value optimized out>
        __FUNCTION__ = "sdap_rebind_proc"
#14 0x00007f48299df6d1 in ldap_new_connection () from
/usr/lib/libldap_r-2.4.so.2
No symbol table info available.
#15 0x00007f48299e0523 in ldap_send_server_request () from
/usr/lib/libldap_r-2.4.so.2
No symbol table info available.
#16 0x00007f48299e11cd in ldap_chase_v3referrals () from
/usr/lib/libldap_r-2.4.so.2
No symbol table info available.
#17 0x00007f48299cbf95 in ?? () from /usr/lib/libldap_r-2.4.so.2
No symbol table info available.
#18 0x00007f48299ccc2d in ldap_result () from /usr/lib/libldap_r-2.4.so.2
No symbol table info available.
#19 0x00007f4825acfcb4 in sdap_process_result (ev=0x221b1f0, pvt=<value
optimized out>)
    at src/providers/ldap/sdap_async.c:178
        sh = 0x2241650
        no_timeout = {tv_sec = 0, tv_usec = 0}
        te = <value optimized out>
        msg = <value optimized out>
        ret = <value optimized out>
        __FUNCTION__ = "sdap_process_result"
#20 0x00007f482b4f1825 in ?? () from /usr/lib/libtevent.so.0
No symbol table info available.
#21 0x00007f482b4f363b in ?? () from /usr/lib/libtevent.so.0
No symbol table info available.
#22 0x00007f482b4f09f0 in _tevent_loop_once () from /usr/lib/libtevent.so.0
No symbol table info available.
#23 0x00007f482b4f0a5b in ?? () from /usr/lib/libtevent.so.0
No symbol table info available.
#24 0x0000000000431e51 in server_loop (main_ctx=0x221c360) at
src/util/server.c:526
No locals.
#25 0x000000000040f8bb in main (argc=5, argv=<value optimized out>)
    at src/providers/data_provider_be.c:1333
        opt = <value optimized out>
        pc = <value optimized out>
        be_domain = 0x2212490 "AALTO"
        srv_name = <value optimized out>
        conf_entry = <value optimized out>
        main_ctx = 0x221c360
        ret = 0
        long_options = {{longName = 0x0, shortName = 0 '\000', argInfo = 4, arg
= 0x644d00, 
            val = 0, descrip = 0x436e64 "Help options:", argDescrip = 0x0}, {
            longName = 0x436e72 "debug-level", shortName = 100 'd', argInfo = 2,

            arg = 0x644de0, val = 0, descrip = 0x436e43 "Debug level",
argDescrip = 0x0}, {
            longName = 0x436e7e "debug-to-files", shortName = 102 'f', argInfo =
0, 
            arg = 0x644de4, val = 0, 
            descrip = 0x437ad8 "Send the debug output to files instead of
stderr", 
            argDescrip = 0x0}, {longName = 0x436e8d "debug-timestamps",
shortName = 0 '\000', 
            argInfo = 2, arg = 0x644cc0, val = 0, descrip = 0x436e4f "Add debug
timestamps", 
            argDescrip = 0x0}, {longName = 0x438458 "domain", shortName = 0
'\000', 
            argInfo = 1, arg = 0x7fffe703f208, val = 0, 
            descrip = 0x437b10 "Domain of the information provider (mandatory)",

            argDescrip = 0x0}, {longName = 0x0, shortName = 0 '\000', argInfo =
0, arg = 0x0, 
            val = 0, descrip = 0x0, argDescrip = 0x0}}
        __FUNCTION__ = "main"