Re: (ITS#4387) slapd-ldap backend leaks descriptors on closed connections on x86_64

["Pierangelo Masarati" <ando@sys-net.it>]

[redirected to OpenLDAP-devel]

> On Sun, 2006-02-05 at 18:57 +0000, hyc@symas.com wrote:
>> Associations are already changed upon completion of a Bind. But
>> obviously you cannot be sharing a single connection across multiple
>> clients when a Bind occurs, because any outstanding operations will be
>> (and must be) invalidated.
>
> I think a reasonable design would be to allow caching all of those
> connections and, on connection destroy, destroy all cached connections
> with that connid, i.e. don't limit connection destroy to the pair
> connid/authzdn.  This way, if for the same connection the same DN is
> used more than once, the cached connection can be exploited.  Mechanisms
> like "idle-timeout" and "conn-ttl" (not released yet) can be used to
> limit the amount of idle connections out there.  Unfortunately, they
> both act when a to be renewed connection is requested, not when it
> expires; the latter could be implemented by adding a periodic task that
> cycles over the connection tree and takes care of expired connections in
> cache.  I'd defer the latter, and provide the former ASAP.  I think a
> reasonable approach would be to have ldap_back_conn_cmp() compare on
> lc_conn first, then on lc_local_dn (i.e. reverse the current behavior).
> ldap_back_conn_destroy() would use a different implementation that just
> compares lc_conn, and avl_delete() should be called with it repeatedly,
> until no hit occurs.  Reverting the testing sequence above would allow
> ordered lookups to be correct in both cases, since cache entry storing
> would use the connid/DN check.

Let's talk about numbers; I've run some bind tests using the following
setup (minimal database, no DB_CONFIG, just 10 entries suitable for
slapd-bind in do_base mode): direct access to master; access thru proxy;
direct access to master, reusing the connection (-I switch), access thru
proxy, reusing the connection [this will soon appear as a regression check
for ITS#4390, as soon as I'm able to reproduce and fix it].  With logging
turned off, I get:

slapd-bind -H ldap://:9011 -b dc=example,dc=com -w foobar -l 500
Done 10000 Binds in 8.318260 seconds.
slapd-bind -H ldap://:9012 -b dc=example,dc=com -w foobar -l 500
Done 10000 Binds in 17.904342 seconds.
slapd-bind -H ldap://:9011 -b dc=example,dc=com -w foobar -l 500 -I
Done 10000 Binds in 1.561626 seconds.
slapd-bind -H ldap://:9012 -b dc=example,dc=com -w foobar -l 500 -I
Done 10000 Binds in 11.544833 seconds.

                  |  separate connection      |      same connection |
------------------+---------------------------+----------------------+
direct to master  |       8.318260                    1.561626
------------------+
thru proxy        |      17.904342                   11.544833
------------------+

It appears that for an usage like Courier's, i.e. many repeated binds with
a given set of identities on the same connection, it might be worth
caching connections by identity rather than by connid, keeping cached
connections alive even when the corresponding client->proxy connection
closes, if they're expected to be reused some time.  Some mechanism like
idle-timeout, conn-ttl or so could be used to limit the amount of idle or
no longer required connections.  This behavior, however, may be very
resource intensive and thus it should be optional.

p.





Ing. Pierangelo Masarati
Responsabile Open Solution
OpenLDAP Core Team

SysNet s.n.c.
Via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
------------------------------------------
Office:   +39.02.23998309          
Mobile:   +39.333.4963172
Email:    pierangelo.masarati@sys-net.it
------------------------------------------