Re: connection management issues

["Kurt D. Zeilenga" <Kurt@OpenLDAP.org>]

Well, I'm taking a third approach to resolving these issues.

I've introduce a connection state (inactive, active, binding, closing)
to our connection structure.  When a bind occurs, the state changes
from active to binding.  If a new op occurs while in state 'binding',
it is placed in a separate list of ops_pending list for this connection.
When each operation completes, it calls routine to reschedule pending
operations.

I'm also using the state to determine if the connection should be
placed in the listener's input fd_set (don't listen if inactive or
closing).

A first cut (w/ only bind handling) has been committed to -devel.

I'm not sorting out the details to handle races upon closing.
I hope to be able to shutdown(fd, SHUT_RD) the stream when a
unbind occurs and defer the close() until all executing op
threads have completed.

This approach preserved the ability to compile under -DNO_THREADS.
It can even be extended to support per connection thread limits.

As I noted in some private discussions, it is possible to extend this
our code to support both thread pools and -DNO_THREADS.  However,
this would still require the architecture to limit itself to a
single listener thread and a single thread pool.  So, -DNO_THREADS
has a bit of a reprive.

>I've come to the conclusion that to resolve the these condition will require
>significant restructuring of the connection management issues.   To resolve
>the bind issue, slapd needs to keep same connection operations posted after
>receiving the bind pending until the bind completes.  This could be done by
>blocking new connections, but this would block the listener which might
>deadlock the connection (if it has a write pending).
>
>I've come to the conclusion that I need a queue of hold pending operations
>while waiting for the bind to complete.  When the bind completes, then this
>queue is flushed.
>
>There are two basic ways this could be handled.
>
>1) The thread which just completed the bind could flush the queue (spinning
>off new threads for each pending operation).
>
>2) A thread pool could be implemented which supported pending operations.
>The thread which just completed the bind would just run though pending
>operations to update their status to 'runnable'.
>
>I believe both are about the same level of complexity and would require
>similiar amounts of time.   With option 1, we maintain -DNO_THREAD but do not
>make any significant architectural improvements.  With option 2 we gain a
>thread pool, but lose -DNO_THREAD support.
>
>Comments?  Suggestions?
>
>Kurt
>
>