Re: ITS#3975: Lightweight Listener

[Jonghyuk Choi <jongchoi@us.ibm.com>]

Great to hear that the patch was already
committed. 

I uploaded a draft paper on the design
and performance evaluation on the lightweight listener design (which also
covers the multi listener design).
Hope that it can help the community
to better understand the implication of the lightweight listener patch
and to continue further discussion on how to further optimize it.

Here are the link to the ITS: http://www.openldap.org/its/index.cgi/Incoming?id=4087;selectid=4087
and the link to the draft paper: ftp://ftp.openldap.org/incoming/OpenLDAP_Conn_Mgmt.pdf

- Jong-Hyuk

------------------------
Jong Hyuk Choi
IBM Thomas J. Watson Research Center - Enterprise Linux Group
P. O. Box 218, Yorktown Heights, NY 10598
email: jongchoi@us.ibm.com
(phone) 914-945-3979    (fax) 914-945-4425   TL: 862-3979

"Kurt D. Zeilenga" <Kurt@OpenLDAP.org> Sent by: owner-openldap-devel@OpenLDAP.org 10/12/2005 08:47 PM

To:        openldap-devel@OpenLDAP.org         cc:                 Subject:        Re: ITS#3975: Lightweight Listener

First, I note, the patch was committed.  I had
intended
to wait a bit, but wasn't as careful as I should have been
in committing some other changes... and once committed,
well, I didn't see much point in reversing the commit.

Second, in the following text, where I say "select",
I mean either select or epoll.

In reviewing the code, my primary concern is that the
listener thread is not as light as possible.  For instance,
some writes appears to be done in the listener.  I'd really
like the listener to simply pass each selected descriptor
to a pooled worker immediately with minimal information
(selected on read, on write, etc.) and then suspend further
selection until the worker says its okay.

But as implemented, I believe the code to solve the
primary problem it was designed to solve: reduce
serialization in the event loop.  In particular,
avoid serialization of connection acceptance and
to allow processing of new operations while a new
connection was being accepted.  Note that the
intent of this code is to reduce the worse case
scenario where one connection denies service to
other connections (intentionally or not).  While
every attempt seems to have been made at reducing
the overhead, that is, to maintain the same level
of latency/throughput in absence of such denials,
the current evidence shows the code's overhead
to result in minimal reduction of throughput.
I suspect with some additional tweaking we can
eliminate most if not all of the remaining overhead
(for instance, removing the writes from the listener
loop should improve throughput).

Enjoy!

Kurt

At 03:53 PM 10/11/2005, Kurt D. Zeilenga wrote:
>Sang Seok Lim (IBM) contributed a patch which modifies
>slapd(8) listener to have a lightweight listener.
>The patch includes a number of changes to intended
>to improve the behavior of the listener.  I am currently
>reviewing the patch.  At present, I find it generally
>suitable for HEAD and hence intend to commit it (behind
>LDAP_DEVEL) in the next few days.
>
>Comments?
>
>Kurt

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature