[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: (ITS#3835) Lightweight Listener Thread

Looks interesting. I have not tested the patch yet, but a quick read 
thru shows a couple problems:

The definition of connection_op_queue() is ifdef'd, but it is invoked 
unconditionally in connection_op_activate().

In slapd_suspend() there is a check "#ifdef SLAP_EVENTS_ARE_INDEXED" but 
this macro is always defined; it is the value (1 or 0) that is 
significant. Judging from the comments I believe the correct test is 
actually "#if !SLAP_EVENTS_ARE_INDEXED" here.

slim@us.ibm.com wrote:
> Full_Name: sang seok lim
> Version: Current HEAD
> OS: SUSE Linux 9
> URL: ftp://ftp.openldap.org/incoming/lightweight_listener.diff
> Submission from: (NULL) (
> This patch enables multi-threaded processing of incoming connection
> establishment and LDAP request reading/parsing so as to make the listener thread
> of slapd more lightweight. Lightening a listener thread will make slapd more
> responsive and robust.
> The current slapd consists of a single listener thread and a thread pool. The
> listener thread is in charge of handling incoming connections, reading/parsing
> LDAP requests, waking up blocked operation, etc. The first two operations are
> CPU-cycles consuming routines and are executed against over all triggered events
> one by one, which limits the parallelism of connection management in slapd. For
> example, in this architecture, if the listener thread is stalled or occupied by
> an abnormal connection, it will hinder processing of other normal connections at
> once.
> As a remedy, this patch enables a listener thread to hand-off the processing of
> each triggered event to the worker threads instead of having the listener thread
> process all triggered events by itself.
> At the code level, slapd_handle_listener() and connection_read() are
> multi-threaded. On the reception of a new connection, the listener does only
> select() on a listening socket and then all necessary work will be done by a
> separate worker thread. Likewise, on the reception of new LDAP requests, it only
> select()s the corresponding event and then submits the event to the worker
> thread pool.
> Performance numbers with this patch are as follows,
> System-under-test:
> H/W: IBM eServer OpenPower with 2 1.5GHz CPUs (SMT) and 16GB memory
> O/S: SUSE Linux 9
> 10K entries (cached in an entry cache)
> # of concurrent connections : 100
> Throughput
> W/O patch: 18,450 op/sec
> W patch: 18,845 op/sec
> Throughput with 4,000 idle TCP connections
> W/O patch:  8,748 op/sec
> W patch: 11,383 op/sec
> Throughput with 2 second delay in one out of a hundred incoming connections: ex)
> 2 seconds delay in reverse DNS look-up
> W/O patch: 505 op/sec
> W patch: 10,659 op/sec
> W patch+SLAP_PERSISTENT_ACCEPT_THREAD(see below): 806 op/sec
> The experiment confirms that with the patch the slapd becomes more responsive
> and robust to an unexpected delay in a listener thread and a large number of
> idle connections.
> In addition, if SLAP_PERSISTENT_ACCEPT_THREAD is turned on in the patch, the
> listener will create a single, persistent, and standalone thread which executes
> slapd_handle_listener() in a loop. It achieves a very high connection accept()
> rate. But its use is confined to a thread-supported platform only and it is not
> multi-threaded so that it lacks parallelism in establishing connections. For
> now, its multi-threading extension is under consideration.
> accept() rate comparison
> This patch has been devised as the second approach to the connection management
> in slapd (you can see the first approach, multi-listener threads in ITS#3665). I
> really appreciate any comments to the patch.
> Sang-Seok Lim
> IBM T.J Watson Research,
> P.O. Box 218, Yorktown Heights, NY 10598
> slim@us.ibm.com

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support