Full_Name: sang seok lim Version: HEAD OS: SUSE Linux 9 URL: ftp://ftp.openldap.org/incoming/sangseok-1025.diff Submission from: (NULL) (129.34.20.23) This patch enables a semaphore-based resource access control in slapd so as to mitigate the listener thread overheads, mainly from select(). The resource is defined as available worker threads in a thread pool at given time. Mitigating the overheads improves the throughput of slapd especially when it handles a number of concurrent connections. The current slapd consists of a single listener thread and a thread pool managing multiple worker threads. The listener thread monitors incoming operations by using select() and submits them to the pool for further processing. Then worker threads process the submitted operations. In the current architecture, even when there is no available worker thread, the listener keeps select()ing, processing the incoming requests, and submitting them to the pool. These submitted requests will not be processed until any worker thread becomes available. The select() operation itself is not cheap, when a number of concurrent connections is handled in slapd. This patch utilizes a lazy semaphore, the variant of a semaphore, in order to eliminate the excessive selecting() overheads of the listener thread. With the semaphore, before submitting an operation, a thread checks if there is any available worker thread by calling ldap_lazy_sem_wait(). If there is no available worker thread, ldap_lazy_sem_wait() will be blocked, putting the thread into sleep. After completing the operation, a worker thread calls ldap_lazy_sem_post() which increases the count of the semaphore and wakes up blocked threads if there is any. In brief, with the semaphore, if there is no available worker thread in the pool, the listener thread will be put into sleep until the predefined (lazyness in newly defined semaphore of this patch) number of threads becomes available. The performance numbers of the patched slapd are as follows, System-under-test: H/W: IBM eServer OpenPower with 2 1.5GHz CPUs (SMT) and 16GB memory O/S: SUSE Linux 9 10K entries (all cached in an entry cache) 10K entry random search ftp://ftp.openldap.org/incoming/sangseok-perf.PNG The bars and the lines represent search throughput and select() rate respectively. The search performance of slapd was improved up to 15% in the range of 128 ~ 10,000 connections due to the significant decrease in select() rate. The experiment confirms that the patch relieves the select() overheads when there is a large number of concurrent connections to slapd. This patch consists of three parts: (1) Semaphore-based load conditioning (within SLAP_SEM_LOAD_CONTROL) (2) Lightweight listener (within SLAP_LIGHTWEIGHT_LISTENER) Please check ITS#3835 to see what it is. http://www.openldap.org/its/index.cgi/Development?id=3835 * This patch obsoletes the previously submitted patch of ITS#3835. (3) Multiple connection array support (within SLAP_MULTI_CONN_ARRAY) - Execution of connection_read() is protected by a global mutex (connections_mutex). So even though the execution is parallelized by multi-threaded processing in ITS#3835, it is still serialized by the mutex which incurs 1-3% CPU idleness. By using this patch, it minimizes the chance of lock conflicting so as to eliminate the CPU idle and increase throughput. It is supplemental to part (2). *Each part can be applied independently from other parts. Recommended combinations, however, are (1)+(2)+(3) for responsiveness and throughput, (1) only for throughput, or (2)+(3) only for responsiveness improvement. I really appreciate any comments to the patch. Sang-Seok Lim IBM T.J Watson Research, P.O. Box 218, Yorktown Heights, NY 10598 slim@us.ibm.com
changed notes changed state Open to Feedback
moved from Incoming to Contrib
moved from Contrib to Development
changed notes
changed notes changed state Feedback to Closed
moved from Development to Archive.Development
committed to HEAD