[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: slapd lightweight dispatcher

Howard Chu wrote:
What's interesting is that we only see about 19% CPU usage on slapd (13% user, 6% kernel), even though there are 32 virtual processors here and 16 worker threads (LWPs) and the database is 100% cached in RAM. Clearly slapd isn't making as good a use of the available CPUs as it could; if it were keeping the threads busy we should see right about 50% CPU usage.

With the lightweight dispatcher and multi-conn array code, we see CPU utilization go up to 45% (20% user, 25% kernel) and throughput goes up to about 8800 authentications per second. With the lightweight dispatcher and the new code with no connections_mutex we get about 8900 authentications per second, and CPU use is at 49% (23% user, 26% kernel).

(We didn't collect numbers for just the lightweight dispatcher by itself yet.)

Ah, the lightweight dispatcher code by itself causes a distinct drop in performance - 5400 authentications/second, and 20% CPU use (12% user, 8% kernel). I guess the increased latency from the thread dispatching is hurting it, given the fully serialized connection handoff. I recall we observed this effect the last time we were benchmarking as well.

I really wish the SLAMD clients were written in C with libldap, this Java stuff is way too slow. It shouldn't take 26 clients to generate this workload; it should take at most 16 clients to keep the 16 server threads busy. Who ever thought that writing performance-critical code in Java was a good idea?

Running the same job against back-null, our clients max out at around 14,500 auths/second, with only 30% CPU usage. So that puts a damper on my efforts to see how far adding more worker threads will scale; it's not even taxing the threads that are there and we don't have any more clients to throw at it. Maybe adding some sleep()s to back-null will give a better idea, just to see how well the frontend scales with more threads.

 -- Howard Chu
 Chief Architect, Symas Corp.  http://www.symas.com
 Director, Highland Sun        http://highlandsun.com/hyc
 OpenLDAP Core Team            http://www.openldap.org/project/