[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: slapd lightweight dispatcher
> Some interesting observations from these tests... With the generic
> 2.3.21 code, throwing 26 SLAMD clients at a SunFire T2000 with a 1GHz
> processor, we get around 6400 authentications per second on a 1 million
> entry database. (Which is pretty respectable, really. SunOne only got
> 6233 for the same configuration)
> http://blogs.sun.com/roller/page/DirectoryManager/20060110
>
> What's interesting is that we only see about 19% CPU usage on slapd (13%
> user, 6% kernel), even though there are 32 virtual processors here and
> 16 worker threads (LWPs) and the database is 100% cached in RAM. Clearly
> slapd isn't making as good a use of the available CPUs as it could; if
> it were keeping the threads busy we should see right about 50% CPU usage.
>
> With the lightweight dispatcher and multi-conn array code, we see CPU
> utilization go up to 45% (20% user, 25% kernel) and throughput goes up
> to about 8800 authentications per second. With the lightweight
> dispatcher and the new code with no connections_mutex we get about 8900
> authentications per second, and CPU use is at 49% (23% user, 26% kernel).
>
> (We didn't collect numbers for just the lightweight dispatcher by itself
> yet.) At this point it seems we've eliminated a bottleneck in the
> frontend, only to find that there's probably a bottleneck in the backend
> as well. (Otherwise, performance ought to keep improving as we add
> worker threads, but it doesn't.) I'm puzzled about why we're seeing so
> much more kernel time vs user time. Hopefully spending some time with
> dtrace will tell us what's going on there. Ideally we should see a tiny
> percentage of kernel time, and we should see performance continue to
> improve as more worker threads are added. (Why stop at just 42% faster
> than The Other Guys, when we can potentially be 100% faster?)
>
> It's worth noting that this SLAMD authentication job consists of a
> search for a uid, followed by a Simple Bind on the returned entry's DN,
> so it's really the equivalent of almost 18,000 searches per second. And
> since we processed 6400 authentications/second using only 19% of the
> CPU, we really ought to be able to get 12,800 auths/second at 40%, and
> so on. (OK, it won't scale perfectly linearly because slapd only has one
> dispatcher thread, and at some point we will probably saturate the
> network interfaces on the server, or the gigabit switch. Also, we're
> struggling to get any more requests out of the clients. I really wish
> the SLAMD clients were written in C with libldap, this Java stuff is way
> too slow. It shouldn't take 26 clients to generate this workload; it
> should take at most 16 clients to keep the 16 server threads busy. Who
> ever thought that writing performance-critical code in Java was a good
> idea?)
Your numbers are quite interesting. About the workload tool issue, we
could do some work to improve the existing test suite in tests/progs. The
current versions in HEAD look much more flexible than ever: now they allow
random search/bind on entries looked up from the database, we're using
them to benchmork stock and custom slapd on different architectures.
Maybe a more consistent and coordinated development would turn them into
something more valuable?.
p.
Ing. Pierangelo Masarati
Responsabile Open Solution
OpenLDAP Core Team
SysNet s.n.c.
Via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
------------------------------------------
Office: +39.02.23998309
Mobile: +39.333.4963172
Email: pierangelo.masarati@sys-net.it
------------------------------------------