Re: ITS#3835 Lightweight Listener Thread

[jwm@horde.net]

On Tue, Aug 02, 2005 at 08:57:37AM -0400, Sang s Lim wrote:
> Could you elaborate on incoming request characteristics of the host.
> - H/W spec of the host.

Sure. They all have dual 2.4GHz Xeon CPUs and 2GB RAM. They run Debian Linux
(sarge) with the most recent Debian kernel, 2.4.27.

We have another machine in the cluster running a 2.6 kernel, which allows us
to use NPTL for threading instead of LinuxThreads. It runs OpenLDAP 2.1.30.
The 2.4 -> 2.6 kernel move did not lower CPU consumption significantly,
perhaps a few percentage points.

> - the number of concurrent connections.

Steady-state is about 600 or 700, but we sometimes see peaks of twice that.

> - incoming request types: search, modify, add, etc.

They're always under heavy search load with relatively few writes; I'm
guessing one modify/add/delete every two or three seconds.

> - incoming request pattern: how many requests per single bind().

We have two different patterns. The first binds, performs a single search
that returns a single entry, then unbinds. The second type persistent and
are long-lived. They bind, then issue hundreds or thousands of searches as
requests come in to the supported service (i.e., one search for each e-mail
message delivered, etc.) These connections can persist for several hours or
even days.

In our case with the lightweight listener code, I suspect a lot of the CPU
consumption is in select(). An strace(1) of it yields heavy select()
activity and not much else; just some small read()s, time(), sched_yield(),
and kill() calls.

john
-- 
John Morrissey          _o            /\         ----  __o
jwm@horde.net        _-< \_          /  \       ----  <  \,
www.horde.net/    __(_)/_(_)________/    \_______(_) /_(_)__