[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: >1024 connections in slapd / select->poll

Volker.Lendecke@SerNet.DE wrote:

On Mon, Nov 15, 2004 at 02:47:34PM -0800, Howard Chu wrote:

lists around. But on a heavily loaded server with thousands of active sessions, the fact is that your list of interesting descriptors is not static. Ultimately the server must iterate across all the thousands of descriptors, because the majority of them are probably active, and the server must insert and delete descriptors from the list continuously, because client sessions tend to come and go continually.

All true, but the argument I think is that epoll has no inherent limits, and is optimal for many idle connections.

OK. But most servers where performance limits are being experienced have a higher proportion of active connections to idle connections. The most widely used protocols - http, smtp, pop, etc. have extremely short connection lifetimes.

With thousands of active connections I think
a central select/poll/epoll/whatever is broken anyway.


Only a SMP machine with
a descriptor-passing architecture might help here.

Descriptor-passing per se isn't necessary as long as you have descriptor inheritance and shared memory. In that case you only need to provide one master process performing select()s on the listener sockets, and a pool of child processes inheriting those listeners. The child processes rotate through performing accept()s so that the live session sockets are only present in the children. You can spawn as many of these child processes as necessary to keep the total number of active descriptors in each process down to a manageable limit, using shared memory to coordinate the rest of the program.

But yes, to your main point, a software design that handles tens of thousands of descriptors in a single process is broken. I think the fact that epoll/kqueue can scale to tens of thousands of descriptors is somewhat academic, since you probably can't do anything useful on those connections in a single process. A typical thread subsystem will let you use a couple thousand threads, nowhere near enough to service every descriptor with a low latency.

(At this point I'm just carrying on this conversation out of theoretical interest. I've already decided to look into migrating to libevent http://www.monkey.org/~provos/libevent/ to take advantage of these newer mechanisms.)

 -- Howard Chu
 Chief Architect, Symas Corp.       Director, Highland Sun
 http://www.symas.com               http://highlandsun.com/hyc
 Symas: Premier OpenSource Development and Support