[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: >1024 connections in slapd / select->poll

Volker.Lendecke@SerNet.DE wrote:

On Wed, Nov 17, 2004 at 01:33:52AM -0800, Howard Chu wrote:

Descriptor-passing per se isn't necessary as long as you have descriptor inheritance and shared memory. In that case you only need to provide one master process performing select()s on the listener sockets, and a pool of child processes inheriting those listeners. The child processes rotate through performing accept()s so that the live session sockets are only present in the children. You can spawn as many of these child processes as necessary to keep the total number of active descriptors in each process down to a manageable limit, using shared memory to coordinate the rest of the program.

Okay, I did not know that you can issue multiple accept()s on a single listening socket.

You don't issue multiple accept()s at once. (Although Unix semantics shouldn't present any problem there.)

How does the system guarantee at least close to even load
distribution? Random?

You can leave it random, or you can explicitly manage the child processes. There are any number of ways - the parent/master can just cycle thru the children in round-robin fashion, that's probably the easiest. This is standard IPC stuff, nothing exotic.

For example:
1) the master creates listener sockets for all the network ports, and creates a pipe for communicating to its children.
2) It forks as many children as desired, at any time during the life of the master. Each child thus has a copy of all of the listeners and the pipe.
3) The master performs a select for read on the listeners, waiting however long. When a new connection arrives,
4) (optional) the master performs a select for write on the pipe with a zero timeout. If it indicates a block, all the current children are busy, fork another child.
5) the master writes a byte into the pipe indicating which listener is ready
6) loop back to 3

In each child process:
1) select for read on the pipe and any active connections
2) do a non-blocking read on the pipe. if a byte is read from the pipe, perform an accept on the corresponding listener descriptor
3) for any active connections - do whatever protocol processing is required
4) loop back to 1.

A scheme like the above will scale to millions of connections if you really want to, and if you have a big enough multiprocessor machine to handle it. Any child process that has more active connections will automatically lower its own probability of getting the next new connection by virtue of being slower to get from step 3 back to step 1, thus balancing the load.

 -- Howard Chu
 Chief Architect, Symas Corp.       Director, Highland Sun
 http://www.symas.com               http://highlandsun.com/hyc
 Symas: Premier OpenSource Development and Support