Re: >1024 connections in slapd / select->poll

[Howard Chu <hyc@symas.com>]

Volker.Lendecke@SerNet.DE wrote:

> On Wed, Nov 17, 2004 at 01:33:52AM -0800, Howard Chu wrote:
>  
>
> > Descriptor-passing per se isn't necessary as long as you have descriptor 
> > inheritance and shared memory. In that case you only need to provide one 
> > master process performing select()s on the listener sockets, and a pool 
> > of child processes inheriting those listeners. The child processes 
> > rotate through performing accept()s so that the live session sockets are 
> > only present in the children. You can spawn as many of these child 
> > processes as necessary to keep the total number of active descriptors in 
> > each process down to a manageable limit, using shared memory to 
> > coordinate the rest of the program.
> >    
>
> Okay, I did not know that you can issue multiple accept()s on a single
> listening socket.
You don't issue multiple accept()s at once. (Although Unix semantics 
shouldn't present any problem there.)

> How does the system guarantee at least close to even load
> distribution? Random?
>  
You can leave it random, or you can explicitly manage the child 
processes. There are any number of ways - the parent/master can just 
cycle thru the children in round-robin fashion, that's probably the 
easiest. This is standard IPC stuff, nothing exotic.

For example:
 1) the master creates listener sockets for all the network ports, and 
creates a pipe for communicating to its children.
 2) It forks as many children as desired, at any time during the life 
of the master. Each child thus has a copy of all of the listeners and 
the pipe.
 3) The master performs a select for read on the listeners, waiting 
however long. When a new connection arrives,
 4) (optional) the master performs a select for write on the pipe with 
a zero timeout. If it indicates a block, all the current children are 
busy, fork another child.
 5) the master writes a byte into the pipe indicating which listener is 
ready
 6) loop back to 3

In each child process:
 1) select for read on the pipe and any active connections
 2) do a non-blocking read on the pipe. if a byte is read from the 
pipe, perform an accept on the corresponding listener descriptor
 3) for any active connections - do whatever protocol processing is 
required
 4) loop back to 1.

A scheme like the above will scale to millions of connections if you 
really want to, and if you have a big enough multiprocessor machine to 
handle it. Any child process that has more active connections will 
automatically lower its own probability of getting the next new 
connection by virtue of being slower to get from step 3 back to step 1, 
thus balancing the load.

--
 -- Howard Chu
 Chief Architect, Symas Corp.       Director, Highland Sun
 http://www.symas.com               http://highlandsun.com/hyc
 Symas: Premier OpenSource Development and Support