[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: >1024 connections in slapd / select->poll

David Boreham wrote:

only present in the children. You can spawn as many of these child processes as necessary to keep the total number of active descriptors in each process down to a manageable limit, using shared memory to coordinate the rest of the program.

What's your goal with the multiple processes ? To work around an OS limit on the max number of
descriptors per process ? Or some other thing ?

(Modern OS versions allow large numbers of descriptors
per process, BTW, and they have threads which allow
exploitation of SMP hardware in a single process).

Not to work around the OS limit - to work around the unwieldiness of large fdset's. This is just an application of the same divide-and-conquer strategy that applies to binary search, trees, and other such log(N) algorithms. Just because an OS allows a single process to use thousands of file descriptors does not mean it's a good idea to do so. At the kernel level it's going to be more efficient to deal with a cluster of average sized processes vs a single process with a huge context. Eventually something is going to come along that causes the entire process to block, and all those threads are going to suspend, doing nothing besides occupy address space, whereas a cluster of processes will provide more scheduling opportunities for useful work to get done; even if one process completely blocks, other processes may continue.

 -- Howard Chu
 Chief Architect, Symas Corp.       Director, Highland Sun
 http://www.symas.com               http://highlandsun.com/hyc
 Symas: Premier OpenSource Development and Support