Re: >1024 connections in slapd / select->poll

[Howard Chu <hyc@symas.com>]

Igor Brezac wrote:

> On Thu, 18 Nov 2004, Howard Chu wrote:
>
> > Volker.Lendecke@SerNet.DE wrote:
> >
> > > On Mon, Nov 15, 2004 at 02:47:34PM -0800, Howard Chu wrote:
> > >
> > > > lists around. But on a heavily loaded server with thousands of 
> > > > active sessions, the fact is that your list of interesting 
> > > > descriptors is not static. Ultimately the server must iterate 
> > > > across all the thousands of descriptors, because the majority of 
> > > > them are probably active, and the server must insert and delete 
> > > > descriptors from the list continuously, because client sessions 
> > > > tend to come and go continually.
> > >
> > > All true, but the argument I think is that epoll has no inherent 
> > > limits, and is
> > > optimal for many idle connections. With thousands of active 
> > > connections I think
> > > a central select/poll/epoll/whatever is broken anyway. Only a SMP 
> > > machine with
> > > a descriptor-passing architecture might help here.
> > Well if you're feeling brave, I've just completed a patch in CVS HEAD 
> > supporting epoll. I haven't tried testing it with a massive number of 
> > connections yet, but the code now passes the regular test suite. It 
> > should be simple enough to add kqueue support as well now (I would 
> > have begun that but I don't have BSD installed anywhere at the 
> > moment). Regular poll can easily be added if you want, but there's 
> > really no reason to. Solaris /dev/poll is a bit more awkward.
>
> Is using libevent not an option?
libevent does too much. After spending several hours tinkering with it I 
gave up; too much overhead required for the single purpose we need to 
fulfill. It was still worthwhile to go through the exercise because it 
gave me a model of how things like kqueue (which I'd never seen before) 
work. But in the places where it overlapped things OpenLDAP does I would 
want to keep our infrastructure, which would mean adapting the libevent 
code, which would mean importing it into our CVS and maintaining it 
ourselves, which would mean a lot of time that's better spent elsewhere. 
....

--
 -- Howard Chu
 Chief Architect, Symas Corp.       Director, Highland Sun
 http://www.symas.com               http://highlandsun.com/hyc
 Symas: Premier OpenSource Development and Support