Re: startup costs was Re: filter preprocessing for performance improvement

[Howard Chu <hyc@symas.com>]

Eric Irrgang wrote:
> I have a big problem whenever I have to restart a server.  Obviously, the 
> first search for objectclass=* is going to enumerate the whole directory. 
> The problem is that even an anonymous user can cause the server to execute 
> this search on the backend even though the ACLs and limits will keep them 
> from getting any results.  All it takes is a few poorly configured client 
> applications to do some sort of poll and I have connections hanging for 
> half an hour until the first objectclass=* search finishes.  I run out of 
> threads and every one of them is constantly trying to get CPU time.
>
> What I currently do is to keep a machine from being accessible by taking 
> it out of the load-balancer's rotation for the half hour or so that it 
> takes for me to do a search for objectclass=*, but I figure there has got 
> to be another way.  I have both eq and pres indexes on objectclass.  It's 
> just that I have a very big directory.

Get rid of the presence index. Never use a presence index for attributes 
that are present on the majority of your entries.

You should also probably look into sizelimit.unchecked in slapd.conf(5).
> I'm not trying to speed up the objectclass=* search.  I'm trying to figure 
> out how to keep it from impacting the server's responsiveness when it is 
> being performed under circumstances where no entries will be returned, 
> such as when sizelimits or ACLs (which are evaluated at the frontend after 
> the backend has performed the operation, right) will block things.  Any 
> suggestions?
>
> One thing I just thought of would be to have a single entry that would 
> always be accessible to any searcher and then set 'limits anonymous 
> size=1'.  Would that cause the backend operation to be canceled once the 
> first entry were returned?  That might save me something.

--
  -- Howard Chu
  Chief Architect, Symas Corp.  http://www.symas.com
  Director, Highland Sun        http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP     http://www.openldap.org/project/