Re: filter preprocessing for performance improvement

[Howard Chu <hyc@symas.com>]

Eric Irrgang wrote:
> I wonder if it might be reasonable for the slapd frontend to do some 
> preprocessing of search filters to recognize no-ops and such.  Most 
> specifically, any search filter of the form (&(objectclass=*)(filter2)) 
> can be reduced to (filter2), right?

(objectclass=*) is already a no-op in back-bdb/back-hdb. I made that 
optimization back in November 2001.

So yes, it's a nice idea, been there, done that.

> I have a problem in that the first time someone performs a search for 
> 'objectclass=*' after slapd is restarted, the server is really bogged 
> down for a while.  Once the search has completed once, this is not a 
> problem. I assume that's due to the IDL cache.  However, I currently 
> have to keep the server unavailable after restarting slapd for upwards 
> of half an hour while I do an 'objectclass=*' search the first time.

Changing the behavior of (objectclass=*) isn't going to make any 
difference. If you specify a filter that is too broad, then slapd has to 
trawl through every entry in the DB because every entry satisfies the 
condition.

> I suppose a followup might be some preevaluation of ACLs before 
> performing easily-recognized searches.  For instance, the backend should 
> not have to cope with a search for objectclass=* if the user doesn't 
> have read access to anything as is often the case with an anonymous bind.

That's not a fair assumption.
--
  -- Howard Chu
  Chief Architect, Symas Corp.  http://www.symas.com
  Director, Highland Sun        http://highlandsun.com/hyc
  Chief Architect, OpenLDAP     http://www.openldap.org/project/