RE: Indexed searches which contain a NOT filter return incomplete results (ITS#1405)

[Charlie Bodycote <Charlie.Bodycote@jumpleads.com> (by way of "Kurt D. Zeilenga" <Kurt@OpenLDAP.org>)]

Kurt, 

Could I respond to your comment on the issue I raised? Sorry, I don't seem to have permission to post a reply on the OpenLDAP database...

You suggested that the 'real' fix to the above problem is to not use an index to support not filters. 

If we do this, are we not effectively saying "we do not support NOT filters". As soon as the DIB gets to any significant size, unindexed searches become effectively useless. 

Is this really acceptable? 

It is indeed quite difficult to maintain the substring index when values are deleted (you have to know exactly what values were deleted, and the current code structure makes this hard).

However, it is easy to do for "pres" indices, and I think we should use that. So how about: 
- for now, only exploit "pres" indices for NOT filter evaluations, but perform other filter evaluations by brute-force 
- in the long term (if someone has a real need), engineer a way to pass the "old" attribute values to the indexing component when removing values.

Does this sound sane? 

Cheers, 

Charlie 

-----Original Message----- 
From: Kurt D. Zeilenga [<mailto:Kurt@OpenLDAP.org>mailto:Kurt@OpenLDAP.org] 
Sent: 24 October 2001 18:16 
To: charlie@jumpleads.com 
Cc: openldap-its@OpenLDAP.org 
Subject: Re: Indexed searches which contain a NOT filter return 
incomplete results (ITS#1405) 

At 08:54 AM 2001-10-24, charlie@jumpleads.com wrote: 
>Full_Name: Charlie Bodycote 
>Version: 1.2.13 
>OS: Redhat Linux 7.1 
>URL: <ftp://ftp.openldap.org/incoming/>ftp://ftp.openldap.org/incoming/ 
>Submission from: (NULL) (217.204.231.158) 
> 
>I have a partial fix (which fixes the problem for the "pres" index, but not for other indices). 

I note that as indices are designed to return candidate entries 
as opposed to matching entries.  While it may be possible to 
alter some indices so that only matched entries are returned, 
it is not feasible to for all indices (consider substrings 
indices).  The 'real' fix is not to use indices in support 
of negated filter components.