Issue 2588 - subinitial and subfinal indexes not working
Summary: subinitial and subfinal indexes not working
Status: VERIFIED FIXED
Alias: None
Product: OpenLDAP
Classification: Unclassified
Component: slapd (show other issues)
Version: unspecified
Hardware: All All
: --- normal
Target Milestone: ---
Assignee: OpenLDAP project
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2003-06-10 15:26 UTC by andreas@canonical.com
Modified: 2014-08-01 21:06 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description andreas@canonical.com 2003-06-10 15:26:17 UTC
Full_Name: Andreas Hasenack
Version: 2.1.21
OS: linux
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (200.181.170.6)


slapd.conf, index I'm using:
index   cn,sn,mail      eq,subinitial,subany,subfinal

I also tried with just "eq,sub", same results.

searches being performed: string, *string*, *string and string*.
"string" and "*string*" are very fast (<1s), while "*string" and "string*" are
very slow (>10s).
Results below, excerpts from the slapd log for the searches above:

Jun 10 11:39:04 matrix slapd[31093]: conn=0 op=1 SRCH base="o=myCompany" scope=2
filter="(mail=Employee-2@section-90.mycompany.com)"
Jun 10 11:39:04 matrix slapd[31093]: conn=0 op=1 SEARCH RESULT tag=101 err=0
nentries=1 text=

Jun 10 11:39:25 matrix slapd[31093]: conn=1 op=1 SRCH base="o=myCompany" scope=2
filter="(mail=*Employee-2@section-90.mycompany.com*)"
Jun 10 11:39:25 matrix slapd[31093]: conn=1 op=1 SEARCH RESULT tag=101 err=0
nentries=1 text=

Jun 10 11:41:23 matrix slapd[31093]: conn=3 op=1 SRCH base="o=myCompany" scope=2
filter="(mail=*Employee-2@section-90.mycompany.com)"
Jun 10 11:41:41 matrix slapd[31093]: conn=3 op=1 SEARCH RESULT tag=101 err=0
nentries=1 text=

Jun 10 11:42:18 matrix slapd[31099]: conn=4 op=1 SRCH base="o=myCompany" scope=2
filter="(mail=Employee-2@section-90.mycompany.com*)"
Jun 10 11:42:43 matrix slapd[31099]: conn=4 op=1 SEARCH RESULT tag=101 err=0
nentries=1 text=

For this test I created a dummy database with about 108k entries in the
following format:
dn: o=myCompany
o: myCompany
objectClass: top
objectClass: organization

dn: ou=Departments, o=myCompany
ou: Departments
objectClass: top
objectClass: organizationalUnit

dn: ou=Section-1, ou=Departments, o=myCompany
ou: Section-1
objectClass: top
objectClass: organizationalUnit

dn: ou=People, ou=Section-1, ou=Departments, o=myCompany
ou: People
objectClass: top
objectClass: organizationalUnit

dn: uid=Employee-1, ou=People, ou=Section-1, ou=Departments, o=myCompany
uid: Employee-1
cn: Employee-1-cn
givenName: Employee-1-gn
sn: Employee-1-sn
mail: Employee-1@Section-1.mycompany.com
uidNumber: 1001
gidNumber: 1001
homeDirectory: /home/Employee-1
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
objectClass: top
objectClass: posixAccount

Then there is Employee-2, 3, etc for Section-1, then all over again for
Section-2 and so on. The only globally unique attribute is the mail one.
As a sidenote, this also happens with at least openldap version 2.0.25 and
2.0.27 (I didn't test other versions).

Comment 1 Kurt Zeilenga 2003-08-20 23:28:33 UTC
changed notes
Comment 2 Howard Chu 2003-09-14 03:43:17 UTC
changed notes
moved from Incoming to Software Bugs
Comment 3 Howard Chu 2003-09-14 04:26:36 UTC
>slapd.conf, index I'm using:
>index   cn,sn,mail      eq,subinitial,subany,subfinal

>I also tried with just "eq,sub", same results.

>searches being performed: string, *string*, *string and string*.
>"string" and "*string*" are very fast (<1s), while "*string" and "string*"
>are very slow (>10s).

The subinitial (and subfinal) index code only helps if the indexed strings
are unique within their first (or last) 4 characters. For the data you're
using, this condition doesn't apply; the majority of your entries all fall
into the same index slot and so they must be checked individually.

This same substring indexing logic has been in place for a long time, which
is why the same behavior is present in 2.0, 2.1, and probably even 2.2, in
all the  backend types that support indexing (back-ldbm, back-bdb, back-hdb).

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support

Comment 4 Howard Chu 2003-09-14 04:26:54 UTC
changed state Open to Feedback
Comment 5 Kurt Zeilenga 2003-10-12 16:14:11 UTC
changed notes
changed state Feedback to Closed
Comment 6 Howard Chu 2004-09-28 00:53:57 UTC
changed notes
changed state Closed to Test
Comment 7 Kurt Zeilenga 2004-10-30 22:29:19 UTC
changed notes
changed state Test to Closed
moved from Software Bugs to Software Enhancements
Comment 8 Howard Chu 2009-02-17 04:49:50 UTC
moved from Software Enhancements to Archive.Software Enhancements
Comment 9 OpenLDAP project 2014-08-01 21:06:57 UTC
Not a bug.  Aspect of frontend indexing design.
Redesinged in HEAD, for release in 2.3