[Date Prev][Date Next] [Chronological] [Thread] [Top]
Re: (ITS#7231) back0hdb search results disappear depending on search base

To: openldap-its@OpenLDAP.org
Subject: Re: (ITS#7231) back0hdb search results disappear depending on search base
From: hyc@symas.com
Date: Sat, 7 Apr 2012 14:38:52 GMT
Auto-submitted: auto-generated (OpenLDAP-ITS)
hume-ol@bofh.ca wrote:
> Full_Name: Brandon Hume
> Version: -OPENLDAP_REL_ENG_2_4-0e03b13
> OS: Redhat AS6
> URL: http://den.bofh.ca/~hume/ldap_missing_results.tar.gz
> Submission from: (NULL) (2610:170:2c00:7:224:1ff:fed4:b4ff)

Thanks for the report and test case. Fixed now in git master.

> Firstly, using RE24 built with the following ./configure:
>
>
> #!/bin/sh
> env CC=gcc CFLAGS="-m64 -ggdb" LDFLAGS="-Wl,-rpath -Wl,/appl/ldap/lib
> -L/appl/ldap/lib" CPPFLAGS="-I/appl/ldap/include" \
>          ./configure --prefix=/appl/ldap \
>                  --libexecdir=/appl/ldap/sbin --localstatedir=/var/run
> --datadir=/appl/ldap/data \
>                  --sysconfdir=/appl/ldap/etc --mandir=/appl/ldap/man
> --with-subdir=no \
>                  --disable-rlookups --with-tls=openssl \
>                  --enable-slapd --enable-crypt --with-cyrus-sasl=no
> --enable-debug \
>                  --enable-dynamic --enable-modules \
>                  --enable-backends=mod --enable-overlays=mod --enable-sql=no \
>                  --enable-perl=no --enable-ndb=no --enable-shell=no
>
> Paired with BerkeleyDB built with the following:
>
> #!/bin/sh
> env CC=gcc CFLAGS="-m64 -ggdb" LDFLAGS="-Wl,-rpath -Wl,/appl/ldap/lib
> -L/appl/ldap/lib" CPPFLAGS="-I/appl/ldap/include" \
>          ../dist/configure --prefix=/appl/ldap
>
> DIT has 131207 entries.  Search results will come and go as you shorten the
> search base.  For example, when searching for "maillocaladdress=vbroken@dal.ca",
> which has the full DN of
> "cn=broken129251,cn=dal,cn=Mailmaps,cn=Services,dc=DAL,dc=CA", the following
> will occur:
>
>      Searching with base=cn=dal,cn=Mailmaps,cn=services,dc=dal,dc=ca: Found.
>      Searching with base=cn=Mailmaps,cn=services,dc=dal,dc=ca: NOT found.
>      Searching with base=cn=services,dc=dal,dc=ca: NOT found.
>      Searching with base=dc=dal,dc=ca: Found.
>
> cn=dal has 123312 children, which is very high.  However, breaking the branch up
> into subcontainers, none of which had more than 10k entries, did not cure the
> problem.
>
> With olcLogLevel=any, the following is notable:
>
>      Successful search:
>
>
> Apr  5 21:09:01 kil-ds-3 slapd[17628]: =>
> hdb_dn2idl("cn=dal,cn=mailmaps,cn=services,dc=dal,dc=ca")
> Apr  5 21:09:03 kil-ds-3 slapd[17628]: =>  bdb_filter_candidates
> Apr  5 21:09:03 kil-ds-3 slapd[17628]: #011AND
> Apr  5 21:09:03 kil-ds-3 slapd[17628]: =>  bdb_list_candidates 0xa0
> Apr  5 21:09:03 kil-ds-3 slapd[17628]: =>  bdb_filter_candidates
> Apr  5 21:09:03 kil-ds-3 slapd[17628]: #011OR
> Apr  5 21:09:03 kil-ds-3 slapd[17628]: =>  bdb_list_candidates 0xa1
> Apr  5 21:09:03 kil-ds-3 slapd[17628]: =>  bdb_filter_candidates
> Apr  5 21:09:03 kil-ds-3 slapd[17628]: #011EQUALITY
> Apr  5 21:09:03 kil-ds-3 slapd[17628]: =>  bdb_equality_candidates (objectClass)
> Apr  5 21:09:03 kil-ds-3 slapd[17628]: =>  key_read
> Apr  5 21:09:03 kil-ds-3 slapd[17628]: bdb_idl_fetch_key: [b49d1940]
> Apr  5 21:09:03 kil-ds-3 slapd[17628]:<= bdb_index_read: failed (-30988)
> Apr  5 21:09:03 kil-ds-3 slapd[17628]:<= bdb_equality_candidates: id=0,
> first=0, last=0
> Apr  5 21:09:03 kil-ds-3 slapd[17628]:<= bdb_filter_candidates: id=0 first=0
> last=0
> Apr  5 21:09:03 kil-ds-3 slapd[17628]: =>  bdb_filter_candidates
> Apr  5 21:09:03 kil-ds-3 slapd[17628]: #011EQUALITY
> Apr  5 21:09:03 kil-ds-3 slapd[17628]: =>  bdb_equality_candidates
> (mailLocalAddress)
> Apr  5 21:09:03 kil-ds-3 slapd[17628]: =>  key_read
> Apr  5 21:09:03 kil-ds-3 slapd[17628]: bdb_idl_fetch_key: [217e141f]
> Apr  5 21:09:03 kil-ds-3 slapd[17628]:<= bdb_index_read 1 candidates
> Apr  5 21:09:03 kil-ds-3 slapd[17628]:<= bdb_equality_candidates: id=1,
> first=129301, last=129301
> Apr  5 21:09:03 kil-ds-3 slapd[17628]:<= bdb_filter_candidates: id=1
> first=129301 last=129301
> Apr  5 21:09:03 kil-ds-3 slapd[17628]:<= bdb_list_candidates: id=1 first=129301
> last=129301
> Apr  5 21:09:03 kil-ds-3 slapd[17628]:<= bdb_filter_candidates: id=1
> first=129301 last=129301
> Apr  5 21:09:03 kil-ds-3 slapd[17628]:<= bdb_list_candidates: id=1 first=129301
> last=129301
> Apr  5 21:09:03 kil-ds-3 slapd[17628]:<= bdb_filter_candidates: id=1
> first=129301 last=129301
> Apr  5 21:09:03 kil-ds-3 slapd[17628]: bdb_search_candidates: id=1 first=129301
> last=129301
> Apr  5 21:09:03 kil-ds-3 slapd[17628]: entry_decode: ""
> Apr  5 21:09:03 kil-ds-3 slapd[17628]:<= entry_decode()
> Apr  5 21:09:03 kil-ds-3 slapd[17628]: =>  test_filter
>
>      Unsuccessful search:
>
>
> Apr  5 21:10:34 kil-ds-3 slapd[17628]: =>
> hdb_dn2idl("cn=mailmaps,cn=services,dc=dal,dc=ca")
> Apr  5 21:10:34 kil-ds-3 slapd[17628]: =>  bdb_filter_candidates
> Apr  5 21:10:34 kil-ds-3 slapd[17628]: #011AND
> Apr  5 21:10:34 kil-ds-3 slapd[17628]: =>  bdb_list_candidates 0xa0
> Apr  5 21:10:34 kil-ds-3 slapd[17628]: =>  bdb_filter_candidates
> Apr  5 21:10:34 kil-ds-3 slapd[17628]: #011OR
> Apr  5 21:10:34 kil-ds-3 slapd[17628]: =>  bdb_list_candidates 0xa1
> Apr  5 21:10:34 kil-ds-3 slapd[17628]: =>  bdb_filter_candidates
> Apr  5 21:10:34 kil-ds-3 slapd[17628]: #011EQUALITY
> Apr  5 21:10:34 kil-ds-3 slapd[17628]: =>  bdb_equality_candidates (objectClass)
> Apr  5 21:10:34 kil-ds-3 slapd[17628]: =>  key_read
> Apr  5 21:10:34 kil-ds-3 slapd[17628]: bdb_idl_fetch_key: [b49d1940]
> Apr  5 21:10:34 kil-ds-3 slapd[17628]:<= bdb_index_read: failed (-30988)
> Apr  5 21:10:34 kil-ds-3 slapd[17628]:<= bdb_equality_candidates: id=0,
> first=0, last=0
> Apr  5 21:10:34 kil-ds-3 slapd[17628]:<= bdb_filter_candidates: id=0 first=0
> last=0
> Apr  5 21:10:34 kil-ds-3 slapd[17628]: =>  bdb_filter_candidates
> Apr  5 21:10:34 kil-ds-3 slapd[17628]: #011EQUALITY
> Apr  5 21:10:34 kil-ds-3 slapd[17628]: =>  bdb_equality_candidates
> (mailLocalAddress)
> Apr  5 21:10:34 kil-ds-3 slapd[17628]: =>  key_read
> Apr  5 21:10:34 kil-ds-3 slapd[17628]: bdb_idl_fetch_key: [217e141f]
> Apr  5 21:10:34 kil-ds-3 slapd[17628]:<= bdb_index_read 1 candidates
> Apr  5 21:10:34 kil-ds-3 slapd[17628]:<= bdb_equality_candidates: id=1,
> first=129301, last=129301
> Apr  5 21:10:34 kil-ds-3 slapd[17628]:<= bdb_filter_candidates: id=1
> first=129301 last=129301
> Apr  5 21:10:34 kil-ds-3 slapd[17628]:<= bdb_list_candidates: id=1 first=129301
> last=129301
> Apr  5 21:10:34 kil-ds-3 slapd[17628]:<= bdb_filter_candidates: id=1
> first=129301 last=129301
> Apr  5 21:10:34 kil-ds-3 slapd[17628]:<= bdb_list_candidates: id=0 first=3
> last=0
> Apr  5 21:10:34 kil-ds-3 slapd[17628]:<= bdb_filter_candidates: id=0 first=3
> last=0
> Apr  5 21:10:34 kil-ds-3 slapd[17628]: bdb_search_candidates: id=0 first=3
> last=0
> Apr  5 21:10:34 kil-ds-3 slapd[17628]: hdb_search: no candidates
> Apr  5 21:10:34 kil-ds-3 slapd[17628]: send_ldap_result: conn=1001 op=1 p=3
> Apr  5 21:10:34 kil-ds-3 slapd[17628]: send_ldap_result: err=0 matched=""
> text=""
> Apr  5 21:10:34 kil-ds-3 slapd[17628]: send_ldap_response: msgid=2 tag=101
> err=0
> Apr  5 21:10:34 kil-ds-3 slapd[17628]: conn=1001 op=1 SEARCH RESULT tag=101
> err=0 nentries=0 text=
>
> I'm including a link to a tarball that contains:
>      - My (simplified) config, that still produces the problem.  I'm managed to
> remove locally-created schemas.
>      - My test LDIF, containing the entries that can produce the problem.
> They've been anonymized, but it doesn't affect the problem.  A server loaded
> with the above config and this DIT will be consistently unable to find the
> "maillocaladdress=vbroken@dal.ca" entry at cn=services,dc=dal,dc=ca search
> base.
>      - My test script, test_ldap.sh, that searches for the above entry at the
> levels in between it and the base naming context (dc=dal,dc=ca).
>
> I can provide more if needed, up to and including the compiled binaries.
>
>


-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/
Prev by Date: Re: (ITS#7233) SLAP_SOCK_NOT_ACTIVE() crash during shutdown
Next by Date: Re: (ITS#7233) SLAP_SOCK_NOT_ACTIVE() crash during shutdown
Index(es):
- Chronological
- Thread