[Date Prev][Date Next]
Re: efficiency of queries returning many entries
Mark Whitehouse wrote:
> I have the following configuration:
> RH Linux 6.2
> openldap 2.0.7
> berkley db 3.2.9
> I have a DB with approx 100K entries. Most of my queries are fast.
> However, whenever I submit a query which contains a large result set (i.e.
> ~10,000 entries) the query takes approx 80 secs.
> The query in question is simple and all the attributes in the query are
> indexed, so I assume the reason the query takes so long is the volume of
> data being processed??
> My question is.... How do I make queries which will return large numbers of
> entries efficient? How do search engines (Yahoo / Google) do this, I have
> noticed that broad queries to search engine sites (which return many
> thousands of queries) are very fast. Do they use some sort of paging
> mechanism where the results of the query are retrieved one part at a time?
> Does any similar mechanism exist in OpenLDAP?
> Any help would be appreciated.
I don't think LDAP would be a good way to implement a search engine. Something
like a WAIS database would be more appropriate. I doubt the vendors you refer
to use LDAP for the search engine, even if they use it to hold certain kinds of