[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
RE: Proxy cache extension for OpenLDAP
Hi,
Thanks for your suggestions on supporting the proxy cache on multiple
backends.
>How about adding parent entries as necessary but leaving them empty or
>marking them as incomplete/invalid so that the cache does not try to
>use the data that it has not actually fetched? The tree held in the
>cache database will then be structurally correct. You might still have
>to bypass checks for the presence of objectClass or other
>mandatory attributes though.
Andrews suggestion above is the easiest to implement but might not be very
scalable for directories with huge number of entries. Even if the cache has
a small fraction of the backend entries, the number of entries required to
have the directory structurally correct could be a large fraction. Also the
empty entry ids will still be added to the subtree and one level indices by
the backends even though they will never be used for answering a query.
>I see the problem, not sure I have a good answer. It seems to me that you
can
>do one of two things - have the cache be an actual mirror/replica of the
>target tree, or have the cache be a flat database. It looks like you want
the
>latter, but leveraging the caching and database interface of back-ldbm. I
>believe it may have been more straightforward just to use libldbm
directly.
>However, you can get the same effect by storing cache entries as a flat
>one-level tree in any of the other backends. Name the cache entries by
some
>completely separate attribute, and store the entries' full DNs as an
explicit
>"dn" attribute inside the entry. Then you search for cache results using
e.g.
>(&(dn=*,ou=base,o=suffix)(cn=foo)(uid=foo))
> -- Howard Chu
In this approach, to support single level searches in the cache and to
avoid substring indexing for the new dn attribute, it might be better to
have two new attributes per entry: dn_subtree: containing all the dns which
have subtrees containing the entry. dn_onelevel: parent dn. However
maintaining indices for these attributes is equivalent to the subtree and
one level indices being maintained in the backends.
>Sounds like just a few checks need to be disabled when the
>caller is a proxy. An Operation flag could be used to
>disable the checks... and a Backend flag could be used to
>indicate the backend supports disabling the checks...
>(sounds a lot like an "internal" control).
>
>This may not be the best approach, just food for thought...
Kurts suggestion above, should make it possible for the caching operations
to be performed, using the backend interfaces for search/add/modify/delete
in the callback functions. The checks to be disabled when the caller is a
proxy are: checking for parent before adding entry, schema check while
adding/modifying entry, checking for descendants before deleting entry. Out
of these, schema check disablement can be avoided, if the objectclass
attribute is always included in the list of projected attributes obtained
from the backend server, and the schema at the cache is changed to have no
other required attribute.
This looks to be the best approach to me. Please let me know your comments.
Apurva Kumar,
Research Staff Member,
IBM India Research Lab
Phone: +91-11-6861100
Fax: +91-11-6861555
"Howard Chu"
<hyc@symas.com> To: Apurva Kumar/India/IBM@IBMIN
cc: <Kurt@OpenLDAP.org>, <openldap-devel@OpenLDAP.org>
09/10/02 01:37 PM Subject: RE: Proxy cache extension for OpenLDAP
> -----Original Message-----
> From: Apurva Kumar [mailto:kapurva@in.ibm.com]
> Hi,
>
> Thanks for your suggestions on the proxy cache code.
>
> The suggestion to use callback facility to support all the backends
without
> modifying their codes can save a lot of work. However I am trying to
figure
> out how to do the following operations required in the cache with this
> facility.
>
> 1) adding an entry without a parent.
> 2) deleting an entry with children (without deleting the children).
> 3) making a search with the search base not in the cache.
>
> These operations are encountered while doing the following:
> 1) adding to the cache, an entry returned from the backend server, which
> does not have its parent in the cache.
> 2) removing an entry whose corresponding queries have been removed by
cache
> replacement.
> 3) while searching the local cache for an answerable query with base
entry
> not in the cache.
>
> For LDBM backend I could achieve the above by implementing three
> additional interfaces for adding/merging, searching and removing. I am
not
> sure if all of these can be taken care of by the existing interfaces for
> search/add/delete/modify.
>
> 1) can probably be achieved by adding as root. For 3) the only solution I
> could think of was to use the backend's suffix as the search base for all
> the cache searches and filter out the entries not in the subtree using
the
> callback function for send_search_entry. This is not very efficient.
>
> Would greatly appreciate any help in solving this problem.
I see the problem, not sure I have a good answer. It seems to me that you
can
do one of two things - have the cache be an actual mirror/replica of the
target tree, or have the cache be a flat database. It looks like you want
the
latter, but leveraging the caching and database interface of back-ldbm. I
believe it may have been more straightforward just to use libldbm directly.
However, you can get the same effect by storing cache entries as a flat
one-level tree in any of the other backends. Name the cache entries by some
completely separate attribute, and store the entries' full DNs as an
explicit
"dn" attribute inside the entry. Then you search for cache results using
e.g.
(&(dn=*,ou=base,o=suffix)(cn=foo)(uid=foo))
-- Howard Chu
Chief Architect, Symas Corp. Director, Highland Sun
http://www.symas.com http://highlandsun.com/hyc
Symas: Premier OpenSource Development and Support