Viewing Archive.Contrib/2062
Full headers

Major security issue: yes  no

Notes:
new patch provided integrated into HEAD
Notification:

Date: Tue, 3 Sep 2002 12:35:42 GMT
From: kapurva@in.ibm.com
To: openldap-its@OpenLDAP.org
Subject: Proxy caching extension for OpenLDAP

Full_Name: Apurva Kumar
Version: 2.1.4
OS: FreeBSD
URL: ftp://ftp.openldap.org/incoming/apurva-kumar-pkg-020903.tgz
Submission from: (NULL) (203.143.133.7)


The tarball contains an LDAP proxy cache extension to OpenLDAP 2.1.4. 

Semantic information is used to determine if the incoming query is contained 
(i.e. is narrower or more restrictive) in any of the stored queries. 

The query containment algorithm works for positive conjunctive queries with 
equality, range (GE and LE) and substring assertions. 

The tarball contains the following: 
1) proxy_cache_patch_2.1.4 (Proxy cache patch for 2.1.4) 
2) usage.ps: User manual with example slapd.conf
3) ldapcache.ps: Design and implementation details. 
4) README
5) INSTALL
6) COPYRIGHT 


The patch modifies the LDAP backend functionality and extends LDBM backend
for caching. 

Please let me know if there are any problems installing/using the proxy
cache code. 

-Apurva Kumar (kapurva@in.ibm.com) 
IBM India Research Lab. 
 

Followup 1

Subject: ITS#2062
To: openldap-its@OpenLDAP.org
From: "Apurva Kumar" <kapurva@in.ibm.com>
Date: Wed, 11 Dec 2002 08:53:25 +0530

Hi,

Based on your feedback on the contribution: "LDAP proxy cache extension for
OpenLDAP" released as a patch to OpenLDAP2.1.4, I have submitted a new
patch incorporating the suggestions. The patch is for OpenLDAP2.1.9.  It
can be accessed at:

ftp://ftp.openldap.org/incoming/apurva-kumar-pkg-021210.tgz

The doc ldapcache.html in the tarball contains design, implementation,
usage info.

This release has the following features:

1) Semantic caching of positive conjunctive LDAP queries
   - Answering of repeat and contained queries.
   - Support for answering queries with equality,
     GE, LE and substring assertions.
   - Answering of queries corresponding to specified
     query templates eg. (cn=), &(cn=)(c=).
2) Attribute level caching.
   - Only required attributes of an entry are cached to
     improve cache utilization.
3) Consistency support
   - TTL based weak consistency support provided.
4) Support for multiple backend types
5) Caching operations implemented using backend APIs.
6) Support for caching multiple directories
7) Can function as a meta-directory cache.
8) Support for multiple database instances for a single
   cache directory tree.

With respect to the initial release, the following major changes have been
made:
1) Implementing the proxy cache using the back-meta rather than back-ldap.
This enables the cache for meta directory caching.
2) TTL based weak consistency support added.
3) Using callback mechanism to make the solution backend independent
(requires a control to be added).
4) Support for access control.

The caching operations are implemented using add, modify, search, delete
backend APIs and the callback mechanism. However as discussed in the
forwarded mail, certain checks in the add, modify, delete and search
functions need to be relaxed.  This is done by adding a control in the
Operation struct which is used in the backends supporting LDAP caching.
Thus minor modifications are required in the backends supporting caching.
The changes required are similar for all backends. A patch for LDBM backend
for these changes is included in the release.

Weak consistency is provided, by associating a TTL value with a query type
(template). After the TTL is over, the query (and data) is removed from the
cache.

Only read permissions need to be specified in the ACL since write
operations pass through the cache.

The glue backend is used to glue together multiple database instances
serving the cache directory .

Further details are provided in ldapcache.html included in the tarball.
Will greatly appreciate comments/feedback on the contribution.

Special thanks to Howard, Kurt and Pierangelo for their suggestions which I
have tried to incorporate.

Thanks,
Apurva Kumar,
Research Staff Member,
IBM India Research Lab
Phone: +91-11-6861100
Fax: +91-11-6861555

-------------------------------------

Hi,

Thanks for your suggestions on the proxy cache code.

The suggestion to use callback facility to support all the backends without
modifying their codes can save a lot of work. However I am trying to figure
out how to do the following operations required in the cache with this
facility.

1) adding an entry without a parent.
2) deleting an entry with children (without deleting the children).
3) making a search with the search base not in the cache.

These operations are encountered while doing the following:
1) adding to the cache, an entry returned from the backend server, which
does not have its parent in the cache.
2) removing an entry whose corresponding queries have been removed by cache
replacement.
3) while searching the local cache for an answerable query with base entry
not in the cache.

For LDBM backend I could achieve the above by implementing  three
additional interfaces for adding/merging, searching and removing. I am not
sure if all of these can be taken care of by the existing interfaces for
search/add/delete/modify.

1) can probably be achieved by adding as root. For 3) the only solution I
could think of was to use the backend's suffix as the search base for all
the cache searches and filter out the entries not in the subtree using the
callback function for send_search_entry. This is not  very efficient.

Would greatly appreciate any help in solving this problem.

Thanks,
Apurva Kumar,
Research Staff Member,
IBM India Research Lab
Phone: +91-11-6861100
Fax: +91-11-6861555






                      "Howard Chu"

                      <hyc@symas.com>               To:       Apurva
Kumar/India/IBM@IBMIN, <openldap-devel@OpenLDAP.org>
                      Sent by:                      cc:

                      owner-openldap-devel@O        Subject:  RE: Proxy
cache extension for OpenLDAP
                      penLDAP.org



                      09/06/02 03:38 PM






> -----Original Message-----
> From: Apurva Kumar [mailto:kapurva@in.ibm.com]
>
> LDAP proxy cache docs in HTML.

Thanks. It's a fascinating idea. The effect of ACLs on cached results is

Followup 2

Subject: ITS#2062
To: openldap-its@OpenLDAP.org
From: "Apurva Kumar" <kapurva@in.ibm.com>
Date: Fri, 14 Feb 2003 16:28:23 +0530

Hi,

I have uploaded a new patch for "Proxy cache extension for OpenLDAP,
ITS#2062". The extensions are all behind #ifdef LDAP_CACHING. The tarball
below contains the proxy cache patch for OpenLDAP-2.1.12 and a document
containing design, implementation and usage info (ldapcache.html).

It can be accessed at:
ftp://ftp.openldap.org/incoming/apurva-kumar-pkg-030214.tgz

Thanks,
Apurva Kumar,
Research Staff Member,
IBM India Research Lab
Phone: +91-11-26861100
Fax: +91-11-26861555

Followup 3

Subject: ITS#2062
To: openldap-its@OpenLDAP.org
From: Apurva Kumar <kapurva@in.ibm.com>
Date: Mon, 19 May 2003 18:48:54 +0530

I have uploaded a patch for the proxy cache contribution. The modifications
take into account the recent  callback, backend interface changes in slapd.

ftp://ftp.openldap.org/incoming/apurva-kumar-030519.patch

Apurva