[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: pcache rfe

Howard Chu wrote: 
Remco Post wrote:

Hi all,

I've been playing with the pcache (HEAD code) and it does seem to be very usefull as it is but:

I'd like to be able to cache only a subset of the data. We have fields like userPassword and loginShell that should preferably allways be retrieved from the slq backend because these values can change quite rappidly (if a user has used all his budget, his shell changes, when more budget is added his shell changes back, comparable things happen to userPassword) so these should not be cached, or cached only for a short time, while other attributes are more static and could be cached for ages (one hour or even more).

Even worse, one client does not even list the requested attributes in it's ldap-queries (I've notified the vendor that I think it should), so non of that clients queries are cachable.

as I see it, currently the pcache does not allow for only part of a query to be retrieved from cache and the rest of the attributes to be retrieved from the backend (in my case sql). It would be nice (I think) if this were possible in the future, I guess this might be quite intrusive in the way the cache works, but it would make the cache more versatile and in my case it makes a huge impact on the usefullness of the cache.


I believe what you suggest would actually completely negate the usefulness of the cache. If any part of the entry is not cached, then all requests for the entry will have to be passed through to the underlying database. Of course, this depends on the actual queries; if there are no queries that request both cached and uncached attributes at the same time then I guess it might be practical.


There is one catch. The way bask-sql works, it has to do a sql query for echt attribute requested. So if I were to do a query for both cached and uncached attributes we could save on the sql queries for the cached attributes, saving possibly a lot of time.

We (Symas) have already developed merging code to coalesce attributes from entries in two separate databases into a single entry. This will be appearing in HEAD soon as the "translucent" overlay. This overlay was briefly mentioned at the OpenLDAP Developer Day conference in San Diego last month. The idea is that you have a remote database that carries authoritative, generic information for the DIT and a local database that contains partial entries carrying attributes to augment the remote entries. This is particularly useful for application-specific local directories that have no write-access to the schema of the remote/authoritative directory, but need to associate application-specific information with the existing entries.

E.g., the master database might have
  dn: cn=Joe User,o=example.org
  objectclass: person
  cn: Joe
  sn: User
  userPassword: mySecret

and the local database might have
  dn: cn=Joe User,o=example.org
  objectclass: CustomAppObject
  customAttr: a bunch of info

Searching on the local database for "Joe User" would return a single merged entry:
dn: cn=Joe User,o=example.org
objectclass: person
objectclass: CustomAppObject
cn: Joe
sn: User
userPassword: mySecret
customAttr: a bunch of info

This feature can probably be incorporated into the proxycache code to achieve what you want. (This statement does not imply that anybody I know of is planning to do so, just that a sufficiently motivated person could do it.)


This does seem to be intresting. I'd like to motivate anybody with the proper skills to do so ;-)

--
Met vriendelijke groeten,

Remco Post

SARA - Reken- en Netwerkdiensten                      http://www.sara.nl
High Performance Computing  Tel. +31 20 592 3000    Fax. +31 20 668 3167

"I really didn't foresee the Internet. But then, neither did the
computer industry. Not that that tells us very much of course - the
computer industry didn't even foresee that the century was going to
end." -- Douglas Adams