Issue 5872 - slapo-cloak
Summary: slapo-cloak
Status: VERIFIED FIXED
Alias: None
Product: OpenLDAP
Classification: Unclassified
Component: contrib (show other issues)
Version: unspecified
Hardware: All All
: --- normal
Target Milestone: ---
Assignee: OpenLDAP project
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-12-25 06:56 UTC by manu@openldap.org
Modified: 2014-08-01 21:03 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description manu@openldap.org 2008-12-25 06:56:55 UTC
Full_Name: Emmanuel Dreyfus
Version: HEAD
OS: NetBSD-4.0
URL: ftp://ftp.openldap.org/incoming/manu-cloak-20081225.patch
Submission from: (NULL) (213.41.141.172)


The  cloak  overlay  to  slapd(8)  allows  the  server to hide specific
attributes, unless explicitely requested by the  client.  This  improve
performance when a client requests all attributes and get a huge binary
attribute that is of no interest for it (think jpegPhoto)

Comment 1 ando@openldap.org 2008-12-26 13:15:11 UTC
changed notes
moved from Incoming to Contrib
Comment 2 ando@openldap.org 2008-12-27 10:45:32 UTC
manu@netbsd.org wrote:

> The  cloak  overlay  to  slapd(8)  allows  the  server to hide specific
> attributes, unless explicitely requested by the  client.  This  improve
> performance when a client requests all attributes and get a huge binary
> attribute that is of no interest for it (think jpegPhoto)

I'd see this overlay more appropriate for contrib/slapd-modules right 
now, until its functionality is considered of general usefulness, which 
doesn't seem to be the case right now.

On a related note, if this can be considered of general usefulness, LDAP 
specs would need to be changed in order to define a finer grain of 
attribute request; something like:

empty or "*" ; all user, except attrs that need to be explicitly req.
"+" ; all operational
<all including attrs that need to be explicitly requested>
<...>

p.


Ing. Pierangelo Masarati
OpenLDAP Core Team

SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
-----------------------------------
Office:  +39 02 23998309
Mobile:  +39 333 4963172
Fax:     +39 0382 476497
Email:   ando@sys-net.it
-----------------------------------

Comment 3 manu@openldap.org 2008-12-27 14:16:35 UTC
Pierangelo Masarati <ando@sys-net.it> wrote:

> I'd see this overlay more appropriate for contrib/slapd-modules right
> now, until its functionality is considered of general usefulness, which
> doesn't seem to be the case right now.

Do you think an overlay in the contrib directory could have its --enable
flag in top level configure?

My feeling about the contrib directory is that it is some kind of evil
swamp where things are much more difficult to automatically build for
inclusion in a package.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@netbsd.org

Comment 4 ando@openldap.org 2008-12-27 14:22:33 UTC
Emmanuel Dreyfus wrote:

>> I'd see this overlay more appropriate for contrib/slapd-modules right
>> now, until its functionality is considered of general usefulness, which
>> doesn't seem to be the case right now.
> 
> Do you think an overlay in the contrib directory could have its --enable
> flag in top level configure?
> 
> My feeling about the contrib directory is that it is some kind of evil
> swamp where things are much more difficult to automatically build for
> inclusion in a package.

It is :).  However, I think only generally useful things should go into 
mainstream.  As such, I remember some overlays moving from overlays/ to 
slapd-modules/ and others moving the other way, based on their 
generality and usefulness.

For example, smbk5pwd should be in overlays/, given its usefulness, but 
it is probably not general enough.  Yours is general (i.e. not tied to a 
specific architecture or interoperation issue), but probably of limited 
usefulness (how many users would actually need it?).

That's my opinion, of course.  I suggest you start from contrib/ and 
then we'll vote for moving it to overlays/.

p.


Ing. Pierangelo Masarati
OpenLDAP Core Team

SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
-----------------------------------
Office:  +39 02 23998309
Mobile:  +39 333 4963172
Fax:     +39 0382 476497
Email:   ando@sys-net.it
-----------------------------------

Comment 5 manu@openldap.org 2008-12-27 14:39:57 UTC
Pierangelo Masarati <ando@sys-net.it> wrote:

> It is :).  However, I think only generally useful things should go into
> mainstream.  As such, I remember some overlays moving from overlays/ to
> slapd-modules/ and others moving the other way, based on their 
> generality and usefulness.

The annoying thing is that the only added value of contrib/ is to make
building more difficult.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@netbsd.org

Comment 6 manu@openldap.org 2008-12-27 15:19:40 UTC
Pierangelo Masarati <ando@sys-net.it> wrote:

> That's my opinion, of course.  I suggest you start from contrib/ and 
> then we'll vote for moving it to overlays/.

How do I book an OID for the config database, for an overlay in contrib?
Is it ok to just keep the comment in servers/slapd/bconfig.c?
 * OLcfgOv{Oc|At}:22                   -> cloak

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@netbsd.org

Comment 7 ando@openldap.org 2008-12-27 15:25:10 UTC
Emmanuel Dreyfus wrote:
> Pierangelo Masarati <ando@sys-net.it> wrote:
> 
>> That's my opinion, of course.  I suggest you start from contrib/ and 
>> then we'll vote for moving it to overlays/.
> 
> How do I book an OID for the config database, for an overlay in contrib?
> Is it ok to just keep the comment in servers/slapd/bconfig.c?
>  * OLcfgOv{Oc|At}:22                   -> cloak

Yes.  Why did you skip :21, BTW?

p.


Ing. Pierangelo Masarati
OpenLDAP Core Team

SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
-----------------------------------
Office:  +39 02 23998309
Mobile:  +39 333 4963172
Fax:     +39 0382 476497
Email:   ando@sys-net.it
-----------------------------------

Comment 8 Kurt Zeilenga 2008-12-27 15:54:41 UTC
On Dec 27, 2008, at 2:46 AM, ando@sys-net.it wrote:

> empty or "*" ; all user, except attrs that need to be explicitly req.
> "+" ; all operational
> <all including attrs that need to be explicitly requested>
> <...>

I note that the specification of '+' does allow a server not to  
provide all operational attributes.  That is, a server is allowed to  
only return some operational attributes when requested by name.

This is not so with '*' (or empty list).  However, that said, I see no  
particular issue with a server choosing to return a particular user  
applications attribute only when requested by name.  I see this simply  
as an administrative restriction... and those are always allowed.

(I also note that use of '*' (or empty list) and '+' should generally  
be limited to requests formed by a human.  It is bad (but all to  
common) practice for application-specific directory clients to ask for  
everything.  They should really only ask for what they are prepared to  
make use of.

-- Kurt

Comment 9 ando@openldap.org 2008-12-27 16:07:33 UTC
Kurt Zeilenga wrote:
> 
> On Dec 27, 2008, at 2:46 AM, ando@sys-net.it wrote:
> 
>> empty or "*" ; all user, except attrs that need to be explicitly req.
>> "+" ; all operational
>> <all including attrs that need to be explicitly requested>
>> <...>
> 
> I note that the specification of '+' does allow a server not to provide 
> all operational attributes.  That is, a server is allowed to only return 
> some operational attributes when requested by name.

... based on how expensive their computation is.  In fact, we do not 
exploit this too much in slapd(8), where '+' usually triggers 
operational all attributes evaluation.  Probably, we should add the 
possibility to configure whether the most expensive are computed or not 
when not explicitly requested.

> This is not so with '*' (or empty list).

well, according to RFC4511, Section 4.5.1.8.:

    Client implementors should note that even if all user attributes are
    requested, some attributes and/or attribute values of the entry may
    not be included in Search results due to access controls or other
    restrictions.

The restrictions we're discussing may well fit into this.

> However, that said, I see no 
> particular issue with a server choosing to return a particular user 
> applications attribute only when requested by name.  I see this simply 
> as an administrative restriction... and those are always allowed.

Exactly.

> (I also note that use of '*' (or empty list) and '+' should generally be 
> limited to requests formed by a human.  It is bad (but all to common) 
> practice for application-specific directory clients to ask for 
> everything.  They should really only ask for what they are prepared to 
> make use of.

p.


Ing. Pierangelo Masarati
OpenLDAP Core Team

SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
-----------------------------------
Office:  +39 02 23998309
Mobile:  +39 333 4963172
Fax:     +39 0382 476497
Email:   ando@sys-net.it
-----------------------------------

Comment 10 manu@openldap.org 2008-12-27 16:08:03 UTC
Pierangelo Masarati <ando@sys-net.it> wrote:

> > How do I book an OID for the config database, for an overlay in contrib?
> > Is it ok to just keep the comment in servers/slapd/bconfig.c?
> >  * OLcfgOv{Oc|At}:22                   -> cloak
> Yes.  Why did you skip :21, BTW?

Can't remeber why. I'll go back to 21.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@netbsd.org

Comment 11 ando@openldap.org 2008-12-27 16:26:18 UTC
manu@netbsd.org wrote:
> Pierangelo Masarati <ando@sys-net.it> wrote:
> 
>>> How do I book an OID for the config database, for an overlay in contrib?
>>> Is it ok to just keep the comment in servers/slapd/bconfig.c?
>>>  * OLcfgOv{Oc|At}:22                   -> cloak
>> Yes.  Why did you skip :21, BTW?
> 
> Can't remeber why. I'll go back to 21.

At a quick glance, you're leaking the callback containing the cloak 
response.  You should remove (and free it) when called with REP_RESULT 
as sr_type, and add a slap_freeself_cb() cleanup hook.  You won't notice 
the memory leak unless you build slapd with SLAP_NO_SL_MALLOC set, or 
unless your overlay operates within a slab memory intensive context. 
Configuration is leaking as well, I couldn't exactly trace where.

p.


Ing. Pierangelo Masarati
OpenLDAP Core Team

SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
-----------------------------------
Office:  +39 02 23998309
Mobile:  +39 333 4963172
Fax:     +39 0382 476497
Email:   ando@sys-net.it
-----------------------------------

Comment 12 ando@openldap.org 2008-12-27 16:29:46 UTC
Also, you shouldn't duplicate the entry until you find an occurrence of 
an attribute that needs to be hidden, otherwise you make slapo-cloak 
unnecessarily resource intensive.

p.


Ing. Pierangelo Masarati
OpenLDAP Core Team

SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
-----------------------------------
Office:  +39 02 23998309
Mobile:  +39 333 4963172
Fax:     +39 0382 476497
Email:   ando@sys-net.it
-----------------------------------

Comment 13 Howard Chu 2008-12-27 16:54:06 UTC
ando@sys-net.it wrote:
> Emmanuel Dreyfus wrote:
>> Pierangelo Masarati<ando@sys-net.it>  wrote:
>>
>>> That's my opinion, of course.  I suggest you start from contrib/ and
>>> then we'll vote for moving it to overlays/.
>> How do I book an OID for the config database, for an overlay in contrib?
>> Is it ok to just keep the comment in servers/slapd/bconfig.c?
>>   * OLcfgOv{Oc|At}:22                   ->  cloak
>
> Yes.  Why did you skip :21, BTW?

No, for contrib modules reserve the OID in contrib/ConfigOIDs

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/

Comment 14 Howard Chu 2008-12-27 17:07:33 UTC
manu@netbsd.org wrote:
> Pierangelo Masarati<ando@sys-net.it>  wrote:
>
>> I'd see this overlay more appropriate for contrib/slapd-modules right
>> now, until its functionality is considered of general usefulness, which
>> doesn't seem to be the case right now.
>
> Do you think an overlay in the contrib directory could have its --enable
> flag in top level configure?

> My feeling about the contrib directory is that it is some kind of evil
> swamp where things are much more difficult to automatically build for
> inclusion in a package.

I suppose so. I put my stuff in there when I don't consider it worth my effort 
to autoconf-iscate it. Things like nssov only work with Linux, so there's no 
point in adding the overhead. Stuff in there really is of too limited use to 
warrant the full effort of maintenance that the core code gets.

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/

Comment 15 manu@openldap.org 2008-12-27 18:15:56 UTC
Howard Chu <hyc@symas.com> wrote:

> I suppose so. I put my stuff in there when I don't consider it worth my effort
> to autoconf-iscate it. Things like nssov only work with Linux, so there's no
> point in adding the overhead. Stuff in there really is of too limited use to
> warrant the full effort of maintenance that the core code gets.

That means the added value of contrib/ is that it will only be
maintained by its contributor and will rot otherwise. 

Overlays that have been autoconf-ified could all reside in
server/slapd/overlays. Non-core thing would be disabled by default, and
would rot if not maintained by their contributors, as they are now...

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@netbsd.org

Comment 16 ando@openldap.org 2008-12-28 22:35:13 UTC
Hallvard B Furuseth wrote:
> ando@sys-net.it writes:
>> On a related note, if this can be considered of general usefulness, LDAP 
>> specs would need to be changed in order to define a finer grain of 
>> attribute request; something like:
>>
>> empty or "*" ; all user, except attrs that need to be explicitly req.
>> "+" ; all operational
>> <all including attrs that need to be explicitly requested>
>> <...>
> 
> Would it be cleaner if slapo-cloak redefines the attributes to be
> operational, or to behave as if they are?  Maybe give them an
> X-AS-OPERATIONAL extension?  Or would that just mess up schema code,
> things like attribute inheritance?

I think things would mess up.  The X-AS-OPERATIONAL could help, although 
it would result in a modification of otherwise standard-track schema items.

Moreover, I see a number of features system administrators could ask 
for; e.g. hide attributes only when matching a URI (base, scope, 
filter), or based on size limit, or based on client's identity and so.

In this case, I'd keep schema out.

p.


Ing. Pierangelo Masarati
OpenLDAP Core Team

SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
-----------------------------------
Office:  +39 02 23998309
Mobile:  +39 333 4963172
Fax:     +39 0382 476497
Email:   ando@sys-net.it
-----------------------------------

Comment 17 ando@openldap.org 2008-12-28 22:39:44 UTC
manu@netbsd.org wrote:
> Howard Chu <hyc@symas.com> wrote:
> 
>> I suppose so. I put my stuff in there when I don't consider it worth my effort
>> to autoconf-iscate it. Things like nssov only work with Linux, so there's no
>> point in adding the overhead. Stuff in there really is of too limited use to
>> warrant the full effort of maintenance that the core code gets.
> 
> That means the added value of contrib/ is that it will only be
> maintained by its contributor and will rot otherwise. 
> 
> Overlays that have been autoconf-ified could all reside in
> server/slapd/overlays. Non-core thing would be disabled by default, and
> would rot if not maintained by their contributors, as they are now...

Didn't mean to start a holy war about contrib/ vs. overlays/.  My 
intention was to isolate overlay development from autoconf-ing, until 
generality and usefulness demand for moving to overlays/.

p.


Ing. Pierangelo Masarati
OpenLDAP Core Team

SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
-----------------------------------
Office:  +39 02 23998309
Mobile:  +39 333 4963172
Fax:     +39 0382 476497
Email:   ando@sys-net.it
-----------------------------------

Comment 18 Michael Ströder 2008-12-29 13:29:57 UTC
ando@sys-net.it wrote:
> Hallvard B Furuseth wrote:
>> ando@sys-net.it writes:
>>> On a related note, if this can be considered of general usefulness, LDAP 
>>> specs would need to be changed in order to define a finer grain of 
>>> attribute request; something like:
>>>
>>> empty or "*" ; all user, except attrs that need to be explicitly req.
>>> "+" ; all operational
>>> <all including attrs that need to be explicitly requested>
>>> <...>
>> Would it be cleaner if slapo-cloak redefines the attributes to be
>> operational, or to behave as if they are?  Maybe give them an
>> X-AS-OPERATIONAL extension?  Or would that just mess up schema code,
>> things like attribute inheritance?
> 
> I think things would mess up.

I'd also recommend *not* to turn user attribute types into operational
attribute types. This would certainly confuse schema-aware clients.

> Moreover, I see a number of features system administrators could ask 
> for; e.g. hide attributes only when matching a URI (base, scope, 
> filter),

Well, that's something many overlays would benefit from.

> or based on size limit,

???

> or based on client's identity and so.

That would be similar (not equal) to using ACLs. That was explicitly not
the case in the original inquiry.

Ciao, Michael.

Comment 19 ando@openldap.org 2008-12-29 13:36:43 UTC
michael@stroeder.com wrote:
> ando@sys-net.it wrote:
>> Hallvard B Furuseth wrote:
>>> ando@sys-net.it writes:
>>>> On a related note, if this can be considered of general usefulness, LDAP 
>>>> specs would need to be changed in order to define a finer grain of 
>>>> attribute request; something like:
>>>>
>>>> empty or "*" ; all user, except attrs that need to be explicitly req.
>>>> "+" ; all operational
>>>> <all including attrs that need to be explicitly requested>
>>>> <...>
>>> Would it be cleaner if slapo-cloak redefines the attributes to be
>>> operational, or to behave as if they are?  Maybe give them an
>>> X-AS-OPERATIONAL extension?  Or would that just mess up schema code,
>>> things like attribute inheritance?
>> I think things would mess up.
> 
> I'd also recommend *not* to turn user attribute types into operational
> attribute types. This would certainly confuse schema-aware clients.
> 
>> Moreover, I see a number of features system administrators could ask 
>> for; e.g. hide attributes only when matching a URI (base, scope, 
>> filter),
> 
> Well, that's something many overlays would benefit from.

In fact, based on recent modifications to slapo-constraint(5) and other 
overlays, I'm considering the opportunity to introduce generic helpers 
to parse and check this type of generic limitations.

> 
>> or based on size limit,

I meant: size of the attribute; only return values whose size does not 
exceed a gien number of octets, unless explicitly requested.

> ???
> 
>> or based on client's identity and so.
> 
> That would be similar (not equal) to using ACLs. That was explicitly not
> the case in the original inquiry.

Not exactly.  We're discussing the issue of limiting what information 
not explicitly requested should be returned.  An administrator could 
decide, say, that anonymous will not get any jpegPhoto, unless 
explicitly requested.  This is not ACL (in the stricter sense of what 
access privilege an identity is granted), but rather resource access policy.

p.


Ing. Pierangelo Masarati
OpenLDAP Core Team

SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
-----------------------------------
Office:  +39 02 23998309
Mobile:  +39 333 4963172
Fax:     +39 0382 476497
Email:   ando@sys-net.it
-----------------------------------

Comment 20 Howard Chu 2008-12-29 13:58:25 UTC
ando@sys-net.it wrote:
> michael@stroeder.com wrote:
>> ando@sys-net.it wrote:
>>> Hallvard B Furuseth wrote:
>>>> ando@sys-net.it writes:
>>>>> On a related note, if this can be considered of general usefulness, LDAP
>>>>> specs would need to be changed in order to define a finer grain of
>>>>> attribute request; something like:
>>>>>
>>>>> empty or "*" ; all user, except attrs that need to be explicitly req.
>>>>> "+" ; all operational
>>>>> <all including attrs that need to be explicitly requested>
>>>>> <...>
>>>> Would it be cleaner if slapo-cloak redefines the attributes to be
>>>> operational, or to behave as if they are?  Maybe give them an
>>>> X-AS-OPERATIONAL extension?  Or would that just mess up schema code,
>>>> things like attribute inheritance?
>>> I think things would mess up.
>> I'd also recommend *not* to turn user attribute types into operational
>> attribute types. This would certainly confuse schema-aware clients.
>>
>>> Moreover, I see a number of features system administrators could ask
>>> for; e.g. hide attributes only when matching a URI (base, scope,
>>> filter),
>> Well, that's something many overlays would benefit from.
>
> In fact, based on recent modifications to slapo-constraint(5) and other
> overlays, I'm considering the opportunity to introduce generic helpers
> to parse and check this type of generic limitations.

We've discussed this before - overlays ought to have a configurable range of 
effect, using an X.500 Subtree Search Specification.

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/

Comment 21 manu@openldap.org 2008-12-30 13:57:42 UTC
Pierangelo Masarati <ando@sys-net.it> wrote:

> At a quick glance, you're leaking the callback containing the cloak 
> response.  You should remove (and free it) when called with REP_RESULT
> as sr_type, and add a slap_freeself_cb() cleanup hook.  You won't notice
> the memory leak unless you build slapd with SLAP_NO_SL_MALLOC set, or
> unless your overlay operates within a slab memory intensive context. 

Here is an update, I hope it addresses the concerns you raised:
ftp://ftp.openldap.org/incoming/manu-cloak-20081230.c

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@netbsd.org

Comment 22 manu@openldap.org 2009-01-09 04:36:07 UTC
I committed the overlay in the contrib folder

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@netbsd.org

Comment 23 Howard Chu 2009-01-09 04:47:45 UTC
manu@netbsd.org wrote:
> I committed the overlay in the contrib folder
>
As I mentioned before, you should be using a config OID from the Contrib 
branch, and noting this in contrib/ConfigOIDs.

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/

Comment 24 manu@openldap.org 2009-01-09 06:03:15 UTC
Howard Chu <hyc@symas.com> wrote:

> > I committed the overlay in the contrib folder
> As I mentioned before, you should be using a config OID from the Contrib
> branch, and noting this in contrib/ConfigOIDs.

Oops, sorry, I must have missed your message. I will take care of that.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@netbsd.org

Comment 25 Quanah Gibson-Mount 2009-01-21 01:16:14 UTC
changed notes
changed state Open to Release
Comment 26 Quanah Gibson-Mount 2009-02-15 02:02:15 UTC
changed notes
changed state Release to Closed
Comment 27 OpenLDAP project 2014-08-01 21:03:28 UTC
contrib
added to HEAD
added to RE24