OpenLDAP
Up to top level
Build   Contrib   Development   Documentation   Historical   Incoming   Software Bugs   Software Enhancements   Web  

Logged in as guest

Viewing Development/6598
Full headers

From: quanah@zimbra.com
Subject: EXOP to return number of children of a (sub)tree
Compose comment
Download message
State:
0 replies:
13 followups: 1 2 3 4 5 6 7 8 9 10 11 12 13

Major security issue: yes  no

Notes:

Notification:


Date: Wed, 21 Jul 2010 18:56:13 +0000
From: quanah@zimbra.com
To: openldap-its@OpenLDAP.org
Subject: EXOP to return number of children of a (sub)tree
Full_Name: Quanah Gibson-Mount
Version: NA
OS: NA
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (75.111.45.108)


It would be extremely useful to have an extended operation that allows querying
the number of children of a given (sub)tree, so that one can avoid iterating
through the entire subtree to determine this number.

Followup 1

Download message
Date: Wed, 21 Jul 2010 12:17:55 -0700
From: Howard Chu <hyc@symas.com>
To: quanah@zimbra.com
CC: openldap-its@openldap.org
Subject: Re: (ITS#6598) EXOP to return number of children of a (sub)tree
quanah@zimbra.com wrote:
> Full_Name: Quanah Gibson-Mount
> Version: NA
> OS: NA
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (75.111.45.108)
>
>
> It would be extremely useful to have an extended operation that allows
querying
> the number of children of a given (sub)tree, so that one can avoid
iterating
> through the entire subtree to determine this number.
>
Might as well ask for the numSubordinates operational attribute to be 
implemented instead, this doesn't seem to merit a new exop. And for 
numSubordinates, see the -devel archives for why we chose not to implement it.

In either case, the server still needs to iterate over all entries internally, 
and the result has to take ACLs and entry disclosure into account.

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/



Followup 2

Download message
Date: Wed, 21 Jul 2010 21:24:49 +0200 (CEST)
Subject: Re: (ITS#6598) EXOP to return number of children of a (sub)tree
From: masarati@aero.polimi.it
To: hyc@symas.com
Cc: openldap-its@openldap.org
> quanah@zimbra.com wrote:
>> Full_Name: Quanah Gibson-Mount
>> Version: NA
>> OS: NA
>> URL: ftp://ftp.openldap.org/incoming/
>> Submission from: (NULL) (75.111.45.108)
>>
>>
>> It would be extremely useful to have an extended operation that allows
>> querying
>> the number of children of a given (sub)tree, so that one can avoid
>> iterating
>> through the entire subtree to determine this number.
>>
> Might as well ask for the numSubordinates operational attribute to be
> implemented instead, this doesn't seem to merit a new exop. And for
> numSubordinates, see the -devel archives for why we chose not to implement
> it.
>
> In either case, the server still needs to iterate over all entries
> internally,
> and the result has to take ACLs and entry disclosure into account.

An exop would allow to easily discriminate between intentional and
"catchall" requests, like "+".  Moreover, it might make sense to
discriminate at least between subtree and onelevel number of subordinates;
this would require two distinct operational attributes, or a parameter in
the exop.

I'm not endorsing either solution, I'm just pointing out possible pros and
cons.

p.



Followup 3

Download message
Date: Wed, 21 Jul 2010 13:37:54 -0700
From: Howard Chu <hyc@symas.com>
To: masarati@aero.polimi.it
CC: openldap-its@openldap.org
Subject: Re: (ITS#6598) EXOP to return number of children of a (sub)tree
masarati@aero.polimi.it wrote:
>> quanah@zimbra.com wrote:
>>> Full_Name: Quanah Gibson-Mount
>>> Version: NA
>>> OS: NA
>>> URL: ftp://ftp.openldap.org/incoming/
>>> Submission from: (NULL) (75.111.45.108)
>>>
>>>
>>> It would be extremely useful to have an extended operation that
allows
>>> querying
>>> the number of children of a given (sub)tree, so that one can avoid
>>> iterating
>>> through the entire subtree to determine this number.
>>>
>> Might as well ask for the numSubordinates operational attribute to be
>> implemented instead, this doesn't seem to merit a new exop. And for
>> numSubordinates, see the -devel archives for why we chose not to
implement
>> it.
>>
>> In either case, the server still needs to iterate over all entries
>> internally,
>> and the result has to take ACLs and entry disclosure into account.
>
> An exop would allow to easily discriminate between intentional and
> "catchall" requests, like "+".  Moreover, it might make sense to
> discriminate at least between subtree and onelevel number of subordinates;
> this would require two distinct operational attributes, or a parameter in
> the exop.
>
> I'm not endorsing either solution, I'm just pointing out possible pros and
> cons.

If you want to be able to distinguish results based on scope then it makes 
more sense to just define a Search control. Like No-op - count the entries 
that would be returned, instead of returning them. (And bypassing the 
sizelimit too, but probably not bypassing timelimit.)

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/



Followup 4

Download message
Date: Sun, 5 Sep 2010 18:17:52 +0200 (CEST)
Subject: Re: (ITS#6598) EXOP to return number of children of a (sub)tree
From: masarati@aero.polimi.it
To: hyc@symas.com
Cc: quanah@zimbra.com, openldap-its@openldap.org
> masarati@aero.polimi.it wrote:
>>> quanah@zimbra.com wrote:
>>>> Full_Name: Quanah Gibson-Mount
>>>> Version: NA
>>>> OS: NA
>>>> URL: ftp://ftp.openldap.org/incoming/
>>>> Submission from: (NULL) (75.111.45.108)
>>>>
>>>>
>>>> It would be extremely useful to have an extended operation that
allows
>>>> querying
>>>> the number of children of a given (sub)tree, so that one can
avoid
>>>> iterating
>>>> through the entire subtree to determine this number.
>>>>
>>> Might as well ask for the numSubordinates operational attribute to
be
>>> implemented instead, this doesn't seem to merit a new exop. And for
>>> numSubordinates, see the -devel archives for why we chose not to
>>> implement
>>> it.
>>>
>>> In either case, the server still needs to iterate over all entries
>>> internally,
>>> and the result has to take ACLs and entry disclosure into account.
>>
>> An exop would allow to easily discriminate between intentional and
>> "catchall" requests, like "+".  Moreover, it might make sense to
>> discriminate at least between subtree and onelevel number of
>> subordinates;
>> this would require two distinct operational attributes, or a parameter
>> in
>> the exop.
>>
>> I'm not endorsing either solution, I'm just pointing out possible pros
>> and
>> cons.
>
> If you want to be able to distinguish results based on scope then it makes
> more sense to just define a Search control. Like No-op - count the entries
> that would be returned, instead of returning them. (And bypassing the
> sizelimit too, but probably not bypassing timelimit.)

I'm drafting the code to implement this no-op search.

The semantics consists in a control request in a search request that
results in returning no intermediate responses, only a final response with
a control response that contains summary information on what the search
would have done.

The request control value is empty.

The response control value contains:
  - the error that the search would have returned if it were performed
regularly (should differ from the one actually returned only in case of
sizelimit exceeded)
  - the number of entries that would be returned
  - the number of search references that would be returned

The search request should request no attributes (in any case no attributes
will be returned); I'm not sure whether it is preferable to simply ignore
requested attrs or the control must require that no attrs ("1.1") be
explicitly requested; I favor the first (ignore req attrs) to ease
interoperability; similarly, attrsOnly can be safely ignored.

The rest of the search should be dealt with as usual (base, scope, filter,
alias dereferencing and so on), as well as ACL.  This allows, for example,
to count immediate children (scope=onelevel), to count subtrees
(scope=subtree); the filter allows to select what is counted; and so on.

The code is almost done; am I missing anything?  In case, let me know so I
can fix it before committing.

Thanks, p.



Followup 5

Download message
Date: Mon, 06 Sep 2010 22:06:13 +0200
From: =?ISO-8859-1?Q?Michael_Str=F6der?= <michael@stroeder.com>
To: masarati@aero.polimi.it
CC: openldap-its@openldap.org
Subject: Re: (ITS#6598) EXOP to return number of children of a (sub)tree
masarati@aero.polimi.it wrote:
>> If you want to be able to distinguish results based on scope then it
makes
>> more sense to just define a Search control. Like No-op - count the
entries
>> that would be returned, instead of returning them. (And bypassing the
>> sizelimit too, but probably not bypassing timelimit.)
> 
> I'm drafting the code to implement this no-op search.

Maybe sizelimit should not be by-passed in all cases.
How about letting the client choose which limits are by-passed?

Ciao, Michael.



Followup 6

Download message
Date: Tue, 7 Sep 2010 00:52:38 +0200 (CEST)
Subject: Re: (ITS#6598) EXOP to return number of children of a (sub)tree
From: masarati@aero.polimi.it
To: michael@stroeder.com
Cc: openldap-its@openldap.org
> masarati@aero.polimi.it wrote:
>>> If you want to be able to distinguish results based on scope then
it
>>> makes
>>> more sense to just define a Search control. Like No-op - count the
>>> entries
>>> that would be returned, instead of returning them. (And bypassing
the
>>> sizelimit too, but probably not bypassing timelimit.)
>>
>> I'm drafting the code to implement this no-op search.
>
> Maybe sizelimit should not be by-passed in all cases.
> How about letting the client choose which limits are by-passed?

Michael,

your idea sounds interesting.  In the current implementation sizelimits
are technically bypassed during the internal search.  Since the operation
returns only a (possibly successful) searchResultDone, it seems
appropriate to return success when sizelimit would be violated, because
from the client's perspective no sizelimit could be exceeded since no
entry is being returned.  The fact that the actual search would be
exceeded is reflected in the additional response code in the control
response's value.

Could you be more precise about what you mean?  One or two use cases or a
not-so-formal specification would suffice.

Thanks, p.



Followup 7

Download message
Date: Thu, 9 Sep 2010 20:44:36 +0200 (CEST)
Subject: ITS#6598
From: masarati@aero.polimi.it
To: openldap-its@openldap.org
Committed to HEAD; please test.  Thanks, p.



Followup 8

Download message
Date: Fri, 17 Feb 2012 15:47:11 -0800
From: Quanah Gibson-Mount <quanah@zimbra.com>
To: masarati@aero.polimi.it, openldap-its@openldap.org
Subject: Re: ITS#6598
--On Thursday, September 09, 2010 6:45 PM +0000 masarati@aero.polimi.it 
wrote:

> Committed to HEAD; please test.  Thanks, p.


Doesn't appear to work to me.

This search returns 15 entries normally:

ldapsearch -x -H ldapi:/// -D cn=config -w zimbra -b "" 
'(objectClass=zimbraAccount)' zimbraId

# numResponses: 16
# numEntries: 15


If you choose the control you get:

zimbra@zre-ldap003:~$ ldapsearch -x -H ldapi:/// -D cn=config -w zimbra -b 
"" -e '1.3.6.1.4.1.4203.666.5.18' '(objectClass=zimbraAccount)' zimbraId
# extended LDIF
#
# LDAPv3
# base <> with scope subtree
# filter: (objectClass=zimbraAccount)
# requesting: zimbraId
#

# search result
search: 2
result: 0 Success
control: 1.3.6.1.4.1.4203.666.5.18 false MAkCAQACAQ8CAQA=

# numResponses: 1

MAkCAQACAQ8CAQA= decodes to a 0 followed by several spaces.  If you try to 
make it critical, it fails:

zimbra@zre-ldap003:~$ ldapsearch -x -H ldapi:/// -D cn=config -w zimbra -b 
"" -e '!1.3.6.1.4.1.4203.666.5.18' '(objectClass=zimbraAccount)' zimbraId
# extended LDIF
#
# LDAPv3
# base <> with scope subtree
# filter: (objectClass=zimbraAccount)
# requesting: zimbraId
#

# search result
search: 2
result: 12 Critical extension is unavailable
text: critical control unavailable in context

# numResponses: 1


--Quanah


--

Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra ::  the leader in open source messaging and collaboration



Followup 9

Download message
Date: Fri, 17 Feb 2012 15:59:47 -0800
From: Quanah Gibson-Mount <quanah@zimbra.com>
To: openldap-its@openldap.org
Subject: Re: ITS#6598
--On Friday, February 17, 2012 11:48 PM +0000 quanah@zimbra.com wrote:

> --On Thursday, September 09, 2010 6:45 PM +0000 masarati@aero.polimi.it
> wrote:
>
>> Committed to HEAD; please test.  Thanks, p.
>
>
> Doesn't appear to work to me.
>
> This search returns 15 entries normally:
>
> ldapsearch -x -H ldapi:/// -D cn=config -w zimbra -b ""
> '(objectClass=zimbraAccount)' zimbraId
>
># numResponses: 16
># numEntries: 15
>
>
> If you choose the control you get:
>
> zimbra@zre-ldap003:~$ ldapsearch -x -H ldapi:/// -D cn=config -w zimbra
> -b  "" -e '1.3.6.1.4.1.4203.666.5.18' '(objectClass=zimbraAccount)'


Also, just to show it is definitely loaded:

ldapsearch -x -H ldap://zre-ldap003.eng.vmware.com -s base -b "" +

dn:
structuralObjectClass: OpenLDAProotDSE
configContext: cn=config
monitorContext: cn=Monitor
namingContexts:
supportedControl: 1.3.6.1.4.1.4203.666.5.18


--Quanah



--

Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra ::  the leader in open source messaging and collaboration



Followup 10

Download message
Date: Sat, 18 Feb 2012 01:32:33 +0100
Subject: Re: ITS#6598
From: masarati@aero.polimi.it
To: quanah@zimbra.com
Cc: openldap-its@openldap.org
> --On Thursday, September 09, 2010 6:45 PM +0000 masarati@aero.polimi.it
> wrote:
>
>> Committed to HEAD; please test.  Thanks, p.
>
>
> Doesn't appear to work to me.
>
> This search returns 15 entries normally:
>
> ldapsearch -x -H ldapi:/// -D cn=config -w zimbra -b ""
> '(objectClass=zimbraAccount)' zimbraId
>
> # numResponses: 16
> # numEntries: 15
>
>
> If you choose the control you get:
>
> zimbra@zre-ldap003:~$ ldapsearch -x -H ldapi:/// -D cn=config -w zimbra -b
> "" -e '1.3.6.1.4.1.4203.666.5.18' '(objectClass=zimbraAccount)' zimbraId
> # extended LDIF
> #
> # LDAPv3
> # base <> with scope subtree
> # filter: (objectClass=zimbraAccount)
> # requesting: zimbraId
> #
>
> # search result
> search: 2
> result: 0 Success
> control: 1.3.6.1.4.1.4203.666.5.18 false MAkCAQACAQ8CAQA=

I think it works as expected:

$ echo -n 'MAkCAQACAQ8CAQA=' | base64 -d | od -x
0000000 0930 0102 0200 0f01 0102 0000

re-ordered:

0x30 ber sequence
0x09 length (9 bytes)

0x02 ber int
0x01 length (1 byte)
0x00 "0" (value of would be search result code)

0x02 ber int
0x01 length (1 byte)
0x0f "15" (number of would be returned entries)

0x02 ber int
0x01 length (1 byte)
0x00 "0" (number of would be returned search refs)

p.

>
> # numResponses: 1
>
> MAkCAQACAQ8CAQA= decodes to a 0 followed by several spaces.  If you try to
> make it critical, it fails:
>
> zimbra@zre-ldap003:~$ ldapsearch -x -H ldapi:/// -D cn=config -w zimbra -b
> "" -e '!1.3.6.1.4.1.4203.666.5.18' '(objectClass=zimbraAccount)' zimbraId
> # extended LDIF
> #
> # LDAPv3
> # base <> with scope subtree
> # filter: (objectClass=zimbraAccount)
> # requesting: zimbraId
> #
>
> # search result
> search: 2
> result: 12 Critical extension is unavailable
> text: critical control unavailable in context
>
> # numResponses: 1
>
>
> --Quanah
>
>
> --
>
> Quanah Gibson-Mount
> Sr. Member of Technical Staff
> Zimbra, Inc
> A Division of VMware, Inc.
> --------------------
> Zimbra ::  the leader in open source messaging and collaboration
>
>
>
>




Followup 11

Download message
Date: Fri, 17 Feb 2012 16:39:08 -0800
From: Quanah Gibson-Mount <quanah@zimbra.com>
To: masarati@aero.polimi.it
cc: openldap-its@openldap.org
Subject: Re: ITS#6598
--On Saturday, February 18, 2012 1:32 AM +0100 masarati@aero.polimi.it 
wrote:

>> --On Thursday, September 09, 2010 6:45 PM +0000 masarati@aero.polimi.it
>> wrote:
>>
>>> Committed to HEAD; please test.  Thanks, p.
>>
>>
>> Doesn't appear to work to me.
>>
>> This search returns 15 entries normally:
>>
>> ldapsearch -x -H ldapi:/// -D cn=config -w zimbra -b ""
>> '(objectClass=zimbraAccount)' zimbraId
>>
>> # numResponses: 16
>> # numEntries: 15
>>
>>
>> If you choose the control you get:
>>
>> zimbra@zre-ldap003:~$ ldapsearch -x -H ldapi:/// -D cn=config -w zimbra
>> -b "" -e '1.3.6.1.4.1.4203.666.5.18' '(objectClass=zimbraAccount)'
>> zimbraId
>> # extended LDIF
>> #
>> # LDAPv3
>> # base <> with scope subtree
>> # filter: (objectClass=zimbraAccount)
>> # requesting: zimbraId
>> #
>>
>> # search result
>> search: 2
>> result: 0 Success
>> control: 1.3.6.1.4.1.4203.666.5.18 false MAkCAQACAQ8CAQA=
>
> I think it works as expected:
>
> $ echo -n 'MAkCAQACAQ8CAQA=' | base64 -d | od -x
> 0000000 0930 0102 0200 0f01 0102 0000
>
> re-ordered:
>
> 0x30 ber sequence
> 0x09 length (9 bytes)
>
> 0x02 ber int
> 0x01 length (1 byte)
> 0x00 "0" (value of would be search result code)
>
> 0x02 ber int
> 0x01 length (1 byte)
> 0x0f "15" (number of would be returned entries)
>
> 0x02 ber int
> 0x01 length (1 byte)
> 0x00 "0" (number of would be returned search refs)

ah, ok thanks.  :)

--Quanah



--

Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra ::  the leader in open source messaging and collaboration



Followup 12

Download message
Date: Fri, 17 Feb 2012 16:48:31 -0800
From: Quanah Gibson-Mount <quanah@zimbra.com>
To: openldap-its@openldap.org, masarati@aero.polimi.it
Subject: Re: ITS#6598
--On Saturday, February 18, 2012 12:41 AM +0000 quanah@zimbra.com wrote:

> --On Saturday, February 18, 2012 1:32 AM +0100 masarati@aero.polimi.it
> wrote:

> ah, ok thanks.  :)


Any particularly reason why the control can't be made critical?

Thanks,
Quanah

--

Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra ::  the leader in open source messaging and collaboration



Followup 13

Download message
Date: Wed, 11 Apr 2012 00:51:37 +0200
Subject: Re: ITS#6598
From: masarati@aero.polimi.it
To: quanah@zimbra.com
Cc: openldap-its@openldap.org
> --On Saturday, February 18, 2012 12:41 AM +0000 quanah@zimbra.com wrote:
>
>> --On Saturday, February 18, 2012 1:32 AM +0100 masarati@aero.polimi.it
>> wrote:
>
>> ah, ok thanks.  :)
>
>
> Any particularly reason why the control can't be made critical?

No.  Now it can.  p.


Up to top level
Build   Contrib   Development   Documentation   Historical   Incoming   Software Bugs   Software Enhancements   Web  

Logged in as guest


The OpenLDAP Issue Tracking System uses a hacked version of JitterBug

______________
© Copyright 2013, OpenLDAP Foundation, info@OpenLDAP.org