Re: (ITS#5472) ldap_get_values() should handle paged results from LDAP/AD

[Kurt@OpenLDAP.org]

On Jan 26, 2011, at 11:23 AM, masarati@aero.polimi.it wrote:

>>=20
>> On Jan 26, 2011, at 9:15 AM, masarati@aero.polimi.it wrote:
>>=20
>>> The complexity of handling this nonsense in libldap seems not worth =
the
>>> effort;
>>=20
>> Ando, you are too kind to refer this extension as 'nonsense'.  It's =
worse
>> than nonsense, it's crap.
>>=20
>> The spec you pointed at says: "LDAP servers often place limits on the
>> maximum number of attribute values that can be retrieved in a single
>> query."
>=20
> I'd reword that as "One specific LDAP server places limits on the =
maximum
> number of attribute values that can be retrieved in a single query."
>=20
>> The spec fails to say why would a server want to do that.  I
>> cannot think of any good reason to do precisely that.  While I can
>> understand a desire to place limits on what clients can retrieve, I =
don't
>> see why one would distinguish between a single query and a set of =
queries.
>> I can also see it desirable to allow a client to page through results
>> (but that's a different matter, this is about forcing paging onto
>> clients).
>>=20
>> It is incredibly stupid to require clients to use multiple operations =
to
>> obtain information for which the protocol allowed to be fetched in a
>> single operation, and this is why I refer to this as "crap".
>=20
> :)
>=20
>> A major problem with paging, whether server-forced or at =
client-option, is
>> result set coherency due to changes to the DIT between when various =
page
>> operations are obtain.  This can lead to problems such a member of a =
group
>> not appearing in the results.  While one might argue client might be =
able
>> to reasonable deal with coherency issues, most client developers =
won't.
>> This will lead to a range of security issues creeping into directory
>> applications.
>>=20
>> So while I don't generally oppose introduction of paging extensions =
at the
>> client option, I think the specifications should warn that the =
clients
>> using them need to be designed to handle data coherency issues. =
However,
>> most designers of such extensions seem to have put no thought into =
this
>> issue at all.
>=20
> Well, I don't see paged results as a way to introduce more =
inconsistency
> than a regular search operation, since nowhere in the specs I see that =
the
> collection of the entries returned by a search are consistent with the
> entries matching the search request at a specific time (e.g. when the
> operation was initiated or so).  As a consequence, clients need be =
aware
> of the fact that search results can be inherently inconsistent, paging =
or
> not.

As LDAP is defined as an access protocol to an X.500 directory, it =
inherits some data consistency expectations.  In generally expected that =
entry-level access have ACID properties.  That is, each successful =
modify operation creates a new DIT state and a read operation returns =
information consistent with a particular state.  That is, say the DIT =
has state X and there is modify operation that produces state Y and a =
concurrent issued read operation (and no other operations), the read =
operation will return either state X or state Y depending on whether it =
is processed before or after the modify operation.  The read ought not =
provide some intermediate state.

A search operation acts upon a collection of entries, there is a =
possibility of inconsistencies between different entries (but not within =
them), especially in distributed DITs.

However, as we're talking about values of an attribute of a particular =
entry, this is a non-issue in this discussion.