RE: Server Side Sorting, Paged Results, and Referrals

[Chris Weider <cweider@microsoft.com>]

There's a more general principal here that I think we need to hash out.
Let's say that a new control is published. Should that writeup specify how
the control is handled across referrals? Should that writeup specify what
happens when the client is expecting a single data stream from the API (a la
sort)? Or should this all be handled in the API document? Or should this all
be handled in a separate document for each extension? 

My personal take on this is that the writeup should specify how a control is
handled across referrals, and how it is handled when the client is expecting
a single data stream from the API. But we should write a document that acts
as an 'extension writer's guide' to make sure that these issues get
addressed in every extension document.
Chris

> -----Original Message-----
> From:	Kim J. Worm [SMTP:kjworm@vnet.IBM.COM]
> Sent:	Tuesday, March 03, 1998 4:59 AM
> To:	Chris Weider; ietf-asid@umich.edu
> Subject:	Server Side Sorting, Paged Results, and Referrals
> 
> Chris Weider wrote:
> 
> | Hi Kim:
> |   This is a good point. Addressing the immediate concern of the
> interaction
> | of paged results with referral chasing, one possible implementation is
> that
> | the client saves off all the referrals as it receives them and chases
> each
> | in turn when the current search is finished. It then applies the paged
> | result control to each chased referral in turn. There are some problems
> with
> | this, but this method should work and is what we're pursuing. Note that
> this
> | concern also applies to the virtual list view control as well.
> |
> | The problem of how a control gets appended across referrals is tricky.
> The
> | sort control has similar problems to the paged result control, as it
> implies
> | a merge sort on the client side.
> |
> | We'll have to take a careful look at controls across referral chasing as
> we
> | update the API spec.
> | Chris
> 
> Adding sorting to the ongoing discussion further complicates thing.
> 
> When you consider the possibility of the C LDAP API involving itself with
> coordinating search output sorted by numerous servers which have been
> contacted because of referral chaining, it seems as if the C LDAP API
> would need to run each search to completion before it could guarantee
> a valid sort.  This is true because of the possibility of search
> references being returned which when chased, will return their own
> set of sorted results.  Either the C LDAP API would be responsible for
> merging these separately sorted result sets into a single sorted result
> set, or the sorting extension becomes "advisory" in nature, meaning
> the client app asks for it, hopes for it, but no guarantees are made
> that the results will actually come back sorted.
> 
> Lets assume we want to implement a C LDAP API which guarantees sorting.
> Now suppose a client app asked for BOTH sorted results and simple paged
> results.  Having the C LDAP API gather the entire set of search results
> to guarantee a proper sort seems to defeat the intent of requesting
> paged results.
> 
> I would think that a possible improvement here would be that if the
> server side sorting control is used, then the server should (MUST?)
> send back all its search references first before it starts sending
> search entries.  In this way, a C LDAP API can be implemented that
> contacts all the servers in the referral chaining up front.  It then
> does not need to store the entire set of results to guarantee sort.
> When combined with simple paged results, the C LDAP API can stage the
> output from each server by holding a single set of paged results for
> each server, and still ensure a correct sort.
> 
> Do we want to update the server side sorting draft or the one for simple
> paged results to require servers to send the searchReferences first when
> both extensions are used together?
> 
> I notice that draft-ietf-asid-ldapv3-sorting-00.txt expired almost five
> months ago. Is there a newer version of this, or a revision coming?
> 
> One more point on the referenced sorting draft ... regarding the
> server sending back sorted results, the term SHOULD is used alot where
> MUST would seem more appropriate.  For example:
> 
> "The  sortKeyRequestControl  specifies one  or more  attribute types and
>  matching rules for the results returned by a search request. The server
>  SHOULD return all results for the search request in the order specified
>  by the sort keys." ...
> 
> In fact, with all the SHOULDs in section 4. "Client-Server Interaction",
> the sorting draft ultimately ends up with the following conclusion:
> 
> "The client application is assured that the results are sorted in the
>  specified key order if and only if the result code in the
>  sortKeyResponseControl is success.  If the server omits the
>  sortKeyResponseControl from the searchResultDone message, the client
>  SHOULD assume that the sort control was ignored by the server."
> 
> The one that really makes no sense to me is:
> 
> "3 - If the server supports this sorting control but for some reason
>  cannot sort the search results using the specified sort keys and the
>  client specified TRUE for the control's criticality field, then the
>  server SHOULD do the following: return unavailableCriticalExtension as
>  a return code in the searchResultDone message; include the
>  sortKeyResponseControl in the searchResultDone message, and not send
>  back any search result entries.
> 
> If the client said the control was critical, then it seems to me the
> server MUST obey what the control says.  It seems pretty nasty to
> have an application request sorted results, say its critical, and
> then be given back all the results and at the very end, be told
> ... "oh yeah ... the results weren't sorted ... gotcha!"
> 
> With this approach, it would seem there is no way a C LDAP API could
> guarantee sorted results without gathering up all the search results
> and sorting them itself.
> 
> One other observation ... it is interesting that the sorting draft
> has a reference to servers chaining requests to other servers:
> 
> "Servers that chain requests to other LDAP servers should ensure that
>  the server satisfying the client's request sort the entire result set
>  prior to sending back the results."
> 
> This appears right at the end of section 4.  However, the topic of
> client side chasing of referrals is not mentioned at all.
> 
> Regards,
> Kim J. Worm