Re: StartTLS responseName became optional

["Kurt D. Zeilenga" <Kurt@OpenLDAP.org>]

At 12:26 AM 7/25/2005, Hallvard B Furuseth wrote:
>Kurt D. Zeilenga writes:
>>At 05:34 PM 7/22/2005, Hallvard B Furuseth wrote:
>>> I've just learned that the StartTLS responseName changed
>>> from mandatory to optional in Protocol-26, but I cannot
>>> find anything about it in the LDAPbis archive.
>>
>> I believe this issue was previously discussed on the list
>> and, IIRC, in face-to-face meetings.  I am on the road at
>> present so cannot provide URLs.
>
>The closest I can find is a thread 'starttls result codes' after
>protocol-26 was released, but that did not mention responseName.
>
>Do anyone else remember?  The dates of the drafts seems to put it
>near the Control Combination wars, so I'm not surprised if I lost
>it.  Or even if I participated and then forgot it:-)
>
>I'll wait some days before responding after this (if I remember
>to wait:-), in case someone finds the old thread first.

Note that I, as chair, have not nixed this thread.  Hence,
please feel free to discuss it.  I do note that the document
has already been approved by the IESG, there will be some
resistance to changes.   However, let's discuss the issues
and if the WG reaches a timely conclusion that a change
is necessary (not just appropriate), I will take the change
up with our AD.

In the remainder of this thread is with chair hat off.


>BTW, this change is not listed in Appendix C, "Changes".

Point noted.

Let's step back a bit and discuss extended operations in
general.

It should be clear that upon detection of a
problem with a request that the server may immediately
return a error response.  In generating this error
response, it is not required to further examine the
request PDU.

The RFC 2251 statement:
   If the server does not recognize the request name, it MUST return   
   only the response fields from LDAPResult, containing the
   protocolError result code.  The response field is
   absent.

means that during the examination of the request name
field, if it finds the value to be unrecognized, its
response should be a protocol error and the responseName
and response[Value] fields should be absent.

However, RFC 2251 obviously missed the point that a server
can detect an error BEFORE or DURING examination of the
request name field.  It should be obvious that in these
cases the server should return a response with an
appropriate resultCode but no responseName/response[Value]
field.

Less obvious is that the server could have a problem,
such as some internal fault, AFTER examination of the
request name field which disallows it from sending the
responseName/responseValue field.

It should be noted that to the protocol, it is irrelevant
whether the problem occurred before, during, or after
examination as the order of server processing is not
dicated.
 
The RFC 2830 statement
   A Start TLS extended response MUST contain a responseName field which
   MUST be set to the same string as that in the responseName field
   present in the Start TLS extended request.

is seriously flawed.  Even if we take this as applying to
servers implementations of RFC 2830, it should be obvious
that a server implementation of RFC 2830 could run into
situations where it needs to fall back to RFC 2251 allowed
behavior, such as returning other without responseName
on an internal error detected before or during examination
of the request name field.

This statement is also flawed in that it can be taken as a
client requirement to verify a responseName field is present
and contains a particular value.  I do not believe that was
ever intended.  And the few clients which did attempt to
verify this value did so in an incorrect manner.  They
required the field to be present when RFC 2251 allowed it
to be absent, hence causing interoperability problems.

These interoperability problems cannot be solved by requiring
the responseName in supporting servers, aa the problem spans
servers which do not support this extended operation.  We
need to make it clear that the responseName field in
in extended responses is optional.  If we mandate it any
case, clients will be developed which verify it unnecessarily,
but verify it improperly (as our operational experience
shows).

Secondly, we need to make LDAP easier to implement.
Mandating a responseName makes it difficult to implement
generic error handling of extended operations (as one
must special case the response to the request).

>>> (...) Thus, for backwards compatibility I think the spec should
>>> be that servers still MUST send the responseName with any other
>>> result code, though it wouldn't hurt to also say that clients
>>> SHOULD or MUST accept responses without the responseName.
>>
>> I disagree.  Adding a MUST here will lead to clients being
>> developed which are not accepting of the current practice of
>> some servers not to provide a responseName in this case.
>
>Eh?  That's why I suggested to add that clients SHOULD/MUST accept
>responses without responseName.  And we wouldn't be _adding_ the
>MUST for servers, we'd be restoring it after less than a year's
>absence from a draft.  People have had five years to produce such
>RFC 2830 clients.

Sorry, I was disagreeing with the "servers still MUST send
responseName" as clearly there are cases where they cannot
reasonably be expected to do so.  As far as the "clients
SHOUDL or MUST accept", yes, we should make it clear that
clients should expect the unexpected.

>> And if such clients did exist already, they likely have other
>> problems.
>
>Well, if common practice is to break RFC 2830 in this respect, I
>suppose such clients might work poorly with servers that break the
>standard in other common ways too.  Or do you mean something else?

My point here is that it likely that clients verifying
the requestName would not fully consider all the cases
where it reasonable and acceptable for a server to return
an extended response without a responseName.

>>> While it seems pointless of RFC 2830 to require the responseName,
>>
>> Not only is it pointless, it is very bad for clients to
>> require a responseName here.  Obviously there is the unrecongized
>> case where the server cannot provide a responseName.
>
>In that case RFC 2251 says ExtendedResponse.responseName
>should be absent, so clients that require responseName even
>with protocolError are broken.  So far we agree.  But -

Clients which require responseName with various other
result codes are also broken.

>> But there are likely other cases where a server cannot
>> generate a responseName for some recognized control, for
>> instance when the server fronts for a distributed system.

s/control/extended operation/


>I don't understand, even after your next message.  This is just a
>matter of composing a correct response.  How does a control or a
>distributed system make it difficult to insert the responseName?

Because of lack of complete knowledge of what is or isn't
supported by others in the distributed system.

>[Pasting from your next message]
>
>> If it was not clear in my previous response, there are other
>> cases where it is appropriate for a server not supporting an
>> extended operation to return a result code other than
>> protocolError.
>
>Not according to [Protocol] 4.12:
>
>   If the Extended operation associated with the requestName is not
>   supported by the server, the server MUST NOT provide a responseName
>   nor a responseValue and MUST return with resultCode set to
>   protocolError.

As with all other texts prescribing some error handling, this
text only applies if this is the error that's being handled.
If some other is being handled, such as an internal error,
this text simply doesn't apply.

Note that [protocol] else says that servers are to return an
result code that indicates the nature of the problem they
detected.  They should return busy when busy, other when
they have an internal error, etc.., irregardless of whether
they recognize the requestName.   If they don't recongnize
the requestName, then clearly they cannot provide a responseName.
And if they have yet to examine the requestName, they are
not obligated to do so merely to provide a responseName.
This should be obvious because if there was some obligation
to do so.  If not obvious to you, consider a condition that
prevent the server from determining the responseName to
return, such as an internal problem.  This condition should
not preclude the server from responding.  Hence, it must
be allowed to respond without a responseName in cases
where it is unable to determine what responseName to
provide.

>I guess it depends on what you mean with "appropriate", and if you
>mean something else with "support" than the draft does.  There are
>times when it's appropriate for software to violate standards, but
>it's not appropriate according to the standard.  And nothing
>prevents software which violates the Extended Operation spec from
>violating the StartTLS spec too, no matter what [Protocol] says.

You are, IMO, misinterpreting the text.  That is, the
specification allows a server not supporting an extended
operation to return a result code other than protocol error.

The text you quote means that when a server does examine
the requestName and finds a value it does not recognize,
it should respond with a protocolError with no responseName
and no responseValue.

This text (or any text detailing a particular error case)
should not be taken to mean that if a server were to
detect some other error that it is not free to return a
response indicative of the nature of that error.

>> For instance, it would be reasonable for a server which is busy to
>> return busy, or for a server which believes another server is better
>> able to response to return a referral, or a server which has an
>> internal error to return other, or operationsError when the request
>> cannot be processed due to issues with other operations.
>
>Sure.  But I don't see why any of that makes it difficult to
>insert the responseName.

Maybe the internal error is that the internal data structure
in the server holding information about the extended operation
indicates that a responseName should be provided in responses
but does not include the response name to be provided.  It seems
quite difficult to provide the responseName in this case.

>The server can't e.g. respond to an
>extended operation with referral if it doesn't even (bother to)
>recognize the extended operation.

The server most certainly can respond with a referral to
any extended operation, including those it does not recongize
the requestName of.

>Possibly that's a problem, but
>if so the place to fix it is section 4.12 (Extended Operation).
>If it does recognize the operation, it knows enough to compose a
>correct response.
>
>Well, except for severe internal errors.  But if the server is too
>confused it can always drop the connection.

It would be odd a server to drop the connection when it is
willing and able to accept further requests.

>> A client which assumes that responseName is absent only when the
>> resultCode is protocolError is broken.
>
>Not according to RFC 2830.

A client cannot assume the server implements RFC 2830.
RFC 2830 doesn't trump the general LDAP error handling
semantics.

Upon detection of an error, the server returns an error
response which indicates the nature of the error.  It
should be obvious that if it has not yet examined the
requestName field, or the problem occurred during that
examination, that it won't be able to provide a responseName.

>[Back to previous message]
>
>>> cleaning that up isn't worth breaking clients that also the
>>> require responseName, if any.
>>
>> This is not merely protocol clean.  This is fixing existing
>> broken clients.
>
>Well, it's telling and encouraging broken clients to get fixed.
>But as you see, I think it can also break correct clients.

While certainly no check is needed, a client could check
responseName on success.  But I know of no client which only
checks on success.  A client which checks on any non-success
result code is broken.

>I think it's better to encourage both clients and servers to
>address the problem,

The problem is that StartTLS should never have specified
a responseName.  responseNames are extraneous in extended
operations.  Servers should not be obligated to provide them
(as clearly there are cases which they cannot).   Clients
certainly should not check that they exist (because of the
above).

>so they will work together as soon as at
>least one of them is fixed.

We should do is make it clear that responseName in
extended operation responses are optional.

>> The MUST lead to development of clients which absolutely
>> required the responseName, even from servers which don't
>> implement the extension.
>
>I suggested a fix which covered that:  The StartTLS section
>can say clients MUST accept an absent responseName.

I suggest that this be generalized and stated for
all extended operations.  The problem is not just with
StartTLS.   However, I wouldn't have a problem with
reenforcing the general statement in StartTLS given
our experiences with StartTLS.

>> Another issue is that StartTLS technical specification,
>> because of its inclusion in the core TS, is used as a
>> template for future extended operation specifications.
>> Having it do something which future specifications should
>> not do is a bad thing.
>
>RFC 2830 is a separate RFC detailing one extended operation, so it
>can be used as a template for other such documents.  The LDAPbis
>StartTLS spec won't be that.

The form of the StartTLS specification may not be as usable as a
template for other extension specifications, but the
specified syntax and semantics for StartTLS are still likely
be used as a basis for future extended operations.

>If an extended operation spec should normally say the responseName
>should be absent, that can be mentioned in Section 4.12 (Extended
>Operation) or in the separate document you sometimes suggest which
>could describe how to write extensions.

IIRC, draft-zeilenga-ldap-ext covers this.  BTW, this I-D is
ripe for discussion (on ldapext@ietf.org please).

>And the StartTLS spec
>could mention that servers should include the responseName "for
>the sake of backward compatibility".

I rather not include such.