(ITS#6742) Patch - Mozilla NSS - centralize and improve certificate verification

[rmeggins@redhat.com]

Full_Name: Rich Megginson
Version: 2.4.23 (current CVS HEAD)
OS: RHEL6
URL: ftp://ftp.openldap.org/incoming/openldap-2.4.23-centralize-and-improve-cert-verification-20101209.patch
Submission from: (NULL) (76.113.111.209)


The changes are to make Mozilla NSS certificate verification work like OpenSSL. 
The former is using Fedora 14 with Mozilla NSS 3.12.8, the latter, Fedora 13
with OpenLDAP 2.4.21 and OpenSSL 1.0.0b.  I tested the following scenarios:

* self signed server + CA cert as generated by openssl req -x509
* server cert generated by self signed CA (/etc/pki/tls/misc/CA)

I tested various combinations of certs, with and with CA certs, on both the
server and the client, and verified that the F-14 system worked the same as the
F-13, excepting differences in error messages.

I added a new function - tlsm_verify_cert() - that could handle different types
of cert verification.  I replaced other calls to CERT_Verify* with this function
and consolidated exception and error handling there.

In order to make error handling more flexible, I use the
CERT_VerifyCertificate() function which takes a logging parameter.  The log
contains a list of the errors found along with each cert that caused a problem
(a cert may be listed in the log multiple times if a cert had more than one
problem).  The use of logging also allows cert verification to continue to
search for all problems, rather than just returning at the first problem.  This
allows fine grained error handling which is needed for the following problem:
OpenSSL allows CA certs with the Basic Constraints extension set with the CA
flag set to FALSE.  NSS does not.  If tlsm_verify_cert finds this case, it can
just log a warning and allow it.  I also added the ability to specify an error
code to ignore in cert verification, which is needed when using a self signed CA
cert for the server cert.

I also changed the way trust is handled.  In OpenSSL, certs specified by the
user (e.g. in TLS_CACERT, TLS_CACERTDIR, TLS_CERT) are explicitly trusted.  So,
for example, you can specify a server or client cert without specifying the CA
cert issuer, and OpenSSL will allow it.  NSS will allow it if the cert is
specified as trusted.  I changed the add_cert code to accept an istrusted flag
so that you can specify if a PEM cert is trusted.  The PEM NSS code stores
trusted certs in a special slot, so the code had to be changed to accommodate
that.

Finally, I added a default error message which has the error code if an error
message was not already set by the library.