Proper ACL's?

["Brian" <brianb@sboss.com>]

I'm having what must be a really simple issue with ACL's in OpenLDAP and
allowing users to authenticate with ssh.  If I have no ACL's, it works
fine.  If I put in something like this:

access to dn="" by * read
access to attr=userpassword
   by self write
   by anonymous auth

access to *
    by self write
    by users read

Then users can't authenticate with ssh.

I'm using OpenLDAP 2.0.27 that ships with Redhat EL Advanced Server. 
Upgrading to OpenLDAP 2.1 is not an option at this time.

My /etc/openldap/slapd.conf has:

include         /etc/openldap/schema/core.schema
include         /etc/openldap/schema/cosine.schema
include         /etc/openldap/schema/inetorgperson.schema
include         /etc/openldap/schema/nis.schema
include         /etc/openldap/schema/redhat/rfc822-MailMember.schema
include         /etc/openldap/schema/redhat/autofs.schema
include         /etc/openldap/schema/redhat/kerberosobject.schema
access to dn="" by * read
access to attr=userpassword
   by self write
   by anonymous auth
access to *
    by self write
    by users read
database        ldbm
suffix          "dc=nc,dc=REDACTED,dc=com"
rootdn          "cn=admin,dc=nc,dc=REDACTED,dc=com"
directory       /var/lib/ldap
index   objectClass,uid,uidNumber,gidNumber,memberUid   eq
index   cn,mail,surname,givenname                       eq,subinitial
TLSCertificateFile      /etc/openldap/ssl-cert/slapd.pem
TLSCertificateKeyFile   /etc/openldap/ssl-cert/slapd.pem
TLSCACertificateFile    /etc/openldap/ssl-cert/slapd.pem

My /etc/ldap.conf is:
host 127.0.0.1
base dc=nc,dc=REDACTED,dc=com
pam_filter objectclass=account
pam_groupdn cn=PAM,ou=Group,dc=nc,dc=REDACTED,dc=com
ssl no
pam_password md5

My /etc/pam.d/system-auth is:
auth        required      /lib/security/pam_env.so
auth        sufficient    /lib/security/pam_unix.so likeauth nullok
auth        sufficient    /lib/security/pam_ldap.so use_first_pass
auth        required      /lib/security/pam_deny.so

account     sufficient      /lib/security/pam_unix.so
account     [default=bad success=ok user_unknown=ignore service_err=ignore
system_err=ignore] /lib/security/pam_ldap.so

password    required      /lib/security/pam_cracklib.so retry=3 type=
password    sufficient    /lib/security/pam_unix.so nullok use_authtok
password    sufficient    /lib/security/pam_ldap.so use_authtok
password    required      /lib/security/pam_deny.so

session     required      /lib/security/pam_limits.so
session     required      /lib/security/pam_unix.so
session     optional      /lib/security/pam_ldap.so

Without ACL's, LDAP works great for authentication. Any pointers as to
what I'm doing wrong?

--
Brian