ldapi:/// without TLS; ldap:// with TLS?

[Tom <tom@foscore.com>]

I'm running OpenLDAP 2.4 on CentOS. I'm trying to set it up so clients 
can use the ldapi:/// socket without TLS, but any clients using ldap:// 
must use TLS.
I believe that the relevant olc variables are olcLocalSSF and 
olcSecurity. I can't get it to work - either TLS is required no matter 
which URI I use, or clients can connect without TLS at all.
According to the docs, if I set olcLocalSSF to 128, and olcSecurity to 
ssf=128, it should work, but it's not. I can only connect without TLS if 
I delete the olcSecurity attribute, which allows anyone to connect 
without TLS. What am I dong wrong?