The system currently has around 30 olcAccess stanzas, the majority of which utilize the set notation to grant access based on groups. I decided it might be wise to test the speed of the search using ldapi:/// and the account and it does appear to suffer from the same latency issues, so I this does likely stem from ALCs.
set notation is extremely inefficient for ACL evaluation. I would suggest rewriting your ACLs to not use sets.
--Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration