I’m running OpenLDAP 2.4.17 with 3 servers configured in multi-master mode (server 1 replicates to server 2 and server 3, server 2 replicates to server 1 and server 3, and server 3 replicates to server 1 and server 2). While executing shell scripts to test replication, all 3 slapd instances began returning error code 32 to all requests. No information could be retrieved from any of the 3 LDAP servers using ldapsearch, Apache Directory Studio, LDAP Administrator tool, or the LDAP Browser tool. The slapd log records with debug level = sync contained numerous “findbase failed! 32” errors. Starting and restarting the slapd instances still resulted in error code 32 being returned for any LDAP request. I had to restore the LDAP databases to be able to view/update records in any of the 3 LDAP databases.
Server 1 configuration snippet:
overlay syncprov
syncprov-checkpoint 100 15
syncprov-sessionlog 5000
syncrepl rid=001
provider=ldap://localhost:3892
type=refreshAndPersist
retry="60 60 300 +"
searchbase="dc=authentx"
scope=sub
schemachecking=off
bindmethod=simple
binddn="cn=xroot,dc=authentx"
credentials="980170contact"
syncrepl rid=002
provider=ldap://localhost:3893
type=refreshAndPersist
retry="60 60 300 +"
searchbase="dc=authentx"
scope=sub
schemachecking=off
bindmethod=simple
binddn="cn=xroot,dc=authentx"
credentials="980170contact"
mirrormode on
Server 2 configuration snippet:
overlay syncprov
syncprov-checkpoint 100 15
syncprov-sessionlog 5000
syncrepl rid=001
provider=ldap://localhost:3891
type=refreshAndPersist
retry="60 60 300 +"
searchbase="dc=authentx"
scope=sub
schemachecking=off
bindmethod=simple
binddn="cn=xroot,dc=authentx"
credentials="980170contact"
syncrepl rid=002
provider=ldap://localhost:3893
type=refreshAndPersist
retry="60 60 300 +"
searchbase="dc=authentx"
scope=sub
schemachecking=off
bindmethod=simple
binddn="cn=xroot,dc=authentx"
credentials="980170contact"
mirrormode on
Server 3 configuration snippet:
overlay syncprov
syncprov-checkpoint 100 15
syncprov-sessionlog 5000
syncrepl rid=001
provider=ldap://localhost:3891
type=refreshAndPersist
retry="60 60 300 +"
searchbase="dc=authentx"
scope=sub
schemachecking=off
bindmethod=simple
binddn="cn=xroot,dc=authentx"
credentials="980170contact"
syncrepl rid=002
provider=ldap://localhost:3892
type=refreshAndPersist
retry="60 60 300 +"
searchbase="dc=authentx"
scope=sub
schemachecking=off
bindmethod=simple
binddn="cn=xroot,dc=authentx"
credentials="980170contact"
mirrormode on
Snippets from log file from server 1 around the slapd error 32 problem time:
syncprov_sendresp: to=002, cookie=rid=001,sid=001,csn=20090811143737.466654Z#000000#001#000000
syncprov_sendresp: to=003, cookie=rid=001,sid=001,csn=20090811143737.466654Z#000000#001#000000
slap_queue_csn: queing 0xa2452d98 20090811143737.567781Z#000000#001#000000
syncprov_sendresp: to=002, cookie=rid=001,sid=001,csn=20090811143737.567781Z#000000#001#000000
slap_graduate_commit_csn: removing 0xa17098a8 20090811143737.567781Z#000000#001#000000
syncprov_sendresp: to=003, cookie=rid=001,sid=001,csn=20090811143737.567781Z#000000#001#000000
findbase failed! 32
findbase failed! 32
findbase failed! 32
findbase failed! 32
findbase failed! 32
findbase failed! 32
findbase failed! 32
findbase failed! 32
findbase failed! 32
findbase failed! 32
findbase failed! 32
findbase failed! 32
findbase failed! 32
findbase failed! 32
findbase failed! 32
findbase failed! 32
connection_input: conn=456 deferring operation: binding
findbase failed! 32
findbase failed! 32
findbase failed! 32
findbase failed! 32
findbase failed! 32
findbase failed! 32
connection_input: conn=461 deferring operation: binding
Snippets from log file from server 2 around the slapd error 32 problem time:
slap_queue_csn: queing 0xa1709520 20090811143737.567781Z#000000#001#000000
slap_graduate_commit_csn: removing 0xa1709650 20090811143737.567781Z#000000#001#000000
syncrepl_updateCookie: rid=001 be_modify failed (32)
do_syncrepl: rid=001 rc 32 retrying (59 retries left)
syncprov_sendresp: to=003, cookie=rid=002,sid=002,csn=20090811143737.567781Z#000000#001#000000
do_syncrep2: cookie=rid=002,sid=003,csn=20090811143737.567781Z#000000#001#000000
syncrepl_entry: rid=002 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_DELETE)
syncrepl_entry: rid=002 be_search (32)
syncrepl_entry: rid=002 (null)
slap_queue_csn: queing 0xa1714320 20090811143737.567781Z#000000#001#000000
slap_graduate_commit_csn: removing 0xa1716558 20090811143737.567781Z#000000#001#000000
do_syncrepl: rid=002 rc 32 retrying (59 retries left)
findbase failed! 32
do_syncrep2: rid=001 LDAP_RES_SEARCH_RESULT
do_syncrep2: rid=001 LDAP_RES_SEARCH_RESULT (32) No such object
do_syncrep2: rid=001 (32) No such object
do_syncrepl: rid=001 rc -2 retrying (59 retries left)
do_syncrep2: rid=002 LDAP_RES_SEARCH_RESULT
do_syncrep2: rid=002 LDAP_RES_SEARCH_RESULT (32) No such object
do_syncrep2: rid=002 (32) No such object
do_syncrepl: rid=002 rc -2 retrying (59 retries left)
findbase failed! 32
Snippets from log file from server 3 around the slapd error 32 problem time:
slap_graduate_commit_csn: removing 0x835e498 20090811143737.466654Z#000000#001#000000
do_syncrep2: cookie=rid=001,sid=001,csn=20090811143737.567781Z#000000#001#000000
syncrepl_entry: rid=001 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_DELETE)
syncrepl_entry: rid=001 be_search (0)
syncrepl_entry: rid=001 dc=authentx
slap_queue_csn: queing 0xa160ee80 20090811143737.567781Z#000000#001#000000
syncprov_matchops: skipping original sid 001
slap_graduate_commit_csn: removing 0xa16134f8 20090811143737.567781Z#000000#001#000000
syncrepl_entry: rid=001 be_delete dc=authentx (0)
slap_queue_csn: queing 0xa160ee80 20090811143737.567781Z#000000#001#000000
slap_graduate_commit_csn: removing 0xa16134f8 20090811143737.567781Z#000000#001#000000
syncrepl_updateCookie: rid=001 be_modify failed (32)
do_syncrepl: rid=001 rc 32 retrying (59 retries left)
syncprov_sendresp: to=002, cookie=rid=002,sid=003,csn=20090811143737.567781Z#000000#001#000000
do_syncrep2: cookie=rid=002,sid=002,csn=20090811143737.567781Z#000000#001#000000
syncrepl_entry: rid=002 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_DELETE)
syncrepl_entry: rid=002 be_search (32)
syncrepl_entry: rid=002 (null)
slap_queue_csn: queing 0x8338688 20090811143737.567781Z#000000#001#000000
slap_graduate_commit_csn: removing 0x8360318 20090811143737.567781Z#000000#001#000000
do_syncrepl: rid=002 rc 32 retrying (59 retries left)
do_syncrep2: rid=002 LDAP_RES_SEARCH_RESULT
do_syncrep2: rid=002 LDAP_RES_SEARCH_RESULT (32) No such object
do_syncrep2: rid=002 (32) No such object
do_syncrepl: rid=002 rc -2 retrying (59 retries left)
findbase failed! 32
I have been unable to reproduce this problem. Any suggestions on how to prevent this problem from occurring and how to recover when this problem occurs?
Thanks,
Barry Colston