[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: ldap SQL backend and syncrepl



Hello Jonathan,

Thanks for your answers.

2009/4/21 Jonathan Clarke <jonathan@phillipoux.net>
Hi,


On 21.04.2009 15:57, Frederic Bouy wrote:
Hello,

For performance issues (millions or records) I have two ldap servers:
- one master with an sql backend (postgres) to allow easy data manipulation
- one slave to anwer ldap queries and provide good response time

When lauching the slave ("./slapd -f /usr/local/openldap/etc/openldap/slapd-front.conf -h "ldap://localhost:3890" -d 1") I got a first non blocking error I don't really understand:
" => bdb_dn2id("dc=lnp")
<= bdb_dn2id: get failed: DB_NOTFOUND: No matching key/data pair found (-30988)"

This looks like the entry "dc=lnp" doesn't exist in your database.

I think that this entry exist, at least according to my db I got:

LNP=# select * from ldap_entries where id=1;
 id |   dn   | oc_map_id | parent | keyval
----+--------+-----------+--------+--------
  1 | dc=lnp |         1 |      0 |      1

LNP=# select * from ldap_oc_mappings where id=1;
 id |   name   |  keytbl   | keycol |        create_proc        |           delete_proc            | expect_return
----+----------+-----------+--------+---------------------------+----------------------------------+---------------
  1 | dcObject | dc_object | id     | SELECT create_dc_object() | DELETE FROM dc_object WHERE id=? |             0

LNP=# select * from dc_object where id=1;
 id | dc  | description
----+-----+-------------
  1 | lnp |

But when doing "ldapsearch -b "dc=lnp" " I got the followin result:

# extended LDIF
#
# LDAPv3
# base <dc=lnp> with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#

# search result
search: 2
result: 80 Other (e.g., implementation specific) error
text: bad referral object

# numResponses: 1

but I can not find any error in the log:

slap_listener_activate(8):
>>> slap_listener(ldap:///)
connection_get(10): got connid=0
connection_read(10): checking for input on id=0
ber_get_next
ber_get_next: tag 0x30 len 12 contents:
ber_get_next
conn=0 op=0 do_bind
ber_scanf fmt ({imt) ber:
ber_scanf fmt (m}) ber:
>>> dnPrettyNormal: <>
<<< dnPrettyNormal: <>, <>
do_bind: version=3 dn="" method=128
send_ldap_result: conn=0 op=0 p=3
send_ldap_response: msgid=1 tag=97 err=0
ber_flush2: 14 bytes to sd 10
do_bind: v3 anonymous bind
connection_get(10): got connid=0
connection_read(10): checking for input on id=0
ber_get_next
ber_get_next: tag 0x30 len 43 contents:
ber_get_next
conn=0 op=1 do_search
ber_scanf fmt ({miiiib) ber:
>>> dnPrettyNormal: <dc=lnp>
<<< dnPrettyNormal: <dc=lnp>, <dc=lnp>
ber_scanf fmt (m) ber:
ber_scanf fmt ({M}}) ber:
==> limits_get: conn=0 op=1 dn="[anonymous]"
==>backsql_search(): base="dc=lnp", filter="(objectClass=*)", scope=2, deref=0, attrsonly=0, attributes to load: all
==>backsql_get_db_conn()
==>backsql_open_db_handle()
<==backsql_open_db_handle()
<==backsql_get_db_conn()
==>backsql_dn2id("dc=lnp") matched expected
   backsql_dn2id("dc=lnp"): id_query "SELECT id,keyval,oc_map_id,dn FROM ldap_entries WHERE dn=?"
   backsql_dn2id("dc=lnp"): id=1 keyval=1 oc_id=1 dn=dc=lnp
>>> dnPrettyNormal: <dc=lnp>
<<< dnPrettyNormal: <dc=lnp>, <dc=lnp>
<==backsql_dn2id("dc=lnp"): err=0
==>backsql_id2entry()
backsql_id2entry(): retrieving all attributes
==>backsql_get_attr_vals(): oc="dcObject" attr="dc" keyval=1
backsql_get_attr_vals(): number of values in query: 1
<==backsql_get_attr_vals()
==>backsql_get_attr_vals(): oc="dcObject" attr="objectClass" keyval=1
backsql_get_attr_vals(): number of values in query: 1
<==backsql_get_attr_vals()
<==backsql_id2entry()
send_ldap_result: conn=0 op=1 p=3
send_ldap_response: msgid=2 tag=101 err=80
ber_flush2: 33 bytes to sd 10
<==backsql_search()
connection_get(10): got connid=0
connection_read(10): checking for input on id=0
ber_get_next
ber_get_next: tag 0x30 len 5 contents:
ber_get_next
ber_get_next on fd 10 failed errno=0 (Success)
connection_closing: readying conn=0 sd=10 for close
connection_close: deferring conn=0 sd=10
conn=0 op=2 do_unbind
connection_resched: attempting closing conn=0 sd=10
connection_close: conn=0 sd=10


As lnp is supposed to be the root is has a ldap_entries.parent=0. Could it be linked? or is it a normal behavior?
 


According to the fact that the following command is successfull:
"ldapwhoami -H "ldap://localhost:389" -D "cn=manager,dc=lnp" -w secret"

This works because the "cn=manager,dc=lnp" user and password are defined in your slapd.conf. It would be the same if you had no data in the database at all.

database        bdb
suffix          "dc=lnp"
rootdn          "cn=Manager,dc=lnp"
rootpw          secret

[...]

syncrepl rid=001
       provider=ldap://localhost:389
       bindmethod=simple
       type=refreshAndPersist
       searchbase="ou=31,dc=lnp"

You have configured a database on suffix "dc=lnp", but you're only replicating the subtree "ou=31,dc=lnp". Therefore, there is no LDAP entry "dc=lnp", which is where syncrepl stores it's contextCSN. This probably explains the errors you're seeing.

Either create a simple "dc=lnp" entry yourself, or replicate that entry from the master aswell.

I have changed on the slave the slapd.conf to "searchbase="dc=lnp"" and I still get this non blocking issue in the log file when starting the slave ( ./slapd -f /usr/local/openldap/etc/openldap/slapd-front.conf -h "ldap://localhost:3890" -d 1):
bdb_dn2entry("dc=lnp")
=> bdb_dn2id("dc=lnp")
<= bdb_dn2id: get failed: DB_NOTFOUND: No matching key/data pair found (-30988)

but the replication issue does not seem to be exactly the same one. I know get:

onnection_get(11): got connid=0
=>do_syncrepl rid=001
=>do_syncrep2 rid=001
ldap_result ld 0x87083d0 msgid 2
wait4msg ld 0x87083d0 msgid 2 (timeout 0 usec)
wait4msg continue ld 0x87083d0 msgid 2 all 0
** ld 0x87083d0 Connections:
* host: localhost  port: 389  (default)
  refcnt: 2  status: Connected
  last used: Wed Apr 22 10:06:54 2009


** ld 0x87083d0 Outstanding Requests:
 * msgid 2,  origid 2, status InProgress
   outstanding referrals 0, parent count 0
  ld 0x87083d0 request count 1 (abandoned 0)
** ld 0x87083d0 Response Queue:
   Empty
  ld 0x87083d0 response count 0
ldap_chkResponseList ld 0x87083d0 msgid 2 all 0
ldap_chkResponseList returns ld 0x87083d0 NULL
ldap_int_select
read1msg: ld 0x87083d0 msgid 2 all 0
ber_get_next
ber_get_next: tag 0x30 len 12 contents:
read1msg: ld 0x87083d0 msgid 2 message type search-result
ber_scanf fmt ({eAA) ber:
read1msg: ld 0x87083d0 0 new referrals
read1msg:  mark request completed, ld 0x87083d0 msgid 2
request done: ld 0x87083d0 msgid 2
res_errno: 32, res_error: <>, res_matched: <>
ldap_free_request (origid 2, msgid 2)
ldap_free_connection 0 1
ldap_free_connection: refcnt 1
ldap_parse_result
ber_scanf fmt ({iAA) ber:
ber_scanf fmt (}) ber:
ldap_err2string
do_syncrep2: rid=001 LDAP_RES_SEARCH_RESULT (32) No such object
ldap_err2string
ldap_err2string
do_syncrep2: rid=001 (32) No such object
ldap_err2string
ldap_msgfree
connection_get(11): got connid=0
ldap_free_connection 1 1
ldap_send_unbind
ber_flush2: 7 bytes to sd 11
ldap_free_connection: actually freed
do_syncrepl: rid=001 quitting

Any clue, advices would be greatly apreciated.
Thanks

Frederic


 


Regards,
Jonathan

--
--------------------------------------------------------------
Jonathan Clarke - jonathan@phillipoux.net
--------------------------------------------------------------
Ldap Synchronization Connector (LSC) - http://lsc-project.org
--------------------------------------------------------------