moved from Incoming to Software Bugs

changed notes
changed state Open to Feedback

changed notes
changed state Feedback to Suspended

changed notes
changed state Suspended to Test

> Full_Name: Patrick Cernko
> Version: 2.4.21
> OS: Linux
> URL:
> http://www.mpi-inf.mpg.de/~pcernko/syncrepl_fails_if_attribute_has_no_equalty_maching_rule.tar.bz2
> Submission from: (NULL) (139.19.3.67)
>
>
> Hi OpenLDAP-Programmers,
>
> although some related bug reports were posted in the past years, this
> issue
> still seems not to be fixed:
>
> OpenLDAP's syncrepl replication fails if it comes to removing one of a
> multi-value attribes of an object if this attribute does not have a
> equality
> matching rule defined in the schema. Modification works on the master
> server but
> fails to replicate to the slave(s) leaving them with the dit version just
> before
> and not replicating any other changes any more.
>
> I compiled a small example how to reproduce the problem with the (maybe
> "well-known") nisNetgroupTriple attribute from nis.schema as shipped with
> openldap-2.4.21:
>
> - extract the files from
> http://www.mpi-inf.mpg.de/~pcernko/syncrepl_fails_if_attribute_has_no_equalty_maching_rule.tar.bz2
> into an empty working directory
> - download the tarball to that working dir and configure similar to the
> just
> extracted configure.sh
> - build and install it to get an install subdir in the working dir
> - start the master using sh master.sh in the working dir
>   This will create the "master" database in your working dir, slapadd the
> dit.ldif and start a master slapd with a unix socket /tmp/master.
> - start the slave using sh slave.sh
>   This will create "slave" database in your working dir and start the
> slave
> slapd with a unix socket /tmp/slave which will immediately start syncing
> from
> the running master
> - if you want you can search the 2 servers with
>   install/bin/ldapsearch -H ldapi://%2ftmp%2fmaster/ -b ou=openldap -LLL
>   resp.
>   install/bin/ldapsearch -H ldapi://%2ftmp%2fslave/ -b ou=openldap -LLL
> - use "python mod.py" to remove one of the nisNetgroupTriple attributes
> from the
> netgroup object
> - you will see a debug message from the slave server like this:
>
> bdb_modify_internal: delete nisNetgroupTriple
> bdb_modify_internal: 18 modify/delete: nisNetgroupTriple: no equality
> matching
> rule
> send_ldap_result: err=18 matched="" text="modify/delete:
> nisNetgroupTriple: no
> equality matching rule"
> null_callback : error code 0x12
> slap_graduate_commit_csn: removing 0x29931c0
> 20100120124712.028626Z#000000#000#000000
> syncrepl_entry: rid=001 be_modify cn=netgroup,ou=openldap (18)
> syncrepl_entry: rid=001 be_modify failed (18)
>
>   if not, use "python mod.py add" to re-add the removed nisNetgroupTriple
> and
> try again. Sometimes, it seems to work (race condition?).
> - you can also verify the difference with:
>   install/bin/ldapsearch -H ldapi://%2ftmp%2fmaster/ -b ou=openldap -LLL
> cn=netgroup
>   install/bin/ldapsearch -H ldapi://%2ftmp%2fslave/ -b ou=openldap -LLL
> cn=netgroup
> - from now on, the slave will fail to sync any succeeding changes, as it
> continuously tries to sync the modification above first. :-(
>
> Now, obviously having an attribute without equality matching rule is not a
> good
> idea in the first place. However, the (official?) nis.schema comes with
> such an
> attribute which we must use like this (e.g. we would have to patch the
> Solaris
> kernel to change it for the Solaris NFS server).
>
> It is possible to modify a nisNetgroup object like this: according to the
> debugging output, I guess the master removes simply ALL nisNetgroupTriple
> attributes and adds only the attributes that should remain. When the
> corresponding syncrepl_entry is handled by the slave, it seems the it in
> contrast tries to really only remove the 2-be-removed attribute which does
> not
> work as it cannot identify the right one due to a missing eq-match
> operator.
>
> I suggest the best way to solve this issue is to try to get syncrepl do
> exactly
> the same actions (not resulting changes) as the server does.

I see the issue, and I've isolated it to the case where you're removing a
value.  In that case, in attr_cmp(), nn == 0 and no < o; the original test
for modifying an attribute without matching rule fails.  I've fixed it
according to this patch, not sure whether it's appropriate for other cases

<ftp://ftp.openldap.org/incoming/pierangelo-masarati-2010-04-17-syncrepl-no-mr.1.patch>

Please test and review.  Thanks, p.

> I see the issue, and I've isolated it to the case where you're removing a
> value.  In that case, in attr_cmp(), nn == 0 and no < o; the original test
> for modifying an attribute without matching rule fails.  I've fixed it
> according to this patch, not sure whether it's appropriate for other cases
>
> <ftp://ftp.openldap.org/incoming/pierangelo-masarati-2010-04-17-syncrepl-no-mr.1.patch>
>
> Please test and review.  Thanks, p.

Please disregard, this patch is breaking other things (basically,
producer's and consumer's entries are equivalent, but attrs are listed in
a different order; this conforms to the LDAP model but breaks the
original, preferable behavior).  Working at a better fix.

p.

Should now be fixed in HEAD; please test.  Thanks, p.

changed notes
changed state Test to Release

changed notes
changed state Release to Closed

fixed in HEAD
fixed in RE24