[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: (ITS#6641) Syncrepl failure with 'overlay unique'

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09/07/2010 03:26 PM, andrew.findlay@skills-1st.co.uk wrote:
> On Tue, Sep 07, 2010 at 05:09:07AM -0700, Howard Chu wrote:
> 
>>> We've talked about doing this isolation in the first refresh upon slapd
>>> startup. That might still be a good idea.
> 
> It would certainly help to keep the apparent promises made by things
> like the uniqueness overlay. Alternatively you could take the view
> that the data will converge eventually and that is all that the LDAP
> standards promise.

We've hit a similar problem and decided to go this way, i.e. allowing
the replication to bypass the uniqueness constraints.

In RFC 4533, there are only two claims regarding DIT consistency:

<quote>
1.2 Intended Usage
Upon completion of each synchronization stage of the operation, all
information to construct a synchronized client copy of the content has
been provided to the client [...]. Except for transient inconsistencies
due to concurrent operation (or other) processing at the server, the
client copy is an accurate reflection of the content held by the server.
Transient inconsistencies will be resolved by subsequent synchronization
operations.

[...]

This protocol is not intended to be used in applications requiring
transactional data consistency.
</quote>

There is no claim that the DIT should stay consistent wrt. any
structural restrictions during one synchronization operation. It might
be worth noting that trying to replicate the data through as one
transaction, like Andrew suggested (quoted below), would still have been
impossible without a change in the overlay to allow it to pass through.

>>> Doing it on every refresh seems far more problematic, because without some
>>> type of multi-version concurrency control, that means making the server
>>> non-responsive until the refresh completes.
> 
> That may not be a problem with refresh-and-persist, as in normal
> circumstances I would expect updates to arrive at the consumer in the
> same order they hit the supplier (so this bug could not trigger). More
> difficult for scheduled refresh mode though. Could the consumer server
> simply write-lock every entry involved in the refresh while it processes
> the list, and then commit the whole lot in one DB transaction?
> 
> Andrew

I have put a preliminary version of patches that modify the unique
overlay here
ftp://ftp.openldap.org/incoming/ondrej-kuznik-20101109-unique_bypass_v1.tgz

They add a new configuration attribute olcUniqueAllowManageBypass (it is
prohibitively long for a name, though) that, if set to TRUE, triggers
the uniqueness checks not to be performed if the operation has manage
privilegies on the entry. There are three separate patches,
configuration code regarding the new attribute, the checks in
unique_{add,modify,modrdn} and manpage modifications.

Some things that should be sorted out before this is complete from my
point of view:
1. While there might be a way to find out whether an operation comes
   from replication or is really issued by a rootdn, I have not found
   any. As a workaround, I used the "manage" privilege to the entry as
   a trigger, since this privilege already allows to change the
   structural objectclass of an entry, operation otherwise prohibited
   by RFC 4512.
2. When performing a modifyDN operation, the privilege is checked
   against the entry to be changed instead of the new entry. I found
   this approach more appropriate than modifying the entry returned by
   overlay_entry_get_ov as I do not yet understand the implications of
   doing that and do not know whether it is possible and ok to create a
   phony entry by hand just for the call to access_allowed. This is
   clearly marked as a "FIXME" comment in the patch.
3. The entry attribute used for acl checking might be too broad.
   However, deriving the attributes needed for each domain seemed too
   complex. At least when this is the first version of a patch for
   which I have no indication whether even the general idea is
   considered worth pursuing.
4. The code is similar in all three cases so the first attempt was to
   move it into unique_search, but there is not enough information
   available there and would have to be provided by each of the three
   functions anyway.

Howard and/or others, do you consider this solution valid for this ITS?
If yes, could you help me address the things that should be sorted out?

Last, but not least, the IPR stuff:
The attached modifications to OpenLDAP Software are subject to the
following notice:
Copyright 2010 Acision
Redistribution and use in source and binary forms, with or without
modification, are permitted only as authorized by the OpenLDAP Public
License.

Regards,
Ondrej Kuznik
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkzZX1QACgkQ9GWxeeH+cXuS6gCdEL7txhDF6ukM3jGAcdZNXuQo
lkYAoJXgT8T67OAkETtvYsBuehTV2gO5
=t7jk
-----END PGP SIGNATURE-----

This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you.