Issue 8528 - Incorrect results on replace op for olcAccess
Summary: Incorrect results on replace op for olcAccess
Status: VERIFIED INVALID
Alias: None
Product: OpenLDAP
Classification: Unclassified
Component: slapd (show other issues)
Version: 2.4.44
Hardware: All All
: --- normal
Target Milestone: 2.5.1
Assignee: OpenLDAP project
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-11-03 23:41 UTC by Quanah Gibson-Mount
Modified: 2021-02-08 17:52 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description Quanah Gibson-Mount 2016-11-03 23:41:54 UTC
Full_Name: Quanah Gibson-Mount
Version: 2.4.44
OS: N/A
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (47.208.148.26)


When doing a full replace on all values for olcAccess, things work incorrectly
if the values provided are not in numeric sorted order.  This is problematic
when using tools like ldapvi who do alphabetic sort.  It is trivial to reproduce
the problem using the following example LDIFs:

cat > replace_ordered.ldif <<EOF
dn: olcDatabase={1}mdb,cn=config
changetype: modify
replace: olcAccess
olcAccess: {0}to dn.exact="cn=0" by * none
olcAccess: {1}to dn.exact="cn=1" by * none
olcAccess: {2}to dn.exact="cn=2" by * none
olcAccess: {3}to dn.exact="cn=3" by * none
olcAccess: {4}to dn.exact="cn=4" by * none
olcAccess: {5}to dn.exact="cn=5" by * none
olcAccess: {6}to dn.exact="cn=6" by * none
olcAccess: {7}to dn.exact="cn=7"yby * none
-
EOF

cat > replace_ordered_mixup.ldif <<EOF
dn: olcDatabase={1}mdb,cn=config
changetype: modify
replace: olcAccess
olcAccess: {7}to dn.exact="cn=7" by * none
olcAccess: {1}to dn.exact="cn=1" by * none
olcAccess: {4}to dn.exact="cn=4" by * none
olcAccess: %7%7}to dn.exact="cn=3" by * none
olcAccess: {5}to dn.exact="cn=5" by * none
olcAccess: {0}to dn.exact="cn=0" by * none
olcAccess: {6}to dn.exact="cn=6" by * none
olcAccess: {2}to dn.exact="cn=2" by * none
-
EOF

With the initial config as:

olcAccess: {0}to attrs=userPassword by self write by anonymous auth by * none
olcAccess: {1}to attrs=shadowLastChange by self write by * read
olcAccess: {2}to * by * read

When the ordered version is done, the correct result occurs:D%D

olcAccess: {0}to dn.exact="cn=0" by * none
olcAccess: {1}to dn.exact="cn=1" by * none
olcAccess: {2}to dn.exact="cn=2" by * none
olcAccess: {3}to dn.exact="cn=3" by * none
olcAccess: {4}to dn.exact="cn=4" by * none
olcAccess: {5}to dn.exact="cn=5" by * none
Comment 1 Quanah Gibson-Mount 2016-11-03 23:43:46 UTC
--On Friday, November 04, 2016 12:41 AM +0000 openldap-its@OpenLDAP.org 
wrote:

Here's the full text, since the web form apparently ate it:

When doing a full replace on all values for olcAccess, things work 
incorrectly if the values provided are not in numeric sorted order.  This 
is problematic when using tools like ldapvi who do alphabetic sort.  It is 
trivial to reproduce the problem using the following example LDIFs:

cat > replace_ordered.ldif <<EOF
dn: olcDatabase={1}mdb,cn=config
changetype: modify
replace: olcAccess
olcAccess: {0}to dn.exact="cn=0" by * none
olcAccess: {1}to dn.exact="cn=1" by * none
olcAccess: {2}to dn.exact="cn=2" by * none
olcAccess: {3}to dn.exact="cn=3" by * none
olcAccess: {4}to dn.exact="cn=4" by * none
olcAccess: {5}to dn.exact="cn=5" by * none
olcAccess: {6}to dn.exact="cn=6" by * none
olcAccess: {7}to dn.exact="cn=7" by * none
-
EOF

cat > replace_ordered_mixup.ldif <<EOF
dn: olcDatabase={1}mdb,cn=config
changetype: modify
replace: olcAccess
olcAccess: {7}to dn.exact="cn=7" by * none
olcAccess: {1}to dn.exact="cn=1" by * none
olcAccess: {4}to dn.exact="cn=4" by * none
olcAccess: {3}to dn.exact="cn=3" by * none
olcAccess: {5}to dn.exact="cn=5" by * none
olcAccess: {0}to dn.exact="cn=0" by * none
olcAccess: {6}to dn.exact="cn=6" by * none
olcAccess: {2}to dn.exact="cn=2" by * none
-
EOF

With the initial config as:

olcAccess: {0}to attrs=userPassword by self write by anonymous auth by * 
none
olcAccess: {1}to attrs=shadowLastChange by self write by * read
olcAccess: {2}to * by * read

When the ordered version is done, the correct result occurs:

olcAccess: {0}to dn.exact="cn=0" by * none
olcAccess: {1}to dn.exact="cn=1" by * none
olcAccess: {2}to dn.exact="cn=2" by * none
olcAccess: {3}to dn.exact="cn=3" by * none
olcAccess: {4}to dn.exact="cn=4" by * none
olcAccess: {5}to dn.exact="cn=5" by * none
olcAccess: {6}to dn.exact="cn=6" by * none
olcAccess: {7}to dn.exact="cn=7" by * none

However, when the unordered replaced is done, an incorrect result occurs:

olcAccess: {0}to dn.exact="cn=0" by * none
olcAccess: {1}to dn.exact="cn=7" by * none
olcAccess: {2}to dn.exact="cn=2" by * none
olcAccess: {3}to dn.exact="cn=1" by * none
olcAccess: {4}to dn.exact="cn=4" by * none
olcAccess: {5}to dn.exact="cn=3" by * none
olcAccess: {6}to dn.exact="cn=5" by * none
olcAccess: {7}to dn.exact="cn=6" by * none

Since we are doing a replace of all values, it should not be trying to 
reweight the operation. Instead, the values should just be numeric sorted 
and then written out accordingly, so one ends up with the same result as in 
the ordered case.

--Quanah



--

Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
<http://www.symas.com>


Comment 2 Quanah Gibson-Mount 2017-03-20 23:48:31 UTC
moved from Incoming to Software Bugs
Comment 3 Ondřej Kuzník 2020-03-31 08:22:36 UTC
Not saying it's a bad idea, but the interactions with a delete mod might be a little confusing:

changetype: modify
delete: olcAccess
olcAccess: {2}to dn.exact="cn=2" by * none
olcAccess: {1}to dn.exact="cn=1" by * none
olcAccess: {0}to dn.exact="cn=0" by * none

I think the above might fail if sorted. Worse still if you sent this modify request (I think we implement this?):

changetype: modify
delete: olcAccess
olcAccess: {2}
olcAccess: {1}
olcAccess: {0}

You would get the wrong values deleted if we do and you need to make 100% sure that you send this instead:

changetype: modify
delete: olcAccess
olcAccess: {2}
-
delete: olcAccess
olcAccess: {1}
-
delete: olcAccess
olcAccess: {0}
-

Similar with adds.

Would need to reread the draft, but I think the text also needs changing if we want to make this change.

Question is whose job would it be to reorder the values? Frontend or later?
Comment 4 Quanah Gibson-Mount 2020-04-02 16:59:46 UTC
(In reply to Ondřej Kuzník from comment #3)
> Not saying it's a bad idea, but the interactions with a delete mod might be
> a little confusing:

Well, that's why I limited the scope of this bug purely to replace ops where the entire structure is getting replaced.  For add/delete, it must not re-order anything.
Comment 5 Ondřej Kuzník 2020-04-03 08:24:39 UTC
On Thu, Apr 02, 2020 at 04:59:46PM +0000, openldap-its@openldap.org wrote:
>> Not saying it's a bad idea, but the interactions with a delete mod might be
>> a little confusing:
> 
> Well, that's why I limited the scope of this bug purely to replace ops where
> the entire structure is getting replaced.  For add/delete, it must not re-order
> anything.

Not so sure about that, my thinking is a replace: attr should be equal to

delete: attr
-
add: attr
replaced values

And you're now saying that's not the case anymore.
Comment 6 Quanah Gibson-Mount 2020-04-06 20:51:16 UTC
(In reply to Ondřej Kuzník from comment #5)
> On Thu, Apr 02, 2020 at 04:59:46PM +0000, openldap-its@openldap.org wrote:
> >> Not saying it's a bad idea, but the interactions with a delete mod might be
> >> a little confusing:
> > 
> > Well, that's why I limited the scope of this bug purely to replace ops where
> > the entire structure is getting replaced.  For add/delete, it must not re-order
> > anything.
> 
> Not so sure about that, my thinking is a replace: attr should be equal to
> 
> delete: attr
> -
> add: attr
> replaced values
> 
> And you're now saying that's not the case anymore.
Actually I was agreeing with you. ;)  Like in this example:
changetype: modify
delete: olcAccess
olcAccess: {2}
olcAccess: {1}
olcAccess: {0}

It would be a disaster to re-order things, and the correct thing to do is what you proposed, which is to break them down into individual changes like:

changetype: modify
delete: olcAccess
olcAccess: {2}
-
delete: olcAccess
olcAccess: {1}
-
delete: olcAccess
olcAccess: {0}

I.e., reordering it to 0/1/2 would be very bad. ;)
Comment 7 Howard Chu 2020-11-27 22:10:44 UTC
(In reply to Quanah Gibson-Mount from comment #1)
> --On Friday, November 04, 2016 12:41 AM +0000 openldap-its@OpenLDAP.org 
> wrote:
> 
> Here's the full text, since the web form apparently ate it:
> 
> When doing a full replace on all values for olcAccess, things work 
> incorrectly if the values provided are not in numeric sorted order.  This 
> is problematic when using tools like ldapvi who do alphabetic sort.  It is 
> trivial to reproduce the problem using the following example LDIFs:
> 
> cat > replace_ordered.ldif <<EOF
> dn: olcDatabase={1}mdb,cn=config
> changetype: modify
> replace: olcAccess
> olcAccess: {0}to dn.exact="cn=0" by * none
> olcAccess: {1}to dn.exact="cn=1" by * none
> olcAccess: {2}to dn.exact="cn=2" by * none
> olcAccess: {3}to dn.exact="cn=3" by * none
> olcAccess: {4}to dn.exact="cn=4" by * none
> olcAccess: {5}to dn.exact="cn=5" by * none
> olcAccess: {6}to dn.exact="cn=6" by * none
> olcAccess: {7}to dn.exact="cn=7" by * none
> -
> EOF
> 
> cat > replace_ordered_mixup.ldif <<EOF
> dn: olcDatabase={1}mdb,cn=config
> changetype: modify
> replace: olcAccess
> olcAccess: {7}to dn.exact="cn=7" by * none
> olcAccess: {1}to dn.exact="cn=1" by * none
> olcAccess: {4}to dn.exact="cn=4" by * none
> olcAccess: {3}to dn.exact="cn=3" by * none
> olcAccess: {5}to dn.exact="cn=5" by * none
> olcAccess: {0}to dn.exact="cn=0" by * none
> olcAccess: {6}to dn.exact="cn=6" by * none
> olcAccess: {2}to dn.exact="cn=2" by * none
> -
> EOF
> 
> With the initial config as:
> 
> olcAccess: {0}to attrs=userPassword by self write by anonymous auth by * 
> none
> olcAccess: {1}to attrs=shadowLastChange by self write by * read
> olcAccess: {2}to * by * read
> 
> When the ordered version is done, the correct result occurs:
> 
> olcAccess: {0}to dn.exact="cn=0" by * none
> olcAccess: {1}to dn.exact="cn=1" by * none
> olcAccess: {2}to dn.exact="cn=2" by * none
> olcAccess: {3}to dn.exact="cn=3" by * none
> olcAccess: {4}to dn.exact="cn=4" by * none
> olcAccess: {5}to dn.exact="cn=5" by * none
> olcAccess: {6}to dn.exact="cn=6" by * none
> olcAccess: {7}to dn.exact="cn=7" by * none
> 
> However, when the unordered replaced is done, an incorrect result occurs:
> 
> olcAccess: {0}to dn.exact="cn=0" by * none
> olcAccess: {1}to dn.exact="cn=7" by * none
> olcAccess: {2}to dn.exact="cn=2" by * none
> olcAccess: {3}to dn.exact="cn=1" by * none
> olcAccess: {4}to dn.exact="cn=4" by * none
> olcAccess: {5}to dn.exact="cn=3" by * none
> olcAccess: {6}to dn.exact="cn=5" by * none
> olcAccess: {7}to dn.exact="cn=6" by * none
> 
> Since we are doing a replace of all values, it should not be trying to 
> reweight the operation. Instead, the values should just be numeric sorted 
> and then written out accordingly, so one ends up with the same result as in 
> the ordered case.

This ITS is invalid. The frontend is not trying to reweight the operation,
but you're giving it invalid input.

It processes each value in the order it was given. You put {7} first but at that point in time, there aren't 7 values in the attribute, so it goes "at the end".

The 2nd input is {1}, which gets put into the 2nd slot, so you have {7}, {1}.
The 3rd input is {4} but there are only 2 values, so it goes at the end in the 3rd slot.

The 4th input is {3} so it gets inserted into the 3rd slot and pushes the previous value in the 3rd slot into the 4th slot.


The 5th input is {5} so it goes at the end.

The 6th input is {0} so it gets inserted into the 1st slot and pushes everything else out by 1.

The 7th input is {6} so it goes at the end.

The 8th input is {2} so it gets inserted in the 3rd slot and pushes everything else out by 1.


Closing this ITS.