OpenLDAP
Up to top level
Build   Contrib   Development   Documentation   Historical   Incoming   Software Bugs   Software Enhancements   Web  

Logged in as guest

Viewing Development/5534
Full headers

From: abartlet@samba.org
Subject: Samba4 needs internal transactions/consistancy
Compose comment
Download message
State:
0 replies:
6 followups: 1 2 3 4 5 6

Major security issue: yes  no

Notes:

Notification:


Date: Wed, 28 May 2008 00:39:06 GMT
From: abartlet@samba.org
To: openldap-its@OpenLDAP.org
Subject: Samba4 needs internal transactions/consistancy
Full_Name: Andrew Bartlett
Version: CVS HEAD
OS: Fedora 9
URL: http://www.openldap.org/lists/openldap-technical/200803/msg00101.html
Submission from: (NULL) (59.167.251.137)


For Samba4, I need a few things, detailed in the attached URL.

This ITS is for internal transactions and validation - the ability to have a
openldap overlay roll back all the changes so far, because a precondition is not
met.

I need the memberOf and refint modules to ensure that no dangling links ever
exist, even over subtree renames and invalid modifies, and that a transaction
ensures this is always the case. 

This needs to occur even between databases on the server, but I won't ask that
it occur outside the known trees. 


Followup 1

Download message
Date: Tue, 27 May 2008 18:22:15 -0700
From: Howard Chu <hyc@symas.com>
To: abartlet@samba.org
CC: openldap-its@openldap.org
Subject: Re: (ITS#5534) Samba4 needs internal transactions/consistancy
abartlet@samba.org wrote:
> Full_Name: Andrew Bartlett
> Version: CVS HEAD
> OS: Fedora 9
> URL: http://www.openldap.org/lists/openldap-technical/200803/msg00101.html
> Submission from: (NULL) (59.167.251.137)
>
>
> For Samba4, I need a few things, detailed in the attached URL.

The above message thread had some unanswered questions. We may need to have 
each point listed out again.

> This ITS is for internal transactions and validation - the ability to have
a
> openldap overlay roll back all the changes so far, because a precondition
is not
> met.

I think this one is understood, OK. Just a matter of getting the time to do it.

> I need the memberOf and refint modules to ensure that no dangling links
ever
> exist, even over subtree renames and invalid modifies, and that a
transaction
> ensures this is always the case.

I think the proper use of memberOf still needs to be addressed. E.g., it's 
generally a bad idea to search for (memberOf=foo) when you can simply 
enumerate the members inside the "foo" entry. If you give us precise examples 
of the searches and modifications that you'll be using, we may be able to 
narrow the scope of this work.

> This needs to occur even between databases on the server, but I won't ask
that
> it occur outside the known trees.

It's already possible for operations in one database to reference entries in a 
different database, so that aspect of validation should be fine. However, as 
noted before, "validation" is generally bogus to begin with. In particular, 
how do you create entries with circular references? If you disallow references 
to nonexistent entries, you can't set the references until after all of the 
entries have been created. This means that you cannot backup a database that 
has these references and then later reload it in a single pass.
-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/



Followup 2

Download message
Subject: Re: (ITS#5534) Samba4 needs internal transactions/consistancy
From: Andrew Bartlett <abartlet@samba.org>
To: Howard Chu <hyc@symas.com>
Cc: openldap-its@openldap.org
Date: Wed, 28 May 2008 11:31:22 +1000
--=-I6d9E5fOqbwKcJhhvz+z
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Tue, 2008-05-27 at 18:22 -0700, Howard Chu wrote:
> abartlet@samba.org wrote:
> > Full_Name: Andrew Bartlett
> > Version: CVS HEAD
> > OS: Fedora 9
> > URL: http://www.openldap.org/lists/openldap-technical/200803/msg00101.h=
tml
> > Submission from: (NULL) (59.167.251.137)
> >
> >
> > For Samba4, I need a few things, detailed in the attached URL.
>=20
> The above message thread had some unanswered questions. We may need to ha=
ve=20
> each point listed out again.
>=20
> > This ITS is for internal transactions and validation - the ability to
h=
ave a
> > openldap overlay roll back all the changes so far, because a
preconditi=
on is not
> > met.
>=20
> I think this one is understood, OK. Just a matter of getting the time to =
do it.
>=20
> > I need the memberOf and refint modules to ensure that no dangling
links=
 ever
> > exist, even over subtree renames and invalid modifies, and that a
trans=
action
> > ensures this is always the case.
>=20
> I think the proper use of memberOf still needs to be addressed. E.g., it'=
s=20
> generally a bad idea to search for (memberOf=3Dfoo) when you can simply=20
> enumerate the members inside the "foo" entry. If you give us precise exam=
ples=20
> of the searches and modifications that you'll be using, we may be able to=
=20
> narrow the scope of this work.

I'll be passing on any search that a windows client makes, and trying to
return the same result a windows server would return.  Bad ideas still
have to be implemented in my world :-(

> > This needs to occur even between databases on the server, but I won't
a=
sk that
> > it occur outside the known trees.
>=20
> It's already possible for operations in one database to reference entries=
 in a=20
> different database, so that aspect of validation should be fine. However,=
 as=20
> noted before, "validation" is generally bogus to begin with. In particula=
r,=20
> how do you create entries with circular references? If you disallow refer=
ences=20
> to nonexistent entries, you can't set the references until after all of t=
he=20
> entries have been created. This means that you cannot backup a database t=
hat=20
> has these references and then later reload it in a single pass.

An interesting point, but I need to match the windows runtime
behaviour.=20

Andrew Bartlett

--=20
Andrew Bartlett
http://samba.org/~abartlet/
Authentication Developer, Samba Team           http://samba.org
Samba Developer, Red Hat Inc.

--=-I6d9E5fOqbwKcJhhvz+z
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iD8DBQBIPLXqz4A8Wyi0NrsRArp6AJ9OaJP8Cu4MdO69n1k1S8vlBjtPOACdHvDh
t0XbDQzXaJya2LR/bhl1RlQ=
=/FnH
-----END PGP SIGNATURE-----

--=-I6d9E5fOqbwKcJhhvz+z--



Followup 3

Download message
Date: Tue, 27 May 2008 18:43:08 -0700
From: Howard Chu <hyc@symas.com>
To: Andrew Bartlett <abartlet@samba.org>
CC: openldap-its@openldap.org
Subject: Re: (ITS#5534) Samba4 needs internal transactions/consistancy
Andrew Bartlett wrote:
> On Tue, 2008-05-27 at 18:22 -0700, Howard Chu wrote:

>>> This needs to occur even between databases on the server, but I
won't ask that
>>> it occur outside the known trees.
>> It's already possible for operations in one database to reference
entries in a
>> different database, so that aspect of validation should be fine.
However, as
>> noted before, "validation" is generally bogus to begin with. In
particular,
>> how do you create entries with circular references? If you disallow
references
>> to nonexistent entries, you can't set the references until after all of
the
>> entries have been created. This means that you cannot backup a database
that
>> has these references and then later reload it in a single pass.
>
> An interesting point, but I need to match the windows runtime
> behaviour.

Only when it has a visible impact on other clients. What software will break 
if the directory allows you to add new entries that contain dangling 
references? What will break if the directory allows you to modify a reference 
attribute to point to a nonexistent entry?

There's a lot of Windows behavior that is clearly wrong, by any number of 
metrics. You need to be a bit more selective in prioritizing the list of 
things to chase down.

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/



Followup 4

Download message
Subject: Re: (ITS#5534) Samba4 needs internal transactions/consistancy
From: Andrew Bartlett <abartlet@samba.org>
To: Howard Chu <hyc@symas.com>
Cc: openldap-its@openldap.org
Date: Wed, 28 May 2008 12:28:00 +1000
--=-s2sha8beM9nUssvA07HS
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Tue, 2008-05-27 at 18:43 -0700, Howard Chu wrote:
> Andrew Bartlett wrote:
> > On Tue, 2008-05-27 at 18:22 -0700, Howard Chu wrote:
>=20
> >>> This needs to occur even between databases on the server, but
I won't=
 ask that
> >>> it occur outside the known trees.
> >> It's already possible for operations in one database to reference
entr=
ies in a
> >> different database, so that aspect of validation should be fine.
Howev=
er, as
> >> noted before, "validation" is generally bogus to begin with. In
partic=
ular,
> >> how do you create entries with circular references? If you
disallow re=
ferences
> >> to nonexistent entries, you can't set the references until after
all o=
f the
> >> entries have been created. This means that you cannot backup a
databas=
e that
> >> has these references and then later reload it in a single pass.
> >
> > An interesting point, but I need to match the windows runtime
> > behaviour.
>=20
> Only when it has a visible impact on other clients. What software will br=
eak=20
> if the directory allows you to add new entries that contain dangling=20
> references? What will break if the directory allows you to modify a refer=
ence=20
> attribute to point to a nonexistent entry?

Sure, I'm not asking for a change to default behaviours.  I'm listing
the things that our testsuite finds are differences, and looking for
solutions.=20

> There's a lot of Windows behavior that is clearly wrong, by any number of=
=20
> metrics. You need to be a bit more selective in prioritizing the list of=20
> things to chase down.

This is the currently the top priority for an LDAP Backend for Samba4. =20

Andrew Bartlett

--=20
Andrew Bartlett
http://samba.org/~abartlet/
Authentication Developer, Samba Team           http://samba.org
Samba Developer, Red Hat Inc.

--=-s2sha8beM9nUssvA07HS
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iD8DBQBIPMMwz4A8Wyi0NrsRAljCAJsEsn1tsq4BdkdenNOEOF3PIGcDDACfVoUR
APoU1kbv2ljwVBgjyhPbyGQ=
=mXBr
-----END PGP SIGNATURE-----

--=-s2sha8beM9nUssvA07HS--



Followup 5

Download message
Date: Wed, 28 May 2008 08:14:14 -0700
From: Howard Chu <hyc@symas.com>
To: Andrew Bartlett <abartlet@samba.org>
CC: openldap-its@openldap.org
Subject: Re: (ITS#5534) Samba4 needs internal transactions/consistancy
Andrew Bartlett wrote:
> On Tue, 2008-05-27 at 18:43 -0700, Howard Chu wrote:
>> Andrew Bartlett wrote:
>>> On Tue, 2008-05-27 at 18:22 -0700, Howard Chu wrote:
>>>>> This needs to occur even between databases on the server,
but I won't ask that
>>>>> it occur outside the known trees.
>>>> It's already possible for operations in one database to
reference entries in a
>>>> different database, so that aspect of validation should be
fine. However, as
>>>> noted before, "validation" is generally bogus to begin with. In
particular,
>>>> how do you create entries with circular references? If you
disallow references
>>>> to nonexistent entries, you can't set the references until
after all of the
>>>> entries have been created. This means that you cannot backup a
database that
>>>> has these references and then later reload it in a single pass.
>>> An interesting point, but I need to match the windows runtime
>>> behaviour.
>> Only when it has a visible impact on other clients. What software will
break
>> if the directory allows you to add new entries that contain dangling
>> references? What will break if the directory allows you to modify a
reference
>> attribute to point to a nonexistent entry?
>
> Sure, I'm not asking for a change to default behaviours.  I'm listing
> the things that our testsuite finds are differences, and looking for
> solutions.

I don't believe your proposed solution will ever be satisfactory. Entries with 
circular references will also break syncrepl Refresh if the constraint you're 
asking for is enforced. That will clearly have visible impact in many 
deployments. If the only thing that complains with the current behavior is 
your testsuite and not any real world clients, I suggest you just note the 
difference and move on.

>> There's a lot of Windows behavior that is clearly wrong, by any number
of
>> metrics. You need to be a bit more selective in prioritizing the list
of
>> things to chase down.
>
> This is the currently the top priority for an LDAP Backend for Samba4.

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/



Followup 6

Download message
Subject: Re: (ITS#5534) Samba4 needs internal transactions/consistancy
From: Andrew Bartlett <abartlet@samba.org>
To: Howard Chu <hyc@symas.com>
Cc: openldap-its@openldap.org
Date: Thu, 29 May 2008 21:24:56 +1000
--=-8f0GY55a/Qq8n0F+gHZS
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Wed, 2008-05-28 at 08:14 -0700, Howard Chu wrote:
> Andrew Bartlett wrote:
> > On Tue, 2008-05-27 at 18:43 -0700, Howard Chu wrote:
> >> Andrew Bartlett wrote:
> >>> On Tue, 2008-05-27 at 18:22 -0700, Howard Chu wrote:
> >>>>> This needs to occur even between databases on the
server, but I won=
't ask that
> >>>>> it occur outside the known trees.
> >>>> It's already possible for operations in one database to
reference en=
tries in a
> >>>> different database, so that aspect of validation should be
fine. How=
ever, as
> >>>> noted before, "validation" is generally bogus to begin
with. In part=
icular,
> >>>> how do you create entries with circular references? If you
disallow =
references
> >>>> to nonexistent entries, you can't set the references until
after all=
 of the
> >>>> entries have been created. This means that you cannot
backup a datab=
ase that
> >>>> has these references and then later reload it in a single
pass.
> >>> An interesting point, but I need to match the windows runtime
> >>> behaviour.
> >> Only when it has a visible impact on other clients. What software
will=
 break
> >> if the directory allows you to add new entries that contain
dangling
> >> references? What will break if the directory allows you to modify
a re=
ference
> >> attribute to point to a nonexistent entry?
> >
> > Sure, I'm not asking for a change to default behaviours.  I'm listing
> > the things that our testsuite finds are differences, and looking for
> > solutions.
>=20
> I don't believe your proposed solution will ever be satisfactory. Entries=
 with=20
> circular references will also break syncrepl Refresh if the constraint yo=
u're=20
> asking for is enforced.=20

Only if you don't consider them in replication.  If the backlinks are
added on each node, and not replicated, then surely you just need to
import a set of replicated data, and then in the same transaction update
the links.=20

Is there perhaps another way to implement this - say using a
search-based virtual attribute for one half of the problem?

I'm in no position to set your priorities, and my wishlist remains only
that I hope to someday be able to make this work with OpenLDAP, but
these issues remain.

Andrew Bartlett

--=20
Andrew Bartlett
http://samba.org/~abartlet/
Authentication Developer, Samba Team           http://samba.org
Samba Developer, Red Hat Inc.

--=-8f0GY55a/Qq8n0F+gHZS
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iD8DBQBIPpKIz4A8Wyi0NrsRAhu+AJ9fRq1o5INcGiX1ZYJTAmjmBUMBogCfQ8gC
zrbftn69NpgTvb546qKvGKA=
=kiKt
-----END PGP SIGNATURE-----

--=-8f0GY55a/Qq8n0F+gHZS--


Up to top level
Build   Contrib   Development   Documentation   Historical   Incoming   Software Bugs   Software Enhancements   Web  

Logged in as guest


The OpenLDAP Issue Tracking System uses a hacked version of JitterBug

______________
© Copyright 2013, OpenLDAP Foundation, info@OpenLDAP.org