Issue 6683 - slapo-ddds fails with expired branches
Summary: slapo-ddds fails with expired branches
Status: VERIFIED WORKSFORME
Alias: None
Product: OpenLDAP
Classification: Unclassified
Component: slapd (show other issues)
Version: 2.4.23
Hardware: All All
: --- normal
Target Milestone: ---
Assignee: Quanah Gibson-Mount
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-10-25 10:34 UTC by petteri.stenius@ubisecure.com
Modified: 2021-06-23 20:43 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description petteri.stenius@ubisecure.com 2010-10-25 10:34:53 UTC
Full_Name: 
Version: 2.4.23
OS: Linux
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (195.197.205.34)


Hello,

I have a directory with branches of dynamicObject entries. It looks like if the
entryExpireTimestamp value is the same on objects within a branch then DDS
search for expired objects will only find the top-most object. This results in
remove failing with message

DDS dn="cn=top,cn=root,dc=test" is non-leaf; deferring.


To reproduce

OpenLDAP 2.4.23, Berkeley DB 4.6.21

Use slapadd to prepare directory with following

dn: cn=Root,dc=test
objectClass: top
objectClass: applicationProcess
cn: Root

dn: cn=top,cn=Root,dc=test
objectClass: top
objectClass: device
objectClass: dynamicObject
entryTTL: 60
entryExpireTimestamp: 20101024113626Z
cn: top

dn: cn=leaf1,cn=top,cn=Root,dc=test
objectClass: top
objectClass: device
objectClass: dynamicObject
entryTTL: 60
entryExpireTimestamp: 20101024113626Z
cn: leaf1

dn: cn=leaf2,cn=top,cn=Root,dc=test
objectClass: top
objectClass: device
objectClass: dynamicObject
entryTTL: 60
entryExpireTimestamp: 20101024113626Z
cn: leaf2

dn: cn=leaf3,cn=top,cn=Root,dc=test
objectClass: top
objectClass: device
objectClass: dynamicObject
entryTTL: 60
entryExpireTimestamp: 20101024113626Z
cn: leaf3


Relevant slapd.conf entries

database        bdb
suffix          "cn=Root,dc=test"
rootdn          "cn=Root,dc=test"
rootpw          "password"

overlay         dds
dds-default-ttl 3600
dds-min-ttl     60
dds-interval    60
dds-state       true
index           entryExpireTimestamp eq,pres

access to dn.subtree="cn=Root,dc=test"
        by users write
        by * read


Running "slapd -d 1 -d 256" produces following

put_filter: "(&(objectClass=dynamicObject)(entryExpireTimestamp<=20101025082446Z))"
put_filter: AND
put_filter_list "(objectClass=dynamicObject)(entryExpireTimestamp<=20101025082446Z)"
put_filter: "(objectClass=dynamicObject)"
put_filter: simple
put_simple_filter: "objectClass=dynamicObject"
put_filter: "(entryExpireTimestamp<=20101025082446Z)"
put_filter: simple
put_simple_filter: "entryExpireTimestamp<=20101025082446Z"
ber_scanf fmt ({mm}) ber:
ber_scanf fmt ({mm}) ber:
=> bdb_search
bdb_dn2entry("cn=root,dc=test")
=> bdb_dn2id("cn=root,dc=test")
<= bdb_dn2id: got id=0x1
entry_decode: "cn=Root,dc=test"
<= entry_decode(cn=Root,dc=test)
search_candidates: base="cn=root,dc=test" (0x00000001) scope=2
=> bdb_dn2idl("cn=root,dc=test")
=> bdb_equality_candidates (objectClass)
=> key_read
<= bdb_index_read: failed (-30989)
<= bdb_equality_candidates: id=0, first=0, last=0
=> bdb_equality_candidates (objectClass)
=> key_read
<= bdb_index_read 4 candidates
<= bdb_equality_candidates: id=4, first=2, last=5
=> bdb_inequality_candidates (entryExpireTimestamp)
=> key_read
<= bdb_index_read 1 candidates
=> key_read
<= bdb_index_read: failed (-30989)
<= bdb_inequality_candidates: id=1, first=2, last=2
bdb_search_candidates: id=1 first=2 last=2
entry_decode: "cn=top,cn=Root,dc=test"
<= entry_decode(cn=top,cn=Root,dc=test)
=> bdb_dn2id("cn=top,cn=root,dc=test")
<= bdb_dn2id: got id=0x2
send_ldap_result: conn=-1 op=0 p=0
bdb_dn2entry("cn=top,cn=root,dc=test")
=> bdb_dn2id_children("cn=top,cn=root,dc=test")
<= bdb_dn2id_children("cn=top,cn=root,dc=test"):  (0)
send_ldap_result: conn=-1 op=0 p=0
DDS dn="cn=top,cn=root,dc=test" is non-leaf; deferring.
DDS expired=0


ldapsearch "(entryExpireTimestamp=*)" produces

dn: cn=top,cn=Root,dc=test
entryExpireTimestamp: 20101024113626Z

dn: cn=leaf1,cn=top,cn=Root,dc=test
entryExpireTimestamp: 20101024113626Z

dn: cn=leaf2,cn=top,cn=Root,dc=test
entryExpireTimestamp: 20101024113626Z

dn: cn=leaf3,cn=top,cn=Root,dc=test
entryExpireTimestamp: 20101024113626Z


where ldapsearch "(entryExpireTimestamp<=20101024113626Z)" only finds

dn: cn=top,cn=Root,dc=test
entryExpireTimestamp: 20101024113626Z


If I change all timestamps to distinct values then expiration of complete
branches works as expected.


Thanks,
Petteri
Comment 1 petteri.stenius@ubisecure.com 2010-11-09 05:53:15 UTC
Hello,

Further investigation shows this issue is caused by operator LE search
failing with indexed attributes. Also this indexed search issue is NOT
limited to DDS. 

I have reproduced the issue with integerOrderingMatch and
generalizedTimeOrderingMatch. 

The piece of code I find suspicious is in servers/back-bdb/idl.c,
somewhere in the middle it reads

	/* skip presence key on range inequality lookups */
	while (rc == 0 && kptr->size != len) {
		rc = cursor->c_get( cursor, kptr, &data, flags |
DB_NEXT_NODUP );
	}

If I remove this block then LE search works as expected with indexed
attributes. The key here seems to be the DB_NEXT_NODUP flag. This flag
causes the iterator block a few lines below to return partial matches.

Thanks,
Petteri

-----Original Message-----
From: openldap-bugs-bounces@OpenLDAP.org
[mailto:openldap-bugs-bounces@OpenLDAP.org] On Behalf Of
petteri.stenius@ubisecure.com
Sent: Monday, October 25, 2010 1:35 PM
To: openldap-its@openldap.org
Subject: (ITS#6683) DDS fails with expired branches

Full_Name: 
Version: 2.4.23
OS: Linux
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (195.197.205.34)


Hello,

I have a directory with branches of dynamicObject entries. It looks like
if the
entryExpireTimestamp value is the same on objects within a branch then
DDS
search for expired objects will only find the top-most object. This
results in
remove failing with message

DDS dn="cn=top,cn=root,dc=test" is non-leaf; deferring.


To reproduce

OpenLDAP 2.4.23, Berkeley DB 4.6.21

Use slapadd to prepare directory with following

dn: cn=Root,dc=test
objectClass: top
objectClass: applicationProcess
cn: Root

dn: cn=top,cn=Root,dc=test
objectClass: top
objectClass: device
objectClass: dynamicObject
entryTTL: 60
entryExpireTimestamp: 20101024113626Z
cn: top

dn: cn=leaf1,cn=top,cn=Root,dc=test
objectClass: top
objectClass: device
objectClass: dynamicObject
entryTTL: 60
entryExpireTimestamp: 20101024113626Z
cn: leaf1

dn: cn=leaf2,cn=top,cn=Root,dc=test
objectClass: top
objectClass: device
objectClass: dynamicObject
entryTTL: 60
entryExpireTimestamp: 20101024113626Z
cn: leaf2

dn: cn=leaf3,cn=top,cn=Root,dc=test
objectClass: top
objectClass: device
objectClass: dynamicObject
entryTTL: 60
entryExpireTimestamp: 20101024113626Z
cn: leaf3


Relevant slapd.conf entries

database        bdb
suffix          "cn=Root,dc=test"
rootdn          "cn=Root,dc=test"
rootpw          "password"

overlay         dds
dds-default-ttl 3600
dds-min-ttl     60
dds-interval    60
dds-state       true
index           entryExpireTimestamp eq,pres

access to dn.subtree="cn=Root,dc=test"
        by users write
        by * read


Running "slapd -d 1 -d 256" produces following

put_filter:
"(&(objectClass=dynamicObject)(entryExpireTimestamp<=20101025082446Z))"
put_filter: AND
put_filter_list
"(objectClass=dynamicObject)(entryExpireTimestamp<=20101025082446Z)"
put_filter: "(objectClass=dynamicObject)"
put_filter: simple
put_simple_filter: "objectClass=dynamicObject"
put_filter: "(entryExpireTimestamp<=20101025082446Z)"
put_filter: simple
put_simple_filter: "entryExpireTimestamp<=20101025082446Z"
ber_scanf fmt ({mm}) ber:
ber_scanf fmt ({mm}) ber:
=> bdb_search
bdb_dn2entry("cn=root,dc=test")
=> bdb_dn2id("cn=root,dc=test")
<= bdb_dn2id: got id=0x1
entry_decode: "cn=Root,dc=test"
<= entry_decode(cn=Root,dc=test)
search_candidates: base="cn=root,dc=test" (0x00000001) scope=2
=> bdb_dn2idl("cn=root,dc=test")
=> bdb_equality_candidates (objectClass)
=> key_read
<= bdb_index_read: failed (-30989)
<= bdb_equality_candidates: id=0, first=0, last=0
=> bdb_equality_candidates (objectClass)
=> key_read
<= bdb_index_read 4 candidates
<= bdb_equality_candidates: id=4, first=2, last=5
=> bdb_inequality_candidates (entryExpireTimestamp)
=> key_read
<= bdb_index_read 1 candidates
=> key_read
<= bdb_index_read: failed (-30989)
<= bdb_inequality_candidates: id=1, first=2, last=2
bdb_search_candidates: id=1 first=2 last=2
entry_decode: "cn=top,cn=Root,dc=test"
<= entry_decode(cn=top,cn=Root,dc=test)
=> bdb_dn2id("cn=top,cn=root,dc=test")
<= bdb_dn2id: got id=0x2
send_ldap_result: conn=-1 op=0 p=0
bdb_dn2entry("cn=top,cn=root,dc=test")
=> bdb_dn2id_children("cn=top,cn=root,dc=test")
<= bdb_dn2id_children("cn=top,cn=root,dc=test"):  (0)
send_ldap_result: conn=-1 op=0 p=0
DDS dn="cn=top,cn=root,dc=test" is non-leaf; deferring.
DDS expired=0


ldapsearch "(entryExpireTimestamp=*)" produces

dn: cn=top,cn=Root,dc=test
entryExpireTimestamp: 20101024113626Z

dn: cn=leaf1,cn=top,cn=Root,dc=test
entryExpireTimestamp: 20101024113626Z

dn: cn=leaf2,cn=top,cn=Root,dc=test
entryExpireTimestamp: 20101024113626Z

dn: cn=leaf3,cn=top,cn=Root,dc=test
entryExpireTimestamp: 20101024113626Z


where ldapsearch "(entryExpireTimestamp<=20101024113626Z)" only finds

dn: cn=top,cn=Root,dc=test
entryExpireTimestamp: 20101024113626Z


If I change all timestamps to distinct values then expiration of
complete
branches works as expected.


Thanks,
Petteri

Comment 2 Howard Chu 2010-11-09 08:45:09 UTC
Petteri.Stenius@ubisecure.com wrote:
> Hello,
>
> Further investigation shows this issue is caused by operator LE search
> failing with indexed attributes. Also this indexed search issue is NOT
> limited to DDS.=20
>
> I have reproduced the issue with integerOrderingMatch and
> generalizedTimeOrderingMatch.=20
>
> The piece of code I find suspicious is in servers/back-bdb/idl.c,
> somewhere in the middle it reads
>
> 	/* skip presence key on range inequality lookups */
> 	while (rc =3D=3D 0&&  kptr->size !=3D len) {
> 		rc =3D cursor->c_get( cursor, kptr,&data, flags |
> DB_NEXT_NODUP );
> 	}
>
> If I remove this block then LE search works as expected with indexed
> attributes. The key here seems to be the DB_NEXT_NODUP flag. This flag
> causes the iterator block a few lines below to return partial matches.

That implies that there's something else corrupt in the index, because the 
presence key will never be the same size as an equality key.

>
> Thanks,
> Petteri
>
> -----Original Message-----
> From: openldap-bugs-bounces@OpenLDAP.org
> [mailto:openldap-bugs-bounces@OpenLDAP.org] On Behalf Of
> petteri.stenius@ubisecure.com
> Sent: Monday, October 25, 2010 1:35 PM
> To: openldap-its@openldap.org
> Subject: (ITS#6683) DDS fails with expired branches
>
> Full_Name:=20
> Version: 2.4.23
> OS: Linux
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (195.197.205.34)
>
>
> Hello,
>
> I have a directory with branches of dynamicObject entries. It looks like
> if the
> entryExpireTimestamp value is the same on objects within a branch then
> DDS
> search for expired objects will only find the top-most object. This
> results in
> remove failing with message
>
> DDS dn=3D"cn=3Dtop,cn=3Droot,dc=3Dtest" is non-leaf; deferring.
>
>
> To reproduce
>
> OpenLDAP 2.4.23, Berkeley DB 4.6.21
>
> Use slapadd to prepare directory with following
>
> dn: cn=3DRoot,dc=3Dtest
> objectClass: top
> objectClass: applicationProcess
> cn: Root
>
> dn: cn=3Dtop,cn=3DRoot,dc=3Dtest
> objectClass: top
> objectClass: device
> objectClass: dynamicObject
> entryTTL: 60
> entryExpireTimestamp: 20101024113626Z
> cn: top
>
> dn: cn=3Dleaf1,cn=3Dtop,cn=3DRoot,dc=3Dtest
> objectClass: top
> objectClass: device
> objectClass: dynamicObject
> entryTTL: 60
> entryExpireTimestamp: 20101024113626Z
> cn: leaf1
>
> dn: cn=3Dleaf2,cn=3Dtop,cn=3DRoot,dc=3Dtest
> objectClass: top
> objectClass: device
> objectClass: dynamicObject
> entryTTL: 60
> entryExpireTimestamp: 20101024113626Z
> cn: leaf2
>
> dn: cn=3Dleaf3,cn=3Dtop,cn=3DRoot,dc=3Dtest
> objectClass: top
> objectClass: device
> objectClass: dynamicObject
> entryTTL: 60
> entryExpireTimestamp: 20101024113626Z
> cn: leaf3
>
>
> Relevant slapd.conf entries
>
> database        bdb
> suffix          "cn=3DRoot,dc=3Dtest"
> rootdn          "cn=3DRoot,dc=3Dtest"
> rootpw          "password"
>
> overlay         dds
> dds-default-ttl 3600
> dds-min-ttl     60
> dds-interval    60
> dds-state       true
> index           entryExpireTimestamp eq,pres
>
> access to dn.subtree=3D"cn=3DRoot,dc=3Dtest"
>          by users write
>          by * read
>
>
> Running "slapd -d 1 -d 256" produces following
>
> put_filter:
> "(&(objectClass=3DdynamicObject)(entryExpireTimestamp<=3D20101025082446Z)=
> )"
> put_filter: AND
> put_filter_list
> "(objectClass=3DdynamicObject)(entryExpireTimestamp<=3D20101025082446Z)"
> put_filter: "(objectClass=3DdynamicObject)"
> put_filter: simple
> put_simple_filter: "objectClass=3DdynamicObject"
> put_filter: "(entryExpireTimestamp<=3D20101025082446Z)"
> put_filter: simple
> put_simple_filter: "entryExpireTimestamp<=3D20101025082446Z"
> ber_scanf fmt ({mm}) ber:
> ber_scanf fmt ({mm}) ber:
> =3D>  bdb_search
> bdb_dn2entry("cn=3Droot,dc=3Dtest")
> =3D>  bdb_dn2id("cn=3Droot,dc=3Dtest")
> <=3D bdb_dn2id: got id=3D0x1
> entry_decode: "cn=3DRoot,dc=3Dtest"
> <=3D entry_decode(cn=3DRoot,dc=3Dtest)
> search_candidates: base=3D"cn=3Droot,dc=3Dtest" (0x00000001) scope=3D2
> =3D>  bdb_dn2idl("cn=3Droot,dc=3Dtest")
> =3D>  bdb_equality_candidates (objectClass)
> =3D>  key_read
> <=3D bdb_index_read: failed (-30989)
> <=3D bdb_equality_candidates: id=3D0, first=3D0, last=3D0
> =3D>  bdb_equality_candidates (objectClass)
> =3D>  key_read
> <=3D bdb_index_read 4 candidates
> <=3D bdb_equality_candidates: id=3D4, first=3D2, last=3D5
> =3D>  bdb_inequality_candidates (entryExpireTimestamp)
> =3D>  key_read
> <=3D bdb_index_read 1 candidates
> =3D>  key_read
> <=3D bdb_index_read: failed (-30989)
> <=3D bdb_inequality_candidates: id=3D1, first=3D2, last=3D2
> bdb_search_candidates: id=3D1 first=3D2 last=3D2
> entry_decode: "cn=3Dtop,cn=3DRoot,dc=3Dtest"
> <=3D entry_decode(cn=3Dtop,cn=3DRoot,dc=3Dtest)
> =3D>  bdb_dn2id("cn=3Dtop,cn=3Droot,dc=3Dtest")
> <=3D bdb_dn2id: got id=3D0x2
> send_ldap_result: conn=3D-1 op=3D0 p=3D0
> bdb_dn2entry("cn=3Dtop,cn=3Droot,dc=3Dtest")
> =3D>  bdb_dn2id_children("cn=3Dtop,cn=3Droot,dc=3Dtest")
> <=3D bdb_dn2id_children("cn=3Dtop,cn=3Droot,dc=3Dtest"):  (0)
> send_ldap_result: conn=3D-1 op=3D0 p=3D0
> DDS dn=3D"cn=3Dtop,cn=3Droot,dc=3Dtest" is non-leaf; deferring.
> DDS expired=3D0
>
>
> ldapsearch "(entryExpireTimestamp=3D*)" produces
>
> dn: cn=3Dtop,cn=3DRoot,dc=3Dtest
> entryExpireTimestamp: 20101024113626Z
>
> dn: cn=3Dleaf1,cn=3Dtop,cn=3DRoot,dc=3Dtest
> entryExpireTimestamp: 20101024113626Z
>
> dn: cn=3Dleaf2,cn=3Dtop,cn=3DRoot,dc=3Dtest
> entryExpireTimestamp: 20101024113626Z
>
> dn: cn=3Dleaf3,cn=3Dtop,cn=3DRoot,dc=3Dtest
> entryExpireTimestamp: 20101024113626Z
>
>
> where ldapsearch "(entryExpireTimestamp<=3D20101024113626Z)" only finds
>
> dn: cn=3Dtop,cn=3DRoot,dc=3Dtest
> entryExpireTimestamp: 20101024113626Z
>
>
> If I change all timestamps to distinct values then expiration of
> complete
> branches works as expected.
>
>
> Thanks,
> Petteri
>
>
>


-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/

Comment 3 petteri.stenius@ubisecure.com 2010-11-10 11:26:49 UTC
Hi,

Thank you for your reply.

My knowledge about the bdb internals is limited. I have reproduced this
issue and reduced it to a small amount of data (see below). I think I
have done everything right with regards to setting up and preparing the
database and indexes. I've used both ldapmodify and slapadd/slapindex to
prepare the db.

If you think my indexes are corrupt then can you please give me pointers
how to verify?

I don't think there exist test cases for operators LE "<=" or GE ">=" in
the source codes.

Thanks,
Petteri

-----Original Message-----
From: Howard Chu [mailto:hyc@symas.com] 
Sent: Tuesday, November 09, 2010 10:45 AM
To: Petteri Stenius
Cc: openldap-its@openldap.org
Subject: Re: (ITS#6683) DDS fails with expired branches

Petteri.Stenius@ubisecure.com wrote:
> Hello,
>
> Further investigation shows this issue is caused by operator LE search
> failing with indexed attributes. Also this indexed search issue is NOT
> limited to DDS.=20
>
> I have reproduced the issue with integerOrderingMatch and
> generalizedTimeOrderingMatch.=20
>
> The piece of code I find suspicious is in servers/back-bdb/idl.c,
> somewhere in the middle it reads
>
> 	/* skip presence key on range inequality lookups */
> 	while (rc =3D=3D 0&&  kptr->size !=3D len) {
> 		rc =3D cursor->c_get( cursor, kptr,&data, flags |
> DB_NEXT_NODUP );
> 	}
>
> If I remove this block then LE search works as expected with indexed
> attributes. The key here seems to be the DB_NEXT_NODUP flag. This flag
> causes the iterator block a few lines below to return partial matches.

That implies that there's something else corrupt in the index, because
the 
presence key will never be the same size as an equality key.

>
> Thanks,
> Petteri
>
> -----Original Message-----
> From: openldap-bugs-bounces@OpenLDAP.org
> [mailto:openldap-bugs-bounces@OpenLDAP.org] On Behalf Of
> petteri.stenius@ubisecure.com
> Sent: Monday, October 25, 2010 1:35 PM
> To: openldap-its@openldap.org
> Subject: (ITS#6683) DDS fails with expired branches
>
> Full_Name:=20
> Version: 2.4.23
> OS: Linux
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (195.197.205.34)
>
>
> Hello,
>
> I have a directory with branches of dynamicObject entries. It looks
like
> if the
> entryExpireTimestamp value is the same on objects within a branch then
> DDS
> search for expired objects will only find the top-most object. This
> results in
> remove failing with message
>
> DDS dn=3D"cn=3Dtop,cn=3Droot,dc=3Dtest" is non-leaf; deferring.
>
>
> To reproduce
>
> OpenLDAP 2.4.23, Berkeley DB 4.6.21
>
> Use slapadd to prepare directory with following
>
> dn: cn=3DRoot,dc=3Dtest
> objectClass: top
> objectClass: applicationProcess
> cn: Root
>
> dn: cn=3Dtop,cn=3DRoot,dc=3Dtest
> objectClass: top
> objectClass: device
> objectClass: dynamicObject
> entryTTL: 60
> entryExpireTimestamp: 20101024113626Z
> cn: top
>
> dn: cn=3Dleaf1,cn=3Dtop,cn=3DRoot,dc=3Dtest
> objectClass: top
> objectClass: device
> objectClass: dynamicObject
> entryTTL: 60
> entryExpireTimestamp: 20101024113626Z
> cn: leaf1
>
> dn: cn=3Dleaf2,cn=3Dtop,cn=3DRoot,dc=3Dtest
> objectClass: top
> objectClass: device
> objectClass: dynamicObject
> entryTTL: 60
> entryExpireTimestamp: 20101024113626Z
> cn: leaf2
>
> dn: cn=3Dleaf3,cn=3Dtop,cn=3DRoot,dc=3Dtest
> objectClass: top
> objectClass: device
> objectClass: dynamicObject
> entryTTL: 60
> entryExpireTimestamp: 20101024113626Z
> cn: leaf3
>
>
> Relevant slapd.conf entries
>
> database        bdb
> suffix          "cn=3DRoot,dc=3Dtest"
> rootdn          "cn=3DRoot,dc=3Dtest"
> rootpw          "password"
>
> overlay         dds
> dds-default-ttl 3600
> dds-min-ttl     60
> dds-interval    60
> dds-state       true
> index           entryExpireTimestamp eq,pres
>
> access to dn.subtree=3D"cn=3DRoot,dc=3Dtest"
>          by users write
>          by * read
>
>
> Running "slapd -d 1 -d 256" produces following
>
> put_filter:
>
"(&(objectClass=3DdynamicObject)(entryExpireTimestamp<=3D20101025082446Z
)=
> )"
> put_filter: AND
> put_filter_list
>
"(objectClass=3DdynamicObject)(entryExpireTimestamp<=3D20101025082446Z)"
> put_filter: "(objectClass=3DdynamicObject)"
> put_filter: simple
> put_simple_filter: "objectClass=3DdynamicObject"
> put_filter: "(entryExpireTimestamp<=3D20101025082446Z)"
> put_filter: simple
> put_simple_filter: "entryExpireTimestamp<=3D20101025082446Z"
> ber_scanf fmt ({mm}) ber:
> ber_scanf fmt ({mm}) ber:
> =3D>  bdb_search
> bdb_dn2entry("cn=3Droot,dc=3Dtest")
> =3D>  bdb_dn2id("cn=3Droot,dc=3Dtest")
> <=3D bdb_dn2id: got id=3D0x1
> entry_decode: "cn=3DRoot,dc=3Dtest"
> <=3D entry_decode(cn=3DRoot,dc=3Dtest)
> search_candidates: base=3D"cn=3Droot,dc=3Dtest" (0x00000001) scope=3D2
> =3D>  bdb_dn2idl("cn=3Droot,dc=3Dtest")
> =3D>  bdb_equality_candidates (objectClass)
> =3D>  key_read
> <=3D bdb_index_read: failed (-30989)
> <=3D bdb_equality_candidates: id=3D0, first=3D0, last=3D0
> =3D>  bdb_equality_candidates (objectClass)
> =3D>  key_read
> <=3D bdb_index_read 4 candidates
> <=3D bdb_equality_candidates: id=3D4, first=3D2, last=3D5
> =3D>  bdb_inequality_candidates (entryExpireTimestamp)
> =3D>  key_read
> <=3D bdb_index_read 1 candidates
> =3D>  key_read
> <=3D bdb_index_read: failed (-30989)
> <=3D bdb_inequality_candidates: id=3D1, first=3D2, last=3D2
> bdb_search_candidates: id=3D1 first=3D2 last=3D2
> entry_decode: "cn=3Dtop,cn=3DRoot,dc=3Dtest"
> <=3D entry_decode(cn=3Dtop,cn=3DRoot,dc=3Dtest)
> =3D>  bdb_dn2id("cn=3Dtop,cn=3Droot,dc=3Dtest")
> <=3D bdb_dn2id: got id=3D0x2
> send_ldap_result: conn=3D-1 op=3D0 p=3D0
> bdb_dn2entry("cn=3Dtop,cn=3Droot,dc=3Dtest")
> =3D>  bdb_dn2id_children("cn=3Dtop,cn=3Droot,dc=3Dtest")
> <=3D bdb_dn2id_children("cn=3Dtop,cn=3Droot,dc=3Dtest"):  (0)
> send_ldap_result: conn=3D-1 op=3D0 p=3D0
> DDS dn=3D"cn=3Dtop,cn=3Droot,dc=3Dtest" is non-leaf; deferring.
> DDS expired=3D0
>
>
> ldapsearch "(entryExpireTimestamp=3D*)" produces
>
> dn: cn=3Dtop,cn=3DRoot,dc=3Dtest
> entryExpireTimestamp: 20101024113626Z
>
> dn: cn=3Dleaf1,cn=3Dtop,cn=3DRoot,dc=3Dtest
> entryExpireTimestamp: 20101024113626Z
>
> dn: cn=3Dleaf2,cn=3Dtop,cn=3DRoot,dc=3Dtest
> entryExpireTimestamp: 20101024113626Z
>
> dn: cn=3Dleaf3,cn=3Dtop,cn=3DRoot,dc=3Dtest
> entryExpireTimestamp: 20101024113626Z
>
>
> where ldapsearch "(entryExpireTimestamp<=3D20101024113626Z)" only
finds
>
> dn: cn=3Dtop,cn=3DRoot,dc=3Dtest
> entryExpireTimestamp: 20101024113626Z
>
>
> If I change all timestamps to distinct values then expiration of
> complete
> branches works as expected.
>
>
> Thanks,
> Petteri
>
>
>


-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/

Comment 4 Quanah Gibson-Mount 2017-04-07 23:41:09 UTC
moved from Incoming to Software Bugs
Comment 5 Quanah Gibson-Mount 2020-03-19 22:11:31 UTC
Needs to be validated with the provided test case against back-mdb
Comment 6 Quanah Gibson-Mount 2021-06-23 20:43:29 UTC
Updated the configuration to use slapd-mdb as the backend, and the data.ldif to use the year 2021 instead of 2010 so the entries wouldn't expire.

Works correctly for me.

slapd.conf:
------------------
include         /opt/symas/etc/openldap/schema/core.schema

pidfile         /var/symas/run/slapd.pid
argsfile        /var/symas/run/slapd.args

moduleload      back_mdb.la
moduleload      dds.la

access to dn.base="" by * read
access to dn.base="cn=Subschema" by * read

database config
rootpw secret

database        mdb
maxsize         1073741824
suffix          "cn=Root,dc=test"
rootdn          "cn=Root,dc=test"
rootpw          "password"
directory       /var/symas/openldap-data
index   objectClass     eq
index           entryExpireTimestamp eq,pres


access to dn.subtree="cn=Root,dc=test"
        by users write
        by * read

overlay         dds
dds-default-ttl 3600
dds-min-ttl     60
dds-interval    60
dds-state       true

database monitor



data.ldif
-----------------------------
dn: cn=Root,dc=test
objectClass: top
objectClass: applicationProcess
cn: Root

dn: cn=top,cn=Root,dc=test
objectClass: top
objectClass: device
objectClass: dynamicObject
entryTTL: 60
entryExpireTimestamp: 20211024113626Z
cn: top

dn: cn=leaf1,cn=top,cn=Root,dc=test
objectClass: top
objectClass: device
objectClass: dynamicObject
entryTTL: 60
entryExpireTimestamp: 20211024113626Z
cn: leaf1

dn: cn=leaf2,cn=top,cn=Root,dc=test
objectClass: top
objectClass: device
objectClass: dynamicObject
entryTTL: 60
entryExpireTimestamp: 20211024113626Z
cn: leaf2

dn: cn=leaf3,cn=top,cn=Root,dc=test
objectClass: top
objectClass: device
objectClass: dynamicObject
entryTTL: 60
entryExpireTimestamp: 20211024113626Z
cn: leaf3


results:

/opt/symas/bin/ldapsearch -x -LLL -b "cn=root,dc=test" "(entryExpireTimestamp=*)"
dn: cn=top,cn=Root,dc=test
objectClass: top
objectClass: device
objectClass: dynamicObject
cn: top

dn: cn=leaf1,cn=top,cn=Root,dc=test
objectClass: top
objectClass: device
objectClass: dynamicObject
cn: leaf1

dn: cn=leaf2,cn=top,cn=Root,dc=test
objectClass: top
objectClass: device
objectClass: dynamicObject
cn: leaf2

dn: cn=leaf3,cn=top,cn=Root,dc=test
objectClass: top
objectClass: device
objectClass: dynamicObject
cn: leaf3


/opt/symas/bin/ldapsearch -x -LLL  -b "cn=root,dc=test" "(entryExpireTimestamp<=20211024113626Z)"
dn: cn=top,cn=Root,dc=test
objectClass: top
objectClass: device
objectClass: dynamicObject
cn: top

dn: cn=leaf1,cn=top,cn=Root,dc=test
objectClass: top
objectClass: device
objectClass: dynamicObject
cn: leaf1

dn: cn=leaf2,cn=top,cn=Root,dc=test
objectClass: top
objectClass: device
objectClass: dynamicObject
cn: leaf2

dn: cn=leaf3,cn=top,cn=Root,dc=test
objectClass: top
objectClass: device
objectClass: dynamicObject
cn: leaf3