Issue 5859 - back-hdb entrycache problem
Summary: back-hdb entrycache problem
Status: VERIFIED FIXED
Alias: None
Product: OpenLDAP
Classification: Unclassified
Component: slapd (show other issues)
Version: unspecified
Hardware: All All
: --- normal
Target Milestone: ---
Assignee: OpenLDAP project
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-12-15 16:16 UTC by Quanah Gibson-Mount
Modified: 2014-08-01 21:04 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description Quanah Gibson-Mount 2008-12-15 16:16:32 UTC
Full_Name: Quanah Gibson-Mount
Version: 2.4.13+patches
OS: Linux 2.6
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (75.111.29.239)


In using 2.4.13 + the slapadd fix from Ralf, ITS#5841,
ITS#5842,ITS#5849,ITS#5850, ITS#5854, I found the following:

Create an database with back-hdb with a few subtrees, with several entries under
that subtree (My database is rooted at "".  I have a "cn=zimbra" tree, with
"cn=zimlets,cn=zimbra" underneath it).

Now, create a brand new entry under one of those subtrees.

The new entry will not be returned in queries rooted at the upper level
("cn=zimbra").  It will show up at its level ("cn=zimlets,cn=zimbra").  If you
stop/restart slapd to flush the entry/idl cache, things work correctly.



Comment 1 Quanah Gibson-Mount 2008-12-15 17:51:48 UTC
--On Monday, December 15, 2008 4:16 PM +0000 openldap-its@OpenLDAP.org 
wrote:

Appears to be broken in 2.4.13 as well.

Search prior to adding zimbra-3 entry:

[zimbra@freelancer ~]$ ldapsearch -x -h freelancer -b "cn=zimbra" -D 
"uid=zimbra,cn=admins,cn=zimbra" -W

# com_zimbra_license, zimlets, zimbra
dn: cn=com_zimbra_license,cn=zimlets,cn=zimbra
zimbraZimletTarget: admin-main
zimbraZimletDescription: License Management for Admin UI
zimbraZimletVersion: 5.2
objectClass: zimbraZimletEntry
zimbraZimletIndexingEnabled: TRUE
zimbraZimletKeyword: com_zimbra_license
cn: com_zimbra_license
zimbraZimletIsExtension: TRUE
zimbraZimletPriority: 1
zimbraZimletEnabled: TRUE

# zimbra-2, zimlets, zimbra
dn: cn=zimbra-2,cn=zimlets,cn=zimbra
cn: zimbra-2
objectClass: zimbraZimletEntry
zimbraZimletVersion: 1.0
zimbraZimletEnabled: TRUE

# search result
search: 2
result: 0 Success

# numResponses: 46
# numEntries: 45


Then we add the new entry:


[zimbra@freelancer ~]$ ldapadd -f /tmp/test.ldif -x -h freelancer -D 
"uid=zimbra,cn=admins,cn=zimbra" -W
Enter LDAP Password:
adding new entry "cn=zimbra-3,cn=zimlets,cn=zimbra"


Now we search again:

# com_zimbra_license, zimlets, zimbra
dn: cn=com_zimbra_license,cn=zimlets,cn=zimbra
zimbraZimletTarget: admin-main
zimbraZimletDescription: License Management for Admin UI
zimbraZimletVersion: 5.2
objectClass: zimbraZimletEntry
zimbraZimletIndexingEnabled: TRUE
zimbraZimletKeyword: com_zimbra_license
cn: com_zimbra_license
zimbraZimletIsExtension: TRUE
zimbraZimletPriority: 1
zimbraZimletEnabled: TRUE

# zimbra-2, zimlets, zimbra
dn: cn=zimbra-2,cn=zimlets,cn=zimbra
cn: zimbra-2
objectClass: zimbraZimletEntry
zimbraZimletVersion: 1.0
zimbraZimletEnabled: TRUE

# search result
search: 2
result: 0 Success

# numResponses: 46
# numEntries: 45

Same result, this is bad.

Now we stop/restart slapd:

[zimbra@freelancer ~]$ ldap stop
Killing slapd with pid 15613 done.
[zimbra@freelancer ~]$ ldap start
Started slapd: pid 20224


search again:


# zimbra-2, zimlets, zimbra
dn: cn=zimbra-2,cn=zimlets,cn=zimbra
cn: zimbra-2
objectClass: zimbraZimletEntry
zimbraZimletVersion: 1.0
zimbraZimletEnabled: TRUE

# zimbra-3, zimlets, zimbra
dn: cn=zimbra-3,cn=zimlets,cn=zimbra
cn: zimbra-3
objectClass: zimbraZimletEntry
zimbraZimletVersion: 1.0
zimbraZimletEnabled: TRUE

# search result
search: 2
result: 0 Success

# numResponses: 46
# numEntries: 45


And there is our missing entry.

--Quanah

--

Quanah Gibson-Mount
Release and QA Engineer
<http://www.openldap.org>

Comment 2 Ralf 2008-12-17 14:03:32 UTC
This problem seems to be only present when using an empty suffix. I wasn't 
able to reproduce it with a non-empty suffix at least.

It seems to be a problem with the idlcache. After setting "idlcachesize" to 0 
the problem went a way for me.

And yes, it seems to have been in the RE24 tree for a longer time already 
(2.4.12 was affected as well).

-- 
Ralf

Comment 3 Ralf 2008-12-17 19:48:37 UTC
Am Mittwoch 17 Dezember 2008 15:03:49 schrieb rhafer@suse.de:
> This problem seems to be only present when using an empty suffix. I wasn't
> able to reproduce it with a non-empty suffix at least.
>
> It seems to be a problem with the idlcache. After setting "idlcachesize" to
> 0 the problem went a way for me.
>
> And yes, it seems to have been in the RE24 tree for a longer time already
> (2.4.12 was affected as well).
Digging a bit deeper into this, I found that only back-hdb is affected  and 
that it is most probably a bug in hdb_dn2id_add(). It seems that after adding  
a new entry, hdb_dn2id_add() fails to correctly update the IDLs in the idl-
cache for the entry's parent entries.

The following patch seems to fix the problem for me:
--- dn2id.c     31 Oct 2008 23:23:59 -0000      1.137.2.13                           
+++ dn2id.c     17 Dec 2008 19:45:32 -0000                                           
@@ -620,6 +620,11 @@                                                                 
                                tmp[1] = eip->bei_id;
                                bdb_idl_cache_add_id( bdb, db, &key, e->e_id 
);
                        }
+                        if ( !op->o_bd->be_suffix[0].bv_len && eip)
+                        {
+                               tmp[1] = eip->bei_id;
+                               bdb_idl_cache_add_id( bdb, db, &key, e->e_id 
);
+                        }
                }
        }

I suspect that hdb_dn2id_delete() has a similar issue.

-- 
Ralf

Comment 4 Ralf 2009-01-16 14:19:24 UTC
moved from Incoming to Software Bugs
Comment 5 Howard Chu 2009-01-26 09:21:05 UTC
changed notes
changed state Open to Test
Comment 6 Quanah Gibson-Mount 2009-01-26 21:28:26 UTC
changed notes
changed state Test to Release
Comment 7 Quanah Gibson-Mount 2009-02-15 02:06:35 UTC
changed notes
changed state Release to Closed
Comment 8 OpenLDAP project 2014-08-01 21:04:19 UTC
fixed in HEAD
fixed in RE24