OpenLDAP
Up to top level
Build   Contrib   Development   Documentation   Historical   Incoming   Software Bugs   Software Enhancements   Web  

Logged in as guest

Viewing Development/6243
Full headers

From: quanah@zimbra.com
Subject: back-monitor fails to report entry cache usage
Compose comment
Download message
State:
0 replies:
9 followups: 1 2 3 4 5 6 7 8 9

Major security issue: yes  no

Notes:

Notification:


Date: Wed, 05 Aug 2009 17:35:57 +0000
From: quanah@zimbra.com
To: openldap-its@OpenLDAP.org
Subject: back-monitor fails to report entry cache usage
Full_Name: Quanah Gibson-Mount
Version: 2.4.17
OS: Linux 2.6
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (69.235.224.133)


Querying back monitor to examine the values of the various caches reports the
entry cache as empty:

# Frontend, Databases, Monitor
dn: cn=Frontend,cn=Databases,cn=Monitor
structuralObjectClass: monitoredObject
creatorsName: cn=config
modifiersName: cn=config
createTimestamp: 20090805044038Z
modifyTimestamp: 20090805044038Z
monitoredInfo: frontend
monitorIsShadow: FALSE
namingContexts:
readOnly: FALSE
olmBDBEntryCache: 0
olmBDBDNCache: 592
olmBDBIDLCache: 502
olmDbDirectory: /opt/zimbra/data/ldap/hdb/db/
entryDN: cn=Frontend,cn=Databases,cn=Monitor
subschemaSubentry: cn=Subschema
hasSubordinates: FALSE


Followup 1

Download message
Date: Wed, 05 Aug 2009 11:03:35 -0700
From: Quanah Gibson-Mount <quanah@zimbra.com>
To: openldap-its@openldap.org
Subject: Re: (ITS#6243) back-monitor fails to report entry cache usage

--On August 5, 2009 5:35:57 PM +0000 quanah@zimbra.com wrote:

> Full_Name: Quanah Gibson-Mount
> Version: 2.4.17
> OS: Linux 2.6
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (69.235.224.133)
>
>
> Querying back monitor to examine the values of the various caches reports
> the entry cache as empty:
>
># Frontend, Databases, Monitor
> dn: cn=Frontend,cn=Databases,cn=Monitor
> structuralObjectClass: monitoredObject
> creatorsName: cn=config
> modifiersName: cn=config
> createTimestamp: 20090805044038Z
> modifyTimestamp: 20090805044038Z
> monitoredInfo: frontend
> monitorIsShadow: FALSE
> namingContexts:
> readOnly: FALSE
> olmBDBEntryCache: 0
> olmBDBDNCache: 592
> olmBDBIDLCache: 502
> olmDbDirectory: /opt/zimbra/data/ldap/hdb/db/
> entryDN: cn=Frontend,cn=Databases,cn=Monitor
> subschemaSubentry: cn=Subschema
> hasSubordinates: FALSE


This may be specific to glued databases (databases rooted at "").  I see 
that it's reporting the caches as under the Frontend database, instead of 
the actual BDB database, which is database 3, and shows no cache 
information:

# Database 3, Databases, Monitor
dn: cn=Database 3,cn=Databases,cn=Monitor
structuralObjectClass: monitoredObject
creatorsName: cn=config
modifiersName: cn=config
createTimestamp: 20090805044038Z
modifyTimestamp: 20090805044038Z
monitoredInfo: hdb
monitorIsShadow: FALSE
namingContexts:
readOnly: FALSE
monitorOverlay: accesslog
monitorOverlay: syncprov
entryDN: cn=Database 3,cn=Databases,cn=Monitor
subschemaSubentry: cn=Subschema
hasSubordinates: TRUE

I see the same behavior on the replica, where it is Database 2 instead of 
database 3, as it has no accesslog DB.  On the master, the accesslog DB 
caches are correct:

# Database 2, Databases, Monitor
dn: cn=Database 2,cn=Databases,cn=Monitor
structuralObjectClass: monitoredObject
creatorsName: cn=config
modifiersName: cn=config
createTimestamp: 20090805044038Z
modifyTimestamp: 20090805044038Z
monitoredInfo: hdb
monitorIsShadow: FALSE
namingContexts: cn=accesslog
readOnly: FALSE
monitorOverlay: syncprov
olmBDBEntryCache: 297
olmBDBDNCache: 517
olmBDBIDLCache: 9
olmDbDirectory: /opt/zimbra/data/ldap/accesslog/db/
entryDN: cn=Database 2,cn=Databases,cn=Monitor
subschemaSubentry: cn=Subschema
hasSubordinates: TRUE

--Quanah


--

Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra ::  the leader in open source messaging and collaboration



Followup 2

Download message
Date: Wed, 5 Aug 2009 21:17:27 +0200 (CEST)
Subject: Re: (ITS#6243) back-monitor fails to report entry cache usage
From: masarati@aero.polimi.it
To: quanah@zimbra.com
Cc: openldap-its@openldap.org
> This may be specific to glued databases (databases rooted at "").

The problem could be partially addressed by telling back-bdb (who's
maintaining this data in the monitor backend) to check subordinates as
soon as it discovers it's a glue instance.  However, this poses two
different problems:

- is the aggregate information resulting from adding all glued databases
cache usage still useful?

- what happens if heterogeneous databases are glued?  Significantly, what
if the superior database is not bdb/hdb?

Probably, the monitor database should also present subordinate databases
as separate entries.

p.



Followup 3

Download message
Date: Thu, 06 Aug 2009 09:40:00 -0700
From: Quanah Gibson-Mount <quanah@zimbra.com>
To: masarati@aero.polimi.it, openldap-its@openldap.org
Subject: Re: (ITS#6243) back-monitor fails to report entry cache usage

--On August 5, 2009 7:17:43 PM +0000 masarati@aero.polimi.it wrote:

>> This may be specific to glued databases (databases rooted at "").
>
> The problem could be partially addressed by telling back-bdb (who's
> maintaining this data in the monitor backend) to check subordinates as
> soon as it discovers it's a glue instance.  However, this poses two
> different problems:
>
> - is the aggregate information resulting from adding all glued databases
> cache usage still useful?
>
> - what happens if heterogeneous databases are glued?  Significantly, what
> if the superior database is not bdb/hdb?
>
> Probably, the monitor database should also present subordinate databases
> as separate entries.

Interestingly, in my case, there's only one real database in play for the 
glue as it is, since it's rooted at "".  All other database definitions 
come before it (cn=config, cn=accesslog, cn=monitor).

I'm also curious why back-monitor develops stats for the other caches but 
specifically not for entry cache.

--Quanah


--

Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra ::  the leader in open source messaging and collaboration



Followup 4

Download message
Date: Thu, 6 Aug 2009 19:25:16 +0200 (CEST)
Subject: Re: (ITS#6243) back-monitor fails to report entry cache usage
From: masarati@aero.polimi.it
To: "Quanah Gibson-Mount" <quanah@zimbra.com>
Cc: openldap-its@openldap.org
>
>
> --On August 5, 2009 7:17:43 PM +0000 masarati@aero.polimi.it wrote:
>
>>> This may be specific to glued databases (databases rooted at "").
>>
>> The problem could be partially addressed by telling back-bdb (who's
>> maintaining this data in the monitor backend) to check subordinates as
>> soon as it discovers it's a glue instance.  However, this poses two
>> different problems:
>>
>> - is the aggregate information resulting from adding all glued
databases
>> cache usage still useful?
>>
>> - what happens if heterogeneous databases are glued?  Significantly,
>> what
>> if the superior database is not bdb/hdb?
>>
>> Probably, the monitor database should also present subordinate
databases
>> as separate entries.
>
> Interestingly, in my case, there's only one real database in play for the
> glue as it is, since it's rooted at "".  All other database definitions
> come before it (cn=config, cn=accesslog, cn=monitor).

but... are you actually gluing something?  In any case, this is not
specific to the case of empty suffix in the glue database.  I could easily
reproduce it with a "normal" glued setup, and I was about to start fixing
things when the two above questions came to my mind.

> I'm also curious why back-monitor develops stats for the other caches but
> specifically not for entry cache.

I haven't looked in detail, but it makes sense that some operation occurs
within the glue database which requires caching something, but not
entries.

- The entry cache monitor shows the value of bdb->bi_cache.c_cursize;

- the DN cache monitor shows the value of bdb->bi_cache.c_eiused, which
should be the number of entryinfo structures used;

- the IDL cache monitor shows the value of bdb->bi_idl_cache_size.

In my very simple tests, I only saw something populating the DN cache,
which means some internal operation required to allocate some entryinfo
structures that remain 'round.

In any case, it's only showing information related to its database
structure, it is by no means collecting info from the glued databases.

p.



Followup 5

Download message
Date: Thu, 06 Aug 2009 10:33:03 -0700
From: Quanah Gibson-Mount <quanah@zimbra.com>
To: masarati@aero.polimi.it
cc: openldap-its@openldap.org
Subject: Re: (ITS#6243) back-monitor fails to report entry cache usage

--On August 6, 2009 7:25:16 PM +0200 masarati@aero.polimi.it wrote:


>> I'm also curious why back-monitor develops stats for the other caches
but
>> specifically not for entry cache.
>
> I haven't looked in detail, but it makes sense that some operation occurs
> within the glue database which requires caching something, but not
> entries.
>
> - The entry cache monitor shows the value of bdb->bi_cache.c_cursize;
>
> - the DN cache monitor shows the value of bdb->bi_cache.c_eiused, which
> should be the number of entryinfo structures used;
>
> - the IDL cache monitor shows the value of bdb->bi_idl_cache_size.
>
> In my very simple tests, I only saw something populating the DN cache,
> which means some internal operation required to allocate some entryinfo
> structures that remain 'round.
>
> In any case, it's only showing information related to its database
> structure, it is by no means collecting info from the glued databases.

Ok, this makes sense, as the numbers it was reporting definitely seemed 
smaller than I was expecting.  I bet they are for pieces in the rootDSE.

--Quanah

--

Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra ::  the leader in open source messaging and collaboration



Followup 6

Download message
Date: Thu, 13 Aug 2009 10:35:10 -0700
From: Quanah Gibson-Mount <quanah@zimbra.com>
To: masarati@aero.polimi.it
cc: openldap-its@openldap.org
Subject: Re: (ITS#6243) back-monitor fails to report entry cache usage
--On Thursday, August 06, 2009 7:25 PM +0200 masarati@aero.polimi.it wrote:

>>
>>
>> --On August 5, 2009 7:17:43 PM +0000 masarati@aero.polimi.it wrote:
>>
>>>> This may be specific to glued databases (databases rooted at
"").
>>>
>>> The problem could be partially addressed by telling back-bdb (who's
>>> maintaining this data in the monitor backend) to check subordinates
as
>>> soon as it discovers it's a glue instance.  However, this poses two
>>> different problems:
>>>
>>> - is the aggregate information resulting from adding all glued
databases
>>> cache usage still useful?
>>>
>>> - what happens if heterogeneous databases are glued? 
Significantly,
>>> what
>>> if the superior database is not bdb/hdb?
>>>
>>> Probably, the monitor database should also present subordinate
databases
>>> as separate entries.
>>
>> Interestingly, in my case, there's only one real database in play for
the
>> glue as it is, since it's rooted at "".  All other database definitions
>> come before it (cn=config, cn=accesslog, cn=monitor).
>
> but... are you actually gluing something?  In any case, this is not
> specific to the case of empty suffix in the glue database.  I could easily
> reproduce it with a "normal" glued setup, and I was about to start fixing
> things when the two above questions came to my mind.
>
>> I'm also curious why back-monitor develops stats for the other caches
but
>> specifically not for entry cache.
>
> I haven't looked in detail, but it makes sense that some operation occurs
> within the glue database which requires caching something, but not
> entries.
>
> - The entry cache monitor shows the value of bdb->bi_cache.c_cursize;
>
> - the DN cache monitor shows the value of bdb->bi_cache.c_eiused, which
> should be the number of entryinfo structures used;
>
> - the IDL cache monitor shows the value of bdb->bi_idl_cache_size.
>
> In my very simple tests, I only saw something populating the DN cache,
> which means some internal operation required to allocate some entryinfo
> structures that remain 'round.
>
> In any case, it's only showing information related to its database
> structure, it is by no means collecting info from the glued databases.

Any chance we can get this fixed for 2.4.18 for the real backends?  I think 
making it so you can see the counters per-real backend should definitely be 
in place.  How to handle the frontend is interesting and definitely should 
be addressed as well but isn't (to me at least) quite as crucial at this 
moment. ;)

--Quanah



--

Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra ::  the leader in open source messaging and collaboration



Followup 7

Download message
Date: Thu, 13 Aug 2009 19:39:50 +0200 (CEST)
Subject: Re: (ITS#6243) back-monitor fails to report entry cache usage
From: masarati@aero.polimi.it
To: "Quanah Gibson-Mount" <quanah@zimbra.com>
Cc: openldap-its@openldap.org
> Any chance we can get this fixed for 2.4.18 for the real backends?  I
> think
> making it so you can see the counters per-real backend should definitely
> be
> in place.  How to handle the frontend is interesting and definitely should
> be addressed as well but isn't (to me at least) quite as crucial at this
> moment. ;)

A quick hack consists in removing the test for SLAP_GLUE_SUBORDINATE() in
monitor_subsys_database_init().  This will expose subordinate databases
and thus should allow back-bdb and back-hdb to hook their cache stats
stuff.  Probably this is the best approach.  I would complement it by
adding a olmIsSubordinate attribute with TRUE value, and/or a
olmSuperiorSuffix containing the suffix of the superior.

p.



Followup 8

Download message
Date: Thu, 13 Aug 2009 20:56:18 +0200 (CEST)
Subject: Re: (ITS#6243) back-monitor fails to report entry cache usage
From: masarati@aero.polimi.it
To: quanah@zimbra.com
Cc: openldap-its@openldap.org
> A quick hack consists in removing the test for SLAP_GLUE_SUBORDINATE() in
> monitor_subsys_database_init().  This will expose subordinate databases
> and thus should allow back-bdb and back-hdb to hook their cache stats
> stuff.  Probably this is the best approach.  I would complement it by
> adding a olmIsSubordinate attribute with TRUE value, and/or a
> olmSuperiorSuffix containing the suffix of the superior.

An even better approach would be to append subordinate databases below the
superior one...  but I'd leave this for 2.5 :)

p.



Followup 9

Download message
Date: Thu, 13 Aug 2009 12:52:13 -0700
From: Quanah Gibson-Mount <quanah@zimbra.com>
To: masarati@aero.polimi.it
cc: openldap-its@openldap.org
Subject: Re: (ITS#6243) back-monitor fails to report entry cache usage
--On Thursday, August 13, 2009 8:56 PM +0200 masarati@aero.polimi.it wrote:

>
>> A quick hack consists in removing the test for SLAP_GLUE_SUBORDINATE()
in
>> monitor_subsys_database_init().  This will expose subordinate databases
>> and thus should allow back-bdb and back-hdb to hook their cache stats
>> stuff.  Probably this is the best approach.  I would complement it by
>> adding a olmIsSubordinate attribute with TRUE value, and/or a
>> olmSuperiorSuffix containing the suffix of the superior.
>
> An even better approach would be to append subordinate databases below the
> superior one...  but I'd leave this for 2.5 :)

Thanks for handling this. :)

--Quanah

--

Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra ::  the leader in open source messaging and collaboration


Up to top level
Build   Contrib   Development   Documentation   Historical   Incoming   Software Bugs   Software Enhancements   Web  

Logged in as guest


The OpenLDAP Issue Tracking System uses a hacked version of JitterBug

______________
© Copyright 2013, OpenLDAP Foundation, info@OpenLDAP.org