[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: (ITS#6242) PCache cache corruption



I see a few configuration issues that may be related to the problem you
describe.  See below for details.

> karavelov@spnet.net wrote:
>> This is a multi-part message in MIME format.
>> --------------000408020401050805090402
>>
> The attachment did't work. So here is the conf file:

You should provide an URI to data you want to share.  You can upload data
to OpenLDAP's public FTP site.

> # This is the main slapd configuration file. See slapd.conf(5) for more
> # info on the configuration options.
>
> #######################################################################
> # Global Directives:
>
> # Schema and objectClass definitions
> include         /etc/ldap/schema/core.schema
> include         /etc/ldap/schema/cosine.schema
> include         /etc/ldap/schema/nis.schema
> include         /etc/ldap/schema/inetorgperson.schema
> include         /etc/ldap/schema/mailrouter.schema
>
> # Where the pid file is put. The init.d script
> # will not stop the server if you change this.
> pidfile         /var/run/slapd/slapd.pid
>
> # List of arguments that were passed to the server
> argsfile        /var/run/slapd/slapd.args
>
> # Read slapd.conf(5) for possible values
> loglevel    4864

^^^ This is equivalent to 0x100 + 0x200 + 0x1000, which means stats,
stats2 and an undefined log level.

> # Where the dynamically loaded modules are stored
> modulepath    /usr/lib/ldap
> moduleload    back_bdb
> moduleload    back_sql
> moduleload    pcache
>
> # The maximum number of entries that is returned for a search operation
> sizelimit 500
>
> # The tool-threads parameter sets the actual amount of cpu's that is used
> # for indexing.
> tool-threads 8
> threads 2

This may be nonsense.  The default is 16 threads, and you should not use
less unless you know what you're doing.  Slapo-pcache uses internal
threads for cache maintenance; slapd may need to use other threads for
internal purposes.  Setting threads to 2 you risk starvation as soon as
multiple internal tasks need to be done at the same time.

> # Ensure read access to the base for things like
> # supportedSASLMechanisms.  Without this you may
> # have problems with SASL not knowing what
> # mechanisms are available and the like.
> # Note that this is covered by the 'access to *'
> # ACL below too but if you change that as people
> # are wont to do you'll still need this if you
> # want SASL (and possible other things) to work
> # happily.
> access to dn.base="" by * read
>
> # The admin dn has full write access, everyone else
> # can read everything.
>
> access to dn.subtree="dc=hosting,dc=spnet,dc=net"
>        by dn="cn=nss,dc=ldapaccess" read
>
>
> access to *
>        by * auth
>
> # LDIF for bind creditials
> database ldif
> suffix      "dc=ldapaccess"
> rootdn      "dc=ldapaccess"
> directory    "/etc/ldap/ldif"
> rootpw      some-secret
> #readonly        on
>
> #######################################################################
> ## sql database definitions
> ########################################################################
> #
> database sql
> suffix      "dc=spnet,dc=net"
> rootdn      "cn=admin,dc=spnet,dc=net"
> rootpw      bind-secret
> dbname          ISP
> dbuser          ldap
> dbpasswd        sql-secret
> aliasing_quote     \"
> subtree_cond    "UPPER(ldap_entries.dn) LIKE CONCAT('%',UPPER(?))"
> # actiualy it is read only db - it is never used
> insentry_stmt   "INSERT INTO ldap_entries
> (id,dn,oc_map_id,parent,keyval) VALUES (ldap_entry_ids.nextval,?,?,?,?)"
> upper_func      UPPER
> has_ldapinfo_dn_ru no
>
> ##########
> # chache
> ##########
> overlay pcache
> proxycache    bdb 67108864 2 16384 60

Are you seriously going to cache up to 67M entries in a database with 16MB
of Berkeley DB cache? (see below)

> proxyAttrset  0 cn uid uidNumber gidNumber loginShell homeDirectory
> userPassword description gecos memberUid dc spamassassin
> proxyAttrset  1 spamassassin uid cn userPassword dc mailMessageStore
> mailQuota uidNumber gidNumber mailAliasedName
>
> proxytemplate   (&(objectClass=)(uidNumber=))
> 0 3600 3600
> proxytemplate   (&(objectClass=)(gidNumber=))
> 0 3600 3600
> proxytemplate   (&(objectClass=)(uid=))
> 0 3600 3600
> proxytemplate   (&(objectClass=)(memberUid=))
> 0 3600 3600
> proxytemplate   (&(objectClass=)(|(memberUid=)(uniqueMember=)))
> 0 3600 3600
> proxytemplate   (&(objectClass=)(dc=))
> 0 3600 3600
> proxytemplate   (&(objectClass=)(cn=))
> 0 3600 3600
> proxytemplate   (objectClass=)
> 0 3600 3600
> proxytemplate   (uid=)
> 0 3600 3600
> proxytemplate   (cn=)
> 0 3600 3600
> proxytemplate   (dc=)
> 0 3600 3600
>
>
> proxytemplate   (&(objectClass=)(uidNumber=))
> 1 600 300
> proxytemplate   (&(objectClass=)(gidNumber=))
> 1 600 300
> proxytemplate   (&(objectClass=)(uid=))
> 1 600 300
> proxytemplate   (&(objectClass=)(dc=))
> 1 600 300
> proxytemplate   (&(objectClass=)(cn=))
> 1 600 300
> proxytemplate   (objectClass=)
> 1 600 300
> proxytemplate   (uid=)
> 1 600 300
> proxytemplate   (cn=)
> 1 600 300

I hope line wraps are related to cut'n'paste in the mailer.  Continuation
lines need to start with a blank in slapd.conf
>
> # dbd backend config
>
> directory /var/tmp/
>
> index           objectClass eq
> index           uid eq
> index           uidNumber eq
> index           gidNumber eq
> index           memberUid eq
> index           dc pres,eq,sub
> index           cn pres,eq,sub
>
> checkpoint      128 60
> cachesize       163840
> idlcachesize    163840
> dirtyread

^^^ you shouldn't use this parameter if you look for search result
consistency

> dbconfig        set_cachesize 0 16777216 1

as mentioned earlier, this value makes little sense, since a 16MB cache is
going to perform poorly as soon as your database is less than minimal. 
Read Berkeley DB documentation for details about the meaning of this
configuration.  A sane starting value, also depending on the amount of RAM
of your system, is 256MB.

> dbconfig        set_lg_regionmax 262144
> dbconfig        set_lg_bsize 2097152

In conclusion, there is nothing blatantly wrong, but some tuning might
improve your configuration.  As per the issue you're complaining about,
you should investigate it a little further, before we can determine
whether it's caused by a bug in the software and, in case, track it.  For
example, you should determine whether it repeats with a given data set,
and possibly provide logs at stats, stats2, pcache level of a
malfunctioning event.

p.