[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: Openldap overloading



2012/9/28 Nick Milas <nick@eurobjects.com>:
> Hi,
>
> I am running a v2.4.31 consumer on CentOS 5.8 to serve user accounts (and
> aliases) on a Postfix mail server running locally. It has been running for a
> long time without problems.
>
> Today, after a user sent (on 14:53:39) a mass mail (through a group alias,
> implemented using ldap dynlist), Postfix stalled and the server (a VM under
> KVM) became overloaded. I noticed that openldap was using all the cpu:
>
> # top
> top - 15:30:01 up 81 days,  2:11,  1 user,  load average: 113.58, 114.36,
> 104.02
> Tasks: 460 total,   3 running, 457 sleeping,   0 stopped,   0 zombie
> Cpu(s): 98.9%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  1.1%hi, 0.0%si,
> 0.0%st
> Mem:   3089988k total,  3074912k used,    15076k free,    12180k buffers
> Swap:  2064376k total,       92k used,  2064284k free,  1909976k cached
>
>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+ COMMAND
>  2209 ldap      18   0  577m  17m 8952 S 93.4  0.6  55:03.67 slapd
> ...
>
> I had to stop and restart openldap manually, and after that I only found in
> the log (nothing has been logged earlier):
>
> Sep 28 15:00:07 mail slapd[2209]: connection_input: conn=14847 deferring
> operation: too many executing
> Sep 28 15:00:38 mail slapd[2209]: connection_input: conn=19285 deferring
> operation: too many executing
> Sep 28 15:32:46 mail slapd[2209]: connection_input: conn=19419 deferring
> operation: binding
> Sep 28 15:32:47 mail slapd[2209]: connection_input: conn=19419 deferring
> operation: binding
> Sep 28 15:32:57 mail slapd[4484]: [INFO] Using /etc/default/slapd for
> configuration
> Sep 28 15:32:57 mail slapd[4489]: [INFO] Halting OpenLDAP...
> Sep 28 15:32:57 mail slapd[2209]: daemon: shutdown requested and initiated.
> Sep 28 15:32:57 mail slapd[2209]: slapd shutdown: waiting for 1
> operations/tasks to finish
> Sep 28 15:33:03 mail slapd[2209]: slapd stopped.
> Sep 28 15:33:05 mail slapd[4510]: [OK] OpenLDAP stopped after 7 seconds
> Sep 28 15:33:05 mail slapd[4511]: [INFO] No data backup done
> Sep 28 15:33:12 mail slapd[4529]: [INFO] Using /etc/default/slapd for
> configuration
> Sep 28 15:33:12 mail slapd[4534]: [INFO] Launching OpenLDAP configuration
> test...
> Sep 28 15:33:16 mail slapd[4568]: [OK] OpenLDAP configuration test
> successful
> Sep 28 15:33:16 mail slapd[4578]: [INFO] No db_recover done
> Sep 28 15:33:16 mail slapd[4579]: [INFO] Launching OpenLDAP...
> Sep 28 15:33:16 mail slapd[4580]: [OK] File descriptor limit set to 1024
> Sep 28 15:33:17 mail slapd[4581]: @(#) $OpenLDAP: slapd 2.4.31 (Apr 26 2012
> 19:53:11) $
> clement@localhost.localdomain:/home/clement/build/BUILD/openldap-2.4.31/servers/slapd
> ...
>
> Possibly, a number of parallel group alias uses, caused a large number of
> LDAP queries by Postfix. Can you please advise on what may have caused
> OpenLDAP overloading, and on how can we avoid it from happening again? Any
> config changes?
>
> My config follows.
>
> Thanks in advance for your time and assistance.
>
> Regards,
> Nick
>
> # cat /usr/local/openldap/var/openldap-data/DB_CONFIG
> #====================================================================
> # BDB configuration
> #
> # Provided by LTB-project (http://www.ltb-project.org)
> #====================================================================
>
> #====================================================================
> # Cache size for DB files
> #====================================================================
> set_cachesize           1       0       1
>
> #====================================================================
> # Flags
> #====================================================================
> #set_flags              DB_TXN_WRITE_NOSYNC
> #set_flags              DB_TXN_NOSYNC
> set_flags               DB_LOG_AUTOREMOVE
>
> #====================================================================
> # Logs
> #====================================================================
> # Size
> set_lg_regionmax        1048576
> set_lg_max              10485760
> set_lg_bsize            2097152
>
> # Directory
> set_lg_dir      /usr/local/berkeleydb/openldap-logs
>
> ************************************************************************
>
> # cat /usr/local/openldap/etc/openldap/slapd.conf
> #
> include /usr/local/openldap/etc/openldap/schema/core.schema
> include /usr/local/openldap/etc/openldap/schema/cosine.schema
> include /usr/local/openldap/etc/openldap/schema/inetorgperson.schema
> include         /usr/local/openldap/etc/openldap/schema/nis.schema
> include /usr/local/openldap/etc/openldap/schema/eduperson.schema
> include /usr/local/openldap/etc/openldap/schema/postfix.schema
> include /usr/local/openldap/etc/openldap/schema/dyngroup.schema
> include /usr/local/openldap/etc/openldap/schema/misc.schema
> include /usr/local/openldap/etc/openldap/schema/ppolicy.schema
> include /usr/local/openldap/etc/openldap/schema/schac-20090326-1.4.0.schema
> include /usr/local/openldap/etc/openldap/schema/dnsdomain2.schema
> include /usr/local/openldap/etc/openldap/schema/proftpd-quota.schema
> include /usr/local/openldap/etc/openldap/schema/kerberos.schema
> include /usr/local/openldap/etc/openldap/schema/localemail.schema
> include /usr/local/openldap/etc/openldap/schema/entryaccess.schema
>
> pidfile         /usr/local/openldap/var/run/slapd.pid
> argsfile        /usr/local/openldap/var/run/slapd.args
>
> modulepath      /usr/local/openldap/lib64
>
> loglevel sync
>
> sizelimit unlimited
> timelimit unlimited
>
> TLSCipherSuite HIGH:MEDIUM:+SSLv2
>
> TLSCACertificateFile /usr/local/openldap/etc/openldap/cacerts/chain.pem
> TLSCertificateFile /usr/local/openldap/etc/openldap/cacerts/cert.pem
> TLSCertificateKeyFile /usr/local/openldap/etc/openldap/cacerts/key.pem
>
> TLSVerifyClient never
>
> #######################################################################
> # ldbm and/or bdb database definitions
> #######################################################################
>
> database        hdb
> suffix          "dc=example,dc=com"
> rootdn          "cn=Manager,dc=example,dc=com"
> rootpw          secret
>
> ########
> # ACLs #
> ########
> include      /usr/local/openldap/etc/openldap/acl.conf
>
> directory    /usr/local/openldap/var/openldap-data
>
> index   objectClass              eq,pres
> index   employeeType             pres,eq
> index   cn                       eq,pres,sub
> index   sn,givenname             eq,pres,sub
> index   mail                     eq,pres,sub
> index   uid                      eq,pres
> index   ou                       eq,pres
> index   mailacceptinggeneralid   eq,pres
> index   owner                    eq
> index   entryCSN,entryUUID       eq
> index   vacationActive           eq
> index   associatedDomain         pres,eq,sub
> index   dc                       eq
> index   emailLocalAddress        eq,pres,sub
>
> overlay dynlist
> dynlist-attrset nisMailAlias labeledURI
> dynlist-attrset groupOfURLs labeledURI member
>
> syncrepl rid=111
>         provider=ldaps://ldap.example.com
>         tls_reqcert=never
>         type=refreshAndPersist
>         retry="60 15 180 +"
>         searchbase="dc=example,dc=com"
>         schemachecking=off
>         bindmethod=simple
>         binddn="uid=FullReplAcc1,ou=System,dc=example,dc=com"
>         credentials="mypassword"
>
> database monitor
>
> access to *
>    by dn.exact="cn=Manager,dc=example,dc=com" read
>    by * none
>
> *********************************************************************
>
> # ls -la /usr/local/openldap/var/openldap-data/
> total 14120
> drwxr-xr-x 2 ldap ldap     4096 Sep 28 15:33 .
> drwxr-xr-x 4 ldap ldap     4096 Apr 26 20:56 ..
> -rw-r--r-- 1 ldap ldap     4096 Sep 28 15:33 alock
> -rw------- 1 ldap ldap  1261568 Sep 28 15:32 associatedDomain.bdb
> -rw------- 1 ldap ldap   512000 Sep 28 15:32 cn.bdb
> -rw------- 1 ldap ldap    24576 Sep 28 15:33 __db.001
> -rw------- 1 ldap ldap  1294336 Sep 28 16:12 __db.002
> -rw------- 1 ldap ldap 32776192 Sep 28 16:12 __db.003
> -rw------- 1 ldap ldap  3145728 Sep 28 16:11 __db.004
> -rw------- 1 ldap ldap   729088 Sep 28 16:12 __db.005
> -rw------- 1 ldap ldap    32768 Sep 28 16:11 __db.006
> -rw-r--r-- 1 ldap ldap      924 Apr 26 21:01 DB_CONFIG
> -rw------- 1 ldap ldap      845 Apr 26 20:56 DB_CONFIG.example
> -rw------- 1 ldap ldap    61440 Sep 28 15:32 dc.bdb
> -rw------- 1 ldap ldap   339968 Sep 28 15:33 dn2id.bdb
> -rw------- 1 ldap ldap   212992 Sep 28 15:33 emailLocalAddress.bdb
> -rw------- 1 ldap ldap    20480 Sep 28 15:33 employeeType.bdb
> -rw------- 1 ldap ldap   118784 Sep 28 15:33 entryCSN.bdb
> -rw------- 1 ldap ldap    81920 Sep 28 15:33 entryUUID.bdb
> -rw------- 1 ldap ldap    90112 Sep 28 15:32 givenName.bdb
> -rw------- 1 ldap ldap  2457600 Sep 28 15:33 id2entry.bdb
> -rw------- 1 ldap ldap    24576 Jul  9 13:13 mailacceptinggeneralid.bdb
> -rw------- 1 ldap ldap   212992 Sep 28 15:33 mail.bdb
> -rw------- 1 ldap ldap   266240 Sep 28 15:33 objectClass.bdb
> -rw------- 1 ldap ldap    40960 Sep 28 15:33 ou.bdb
> -rw------- 1 ldap ldap     8192 Sep 28 15:32 owner.bdb
> -rw------- 1 ldap ldap   253952 Sep 28 15:32 sn.bdb
> -rw------- 1 ldap ldap    28672 Sep 28 15:33 uid.bdb
> -rw------- 1 ldap ldap     8192 Sep 25  2011 vacationActive.bdb
>
> ***************************************************************************
>

Hi,

try to set sortvals parameter like this:

sortvals uniqueMember

See http://www.openldap.org/lists/openldap-technical/200808/msg00033.html


Clément.