Re: loglevel expected performance impact

[Meike Stone <meike.stone@googlemail.com>]

If we talk about syslog ..
SuSE (opensuse/SLES) writes local4 in /var/log/localmessages and
/var/log/messages!!
Best way here to write messages in separate file is:

#### part from syslog-ng.conf #####

filter f_ldap       { program(slapd);};

#change original lines:
filter f_local      { facility(local0, local1, local2, local3, local4,
local5, local6, local7) and not filter(f_ldap); };
filter f_messages   { not facility(news, mail) and not
filter(f_iptables) and not filter(f_ldap); };

#
# LDAP in separate file
#
destination ldaplog { file("/var/log/ldap.log"
                      owner(ldap) group(ldap)); };
log { source(src); filter(f_ldap); destination(ldaplog); };

### end ######

Additionally there is a sync option for syslg-ng. It can be set in the
options { ...} line.


BTW a few other thoughts, who may influence the speed if you turn on
logging...:
* How many ldap requests per second you get on LDAP?
   - you can check this via monitor db
      ~# ldapsearch  ... -b"cn=Search,cn=Operations,cn=Monitor" monitorCounter
      ~# ldapsearch ... -b"cn=Uptime,cn=Time,cn=Monitor" monitoredInfo
      then you can calculate the average of your req/s ...

* How is sized your system (especially CPU, RAM, disks, raid,
controller, partitions, ..)
* database size
* Do you use shm_key (takes away IO from your disk)?

We log (level 256) since a short time too, with about (max) 1200 ldap
req/sec and the system and IO is mostly idle :-)

Kindly regards Meike


2012/6/13 Maucci, Cyrille <cyrille.maucci@hp.com>:
> How does your syslog config look like ?
> Did you prefix the filename with the magic - symbol in order to ask for async I/Os on that file?
>
> E.g.
>
> *.info;mail.none;authpriv.none;cron.none                        -/var/log/messages
>
> mail.*                                                          -/var/log/maillog
>
> local4.*                                                                -/var/log/slapd.log
>
> Instead of
>
> *.info;mail.none;authpriv.none;cron.none                        /var/log/messages
>
> mail.*                                                          /var/log/maillog
>
> local4.*                                                                /var/log/slapd.log
>
> ++Cyrille
>
> Man syslog.conf
>   Regular File
>       Typically  messages  are  logged  to real files.  The file has to be specified with full
>       pathname, beginning with a slash ??/??.
>
>       You may prefix each entry with the minus ??-?? sign to omit syncing the file after every
>       logging.   Note  that  you  might  lose information if the system crashes right behind a
>       write attempt.  Nevertheless this might give you back some  performance,  especially  if
>       you run programs that use logging in a very verbose manner.
> -----Original Message-----
> From: openldap-technical-bounces@OpenLDAP.org [mailto:openldap-technical-bounces@OpenLDAP.org] On Behalf Of Berend De Schouwer
> Sent: Wednesday, June 13, 2012 12:28 PM
> To: openldap-technical@openldap.org
> Subject: loglevel expected performance impact
>
> I'm running some 2.4.23 servers, and I've encountered some slowdown on loglevel other than 0.  Even 256 (stats; recommended) impacts about a 4x slowdown on queries.  Logging is to syslog.
>
> Running ldapsearch slows from 0.005-0.010 seconds to about 0.030-0.040 seconds; and that includes loading the binary.  That's from localhost to remove potential DNS lookups.
>
> I stumbled across this when logging was wrong, and the slowdown was 100x.
>
> I'm aware that 2.4.23 isn't the latest version.  I'm also quite happy, for now, to run loglevel 0.
>
> I'm wondering if this is the expected behaviour, given that it's the recommended configuration.  Or should I go dig to find the slowdown?
>
> (I did check the indexes, and db_stats, etc.  All seems fine.)
>
> I apologise for the disclaimer,
> Berend
>
>
>
>
>
> CONFIDENTIALITY NOTICE
> The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential information of Argility (Proprietary) Limited and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intended addressee is prohibited.
> If you are not the intended addressee please notify the writer immediately and destroy the e-mail. Argility (Proprietary) Limited and its subsidiaries distance themselves from and accept no liability for unauthorised use of their e-mail facilities or e-mails sent other than strictly for business purposes.
>
>