2.6.0 added the new feature allowing using a logfile for all debug/loglevel messages and bypassing syslog entirely. However, there is no documentation on the new settings or examples of how to do this in the admin guide, and the man page sections on the new parameters for the logfile side do not note at when/how they enable bypassing syslog.
> and the man page sections on the new parameters for the logfile side do not note at when/how they enable bypassing syslog. Here's what's currently listed on manpage. Not sure what the ask is here? What's missing? ``` olcLogFile: <filename> Specify a file for recording slapd debug messages. By default these messages only go to stderr, are not recorded anywhere else, and are unrelated to messages exposed by the olcLogLevel configuration pa‐ rameter. Specifying a logfile copies messages to both stderr and the logfile. olcLogFileFormat: debug | syslog-utc | syslog-localtime Specify the prefix format for messages written to the logfile. The debug format is the normal format used for slapd debug messages, with a timestamp in hexadecimal, followed by a thread ID. The other op‐ tions are to use syslog(3) style prefixes, with timestamps either in UTC or in the local timezone. The default is debug format. olcLogFileOnly: TRUE | FALSE Specify that debug messages should only go to the configured logfile, and not to stderr. ```
>However, there is no documentation on the new settings or examples of how to do this in the admin guide Will add the param descriptions and overview of new logger here: https://www.openldap.org/doc/admin26/tuning.html#Logging
(In reply to Shawn McKinney from comment #1) > > and the man page sections on the new parameters for the logfile side do not note at when/how they enable bypassing syslog. > > Here's what's currently listed on manpage. Not sure what the ask is here? > What's missing? > > ``` > olcLogFile: <filename> > Specify a file for recording slapd debug messages. By > default these messages only go to stderr, are not recorded anywhere else, > and are unrelated to messages exposed by the olcLogLevel configuration pa‐ > rameter. Specifying a logfile copies messages to both stderr > and the logfile. > > olcLogFileFormat: debug | syslog-utc | syslog-localtime > Specify the prefix format for messages written to the logfile. > The debug format is the normal format used for slapd debug messages, with a > timestamp in hexadecimal, followed by a thread ID. The other op‐ > tions are to use syslog(3) style prefixes, with timestamps > either in UTC or in the local timezone. The default is debug format. > > olcLogFileOnly: TRUE | FALSE > Specify that debug messages should only go to the configured > logfile, and not to stderr. > ``` Which one actually disables syslog integration from slapd? olcLogFile specifies a logfile that gets logging messages and stderr debug messages (by default) olcLogFileFormat sets the format to use olcLogFileOnly stops snding debug messages to stderr but what actually disables messages from going to syslog? The last time I set this up, I ended up with messages going to both syslog and the logfile.
None of them do. Use -s0 or olcLoglevel 0 to turn off syslog, same as always.
... never mind. setting olcLogfileOnly will turn off syslog.
(In reply to Howard Chu from comment #5) > ... never mind. setting olcLogfileOnly will turn off syslog. Confirms my observation. I'll explicitly point it out in the docs.
(In reply to Shawn McKinney from comment #6) > (In reply to Howard Chu from comment #5) > > ... never mind. setting olcLogfileOnly will turn off syslog. > > Confirms my observation. I'll explicitly point it out in the docs. To be clear: the olcLogFile description states explicitly that it only affects debug messages. But when olcLogFileOnly is set, then all messages that would be generated according to the current olcLogLevel will go to the logfile (instead of syslog). IMO all of this is a stupid hack, but it was done at a customer's request. Even though they could have just used -s0 and been done with it. Now the code is more complex and the behavior description must be expanded as well. Idiocy that we'll be stuck with forever after.
>logfile <filename> >Specify a file for recording slapd debug messages. By default these messages >only go to stderr, are not recorded anywhere else, and are unrelated to >messages exposed by the loglevel configuration parameter. Specifying a logfile >copies messages to both stderr and the logfile. Confused by phrasing "and are unrelated to messages exposed by the loglevel configuration parameter". How are they unrelated to the loglevel setting? IOW - Doesn't loglevel always control the granularity of the messages, regardless of whether they are bound for stderr or the logfile? And yes, the 'logFileOnly' flag controls that behavior which is clearly pointed out in the man page and despite what was said on earlier comment.
(In reply to Shawn McKinney from comment #8) > >logfile <filename> > >Specify a file for recording slapd debug messages. By default these messages >only go to stderr, are not recorded anywhere else, and are unrelated to >messages exposed by the loglevel configuration parameter. Specifying a logfile >copies messages to both stderr and the logfile. > > Confused by phrasing "and are unrelated to messages exposed by the loglevel > configuration parameter". How are they unrelated to the loglevel setting? It's talking about slapd debug messages, which are separate from loglevel settings: "Specify a file for recording slapd debug messages." > IOW - Doesn't loglevel always control the granularity of the messages, > regardless of whether they are bound for stderr or the logfile? No, only loglevel messages, not debug messages. > And yes, the 'logFileOnly' flag controls that behavior which is clearly > pointed out in the man page and despite what was said on earlier comment. No, that's not clear at all. What it says is: "logfile-only on | off Specify that debug messages should only go to the configured logfile, and not to stderr." Which is talking only about debug messages and says nothing about loglevel messages which by default get sent to syslog.
(In reply to Quanah Gibson-Mount from comment #9) > (In reply to Shawn McKinney from comment #8) > > >logfile <filename> > > >Specify a file for recording slapd debug messages. By default these messages >only go to stderr, are not recorded anywhere else, and are unrelated to >messages exposed by the loglevel configuration parameter. Specifying a logfile >copies messages to both stderr and the logfile. > > > > Confused by phrasing "and are unrelated to messages exposed by the loglevel > > configuration parameter". How are they unrelated to the loglevel setting? > > It's talking about slapd debug messages, which are separate from loglevel > settings: > > "Specify a file for recording slapd debug messages." > > > IOW - Doesn't loglevel always control the granularity of the messages, > > regardless of whether they are bound for stderr or the logfile? > > No, only loglevel messages, not debug messages. I must be way off the mark because my assumption, 'debug', we're talking about what gets routed to the (new) logfile. stderr messages are routed to syslog. In light of that, possibly faulty assertion, the loglevel setting controls the level of granularity sent to the debug log (in addition to syslog), per my observations. e.g. set loglevel 0, and no messages will be sent. loglevel -1 sends all. What do I have wrong here? > > > And yes, the 'logFileOnly' flag controls that behavior which is clearly > > pointed out in the man page and despite what was said on earlier comment. > > No, that's not clear at all. What it says is: > > "logfile-only on | off > Specify that debug messages should only go to the > configured > logfile, and not to stderr." > > Which is talking only about debug messages and says nothing about loglevel > messages which by default get sent to syslog. Again, stderr is what gets routed to syslog -- ? So, if logfile-only on then it's debug only, and no messages sent to syslog (stderr), regardless of what the loglevel is set to. Obviously, I don't have a firm grasp of openldap log fundamentals. if I'm confused, others must be. So, this is opportunity to make it explicit.
(In reply to Shawn McKinney from comment #10) > Again, stderr is what gets routed to syslog -- ? It depends. Default logging configuration on recent Linux distros can be chain of stderr of process started by systemd (service unit) -> journald -> syslog Yes, that's really confusing and I always have to explain this stuff in detail to my customers. But these OS-specific aspects are IMHO somewhat beyond the scope of OpenLDAP docs.
(In reply to Shawn McKinney from comment #10) > (In reply to Quanah Gibson-Mount from comment #9) > > (In reply to Shawn McKinney from comment #8) > > > >logfile <filename> > > > >Specify a file for recording slapd debug messages. By default these messages >only go to stderr, are not recorded anywhere else, and are unrelated to >messages exposed by the loglevel configuration parameter. Specifying a logfile >copies messages to both stderr and the logfile. > > > > > > Confused by phrasing "and are unrelated to messages exposed by the loglevel > > > configuration parameter". How are they unrelated to the loglevel setting? > > > > It's talking about slapd debug messages, which are separate from loglevel > > settings: > > > > "Specify a file for recording slapd debug messages." > > > > > IOW - Doesn't loglevel always control the granularity of the messages, > > > regardless of whether they are bound for stderr or the logfile? > > > > No, only loglevel messages, not debug messages. > > I must be way off the mark because my assumption, 'debug', we're talking > about what gets routed to the (new) logfile. > stderr messages are routed to syslog. > > In light of that, possibly faulty assertion, the loglevel setting controls > the level of granularity sent to the debug log (in addition to syslog), per > my observations. > > e.g. set loglevel 0, and no messages will be sent. loglevel -1 sends all. > > What do I have wrong here? That's entirely backward. debuglevel controls messages to stderr. loglevel controls messages sent to syslog. The confusing part is that logfile specifies a file to receive debug messages, in addition to stderr. And then the logfile_only directive means debug messages only go to the logfile, and turns off writing to stderr. And then, because some pinheads are too stupid to use debuglevel and still want to use loglevel, but without syslog, the logfile_only directive additionally routes loglevel messages to the logfile instead of syslog.
On 3/17/22 16:22, openldap-its@openldap.org wrote: > That's entirely backward. debuglevel controls messages to stderr. loglevel > controls messages sent to syslog. slapd -s syslog-level and loglevel in slapd.conf are the same parameter?
Clarifications / Level Set (courtesy of Quanah) OpenLDAP's C code has two different types of logging: 1. "DEBUG" messages, which confusingly, are what loglevel controls: ``` # loglevel of ACL: aci.c: Debug( LDAP_DEBUG_ACL, "aciNormalizeRight: perms:attr need to be pairs in '%s'\n", action->bv_val ); # loglevel of STATS: delete.c: Debug( LDAP_DEBUG_STATS, "%s DEL dn="%s"\n", # etc ``` 2. The actual debug messages, that only (normally) go to stderr: like ``` slapmodify.c: fprintf( stderr, "%s: could not %s entry dn="%s" " or (avoiding commandline tools) syncrepl.c: fprintf(stderr, "merge_state: %s si_syncCookie [%d] %d %s\n", syncrepl.c: fprintf(stderr, "merge_state: %s si_cookieState [%d] %d %s\n", ``` When the man page says "debug messages", it's talking about the messages printed to stderr.
(In reply to Shawn McKinney from comment #14) > Clarifications / Level Set (courtesy of Quanah) No. Wrong. Stop with the nonsense. > > OpenLDAP's C code has two different types of logging: > > 1. "DEBUG" messages, which confusingly, are what loglevel controls: > ``` > # loglevel of ACL: > aci.c: Debug( LDAP_DEBUG_ACL, "aciNormalizeRight: perms:attr need > to be pairs in '%s'\n", action->bv_val ); > > # loglevel of STATS: > delete.c: Debug( LDAP_DEBUG_STATS, "%s DEL dn="%s"\n", > > # etc > ``` The Debug() macro checks both debuglevel and loglevel and routes messages accordingly. It is the only official mechanism for either debug or log messages from slapd. > 2. The actual debug messages, that only (normally) go to stderr: > > like > ``` > slapmodify.c: fprintf( stderr, "%s: could not %s > entry dn="%s" " > or (avoiding commandline tools) Calling fprintf(stderr) is only valid for cmdline tools. Not for slapd daemon. > syncrepl.c: fprintf(stderr, "merge_state: %s si_syncCookie [%d] > %d %s\n", > syncrepl.c: fprintf(stderr, "merge_state: %s si_cookieState [%d] > %d %s\n", > ``` Instances of fprintf(stderr) in daemon code are anomalies. > When the man page says "debug messages", it's talking about the messages > printed to stderr. It is talking about messages printed to stderr because of the debuglevel setting. it is not talking about the anomalous fprintf(stderr) messages scattered thru daemon code that don't belong there.
(In reply to Howard Chu from comment #15) > (In reply to Shawn McKinney from comment #14) > > Clarifications / Level Set (courtesy of Quanah) > > No. Wrong. Stop with the nonsense. Ha, exactly, stop the madness ;-P But, seriously, this is indicative of the confusion that has already been pointed out. > > > > OpenLDAP's C code has two different types of logging: > > > > 1. "DEBUG" messages, which confusingly, are what loglevel controls: > > ``` > > # loglevel of ACL: > > aci.c: Debug( LDAP_DEBUG_ACL, "aciNormalizeRight: perms:attr need > > to be pairs in '%s'\n", action->bv_val ); > > > > # loglevel of STATS: > > delete.c: Debug( LDAP_DEBUG_STATS, "%s DEL dn="%s"\n", > > > > # etc > > ``` > > The Debug() macro checks both debuglevel and loglevel and routes messages > accordingly. It is the only official mechanism for either debug or log > messages from slapd. OK, good. Clarity has been achieved, and confirms my observations of loglevel, thank you. > > > 2. The actual debug messages, that only (normally) go to stderr: > > > > like > > ``` > > slapmodify.c: fprintf( stderr, "%s: could not %s > > entry dn="%s" " > > or (avoiding commandline tools) > > Calling fprintf(stderr) is only valid for cmdline tools. Not for slapd > daemon. Got it. > > > syncrepl.c: fprintf(stderr, "merge_state: %s si_syncCookie [%d] > > %d %s\n", > > syncrepl.c: fprintf(stderr, "merge_state: %s si_cookieState [%d] > > %d %s\n", > > ``` > Instances of fprintf(stderr) in daemon code are anomalies. and so should converted to use the macro, right? (Admittedly low priority) > > > When the man page says "debug messages", it's talking about the messages > > printed to stderr. > > It is talking about messages printed to stderr because of the debuglevel > setting. it is not talking about the anomalous fprintf(stderr) messages > scattered thru daemon code that don't belong there. So, this means we now have two mechanisms that do the same thing. To prevent (the terminology) confusion, the old mechanism, debuglevel, should be deprecated IMO. But, that's besides the point (documentation), and also low priority.
(In reply to Shawn McKinney from comment #16) > (In reply to Howard Chu from comment #15) > > > syncrepl.c: fprintf(stderr, "merge_state: %s si_syncCookie [%d] > > > %d %s\n", > > > syncrepl.c: fprintf(stderr, "merge_state: %s si_cookieState [%d] > > > %d %s\n", > > > ``` > > Instances of fprintf(stderr) in daemon code are anomalies. > > and so should converted to use the macro, right? (Admittedly low priority) > > > > > > When the man page says "debug messages", it's talking about the messages > > > printed to stderr. > > > > It is talking about messages printed to stderr because of the debuglevel > > setting. it is not talking about the anomalous fprintf(stderr) messages > > scattered thru daemon code that don't belong there. > > So, this means we now have two mechanisms that do the same thing. To > prevent (the terminology) confusion, the old mechanism, debuglevel, should > be deprecated IMO. No. The fprintf(stderr) instances that you pointed out (e.g. in syncrepl.c) are behind testing #ifdef's that aren't built by default. For the purposes of this discussion, they don't exist. There are not two mechanisms that do the same thing.
(In reply to Howard Chu from comment #17) > (In reply to Shawn McKinney from comment #16) > > (In reply to Howard Chu from comment #15) > > > It is talking about messages printed to stderr because of the debuglevel > > > setting. it is not talking about the anomalous fprintf(stderr) messages > > > scattered thru daemon code that don't belong there. > > > > So, this means we now have two mechanisms that do the same thing. To > > prevent (the terminology) confusion, the old mechanism, debuglevel, should > > be deprecated IMO. > > No. The fprintf(stderr) instances that you pointed out (e.g. in syncrepl.c) > are behind testing #ifdef's that aren't built by default. For the purposes > of this discussion, they don't exist. > > There are not two mechanisms that do the same thing. How are they different? I'm not talking about how they were implemented, obviously. We're discussing their behavior, i.e. does one replace the other from the user's perspective. loglevel ``` loglevel <integer> [...] Specify the level at which debugging statements and operation statistics should be syslogged (currently logged to the syslogd(8) LOG_LOCAL4 facility). They must be considered subsystems rather than increasingly verbose log levels. Some messages with higher priority are logged regardless of the configured loglevel as soon as any logging is configured. Log levels are additive, and available levels are: 1 (0x1 trace) trace function calls 2 (0x2 packets) debug packet handling 4 (0x4 args) heavy trace debugging (function args) 8 (0x8 conns) connection management 16 (0x10 BER) print out packets sent and received 32 (0x20 filter) search filter processing 64 (0x40 config) configuration file processing 128 (0x80 ACL) access control list processing 256 (0x100 stats) connections, LDAP operations, results (recommended) 512 (0x200 stats2) stats2 log entries sent 1024 (0x400 shell) print communication with shell backends 2048 (0x800 parse) entry parsing 16384 (0x4000 sync) LDAPSync replication 32768 (0x8000 none) only messages that get logged whatever ``` debuglevel ``` # /opt/symas/lib/slapd -d ? Installed log subsystems: Any (-1, 0xffffffff) Trace (1, 0x1) Packets (2, 0x2) Args (4, 0x4) Conns (8, 0x8) BER (16, 0x10) Filter (32, 0x20) Config (64, 0x40) ACL (128, 0x80) Stats (256, 0x100) Stats2 (512, 0x200) Shell (1024, 0x400) Parse (2048, 0x800) Sync (16384, 0x4000) None (32768, 0x8000) ``` I did find this in slapd(8), under -d ``` Remember that if you turn on packet logging, packets containing bind passwords will be output, so if you redirect the log to a logfile, that file should be read-protected. ``` I can infer this perhaps means given the same level setting for each, a different content will be output. But this isn't spelled out explicitly, FWICT. Yes, of course the two worked differently prior to 2.6, but we're not considering those differences here.
(In reply to Shawn McKinney from comment #18) > How are they different? I'm not talking about how they were implemented, > obviously. We're discussing their behavior, i.e. does one replace the other > from the user's perspective. I left out that with -d all output can be redirected to the foreground, which can't be done using loglevel. So yes, that difference remains.
(In reply to Shawn McKinney from comment #19) > (In reply to Shawn McKinney from comment #18) > > How are they different? I'm not talking about how they were implemented, > > obviously. We're discussing their behavior, i.e. does one replace the other > > from the user's perspective. > > I left out that with -d all output can be redirected to the foreground, > which can't be done using loglevel. > > So yes, that difference remains. If the logfile_only directive is not used, there is no difference to previous behavior. loglevel controls output to syslog. debug level controls output to stderr. That is how it has always worked, and how it continues to work. If you define a logfile and set logfile_only, then there's effectively no difference, and all output goes only to the logfile, and nowhere else. Is it really so hard to understand that "logfile_only" means to only send output to the logfile?
After RTFM and RTFC (run the code) Observations: 1. When 'logfile-only on' and logfile is not specified: a. no messages to syslog or stderr (already clearly documented on man page, not admin guide) b. this message *always* goes to syslog: '''Mar 22 16:52:52 p01 slapd[158606]: /opt/symas/etc/openldap/slapd.conf: line 10 (logfile-only on)''' i. Which is good for troubleshooting c. have turned off *all* logging 2. When 'logfile-only on' and logfile is specified: a. debug messages -d now go to logfile. i. documented in slapd-config.5 ```Specifying a logfile copies messages to both stderr and the logfile.``` b. both -d and loglevel control logfile granuarlity, and are additive. i. e.g. '-d packets' and 'loglevel stats' both routed to the specified logfile. 3. When logfile-only is not on and a logfile is specified: a. debug messages go to logfile and stderr, loglevel goes to syslog (documented as above) 4. loglevel settings in slapd config have precendence over command line -s, i.e. not additive (undocumented) 5. loglevel or -s 'packets' has no effect FWICT. To get packets, use '-d packets' (undocumented)
(In reply to Shawn McKinney from comment #21) > 5. loglevel or -s 'packets' has no effect FWICT. To get packets, use '-d > packets' (undocumented) Correction, per slapd-config.5 "Note that the packets, BER, and parse levels are only available as debug output on stderr, and are not sent to syslog." A bit confusing. Since they're listed here: ``` loglevel <integer> [...] 2 (0x2 packets) debug packet handling 16 (0x10 BER) print out packets sent and received 2048 (0x800 parse) entry parsing ``` Trying to figure out how to get these into the logfile using loglevel settings.
Updated 1. When logfile-only on and logfile is not specified: a. no messages route to syslog or stderr (documented in slapd-config.5, not admin guide) b. this message *always* goes to syslog: "Mar 22 16:52:52 p01 slapd[158606]: /opt/symas/etc/openldap/slapd.conf: line 10 (logfile-only on)" i. Which is good for troubleshooting c. Turns off *all* logging 2. When logfile-only on and logfile is specified: a. debug messages -d go to logfile. (documented in slapd-config.5 "Specifying a logfile copies messages to both stderr and the logfile.") b. both -d and loglevel control logfile granuarlity, and are additive. i. e.g. -d packets and loglevel stats both routed to the specified logfile. (undocumented) 3. When logfile-only is not on and a logfile is specified: a. debug messages go to logfile and stderr, and are controlled by -d param b. loglevel goes to syslog (only) and are controlled by -s or loglevel. c. a & b are documented in slapd-config.5, not admin guide. d. Settings -d and loglevel have differences. i. e.g. loglevel sync includes stats where -d sync doesn't. (undocumented) 4. loglevel settings in slapd config take precendence over command line -s, i.e. not additive (undocumented) 5. loglevel or -s 'packets BER parse' don't go to the logfile. Use '-d ...' instead (documentation not clear)
Reiterating for newer audiences: the Admin Guide is only a guide, it is not intended to be an exhaustive reference. That's what manpages are for. There is no need for every possible option to be documented in the Guide; that would just be redundant work. As long as all features are documented in manpages that's sufficient; the Guide can be an overview but at all times, admins must refer to the manpages to get full details.