Let syslog(3) use RFC 5424.

[ed]

Summary

With D14926 changing syslogd(8) to parse RFC 5424 formatted syslog
messages, go ahead and also change the syslog(3) libc function to
generate them. Compared to RFC 3164, RFC 5424 has various advantages,
such as sub-second precision for log entry timestamps.

[kib]

Can you provide the examples of old and new default logged lines ?

What is about compatibility ? How the things which expect old format, are expected to cope with the change ? Why a benefit from the change is higher than the broken parsing code used on too many installations ?

lib/libc/gen/syslog.c

138–139

Move 'cnt' declaration into the line with other ints.

180

Style: no local declarations after the first statement.

201

fputs(3), even despite compiler can optimize it.

[ed]

Hi Kostik,

In D14951#314406, @kib wrote:

Can you provide the examples of old and new default logged lines ?

Sure! Consider the following call to syslog(3):

syslog(LOG_CRIT, "This is an example message");

This will normally cause the application to write the following over the UNIX socket:

<10>Apr  3 22:31:11 my-program[756]: This is an example message

After applying this change, it will instead write the following:

<10>1 2018-04-03T22:31:11.290614+02:00 host.domain.com my-program 756 - - This is an example message

What is about compatibility ?

The idea is that this change will only be committed after D14926 lands. This means that if /lib/libc.so.7 (and statically linked executables) are updated, /usr/sbin/syslogd must be updated as well. I think that this requirement is quite reasonable.

Do note that this change only affects the protocol between applications and syslogd(8). It does *not* alter the format of log files written into /var/log. When D14926 lands, syslogd(8) will be capable of parsing both RFC 3164 and RFC 5424 messages, but it will *always* write them to disk in RFC 3164 format. Changing the output format of syslogd(8) will need to be discussed separately.

How the things which expect old format, are expected to cope with the change ? Why a benefit from the change is higher than the broken parsing code used on too many installations ?

The number of consumers of data generated by syslog(3) is small. Most installations of FreeBSD will use syslogd(8) for logging. There are some alternatives, such as syslog-ng and rsyslog. Documentation indicates that these also support RFC 5424 properly. I will, however, check this to be sure.

lib/libc/gen/syslog.c

201

There's a space at the end of the format: "%s "; not "%s". Using fputs() would remove that.

[ed]

Process feedback provided by kib@.

[ed]

In D14951#314438, @ed wrote:

There are some alternatives, such as syslog-ng and rsyslog. Documentation indicates that these also support RFC 5424 properly. I will, however, check this to be sure.

Just tried using these two. They support RFC 5424 perfectly fine. syslog-ng can be configured to parse RFC 5424 messages by configuring the source as follows:

unix-dgram("/var/run/log" flags(syslog-protocol));

In the case of rsyslog things become a bit more clean. Apparently they needed a special parser for the quirky formats generated by different operating systems. This one can be disabled now:

module(load="imuxsock" SysSock.UseSpecialParser="off")

Let me bump __FreeBSD_version and add a notice to UPDATING.

[ed]

Add a note to UPDATING regarding this change.

[kib]

UPDATING

60

I think, if you put this updating note, some background explanation would be more useful. I mean, you need to note that this 'on wire' format is really only on wire for local unix socket between libc and syslogd, and affects only replacements of syslogd. Also you should note, that although not supported, we esp. caution against running old (specify rev) syslogd against new libc.

Then you can explain configuration changes for alternative syslogds etc.

[ed]

Extend the UPDATING note.

[ed]

Thanks for your input, Kostik. What do you think about this version?

[kib]

I would still prefer that the updating entry explicitly mentioned that change only affects local comms between libc and syslog. But the entry is quite wordy already.

[ed]

Make the scope of what is affected a bit more explicit.

[ed]

Oops! Accidentally used the wrong differential revision in the commit message. This was committed as r332100.