Rationale
Message filtering options in syslogd are quite limited:
- facility/level
- host name
- process name While facility and level are pretty straightforward, filtering by host or process names could be quite CPU intensive and/or even unusable: names are evaluated as full string match strictly.
Couple of years ago we've hit a host/process name string length shortage and chose to fix it with a kind of a dirty hack just growing buffer bigger (see rS275729).
Time comes by and now we're limited even by LINE_MAX-sized full string match filter: thanks to rS334719 we're now able to run tons of OpenVPN processes on single socket yet still need to skip logging into all.log from them.
At first I was considering to add shell-like pattern matching globs and/or regular expressions but after careful evaluation I came to the idea that these new matching schemes are not going to fit into constraints of present syntax.
rsyslog-like property-based syntax
Luckily enough, a syslogd alternative, rsyslog, has a good way to express things needed in a configuration file already, Property-based filters (see https://www.rsyslog.com/doc/v8-stable/configuration/filters.html#property-based-filters).
This proposal uses similar syntax, but implements for this moment just a tiny subset of properties that can be judged for filtering purposes.
Property-based filters
(This is a copy&paste from man page)
program, hostname specifications performs exact match filtering against
explicit field only. Property-based filters feature substring and
regular expressions (see re_format(7)) matching against various message
attributes. Filter specification starts with ‘#:’ or ‘:’ followed by
three comma-separated fields property, operator, "value". Value must be
double-quoted. A double quote and backslash must be escaped by a
blackslash.
Following properties are supported as test value:
• ‘msg’ - body of the message received.
• ‘programname’ - program name sent the message
• ‘hostname’ - hostname of message's originator
• ‘source’ - an alias for hostname
Operator specifies a comparison function between propertie's
value against filter's value. Possible operators:
• ‘contains’ - true if filter value is found as a substring of property
• ‘isequal’ - true if filter value is equal to property
• ‘startswith’ - true if property starts with filter value
• ‘regex’ - true if property matches basic regular expression defined
in filter value
• ‘ereregex’ - true if property matches extended regular expression
defined in filter value
Operator may be prefixed by
• ‘!’ - to invert compare logic
• ‘icase_’ - to make comparison function case insensitiveSome usage examples
# Log ipfw messages without syncing after every message.
!ipfw
*.* -/var/log/ipfw
# Log ipfw messages with "Deny" in the message body.
:msg, contains, ".*Deny.*"
*.* /var/log/ipfw.deny
# Reset program name filtering
!*
# Log messages from bird or bird6 into one file
:processname, regex, "^bird6?$"
*.* /var/log/bird-all.log
# Log messages from servers in racks 10-19 in multiple locations, case insensitive
:hostname, icase_ereregex, "^server-(dcA|podB|cdn)-rack1[0-9]{2}\..*"
*.* /var/log/racks10..19.log