Following the introduction of the <compress> configuration in
newsyslog.conf, the -c command line is now redundant. Maintaining
this setting in multiple places could lead to confusion for
administrators, therefore, simplify the configuration management
by centralizing compression settings within the newsyslog.conf
file.
Details
Details
Diff Detail
Diff Detail
- Repository
- rG FreeBSD src repository
- Lint
Lint Not Applicable - Unit
Tests Not Applicable
Event Timeline
Comment Actions
Hi,
First, two high-level remarks:
- About the purpose of this change: I've probably not evaluated enough if removing -c is a good idea, so I'm neutral on this topic. Having command-line switches for what can also be specified in a config file generally makes sense to me (with the command-line switch overriding the configuration file). On the other hand, if the program is not to be launched by hand frequently, there is no such necessity. I agree it's best that administrators tweak a common configuration file rather than specifying command-line switches in rc.conf, with the goal to apply the same environment to manual invocations. That said, I'm not sure it's a reason in itself to remove command-line switches.
- About <compress> : I would really like this directive to be renamed to <compress_override>, since it doesn't affect files that are not already flagged with a compression letter, so the current name is misleading to people that would want to enable compression of all files (which this directive can't do). This would be best done in a separate revision.
The first remark above aside, I'm completely fine with the code changes.
Comment Actions
edit after reviewing the OP.
I think you're saying why have a command-line switch when we have the in-conf-file option? Tradition. Flexibility. Just specify it when newsyslog is launched. I'm sure there are use-cases where newsylog is not launched from cron, but on-demand (for example). The command-line switch allows the user to accomplish the same thing without modifying their .conf files.
- About <compress> : I would really like this directive to be renamed to <compress_override>, since it doesn't affect files that are not already flagged with a compression letter, so the current name is misleading to people that would want to enable compression of all files (which this directive can't do). This would be best done in a separate revision.
I like that change.
Comment Actions
I was just trying to guess the reasoning for the removal of this command-line option, by stating one possible inference, since I anticipate that the use of -c is infrequent and it's always possible for the administrator to change the configuration file (well, I also can see situations where these are not equivalent...). Personally, I don't have a need to remove it, and at the same time I'll likely never use it, in that sense I'm neutral.
Tradition. Flexibility. Just specify it when newsyslog is launched. I'm sure there are use-cases where newsylog is not launched from cron, but on-demand (for example). The command-line switch allows the user to accomplish the same thing without modifying their .conf files.
Yes, I generally agree, that's why I wouldn't have proposed to remove -c. Now, I'm wondering why other people think it should be removed.
IIRC, @karels objected to having it because it caused confusion, wondering which one would apply if both -c and <compress> were used. Personally, I don't think it's the case, as so many utilities already have such kind of overlap, with the command-line switch always overriding the configuration file (this can be considered a requirement by POLA).
I think that someone (probably Mike also) says that specifying -c for some manual runs of newsyslog would cause several rotated files not to be compressed with the same method as that in the configuration file, causing a mess. It's not really a problem in my eye, since I've argued that it's easy to read/grep these files uniformly (e.g., with zgrep), and I generally presume the administrator knows what he's doing (unless there's a big risk of foot shooting, but here the administrator can just re-compress existing files after the fact).
It would be better that people really wanting to keep or remove it speak for themselves (probably here, since this is a public forum).
Comment Actions
I objected to having a knob (-c) to override a knob (<compress>) to override a knob (4 different compression indicators). That seems like a bad design to me. That the precedence wasn't specified was an annoyance, but not the primary objection.
Tradition. Flexibility. Just specify it when newsyslog is launched. I'm sure there are use-cases where newsylog is not launched from cron, but on-demand (for example). The command-line switch allows the user to accomplish the same thing without modifying their .conf files.
Yes, I generally agree, that's why I wouldn't have proposed to remove -c. Now, I'm wondering why other people think it should be removed.
See below.
IIRC, @karels objected to having it because it caused confusion, wondering which one would apply if both -c and <compress> were used. Personally, I don't think it's the case, as so many utilities already have such kind of overlap, with the command-line switch always overriding the configuration file (this can be considered a requirement by POLA).
I think that someone (probably Mike also) says that specifying -c for some manual runs of newsyslog would cause several rotated files not to be compressed with the same method as that in the configuration file, causing a mess. It's not really a problem in my eye, since I've argued that it's easy to read/grep these files uniformly (e.g., with zgrep), and I generally presume the administrator knows what he's doing (unless there's a big risk of foot shooting, but here the administrator can just re-compress existing files after the fact).
It would be better that people really wanting to keep or remove it speak for themselves (probably here, since this is a public forum).
Using -c to specify different compression makes a mess because you end up with a mix of different file types/extensions, and the ones created by -c won't get rotated when the cron job runs. They will be numbered separately, so it's hard to figure out the ordering. It doesn't matter that one utility will read from any of them. Also, if one sysadmin adds -c in the cron file, and another tries to figure out what is going on from the config file, or tries to change the config file, they won't get the desired results. I see no good reason to specify a different compression method for individual invocations of newsyslog. Instead, if changing, I would expect the sysadmin to modify the config files and manually rename and/or recompress the existing compressed files.
Unrelated: I think the changes to document <compress> should be in a separate commit. If you do change the name, it might as well be documented under the new name.
Comment Actions
I totally forgot about this issue. Suppressing -c has just been committed. For the record and perhaps the future, please find some reactions and thoughts.
Unfortunately, "bad design" here was thrown without explanations. Of course, I don't disagree that we want to avoid piling up knobs/tunables, but don't see anything wrong in this precise case (compression indicators are historical and allow to fine-tune the compression for whatever reason necessary, <compress> is a global knob to override them, -c is a command-line override which has final say, as in so many other utilities).
Tradition. Flexibility. Just specify it when newsyslog is launched. I'm sure there are use-cases where newsylog is not launched from cron, but on-demand (for example). The command-line switch allows the user to accomplish the same thing without modifying their .conf files.
Yes, I generally agree, that's why I wouldn't have proposed to remove -c. Now, I'm wondering why other people think it should be removed.
Using -c to specify different compression makes a mess because you end up with a mix of different file types/extensions, and the ones created by -c won't get rotated when the cron job runs. They will be numbered separately, so it's hard to figure out the ordering.
That is a receivable objection to -c. However, it loses ground if we consider just stopping suffixing log files with their compression method (at the expense of some backwards compatibility). For how long have we been having file(1) at our disposal? And less(1) with LESSOPEN and lesspipe.sh? And tar(1) with automatic compression detection? Having a compression suffix really is a relic of the old past. Why not having an option in the configuration file specifying the general rotated files pattern? Aren't administrator grown-up enough to know what they are doing, e.g., not mixing compression suffixes with changes in compression method, especially since there is no real possibility of serious foot-shooting here?
It doesn't matter that one utility will read from any of them.
That matters in practice, even if not the main driving force in the design.
Also, if one sysadmin adds -c in the cron file, and another tries to figure out what is going on from the config file, or tries to change the config file, they won't get the desired results.
As a sysadmin, I for sure will look at the command-line in cron first thing, and then the configuration file. I can't imagine a serious sysadmin not doing the same, else not being able to figure out quickly why the configuration file is overridden.
I see no good reason to specify a different compression method for individual invocations of newsyslog. Instead, if changing, I would expect the sysadmin to modify the config files and manually rename and/or recompress the existing compressed files.
That's the only really considerable objection to -c. Is -c is worth it: Frequency of use cases, complexity, ability to perform the same manually outside of newsyslog(8)? This is a "fundamental" question to ask oneself even before introducing it. From the outside, I tend to think it wasn't really worth it as is on the ground that the uses should be very rare, and changing the compression method after the fact can be served by manually compressing/recompressing existing files easily. It is not in stable/14, so removing it now (before stable/15) will not cause backwards compatibility issues.
Takeaway:
- Suppressing -c seems fine.
- The commit message unfortunately cites the cron(8) issue, which is particularly uncompelling, instead of answering the "fundamental" questions on the worthiness of -c (see above).
- Sometimes, new options/features are introduced or removed without sufficient discussion. People with stakes should speak up and present their case clearly, and be prepared for possible rebuttal. [I don't feel this was done sufficiently in this case (both when adding -c and when removing it), regardless of the fact I'm personally fine with the conclusion and would not have been against keeping -c either.]
Comment Actions
Other discussions:
- https://reviews.freebsd.org/D42961 - the one that started all this, if I recall correctly. I feel the burden of proof to explain myself. Others, please chime in too. That original review has other viewpoints.
- https://reviews.freebsd.org/D43165
- https://reviews.freebsd.org/D43174
One of the primary goals was to avoid the double compression of newsyslog compressing files on a ZFS filesystem which had compression on by default. That was in 2023. That was a common scenario then, and I'm sure the popularity has not decreased.
I could have changed my newsyslog.conf files on multiple hosts. Then with each update, hope I don't miss any merge conflicts. No, it's much easier to be able to specify "don't compress" in one place.
Another goal: For the non-trivial number of users with ZFS compression on: using the same tool when working with rotated and non-rotated log files (e.g. grep) - as it has been pointed out, bzgrep will do this just fine. If this was the only reason, I'd have declined and withdrawn my proposal. However, asking the tool to decompress a file which is already decompressed by the filesystem is a duplication of effect. Better to just let the filesystem do this compression transparently.
Most of all, the change has no affect on any host, unless you use the new feature. Don't like it / don't want it, don't use it. However, please don't strive to deprive the reset of us.
None of these goals alone is enough to convince the most diehard opposition that this change is a good idea. Combined however, it's a good thing. I'm looking forward to the change.
Comment Actions
For those examining the history of newsyslog(8) and the removal of the -c command-line option, I feel it's important to understand the underlying design philosophy that motivated this and related changes. The goal was to establish a more robust, flexible, and administrator-friendly configuration model.
The core principle is that for a given system, there is generally only one optimal compression method for all text-based log files. While different algorithms offer various trade-offs, a system administrator typically chooses a single standard (e.g., zstd for speed, xz for size) to apply universally, or do not compress at all. The configuration system should empower this choice, not complicate it.
Historically, as new compression tools became available in FreeBSD, newsyslog.conf grew flags like J (bzip2), X (xz), and Y (zstd). This approach, while functional, introduced a design flaw: it conflated the intent with the mechanism. A configuration line for a log file was essentially saying, "compress this file using bzip2," rather than the more abstract and durable statement, "this file is considered compressible."
A more extensible and flexible design separates these two concepts. A log file's configuration should only need to declare that it is compressible. A separate, global setting should then define how all compressible files are to be processed. This approach provides several key advantages:
- Centralized Control: The system administrator can change the compression strategy for the entire system by modifying a single line in /etc/newsyslog.conf. If a new, superior compression algorithm emerges, the migration path is trivial, and it would be much easier if one decides to add support for e.g. different compression levels, etc.
- Simplified Package Management: This model reduces the burden on both software maintainers and system administrators. When a third-party package installs a configuration file for its logs, instead of demanding a specific compression method to be used, it mark the log as compressible instead. It doesn't need to bake in a specific compression method that might conflict with the local administrator's policy.
- Reduced Configuration Drift and Conflicts: The historical method created a scenario where packages could dictate compression methods that the administrator did not want. This forced administrators to manually edit package-provided config files. During system updates (e.g. etcupdate(8)), these local modifications would frequently lead to merge conflicts if the upstream package developer decided to change their preferred compression flag. By separating the roles, the administrator's global policy always takes precedence, and package configuration files are less likely to require local changes, resulting in smoother updates.
Ultimately, removing the command-line override and consolidating the compression strategy into a single global configuration option reflects a deliberate move towards a more coherent and manageable system. It places control firmly in the hands of the system administrator and simplifies the long-term maintenance of the system's configuration.