Page MenuHomeFreeBSD

Do not compress log files
AbandonedPublic

Authored by dvl on Dec 7 2023, 3:21 PM.
Tags
None
Referenced Files
Unknown Object (File)
Fri, Apr 26, 2:34 AM
Unknown Object (File)
Apr 1 2024, 1:14 PM
Unknown Object (File)
Feb 29 2024, 10:59 AM
Unknown Object (File)
Feb 4 2024, 4:11 PM
Unknown Object (File)
Dec 26 2023, 10:01 PM

Details

Reviewers
None
Summary

With ZFS now widely used and the default install doing compression, we no
longer have to compress log files to save space.

This is a quality-of-life fix, mainly. to make it easier to search all logs
with the same command. e.g. you don't have to use bzgrep.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 54848
Build 51737: arc lint + arc unit

Event Timeline

dvl requested review of this revision.Dec 7 2023, 3:21 PM

I think it'd be a good idea to add these as an include - that way, it's easier to enable or disable on a case-by-case basis.

I think it'd be a good idea to add these as an include - that way, it's easier to enable or disable on a case-by-case basis.

What do you mean by "as an include"?

With ZFS now widely used and the default install doing compression, we no longer have to compress log files to save space.

There're still people making UFS/UFS2 as the root partition. I think the above conclusion is aggressive.
CC @delphij

This is a quality-of-life fix, mainly. to make it easier to search all logs with the same command. e.g. you don't have to use bzgrep.

For searching logs, agreed.

In D42961#982172, @zlei wrote:

With ZFS now widely used and the default install doing compression, we no longer have to compress log files to save space.

There're still people making UFS/UFS2 as the root partition. I think the above conclusion is aggressive.
CC @delphij

Hmm why me 🤣 I always hated the compression setting here and have all my systems, UFS or ZFS stopped doing compression around 2006, but I wasn't an advocate of doing it one way or another.

Personally, however, I don't think it really matter nowadays even on UFS. Usually, bzip2 achieve a compression ratio of about 11:1 (xz would compress better at expense of slightly slower speed; however, for files at about 1MB, the difference on very old machine is still negligible (within 0.2 seconds per file), and the benefit of faster decompression would be a pretty good argument for using xz as default), but since most of the fast growing files are capped at 1MB, let's say one have fast growing messages and maillog, that would be 14MB vs (let's be generous and assume a compression of 14:1) 1MB, and it's not going to make any difference for any decent modern hardware, even if /var/log is in RAM.

That's said, I'm all *for* disabling compression by default. But I think we should take this a little bit further than simply flipping a default. If I was you I'd probably change the meaning of all compression related flags (J, X, Y, Z) to mean "This file is eligible for compression", and add a new command line option, e.g. --compression= to indicate the compression method to be used, e.g. --compression=legacy means to maintain existing behavior, --compression=none means don't compress at all, --compression=gzip|xz|bzip2|zstd means to use the current Z, X, J, Y behavior, while --compression="gzip -f -9" or --compression=<string> means a custom compression method should be used, and default to --compression=none).

This is a quality-of-life fix, mainly. to make it easier to search all logs with the same command. e.g. you don't have to use bzgrep.

For searching logs, agreed.

In D42961#979432, @0mp wrote:

I think it'd be a good idea to add these as an include - that way, it's easier to enable or disable on a case-by-case basis.

What do you mean by "as an include"?

There are already includes in the file for newsyslog.conf.d/ and the local variant of it.
If I correctly understand how newsyslog works, setting the different values in the include file would override them when the configuration is read by newsyslog?

The solution implemented in D43165 is elegant and is off by default. I prefer that.