Page MenuHomeFreeBSD

ls(1): Change -g and make -n imply -l to implement POSIX
ClosedPublic

Authored by minsoochoo0122_proton.me on Apr 2 2022, 11:25 AM.
Tags
None
Referenced Files
Unknown Object (File)
Thu, Feb 29, 6:21 PM
Unknown Object (File)
Sat, Feb 24, 11:48 PM
Unknown Object (File)
Sun, Feb 18, 8:20 AM
Unknown Object (File)
Sun, Feb 4, 1:08 PM
Unknown Object (File)
Jan 31 2024, 6:26 PM
Unknown Object (File)
Jan 29 2024, 12:03 PM
Unknown Object (File)
Jan 29 2024, 2:38 AM
Unknown Object (File)
Jan 19 2024, 9:03 PM

Details

Summary

Bugzilla bug report in 2004

According to SUSv4 and POSIX, -g, -n, and -o options are required for ls.
This has not been implemented for the compatibility for 4.3 BSD, but I think there is no need to insist it anymore.

f_flags is deleted from case 'o' because file flags are not displayed in the -o option of POSIX.

Test Plan

cd /usr/src/bin/ls
make && make install

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
minsoochoo0122_proton.me added inline comments.
bin/ls/ls.c
378–380

I fixed this by editing usage

378–380

I modified getopt_long on the 280th line.

As disgorged by grep -rH -E 'ls +-[a-zA-A0-9]*o' /usr/src, the following files in src also need changes:
/usr/src/share/skel/dot.shrc:alias ll='ls -laFo'
/usr/src/sys/contrib/zstd/tests/gzip/init.sh: perms=\ls -dgo "$d" 2>/dev/null\ &&
/usr/src/bin/ls/tests/ls_tests.sh: atf_set "descr" "Verify that the output from ls -o prints out the chflag values or '-' if none are set"
/usr/src/bin/ls/tests/ls_tests.sh: -s exit:0 ls -lo a.file
/usr/src/bin/ls/tests/ls_tests.sh: -s exit:0 ls -lo b.file
/usr/src/bin/chflags/chflags.1:You can use "ls -lo" to see the flags of existing files.

Nothing obvious in the ports tree, but that doesn't include actual source code, so I think an exp-run is in order. There's one ls -lo in the handbook (I haven't checked the rest of the doc tree). And of course, that doesn't and can't account for all the unported apps and homegrown ops or devops shell programs out there. With that in mind, I suggest a staged change instead, with introducing -O and adding a deprecation message to -o as the first stage.

Thank you for your help. I edited most of the files, but I cannot find "/usr/src/sys/contrib/zstd/tests/gzip/init.sh".

  • present a deprecation message to -o option
In D34747#790074, @fel1x.mintchoco.development_gmail.com wrote:

Thank you for your help. I edited most of the files, but I cannot find "/usr/src/sys/contrib/zstd/tests/gzip/init.sh".

Once I checked 14-current source as of 12ish hours ago instead of 12.2 source (I thought i was, but I must have gotten confused), I noticed that file is gone indeed, but I found 2 more:

sys/contrib/openzfs/tests/zfs-tests/tests/functional/acl/off/dosmode.ksh; sys/contrib/openzfs/tests/zfs-tests/tests/functional/acl/off/dosmode.ksh: ls -lo $path | awk '{ gsub(",", "\n", $5); print $5 }' | \
tools/test/stress2/misc/fts2.sh; sprintf(help, "ls -lo %s", p->fts_path);

LGTM as far as the manual page and the use of -o/-O in src. I'll leave deciding whether an exp-run is needed to others.

This revision is now accepted and ready to land.Apr 13 2022, 12:37 AM

I'm concerned about changing the interface in a backwards-incompatible manner for something as core as ls but will leave that to others

bin/ls/ls.c
438

Well it doesn't behave the same as -O, so suggesting that seems wrong. Printing warnings that can't be disabled is also often not well received, and it should definitely not go to stdout.

This line is also way too long for the column limit.

bin/ls/ls.h
57

f_sgroup does not show the name of the group, it _disables_ showing the name of the group, which is the opposite of the other variables here, and the opposite of what your comment says

bin/ls/print.c
238

You're missing a space after the comma

This revision now requires review to proceed.Apr 15 2022, 12:16 AM
bin/ls/ls.c
438

Then how about adding a warning message in man page, not through stdout?

bin/ls/ls.c
438

Is the manual page visible enough? (But if the deprecation message stays in source,, yes please to stderr.)

Manual page and UX both LGTM. Still needs src review.

This revision is now accepted and ready to land.Apr 18 2022, 10:50 PM
pstef added inline comments.
bin/ls/ls.c
438

Recently some systems started warning about fgrep being deprecated (this is GNU's doing) and to me that's infuriating. So I vote for moving this information to the manual page only.

minsoochoo0122_proton.me added inline comments.
bin/ls/ls.c
438

Thank you for your suggestion. I like this idea. I removed warning message in stderr and moved it to the man page.

This revision now requires review to proceed.Nov 1 2022, 11:10 PM
This revision is now accepted and ready to land.Nov 2 2022, 2:47 AM

I reflected all the suggestions and it seems that the reviewers don't have further opinion on this. I'll accept this revision just for now, but if this revision needs any modification, please I would like to hear your opinions.

@emaste @jhb can you please take a look and give the approval from src?

I think this is a bad idea. It's one of the cases where in the past we've made a conscious decision to deviate from POSIX and I see little to no value to our users here. These are little used options there's not a lot of demand for and at the same time, it breaks compatibility with old code. On the whole, I think that situation would suggest we do not want this.

In D34747#898852, @imp wrote:

I think this is a bad idea. It's one of the cases where in the past we've made a conscious decision to deviate from POSIX and I see little to no value to our users here. These are little used options there's not a lot of demand for and at the same time, it breaks compatibility with old code. On the whole, I think that situation would suggest we do not want this.

I don't understand what you mean.

FreeBSD is an Open Source, standards-compliant Unix-like operating system

source: FreeBSD Handbook

If this revision is not accepted because FreeBSD has decided not to follow POSIX, then why do you call it "standards-compliant operating system"?

bin/ls/ls.1
348

I think it would be clearer if the second sentence looked more like this:

This behavior changed in
.Fx 14 ,
to conform to
.St -p1003.1-2008 ;
for the previous behavior of displaying file flags, use
.Pq Fl O .
This revision now requires review to proceed.Apr 10 2023, 8:02 PM
In D34747#898912, @fel1x.mintchoco.development_gmail.com wrote:
In D34747#898852, @imp wrote:

I think this is a bad idea. It's one of the cases where in the past we've made a conscious decision to deviate from POSIX and I see little to no value to our users here. These are little used options there's not a lot of demand for and at the same time, it breaks compatibility with old code. On the whole, I think that situation would suggest we do not want this.

I don't understand what you mean.

Breaking people's existing scripts (and with a tool as core as ls people will have scripts that invoke and and parse its output) is a cost. The amount of the cost is relative, but in this case it's a widely used tool (ls) and it's a feature that is also used as it has no standard alternative (there is no way in POSIX to list file flags, so any existing scripts fetching file flags via ls are using this). Those both make the cost higher than in other cases of changing behavior.

FreeBSD is an Open Source, standards-compliant Unix-like operating system

source: FreeBSD Handbook

If this revision is not accepted because FreeBSD has decided not to follow POSIX, then why do you call it "standards-compliant operating system"?

Because FreeBSD's usage here (and BSD in general) pre-dates the standard. All of the UNIX-like OS's have extensions to and deviations from POSIX in part because POSIX itself was originally an attempt to harmonize differences between the BSD and SYSV "forks" of UNIX. FWIW, both NetBSD and OpenBSD implement the compromise I was going to suggest which is to implement -g and -n per POSIX, but leave -o as the custom BSD option that lists file flags.

In D34747#900964, @jhb wrote:

Because FreeBSD's usage here (and BSD in general) pre-dates the standard. All of the UNIX-like OS's have extensions to and deviations from POSIX in part because POSIX itself was originally an attempt to harmonize differences between the BSD and SYSV "forks" of UNIX. FWIW, both NetBSD and OpenBSD implement the compromise I was going to suggest which is to implement -g and -n per POSIX, but leave -o as the custom BSD option that lists file flags.

I like the NetBSD/OpenBSD solution. It breaks nothing, increases standards conformance and lets us have one BSD standard for this.

I'd like to also point out that the -o thing likely is keyed off matching either FreeBSD or *BSD in scripts, so people already cope with the varying standards for this.

In D34747#903407, @imp wrote:
In D34747#900964, @jhb wrote:

Because FreeBSD's usage here (and BSD in general) pre-dates the standard. All of the UNIX-like OS's have extensions to and deviations from POSIX in part because POSIX itself was originally an attempt to harmonize differences between the BSD and SYSV "forks" of UNIX. FWIW, both NetBSD and OpenBSD implement the compromise I was going to suggest which is to implement -g and -n per POSIX, but leave -o as the custom BSD option that lists file flags.

I like the NetBSD/OpenBSD solution. It breaks nothing, increases standards conformance and lets us have one BSD standard for this.

I'd like to also point out that the -o thing likely is keyed off matching either FreeBSD or *BSD in scripts, so people already cope with the varying standards for this.

I want to try this:

# if __POSIX_VISIBLE >= 200809

Can this macro work in this situation?

In D34747#910651, @fel1x.mintchoco.development_gmail.com wrote:
In D34747#903407, @imp wrote:
In D34747#900964, @jhb wrote:

Because FreeBSD's usage here (and BSD in general) pre-dates the standard. All of the UNIX-like OS's have extensions to and deviations from POSIX in part because POSIX itself was originally an attempt to harmonize differences between the BSD and SYSV "forks" of UNIX. FWIW, both NetBSD and OpenBSD implement the compromise I was going to suggest which is to implement -g and -n per POSIX, but leave -o as the custom BSD option that lists file flags.

I like the NetBSD/OpenBSD solution. It breaks nothing, increases standards conformance and lets us have one BSD standard for this.

I'd like to also point out that the -o thing likely is keyed off matching either FreeBSD or *BSD in scripts, so people already cope with the varying standards for this.

I want to try this:

# if __POSIX_VISIBLE >= 200809

Can this macro work in this situation?

No, for multiple reasons.

In D34747#910651, @fel1x.mintchoco.development_gmail.com wrote:
In D34747#903407, @imp wrote:
In D34747#900964, @jhb wrote:

Because FreeBSD's usage here (and BSD in general) pre-dates the standard. All of the UNIX-like OS's have extensions to and deviations from POSIX in part because POSIX itself was originally an attempt to harmonize differences between the BSD and SYSV "forks" of UNIX. FWIW, both NetBSD and OpenBSD implement the compromise I was going to suggest which is to implement -g and -n per POSIX, but leave -o as the custom BSD option that lists file flags.

I like the NetBSD/OpenBSD solution. It breaks nothing, increases standards conformance and lets us have one BSD standard for this.

I'd like to also point out that the -o thing likely is keyed off matching either FreeBSD or *BSD in scripts, so people already cope with the varying standards for this.

I want to try this:

# if __POSIX_VISIBLE >= 200809

Can this macro work in this situation?

That only controls what posix C functions are visible. It is not a conformance level relating to how commands work. Also, even if it were the consensus of the review is that the incompatible change is a hard no.

In D34747#910653, @imp wrote:
In D34747#910651, @fel1x.mintchoco.development_gmail.com wrote:
In D34747#903407, @imp wrote:
In D34747#900964, @jhb wrote:

Because FreeBSD's usage here (and BSD in general) pre-dates the standard. All of the UNIX-like OS's have extensions to and deviations from POSIX in part because POSIX itself was originally an attempt to harmonize differences between the BSD and SYSV "forks" of UNIX. FWIW, both NetBSD and OpenBSD implement the compromise I was going to suggest which is to implement -g and -n per POSIX, but leave -o as the custom BSD option that lists file flags.

I like the NetBSD/OpenBSD solution. It breaks nothing, increases standards conformance and lets us have one BSD standard for this.

I'd like to also point out that the -o thing likely is keyed off matching either FreeBSD or *BSD in scripts, so people already cope with the varying standards for this.

I want to try this:

# if __POSIX_VISIBLE >= 200809

Can this macro work in this situation?

That only controls what posix C functions are visible. It is not a conformance level relating to how commands work. Also, even if it were the consensus of the review is that the incompatible change is a hard no.

Okay, I will work on reverting -o option and update diff file.

bin/ls/print.c
241

Nothing in the new patch ever sets f_sgroup now, so I think you can remove that variable and make this printf unconditional?

English in manual page still LGTM.

This revision is now accepted and ready to land.Jun 19 2023, 5:56 PM
This revision now requires review to proceed.Jul 14 2023, 2:23 PM

The log message should be updated to reflect the effective changes now which I think is roughly "change -g to implement POSIX" and "make -n imply -l".

This revision is now accepted and ready to land.Jul 14 2023, 5:36 PM
minsoochoo0122_proton.me retitled this revision from ls(1): Add -g, -n, and -o options for POSIX standard to ls(1): Change -g and make -n imply -l to implement POSIX.Jul 14 2023, 7:22 PM

Update commit message to ls(1): Change -g and make -n imply -l to implement POSIX

This revision now requires review to proceed.Jul 14 2023, 7:26 PM

Can you confirm your author details, my guess is "Minsoo Choo <fel1x.mintchoco.development@gmail.com>"?

In D34747#934900, @jhb wrote:

Can you confirm your author details, my guess is "Minsoo Choo <fel1x.mintchoco.development@gmail.com>"?

Hi,

I'm switched my email address to minsoochoo0122@proton.me.

Ah, this adds at least one test failure for -g which I will fix before pushing.

This revision was not accepted when it landed; it landed in state Needs Review.Jul 18 2023, 5:05 PM
This revision was automatically updated to reflect the committed changes.