localedef: off by one bug drops the highest code point from every collation table

[becker.greg_att.net]

Summary

localedef(1) writes one fewer entry than it has into the "large" (code point
above UCHAR_MAX) collation table, and records a count one short. The table is
built in code point order, so the entry lost is always the character with the
highest code point in the locale. For that character wcsxfrm(3) returns 0 and
wcscoll(3) gives it no weight: it sorts with the ignorables and compares equal
to unrelated characters.

Test Plan

1. Sort three lines, one of which is U+FF5A (FULLWIDTH LATIN SMALL LETTER Z), the highest code point en_US.UTF-8 declares:
   
   printf 'b\tb\n\357\275\232\tU+FF5A\na\ta\n' | LC_ALL=en_US.UTF-8 sort | cut -f2

2. Repeat with U+FF3A (FULLWIDTH LATIN CAPITAL LETTER Z), the next code point down, declared the same way in the same source file:
   
   printf 'b\tb\n\357\274\272\tU+FF3A\na\ta\n' | LC_ALL=en_US.UTF-8 sort | cut -f2

3. Repeat step 1 under LC_ALL=C, to establish that the ordering comes from the collation data and not from the byte values:
   
   printf 'b\tb\n\357\275\232\tU+FF5A\na\ta\n' | LC_ALL=C sort | cut -f2

---

On 16.0-CURRENT #0 main-n289560-044cae040488: Wed Sep 23 09:44:42 CDT 2026 you can see
that in the "Before: Test 1" test U+FF5A sorts first (which is incorrect), whereas after the fix it is sorted
last. FWIW, I ran these tests on Fedora 43 which agrees with the "After" results.

Before:

Test 1:
U+FF5A
a
b

Test 2:
a
b
U+FF3A

Test 3:
a
b
U+FF5A

After:

Test 1:
a
b
U+FF5A

Test 2:
a
b
U+FF3A

Test 3:
a
b
U+FF5A

$ sudo kyua test usr.bin/locale

usr.bin/locale/locale_test:k_flag_posix -> passed [0.017s]
usr.bin/locale/locale_test:k_flag_unknown_kw -> passed [0.008s]
usr.bin/locale/locale_test:no_flags_posix -> passed [0.016s]

Results file id is usr_tests.20260927-105501-592039
Results saved to /root/.kyua/store/results.usr_tests.20260927-105501-592039.db

3/3 passed (0 broken, 0 failed, 0 skipped)

$ sudo kyua test > ~/tests.out 2>&1

9269/10806 passed (108 broken, 7 failed, 1422 skipped)

$ grep -Ei failed: ~/tests.out
sys/kern/ssl_sendfile:eagain_vs_eof -> failed: /usr/src/tests/sys/kern/ssl_sendfile.c:485: c.sbytes == -1 not met [0.084s]
bin/chflags/chflags_test:outside_symlink_rejected -> failed: atf-check failed; see the output of the test for details [0.014s]
lib/libexecinfo/sigtramp_test:test_backtrace_sigtramp -> failed: /usr/src/lib/libexecinfo/tests/sigtramp_test.c:30: n > 1 not met [0.003s]
lib/libc/stdlib/clearenv_test:clearenvrecreated_system_var_test -> failed: 1 checks failed; see output for more details [0.003s]
lib/libc/stdlib/clearenv_test:clearenvsystem_var_test -> failed: 1 checks failed; see output for more details [0.003s]
sys/audit/file-attribute-access:lpathconf_success -> failed: /usr/src/tests/sys/audit/file-attribute-access.c:771: lpathconf(path, _PC_SYMLINK_MAX) != -1 not met [0.003s]
sys/net/routing/test_routing:test_ecmp_routes_by_event -> failed: atf-check failed; see the output of the test for details [1.183s]

AFAICT, none of the failures look related to my change. I will revert the change and rerun the tests if someone has any concerns.

Probably we should have a simple kyua based regression test for this, but I am not clear on how to write one.