locale: use per-thread-locale's __mb_sb_limit

[yuripv]

Summary

As this adds (yet more stupid) public symbols to libc, I want to make sure this is acceptable before going further with testing on linux cross build.

__mb_sb_limit which defines the highest character that isctype
functions can process describes the global locale as it was introduced
before the xlocale work. It becomes a problem when global locale is set
to the one with e.g. UTF-8 encoding which sets __mb_sb_limit to 128 from
the original 256, and if per-thread locale is set to e.g. ISO8859-1 one,
all of the isctype functions will fail early for [128-255] range even
if the type check should succeed for a character.

To fix this, make isctype functions query the per-thread locale via the
added ___mb_sb_limit function. Sadly this requires making that symbol
public to not expose even more of xlocale internals as isctype helper
functions are inlined.

The change broke bootstrap localedef build (on FreeBSD, Linux cross
buld not tested yet) as it was including _ctype.h from the source tree
which it should not really do anyway. Fix this by extracting the needed
_CTYPE_* defines from the include/_ctype.h.

PR: 265950

Test Plan

See added test case.

buildworld/installworld/run the system

[yuripv]

complete change (except for linux cross build testing)

[kib]

include/_ctype.h

97

lib/libc/Versions.def

52 ↗

(On Diff #127619)

Please commit Versions.def part separately, for easier MFC.

[yuripv]

• Versions.def updated in separate commit
• drop extern as suggested by Konstantin

[imp]

Seems reasonable to me... kib@ is a lot pickier on this stuff (for good reason) than I. Once he's happy, I'm happy.
Are there any locale maintainers we need to loop into the review? I can't think of any, but we should check before rather than after the commit :)

[kib]

I do not know much about runes/locales either.

include/_ctype.h

99

Why do you need this definition? Since you have to change all places, why not call the function by its real name?

[yuripv]

In D41927#956261, @kib wrote:

I do not know much about runes/locales either.

I'm not that worried about the fix itself, more about adding the public symbol to libc, and wanted to hear from you if this is acceptable.

include/_ctype.h

99

This is not really needed, I was going after MB_CUR_MAX design at first, and then just kept it here as "looks better in the checks".

[kib]

In D41927#956264, @yuripv wrote:

In D41927#956261, @kib wrote:

I do not know much about runes/locales either.

I'm not that worried about the fix itself, more about adding the public symbol to libc, and wanted to hear from you if this is acceptable.

If the symbol is needed, why not? The concern is that we must support it forever, with the same or compatible semantic. But then again, this is the question for the locale code maintenance.

[yuripv]

In D41927#956266, @kib wrote:

In D41927#956264, @yuripv wrote:

In D41927#956261, @kib wrote:

I do not know much about runes/locales either.

I'm not that worried about the fix itself, more about adding the public symbol to libc, and wanted to hear from you if this is acceptable.

If the symbol is needed, why not? The concern is that we must support it forever, with the same or compatible semantic. But then again, this is the question for the locale code maintenance.

I *think* I have a better idea that does not need adding new symbols in D41985.