@kevans are you OK with the test case added to libc/regex instead and using sed? :)

Oops, I didn't mean to add Ed back to this review :-)

Reading through the discussion, I do agree with @ed -- we should have just one version in UTF-8, and convert to current encoding on the fly, though I do agree with @AMDmi3 as well -- making sure we use only the characters that can be converted to the related single-byte locale shouldn't be that hard, we don't need to use the fancy unicode chars in messages.

Sorry for the delay. I have spent quite a bit more time on this, including testing. Summary and the change updated accordingly.

In D18334#393097, @asomers wrote:

LGTM. Do you you need someone to commit this for you?

With the comment updated, should I go ahead and commit this? Or we should still wait for actual reviews from @scottl and @mav?

Add the comment for msi_tupelo (and cleanup whitespace issues introduced by the patch).

Now with full context...

Done in rS340491.

diff with full context

Upstream PR: https://github.com/lichray/nvi2/pull/61

In D17950#389310, @bapt wrote:

as this been upstreamed? (https://github.com/lichray/nvi2)

drop unrelated changes

After providing the quick fix, I have started wondering how GNU grep (or libgnuregex, more precisely) does the matching without providing any locale specific data; the answer is pretty simple, not sure why I didn't think about it from the start -- fold the case of *both* sides of comparison, and compare both-upper and/or both-lower.

Please submit diffs with full context.

typos.

In D18297#388056, @pfg wrote:

I am not an expert on the area but I see where the comment comes from:

RFC3629 states on page 2:

The byte-value lexicographic sorting order of UTF-8 strings is the
same as if ordered by character numbers. Of course this is of
limited interest since a sort order based on character numbers is
almost never culturally valid.

The Boyer-Moore fast search algorithm can be used with UTF-8 data.

redo.

There's a reply and action upstream, the patch integrated there is more complete (removing all "bits" stuff from mansearch), will use that one.

Baptiste, anything else you want to see done/answered for this to proceed?

In D17449#375265, @yuripv wrote:

In D17449#375264, @bapt wrote:

First we should ship the said manpage :)

I think this is debattable, but I think upstream is opened enought to accept such change, you should anyway talk to them to see if that will remain a freebsd only patch or not :)

OK, will do.

In D17842#382775, @bapt wrote:

That look sane to me, the thing is I wonder how hard it would be to maintain

I think this makes perfect sense. You might want to update editline(3) as well so it doesn't say it only knows about UTF-8 and C locales.

cleanup done separately; rebase

Just for the note: perl bits here are only the helper tools for someone updating the locale sources in share/, and aren't run during the build, so even if something is broken there (and I tried not to as I used them), it's only bad until there's a need to update the data again where they will be fixed anyway :)