Synchronize libedit with NetBSD and make it unicode aware

[bapt]

Summary

Synchronize libedit with NetBSD
Activate Widechar
Reapply local changes
Fix unicode environment detection (submitted upstream)
Fix el_gets in unicode env (submitted upstream)
This allows to get /bin/sh utf8 aware
Tested with locales fr_FR.UTF-8

The plan is to MFC it after one month

[pfg]

This change is very desirable as we are really missing the unicode support and other small changes in libedit, however I have always resisted it to preserve Jilles' completion code. Updating libedit has usually involved other nasty side effects, a good test is repeatedly resizing the terminal while using /bin/sh.

Assuming it works fine, this is also a good opportunity to finally get in sync the vendor repository with current and bring the changes through there. Please note that we can, and should, do repository moves before applying the changes for some of these files to preserve better the history.

lib/libedit/TEST/tc1.c

279 ↗

(On Diff #3040)

This change is wrong, we use DECONST() instead of NetBSD's UNCONST. We require the include in line 33 for this.

lib/libedit/keymacro.c

1 ↗

(On Diff #3040)

Please repocopy (svn mv) key.c here to preserve the history.

lib/libedit/keymacro.h

1 ↗

(On Diff #3040)

Please repocopy (svn mv) key.h here to preserve the history.

lib/libedit/terminal.c

1 ↗

(On Diff #3040)

Please repocopy (svn mv) term.c here to preserve the history.

lib/libedit/terminal.h

1 ↗

(On Diff #3040)

Please repocopy (svn move) term.h here to preserve the history.

[bapt]

Address @pfg comments

[pfg]

Thank you, the changes are more readable now. I am OK with this if the tests run.
(I didn't look at jilles@ stuff though.)

lib/libedit/TEST/tc1.c

279 ↗

(On Diff #3041)

This still looks wrong:
Setting a const as the first argument to the DECONST beats the purpose of it. Also make sure you include the DECONST definition (sys/cdefs.h) .

[jilles]

I have not tested this, but it looks very good.

I only have a few small remarks.

lib/libedit/TEST/tc1.c

279 ↗

(On Diff #3041)

The __DECONST is needed to remove the const at the level of individual characters. There is no way to make a function that accepts both char ** and const char ** but not arbitrary pointer types, so I think __DECONST is the right solution here.

<sys/cdefs.h> definitions are heavily used by other headers internally, so it is usually not needed to include it directly.

lib/libedit/editrc.5

248 ↗

(On Diff #3041)

Contractions should not be used in man pages.

lib/libedit/el.c

99 ↗

(On Diff #3041)

Calling setlocale() should be left to the application, since it modifies global state.

lib/libedit/filecomplete.c

60 ↗

(On Diff #3041)

'*', '?' and '[' should remain extra quote chars. The rest of this file looks good.

[bapt]

lib/libedit/el.c

99 ↗

(On Diff #3041)

Most applications using libedit is not calling setlocale before, do you have a better way to detect UTF-8 envs?

[bapt]

hum just removing it is enough

[bapt]

Removing setlocale works with bin/sh but not some other libedit consumers :(

[bapt]

Closed by commit rS276881 (authored by @bapt).

[bapt]

actually libreadline also use the same trick to determine unicode (setlocale) so I kept the setlocale call wen committing rS276881.