Change Details

Being annoyed at how incomplete/incorrect our width data file is resulting all kinds of weird issues when using emojis and other wide symbols in terminal and seeing how only select programs are using utf8proc for this (e.g. irssi), I have put together rather naive tooling to get the data from utf8proc's utf8proc_charwidth(). Creating widths.txt this way takes about 15 minutes on rather fast CPU, so it could be possible to rewrite the shell script in perl instead (C has to stay as it is) if this looks to be acceptable approach.

Update our pretty outdated widths.txt with data from utf8proc, which looks to be widely used exactly for the reasons of wcwidth() provided by OS to be unreliable. This implementation consists of simple piece of C code (required as there's only C library in utf8proc), going through 0x0 - 0x110000 character range, and perl script (copied and modified from utf8-rollup.pl) mapping the data into widths.txt format.

Being annoyed at how incomplete/incorrect our width data file is resulting all kinds of weird issues when using emojis and other wide symbols in terminal and seeing how only select programs are using utf8proc for this (e.g.Update our pretty outdated widths.txt with data from utf8proc, irssi), I have put together rather naive tooling to get the data from utf8proc's utf8proc_charwidth(). Creating widths.txt this way takes about 15 minutes on rather fast CPU, so it could be possible to rewrite the shell script in perl instead (C has to stay as it is) if this looks to be acceptable approachwhich looks to be widely used exactly for the reasons of wcwidth() provided by OS to be unreliable. This implementation consists of simple piece of C code (required as there's only C library in utf8proc), going through 0x0 - 0x110000 character range, and perl script (copied and modified from utf8-rollup.pl) mapping the data into widths.txt format.