Change Details

Problem: ``` $ echo μ | bsdgrep -i 'µ' Segmentation fault (core dumped) ``` Why: The character in the pattern 0x00B5 (MICRO SIGN) has unidirectional case mapping to 0x039C (GREEK CAPITAL LETTER MU), which in turn has bidirectional case mapping to 0x03BC (GREEK SMALL LETTER MU). Current check in othercase() doesn't handle it, and we have a infinite recursion: ``` ... frame #93: 0x000000080031bfa3 libc.so.7`p_bracket(p=0x00007fffffffe770) at regcomp.c:904 frame #94: 0x000000080031c538 libc.so.7`ordinary [inlined] bothcases(p=<unavailable>, ch=<unavailable>) at regcomp.c:1142 frame #95: 0x000000080031c4ce libc.so.7`ordinary(p=0x00007fffffffe770, ch=181) at regcomp.c:1158 frame #96: 0x000000080031bfa3 libc.so.7`p_bracket(p=0x00007fffffffe770) at regcomp.c:904 frame #97: 0x000000080031c538 libc.so.7`ordinary [inlined] bothcases(p=<unavailable>, ch=<unavailable>) at regcomp.c:1142 frame #98: 0x000000080031c4ce libc.so.7`ordinary(p=0x00007fffffffe770, ch=181) at regcomp.c:1158 frame #99: 0x000000080031bfa3 libc.so.7`p_bracket(p=0x00007fffffffe770) at regcomp.c:904 ... ``` Proposed fix: Make the other-case-check more robust, and improve checks in CHIN1(), which also requires cutting down the bitmap range from 256 to 128 for multibyte locales, we leave 0-127 characters alone there. This fixes the infinite loop *and* fixes the matching -- originally, this comes from GNU grep test suite.

Problem: ``` $ echo μ | bsdgrep -i 'µ' Segmentation fault (core dumped) ``` Why: We use a bitmap to optimize the first 256 (CHAR_MAX - CHAR_MIN + 1) characters handling. Apparently, the *real* problem here is that one half of lower-pair is in the bitmap we use for first 256 characters (CHAR_MAX - CHAR_MIN), and the other one is outside, which check in othercase() doesn't handle it, and we have a infinite recursion: ``` ... frame #93: 0x000000080031bfa3 libc.so.7`p_bracket(p=0x00007fffffffe770) at regcomp.c:904 frame #94: 0x000000080031c538 libc.so.7`ordinary [inlined] bothcases(p=<unavailable>, ch=<unavailable>) at regcomp.c:1142 frame #95: 0x000000080031c4ce libc.so.7`ordinary(p=0x00007fffffffe770, ch=181) at regcomp.c:1158 frame #96: 0x000000080031bfa3 libc.so.7`p_bracket(p=0x00007fffffffe770) at regcomp.c:904 frame #97: 0x000000080031c538 libc.so.7`ordinary [inlined] bothcases(p=<unavailable>, ch=<unavailable>) at regcomp.c:1142 frame #98: 0x000000080031c4ce libc.so.7`ordinary(p=0x00007fffffffe770, ch=181) at regcomp.c:1158 frame #99: 0x000000080031bfa3 libc.so.7`p_bracket(p=0x00007fffffffe770) at regcomp.c:904 ... ``` Proposed fix: Reduce the actual size of bitmap down to 128 characters (0-127) for multibyte locales. While I was tempted to not use it at all there to be absolutely safe, it will greatly affect performance as 99.99% of regexes used e.g. when building world fall into the POSIX locale characters range; more so there are no known issues (yet?) with that range. It also required a change in computejumps() to reduce the character range as well in UTF-8 case (we can get there in multibyte case only if using UTF-8).

Problem: ``` $ echo μ | bsdgrep -i 'µ' Segmentation fault (core dumped) ``` Why: The character in the pattern 0x00B5 (MICRO SIGN) has unidirectional case mapping to 0x039C (GREEK CAPITAL LETTER MUWe use a bitmap to optimize the first 256 (CHAR_MAX - CHAR_MIN + 1) characters handling. Apparently, the *real* problem here is that one half of lower-pair is in the bitmap we use for first 256 characters (CHAR_MAX - CHAR_MIN), which in turn has bidirectional case mapping to 0x03BC (GREEK SMALL LETTER MU). Currentand the other one is outside, which check in othercase() doesn't handle it, and we have a infinite recursion: ``` ... frame #93: 0x000000080031bfa3 libc.so.7`p_bracket(p=0x00007fffffffe770) at regcomp.c:904 frame #94: 0x000000080031c538 libc.so.7`ordinary [inlined] bothcases(p=<unavailable>, ch=<unavailable>) at regcomp.c:1142 frame #95: 0x000000080031c4ce libc.so.7`ordinary(p=0x00007fffffffe770, ch=181) at regcomp.c:1158 frame #96: 0x000000080031bfa3 libc.so.7`p_bracket(p=0x00007fffffffe770) at regcomp.c:904 frame #97: 0x000000080031c538 libc.so.7`ordinary [inlined] bothcases(p=<unavailable>, ch=<unavailable>) at regcomp.c:1142 frame #98: 0x000000080031c4ce libc.so.7`ordinary(p=0x00007fffffffe770, ch=181) at regcomp.c:1158 frame #99: 0x000000080031bfa3 libc.so.7`p_bracket(p=0x00007fffffffe770) at regcomp.c:904 ... ``` Proposed fix: Make the other-case-check more robustReduce the actual size of bitmap down to 128 characters (0-127) for multibyte locales. While I was tempted to not use it at all there to be absolutely safe, and improve checks in CHIN1(), which also requires cutting down the bitmap range from 256 to 128 for multibyte locales,it will greatly affect performance as 99.99% of regexes used e.g. we leave 0-127 characters alone there. This fixes the infinite loop *and* fixes the matching -- originally, this comes from GNU grep test suitehen building world fall into the POSIX locale characters range; more so there are no known issues (yet?) with that range. It also required a change in computejumps() to reduce the character range as well in UTF-8 case (we can get there in multibyte case only if using UTF-8).