Change Details

Problem: ``` $ echo μ | bsdgrep -i 'µ' Segmentation fault (core dumped) ``` Why: The character in the pattern 0x00B5 (MICRO SIGN) has unidirectional case mapping to 0x039C (GREEK CAPITAL LETTER MU), which in turn has bidirectional case mapping to 0x03BC (GREEK SMALL LETTER MU). Current check in othercase() doesn't handle it, and we have a infinite recursion: ``` ... frame #93: 0x000000080031bfa3 libc.so.7`p_bracket(p=0x00007fffffffe770) at regcomp.c:904 frame #94: 0x000000080031c538 libc.so.7`ordinary [inlined] bothcases(p=<unavailable>, ch=<unavailable>) at regcomp.c:1142 frame #95: 0x000000080031c4ce libc.so.7`ordinary(p=0x00007fffffffe770, ch=181) at regcomp.c:1158 frame #96: 0x000000080031bfa3 libc.so.7`p_bracket(p=0x00007fffffffe770) at regcomp.c:904 frame #97: 0x000000080031c538 libc.so.7`ordinary [inlined] bothcases(p=<unavailable>, ch=<unavailable>) at regcomp.c:1142 frame #98: 0x000000080031c4ce libc.so.7`ordinary(p=0x00007fffffffe770, ch=181) at regcomp.c:1158 frame #99: 0x000000080031bfa3 libc.so.7`p_bracket(p=0x00007fffffffe770) at regcomp.c:904 ... ``` Proposed fix: Detect the unidirectional mappings, and treat such characters as not having a case mapping. This does NOT fix the matching as we currently simply don't have means to tie all possible case conversions together, but fixes the described problem of regcomp() consumers eating a lot of memory and leaving a ~700MB core dump. If/when we'll learn how to use the extended case mapping data from CLDR, this can be revisited.

Problem: ``` $ echo μ | bsdgrep -i 'µ' Segmentation fault (core dumped) ``` Why: The character in the pattern 0x00B5 (MICRO SIGN) has unidirectional case mapping to 0x039C (GREEK CAPITAL LETTER MU), which in turn has bidirectional case mapping to 0x03BC (GREEK SMALL LETTER MU). Current check in othercase() doesn't handle it, and we have a infinite recursion: ``` ... frame #93: 0x000000080031bfa3 libc.so.7`p_bracket(p=0x00007fffffffe770) at regcomp.c:904 frame #94: 0x000000080031c538 libc.so.7`ordinary [inlined] bothcases(p=<unavailable>, ch=<unavailable>) at regcomp.c:1142 frame #95: 0x000000080031c4ce libc.so.7`ordinary(p=0x00007fffffffe770, ch=181) at regcomp.c:1158 frame #96: 0x000000080031bfa3 libc.so.7`p_bracket(p=0x00007fffffffe770) at regcomp.c:904 frame #97: 0x000000080031c538 libc.so.7`ordinary [inlined] bothcases(p=<unavailable>, ch=<unavailable>) at regcomp.c:1142 frame #98: 0x000000080031c4ce libc.so.7`ordinary(p=0x00007fffffffe770, ch=181) at regcomp.c:1158 frame #99: 0x000000080031bfa3 libc.so.7`p_bracket(p=0x00007fffffffe770) at regcomp.c:904 ... ``` Proposed fix: Make the other-case-check more robust, and improve checks in CHIN1(), which also requires cutting down the bitmap range from 256 to 128 for multibyte locales, we leave 0-127 characters alone there. This fixes the infinite loop *and* fixes the matching -- originally, this comes from GNU grep test suite.

Problem: ``` $ echo μ | bsdgrep -i 'µ' Segmentation fault (core dumped) ``` Why: The character in the pattern 0x00B5 (MICRO SIGN) has unidirectional case mapping to 0x039C (GREEK CAPITAL LETTER MU), which in turn has bidirectional case mapping to 0x03BC (GREEK SMALL LETTER MU). Current check in othercase() doesn't handle it, and we have a infinite recursion: ``` ... frame #93: 0x000000080031bfa3 libc.so.7`p_bracket(p=0x00007fffffffe770) at regcomp.c:904 frame #94: 0x000000080031c538 libc.so.7`ordinary [inlined] bothcases(p=<unavailable>, ch=<unavailable>) at regcomp.c:1142 frame #95: 0x000000080031c4ce libc.so.7`ordinary(p=0x00007fffffffe770, ch=181) at regcomp.c:1158 frame #96: 0x000000080031bfa3 libc.so.7`p_bracket(p=0x00007fffffffe770) at regcomp.c:904 frame #97: 0x000000080031c538 libc.so.7`ordinary [inlined] bothcases(p=<unavailable>, ch=<unavailable>) at regcomp.c:1142 frame #98: 0x000000080031c4ce libc.so.7`ordinary(p=0x00007fffffffe770, ch=181) at regcomp.c:1158 frame #99: 0x000000080031bfa3 libc.so.7`p_bracket(p=0x00007fffffffe770) at regcomp.c:904 ... ``` Proposed fix: DetectMake the unidirectional mappingsother-case-check more robust, and treat such characters as not having a case mapping. This does NOT fix the matching as we currently simply don't have means to tie all possible case conversions togetherimprove checks in CHIN1(), which also requires cutting down the bitmap range from 256 to 128 for multibyte locales, but fixes the described problem of regcomp() consumers eating a lot of memory and leaving a ~700MB core dumpwe leave 0-127 characters alone there. If/when we'll learn how to use the extended casThis fixes the infinite loop *and* fixes the mapptching data from CLDR-- originally, this can be revisited. omes from GNU grep test suite.