expr(1): Allow operators as right-hand-side to matching operator

[kevans]

Summary

The grammar is currently structured such that operators need to be interpreted
as literals, rather than other tokens. This causes an error later:

$ expr "/Test" : '/'
expr: syntax error

Fix this by allowing all operators to appear as the right hand side to the matching
operator. This then yields:

$ expr "/Test" : '/'
1

Fixing this also makes the "arithmetic_ops" test pass on FreeBSD, so we can remove the expect
fail as a side effect.

Aside: I don't like the way I've implemented this, but it seems to make for the
cleanest implementation that makes it obvious what's going on. Definitely open
for suggestions of improvements. I tried a couple of different things, and none of them
came up clean.

Mentioned in D11304

Test Plan

Run kyua tests

[jilles]

I looked at what NetBSD and GNU expr do, and they vary in what they accept.

At first, GNU expr seems more lenient, accepting things like expr / : /, but it does not accept expr a : \) which your code does accept. It also accepts expr \( : \) and prints a colon (an alternative would be printing 0).

For NetBSD expr, I read its source code which contains an ambiguous yacc grammar that allows interpreting any operator symbol as a string (for example, it depends on operator precedence to choose between the two possible parse trees for expr \( : \)).

Your change seems at least less ambiguous than what NetBSD has done, although it seems rather tailored to NetBSD's testcases.

I think this change brings complexity for little gain. If you make this change, users will ask for more operator symbols to be interpreted as strings, and it is unclear how this should be done in the general case. Contrast this to the test(1) grammar where POSIX provides a clear set of rules that allows writing scripts passing arbitrary operands without ambiguity problems (for example, test \( = \) compares two strings, yielding false).

Our current implementation of treating any string identical to an operator symbol as that operator may be restrictive, but is at least easy to understand.

What's more, this change is not being made to fix existing code in the wild, and the fixed cases are trivial to implement using a shell case statement (or [ "${x#/}" != "$x" ] for the real case haters).

[kevans]

GNU expr(1) not accepting expr a : \) seems like a bug to me-- ) is a perfectly valid BRE. expr \( : \) not working here is an oversight, and I'll need to think about this one a little more.

This does have more problems than I expected. It's not actually intentionally geared towards the test case -- all of the valid operators in yylex() that return a literal rather than a TOKEN (with exception to GE, LE, NE, I goofed on those) were included.

I don't think we should be expanding the operator selection at all later on to promote more non-compliant behavior, so this selection of literals should be completely static. I've proposed the -E option, but that one is mostly to relax a limitation that POSIX admits is only because of historical applications.