somewhat of a bikeshed comment but an explicit "if (a < b) / else if (b < a) / else" is more immediately clear to me, and a quick check on compilerexplorer showed Clang generated identical code. That said, I'm fine either way and perhaps "(a > b) - (a < b)" is idiomatic.

Yes, (a > b) - (a < b) is the usual idiom for compare functions to avoid tests.

I've tested on Compiler Explorer with latest LLVM 17, 18, 19, and GCC 12, 13 and 14, with -O and -O2, with alternatives: a > b ? 1 : -(a < b) and a full chain of if testing first for a > b and then a < b. Switching to any alternative makes GCC produce test and jump instructions. clang is better, but always generates conditional moves, whereas with the idiom it even doesn't need to (and just uses SETA).

I agree with you it's less readable than just a test. Changing it makes a difference in the produced assembly, albeit probably undetectable in practice in the current context. I'm fine either way.

I did a brief survey of existing code in the tree and find several existing examples of both forms, so let's leave it as is.