An outer loop might handle failure because of an oldval mismatch, but why should it have to handle failure due to the internal implementation details of atomics on a given platform? The strex can fail for reasons such as an interrupt or fault happening between the ldrex and strex. We can loop more efficiently internally than an outer loop can (it costs literally 0 cycles unless the strex actually fails).

The manpage for fcmpset says that some platforms "might" return false even when old==new, and I think that's a documentation bug (and implementation bug if it's actually true on any platform) that should be fixed.

Adding the loop on arm makes the call site bigger by one instruction, which will take 0 cycles in the usual case. I guess it's a cisc-centric background that would make someone declare this code as "large". The biggest one, fcmpset64, is 10 instructions, of which only 6 execute in the usual uncontested case (I wonder if x86 lock cmpxchg is able to run in just 6 cycles without impacting any core other than the one it runs on?). ARM's ability to conditionally execute (or not) most instructions makes for really fast branch-free code in which you get to coast through some instructions at a cost of zero cycles, and most of the ones that aren't skipped are 1 cycle to execute.

The point about returning the found value is why I think it's important to loop internally on a strex access conflict. If you don't do so, then the value returned is the original value you read from there, even though you know (because of the strex failure) that there is now a new value there. If the code returns false on strex failure instead of looping, the caller cannot know whether *oldval is valid or not, and would have to re-read it to be sure.