xdr_string: don't leak strings with xdr_free

[brooks]

Summary

Historically (and in a small amount of older software such as OpenAFS),
developers would attempt tofree XDR strings with
xdr_free((xdrproc_t)xdr_string, &string)

This resulted in xdr_free calling xdr_string with only two intentional
arguments and what ever was left in the third argument register. If the
register held a sufficently small number, xdr_string would return FALSE
and not free the string (noone checks the return values).

Software should instead free strings with:
xdr_free((xdrproc_t)xdr_wrapstring, &string)

Because buggy software exists in the wild, act as though xdr_wrapstring
was used in the XDR_FREE case and plug these leaks.

Effort: CHERI upstreaming
Sponsored by: Innovate UK

[des]

I realized this patch was committed months ago, but it kind of misses the forest for the trees...

lib/libc/xdr/xdr.c

684

assume xdrs->x_op is XDR_FREE...

687

size is uninitialized

705

the original value of maxsize gets overridden here, but it doesn't really make any difference

713

xdr_u_int() here goes straight to return (TRUE), but passing &size to it masks the fact that size is still uninitialized

716

boom, undefined behavior; in practice we randomly return (FALSE) without freeing the string, which is just what this patch was supposed to fix.

719

let's ignore the fact that we just blew up and assume size was initialized to 0; nodesize is now 1

752

mem_free() is a macro that wraps either free(3) or free(9) depending on context and nodesize isn't used, but if our allocator actually cares about its second argument (e.g. calls the C23 function free_sized() instead of plain old free()) we blow up again.

(C23 §7.24.3.4: if the first argument to free_sized() was not previously obtained from malloc(), calloc(), or realloc(), or the second argument is not equal to the requested allocation size, behavior is undefined)

767

pre-existing style issue (missing parentheses)

[des]

This comment was removed by des.

[des]

lib/libc/xdr/xdr.c

716

This is incorrect, I didn't notice the fallthrough from XDR_FREE to XDR_ENCODE... size is not uninitialized