r279908 added logic to Makefile.inc1 to automatically set CROSS_BINUTILS_PREFIX for architectures not supported by in-tree binutils -- arm64 when this was introduced, and subsequently riscv64 in addition.
LLVM's LLD linker is now included in the base system, and is capable of linking world and kernel. Thus, avoid setting CROSS_BINUTILS_PREFIX and requiring the binutils port if WITH_LLD_IS_LD is set.