A writer that fails to allocate a buffer refuses every later message,
the ack and NLMSG_DONE included. nlmsg_end_dump() ignored that,
reported success and suppressed the ack. Either way the request got no
answer and its sender waited for ever. The requests of a whole send(2)
were lost the same way when the first buffer of the writer could not be
allocated.
Neither that buffer nor the ack is allocated with locks held, so let
both wait for memory. Send the ack with ENOMEM for every reply that
ran out of memory: that also ends a dump that nlmsg_end_dump() could
not.
Replies to Linux processes can still be lost when their translation
runs out of memory.
Fixes: 7e5bf68495cc ("netlink: add netlink support")
Fixes: f4d3aa749084 ("netlink: suppress sending NLMSG_ERROR if NLMSG_DONE is already sent")
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")