The limit of 8 KiB comes straight from the 80-ies, when memory sizes were
two orders of magnitude less. Microbenchmarking clearly shows that
increased maximum buffer size is needed to improve bulk transfer
performance over unix(4). Most operating system have larger default
sizes. This brings us another reason for increase. A buggy application,
that treats a stream socket as a datagram socket, shall work on Linux as
long as its messages are smaller than socket buffer size without any
issues 99.99% of time. But on FreeBSD it will instantly fail. In the
ports system there are already applications that suggest increase of this
sysctls for the application to work. We don't want to be a bug catcher
for bad applications in the default configuration.
Note that buffer size increase will not increase memory usage even on a
system with a million unix(4) sockets at normal runtime, as long as
applications are written properly and receivers drain their sockets and
system has enough CPU time for them to do that.