nfscl: Always invalidate buffers for append writes

kib@ reported a problem which was resolved by
reverting commit 867c27c23a5c, which changed the NFS
client to use direct RPCs to the server for
IO_APPEND writes. He also spotted that the
code only invalidated buffer cache buffers
when they were marked NMODIFIED (had been
written into).

This patch modifies the NFS VOP_WRITE() to
always invalidate the buffer cache buffers
and pages for the file when IO_APPEND is
specified. It also includes some cleanup
suggested by kib@.

(cherry picked from commit e4df1036f66dc360606fea053ec04ccabb69df14)