msdosfs: correct handling of vnode pager size on file extension error

If extension fails, vnode pager recorded size might be left increased.
Only update vnode pager when extension is past the point of no rollback.

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D38549