Fix per-connection L2 caching in fast path

r301217 re-added per-connection L2 caching from a previous change,
but it omitted caching in the fast path. Add it.

Reviewed By: gallatin
Approved by: gnn (mentor)
Differential Revision: https://reviews.freebsd.org/D7239