if_vlan locking fixes

[mjoras]

Summary

Previously the locking in if_vlan was not very comprehensive. Particularly
there was very little protection against the destruction of active vlan(4)
interfaces or concurrent modification of a vlan(4) interface. The former
readily produced several different panics.

These changes broaden the scope of the global if_vlan rmlock(9) to avoid these
races. if_vlan should now be much more resilient to the destruction of active
vlan(4) interfaces and other racy situations in the configuration path.

Test Plan

These changes were tested internally (Isilon) by testing the vlan ioctl paths
with WITNESS on, and running a stress test which creates/destroys vlans while
those vlans are active (i.e. have active TCP iperf flows).

[• hselasky]

Could you add a comment with a few words about the locking strategy and order?

[mjoras]

• Fixup comments to be more explicit about the locking strategy.

[mjoras]

In D5825#124451, @hselasky wrote:

Could you add a comment with a few words about the locking strategy and order?

I redid the top comment to be more explicit about the locking. Does that look decent to you?

[ae]

The change looks reasonable to me. Did you measure what penalty added an rmlock on transmit path?