lagg: Fix SIOCSLAGG ioctl

[zlei]

Summary

The data path shall stop pushing data to protocol start or input
routines while changing the aggregation protocol, otherwise it is
possible to access freed protocol private data.

Also, some protocols have stop routine, and the stop routine shall
be invoked prior to detaching the protocol.

Fix them by,

1. Call lagg_stop() to full stop the interface before changing the

aggregation protocol.

2. The access to if_drv_flags lacks proper synchronization. Introduce

sc_state, a driver private state, and access it atomically to prevent
the data path from getting the stale running state.

While here, for the LACP protocol, stop calling lacp_init() within
lacp_attach(). lagg_proto_init() is the generic wrapper of protocol
init routine, and should be used instead.

PR: 289017
Reported by: Gui-Dong Han <[e-mail]>
MFC after: 5 days

Test Plan

Change lagg protocol while pushing traffic over lagg(4) interface.

[markj]

otherwise it is possible to access freed protocol private data.

Which data, exactly? Why can't we fix that by using NET_EPOCH_CALL?

In general we should avoid adding new epoch_wait() and epoch_drain_callbacks() calls. We should also avoid blocking in the data path, otherwise we might as well use an rmlock instead of net_epoch.

sys/net/if_lagg.c

1360

We should really avoid adding more of these barriers.