iflib: Own queue quiescence during power transitions

[kbowling]

Summary

Perform a terminal datapath stop before suspend and shutdown
callbacks, then drain the private configuration taskqueue before
entering low power.  Track power state independently of queue
ownership and prevent built-in admin, IOV, LED, and media-status
callbacks from accessing a suspended device.

Restore driver-specific state while the datapath remains stopped.
Initialize it exactly once on resume when the interface is
administratively up, and keep an administratively-down interface
stopped.  Roll back the driver when suspend or child suspension fails.

Add ifdi_power_prepare() for policy which must be established before
the terminal stop.  Use it to snapshot ixgbe(4) wake policy and preserve
X550EM PHY ordering, and remove the duplicate stop from aq(4).  Other
driver power-method wiring follows separately.

MFC after:      2 weeks

[gallatin]

sys/dev/ixgbe/if_ix.c

4002

What version is this against? As of this am (0abd71f39a5c) ixgbe_setup_low_power_mode() doesn't seem to have a 2nd arg.

<10:06am>flea/gallatin:main>git grep ^ixgbe_setup_low_power_mode
sys/dev/ixgbe/if_ix.c:ixgbe_setup_low_power_mode(if_ctx_t ctx)

[kbowling]

sys/dev/ixgbe/if_ix.c

4002

Apologies I had c004384343be in my merge queue pending validation. Please update main past that when you may and this should then apply.

[kbowling]

ifdi_power_prepare or whatever lands will also be used in a followup commit to disarm the ixgbe firmware recovery task when the device is suspended.

[gallatin]

sys/dev/ixgbe/if_ix.c

4002

thanks.. looking at things in the proper order makes a lot more sense