mlx5en: Report SR-IOV VF status

[kbowling]

Summary

Query each enabled eswitch vport to report its effective MAC and VLAN
configuration and administrative link-state policy.  Serialize the
query with vport enable and disable so VF teardown cannot invalidate
the report in progress.

Propagate firmware query errors from
mlx5_eswitch_get_vport_config() instead of returning a partially
initialized configuration.

Test Plan

Not sure who to route this to at Nvidia these days

[kbowling]

This is a basic implementation, someone more familiar with the driver may look at the nvlist schema and https://reviews.freebsd.org/D58739 which demonstrates driver extensions.

[kib]

I will ask for the internal testing.

How is the VF status accessed?

[kbowling]

Create some VFs with iovctl (can be attached, unattached, passhthru)

On recent main (as of yesterday) run 'ifconfig -v'

Things to watch out for: performance with 0, 1, many VFs. This implementation does some live firmware work, which is different than cached state in my other implementations. The linked ice(4) review above contains an extension syntax example if there is any additional eswitch info that doesn't fit into the schema that should be exposed.

[kib]

sys/dev/mlx5/mlx5_en/mlx5_en_main.c

3716

I think that you should take the state_lock (AKA PRIV_LOCK) around the call.