mlx5en: Report SR-IOV VF status

[kbowling]

Summary

Query each enabled eswitch vport to report its effective MAC and VLAN
configuration and administrative link-state policy.  Serialize the
query with vport enable and disable, and hold the interface state lock
so teardown joins an in-progress report.

Propagate firmware errors from MAC, administrative link-state, and
supported VLAN queries instead of returning a partially initialized
configuration.  When the hardware lacks CVLAN enforcement, report the
remaining status with no imposed access VLAN.

Test Plan

Not sure who to route this to at Nvidia these days

[kbowling]

This is a basic implementation, someone more familiar with the driver may look at the nvlist schema and https://reviews.freebsd.org/D58739 which demonstrates driver extensions.

[kib]

I will ask for the internal testing.

How is the VF status accessed?

[kbowling]

Create some VFs with iovctl (can be attached, unattached, passhthru)

On recent main (as of yesterday) run 'ifconfig -v'

Things to watch out for: performance with 0, 1, many VFs. This implementation does some live firmware work, which is different than cached state in my other implementations. The linked ice(4) review above contains an extension syntax example if there is any additional eswitch info that doesn't fit into the schema that should be exposed.

[kib]

sys/dev/mlx5/mlx5_en/mlx5_en_main.c

3742

I think that you should take the state_lock (AKA PRIV_LOCK) around the call.

[kbowling]

Update against committed netlink API, propagate firmware query failures, tolerate absent CVLAN support, synchronize teardown.

[kbowling]

sys/dev/mlx5/mlx5_en/mlx5_en_main.c

3742

Addressed by holding PRIV_LOCK across validation and firmware snapshot construction

[kib]

sys/dev/mlx5/mlx5_en/mlx5_en_main.c

3611

Why do you claim that trunk cannot have pcp != 0?

3633

Why do you 'continue' there instead of filling something into link_state_policy? It is strange as is.

[kbowling]

sys/dev/mlx5/mlx5_en/mlx5_en_main.c

3611

Keep in mind I know next to nothing about this hw and driver.. think I was seeing the if (!vport-> vlan && !vport->qos) pattern in mlx5_eswitch.c,

Should I just drop the second condition?

3633

How to handle better? the 3 case cover the current enum, invariant or?

[kib]

sys/dev/mlx5/mlx5_en/mlx5_en_main.c

3611

I think yes, drop it. It is not a driver thing BTW, we do support setting pcp on trunks. In this case the non-vlan packets are encapsulated as vlan with vlanid==0 AFAIR, and pcp set.

3633

Well, it is already set to zero. But you continue instead of break, so it will be some surprise for naive developer adding a statement aftter the switch(). If zero is good enough for link_state_policy, then just replace continue with break. Otherwise you would need some value like IFVF_LINK_UNKNOWN_STATE.

[kbowling]

allow trunks to carry pcp, spell out IFVF_LINK_UNKNOWN for code clarity

[kib]

Please wait for the approval from the NVIDIA verification team for the patch.

[kbowling]

Chase 429fbc7c35a6 - allow trunks to display configured PCP. Also distinguish PCP enable so a fixed 0 can be reported in either trunk or access.