cxgbe: Report SR-IOV VF status

[kbowling]

Summary

Retain the effective MAC and VLAN settings accepted by the per-port
t4iov companion and expose them through the corresponding cxgbe ifnet.

Publish, snapshot, and destroy the cache under the existing adapter
synchronized-operation mechanism so status queries cannot race IOV
configuration or teardown.

[kbowling]

This is a basic implementation, someone more familiar with the driver may look at the nvlist schema and https://reviews.freebsd.org/D58739 which demonstrates driver extensions.

[jhb]

sys/dev/cxgbe/adapter.h

315

Maybe struct t4_vf_info? I think that's more consistent with some other cxgbe type names. Really cxgbe would probably call this struct vf_info similar to how struct adapter, but the collision in type names is terrible, so using a t4_ prefix is fine.

sys/dev/cxgbe/t4_iov.c

427

Since you need to do a sync op anyway, just do a single one earlier in the function, and it can use INTR_OK like the existing ones. That is:

has_mac = ...;

if (begin_synchronized_op(adap, NULL, SLEEP_OK | INTR_OK, "t4vfadd") != 0)
   return (ENXIO);

if (has_mac) {
    mac = ...;
    bcopy(...);
    rc = -t4_vf_set_mac(...);
    if (rc != 0) {
        device_printf(...);
        goto out;
}

if (nvlist_exists_number(...)) {
  /* Replace bare "returns" with 'rc = EFOO; goto out;' */
}

However, I think there's also a simpler solution which is a smaller patch. Instead do something like this for the t4iov_add_vf() function:

static int
t4iov_add_vf(...)
{
    const void *mac;
    struct t4iov_softc *sc;
    struct adapter *adap;
    struct t4_iov_vf *vf;
    ...

    adap = ...;
    vf = &sc->sc_port->iov_vfs[vfnum];
    memset(vf, 0, sizeof(*vf));

    ...

    if (nvlist_exists_binary(config, "mac-addr")) {
        ....
        if (rc != 0) {
             ...
        }
        memcpy(vf->mac, ma, ETHER_ADDR_LEN);
   }

   if (nvlist_exists_number(config, "vlan")) {
        ...
        if (vlan == VF_VLAN_TRUNK) {
           vlan = DOT1Q_VID_NULL;
        } else {
           vf->access_vlan = true;
           vf->vlan = vlan;
        }
        ...
    }

    vf->configured = true;
    return (0);
}

The PCI bus driver already needs to synchronize IOV setup / teardown on a given PF, and it should also avoid calling IOV_ADD_VF on the same VF concurrently as a result. Drivers don't need additional protection from that. If the query race is what you are worried about, you can expand the scope of the sync_op as in my first suggestion but still do the inline assignments from my second suggestion.

473

This can't happen unless the PCI bus driver screws up, so I don't think it's worth checking.

sys/dev/cxgbe/t4_main.c

3107

I would add a continue here if the VF isn't configured as there is no point in reporting other fields for an unconfigured VF. It also lets you do the assignments inline in t4iov_add_vf() instead of deferring them all to the end.