PCI function address aliasing support

[br]

Summary

This encodes a 8-bit function address as a bit number in dma_aliases bit string, which is used to program a permissible function source address for DMA in an IOMMU system.

For example Marvel SATA controller uses function number 0 for non-data transfers, and 1 for DMA. In this case both has to be programmed in IOMMU to allow access.

Test Plan

Tested on Neoverse N1 (N1SDP) with Marvel SATA.

[andrew]

Why is it in the pci code rather then iommu? I could imagine broken non-pci devices could also need a DMA alias.

[br]

In D26360#586327, @andrew wrote:

Why is it in the pci code rather then iommu? I could imagine broken non-pci devices could also need a DMA alias.

In non-pci that would be not a function alias, but simply another SID, or a set of SIDs (because only PCI splits the RID/SID to bus:device:function) . And we can't have an alias for a bus or device address, because that would be a different IOMMU domain.
This is obtained from Linux as is. In linux they convert this PCI aliases to the list of ids in the local IOMMU system (SIDs in case of SMMU). And I think this is a right approach.
So the "function address" is property of PCI only. There could be multiple sources that all together build the list of SIDs, and PCI is simply one of them.
So I think in !PCI we should have a different way of aliasing, the way how aliases constructed, etc

[kib]

Well, this is not the right approach. I am guilty of the buswide_ctx() of course, but this one goes much further.

Proper solution is to finish my patch to support DMAR (IOMMU ?) domains, where single domain with its page table serves some number of contexts. Then multi-function devices that must share translations should be put into common domain instead of creating per-function domains.

[br]

In D26360#587123, @kib wrote:

Well, this is not the right approach. I am guilty of the buswide_ctx() of course, but this one goes much further.

Proper solution is to finish my patch to support DMAR (IOMMU ?) domains, where single domain with its page table serves some number of contexts. Then multi-function devices that must share translations should be put into common domain instead of creating per-function domains.

We don't create a per-function domain. This is just to setup a Stream Table Entry (STE) alias in the 2-level walk table.
Marvel SATA issues non-dma requests from SID 0x600, and DMA from 0x601. So SMMU simply could not find a valid STE in the position of 0x601 in the 2-level walk table.
The solution is just to copy the same STE to a required position in the walk table and everything works (this is what Linux does as well).

[kib]

In D26360#587442, @br wrote:

In D26360#587123, @kib wrote:

Well, this is not the right approach. I am guilty of the buswide_ctx() of course, but this one goes much further.

Proper solution is to finish my patch to support DMAR (IOMMU ?) domains, where single domain with its page table serves some number of contexts. Then multi-function devices that must share translations should be put into common domain instead of creating per-function domains.

We don't create a per-function domain. This is just to setup a Stream Table Entry (STE) alias in the 2-level walk table.
Marvel SATA issues non-dma requests from SID 0x600, and DMA from 0x601. So SMMU simply could not find a valid STE in the position of 0x601 in the 2-level walk table.
The solution is just to copy the same STE to a required position in the walk table and everything works (this is what Linux does as well).

In essence, you just reworded what I wrote.

Let me be more explicit: I believe such cases should be handled by creating domains which contain more than one context.

[br]

In D26360#588498, @kib wrote:

In D26360#587442, @br wrote:

In D26360#587123, @kib wrote:

Well, this is not the right approach. I am guilty of the buswide_ctx() of course, but this one goes much further.

Proper solution is to finish my patch to support DMAR (IOMMU ?) domains, where single domain with its page table serves some number of contexts. Then multi-function devices that must share translations should be put into common domain instead of creating per-function domains.

We don't create a per-function domain. This is just to setup a Stream Table Entry (STE) alias in the 2-level walk table.
Marvel SATA issues non-dma requests from SID 0x600, and DMA from 0x601. So SMMU simply could not find a valid STE in the position of 0x601 in the 2-level walk table.
The solution is just to copy the same STE to a required position in the walk table and everything works (this is what Linux does as well).

In essence, you just reworded what I wrote.

Let me be more explicit: I believe such cases should be handled by creating domains which contain more than one context.

Do you mean that each function alias should be in a separate iommu context?

[kib]

In D26360#588657, @br wrote:

In D26360#588498, @kib wrote:

In D26360#587442, @br wrote:

In D26360#587123, @kib wrote:

Well, this is not the right approach. I am guilty of the buswide_ctx() of course, but this one goes much further.

Proper solution is to finish my patch to support DMAR (IOMMU ?) domains, where single domain with its page table serves some number of contexts. Then multi-function devices that must share translations should be put into common domain instead of creating per-function domains.

We don't create a per-function domain. This is just to setup a Stream Table Entry (STE) alias in the 2-level walk table.
Marvel SATA issues non-dma requests from SID 0x600, and DMA from 0x601. So SMMU simply could not find a valid STE in the position of 0x601 in the 2-level walk table.
The solution is just to copy the same STE to a required position in the walk table and everything works (this is what Linux does as well).

In essence, you just reworded what I wrote.

Let me be more explicit: I believe such cases should be handled by creating domains which contain more than one context.

Do you mean that each function alias should be in a separate iommu context?

There are no aliases in PCIe spec or DMAR. Yes, DMAR spec makes 1:1 correspondence between context and RID. Domain can group several contexts that share page tables, making all RIDs from domain using identical translations. I believe AMD IOMMU is same.

[br]

In D26360#588838, @kib wrote:

In D26360#588657, @br wrote:

In D26360#588498, @kib wrote:

In D26360#587442, @br wrote:

In D26360#587123, @kib wrote:

Well, this is not the right approach. I am guilty of the buswide_ctx() of course, but this one goes much further.

Proper solution is to finish my patch to support DMAR (IOMMU ?) domains, where single domain with its page table serves some number of contexts. Then multi-function devices that must share translations should be put into common domain instead of creating per-function domains.

We don't create a per-function domain. This is just to setup a Stream Table Entry (STE) alias in the 2-level walk table.
Marvel SATA issues non-dma requests from SID 0x600, and DMA from 0x601. So SMMU simply could not find a valid STE in the position of 0x601 in the 2-level walk table.
The solution is just to copy the same STE to a required position in the walk table and everything works (this is what Linux does as well).

In essence, you just reworded what I wrote.

Let me be more explicit: I believe such cases should be handled by creating domains which contain more than one context.

Do you mean that each function alias should be in a separate iommu context?

There are no aliases in PCIe spec or DMAR. Yes, DMAR spec makes 1:1 correspondence between context and RID. Domain can group several contexts that share page tables, making all RIDs from domain using identical translations. I believe AMD IOMMU is same.

But the AHCI driver request only one tag ?
So it creates a controller-wide DMA tag, and uses it to make a child tags for data transfers/etc.
We have only one request from it to acpi_iommu_get_dma_tag().

[jhb]

I don't think this belongs in PCI either, but I think we need a way for a driver to get a reference to the IOMMU backing a tag it has created (bus_dma_tag_get_iommu_domain) or some such and then we need a way to add other devices to that domain. I don't think Ruslan has shared the associated Linux bug, but it was actually found on Intel with DMAR using this Marvell controller (and is just as broken when using SMMU on Arm) that initiates some DMA request from function 1. However, there is no function 1 in lspci output, which means Marvell is just setting random bits in PCI-e headers for devices that don't exist (so you can't shut them up by clearing their busmaster_en bit, you can't FLR them, you could never use them for PCI passthrough. Apparently Marvell engineers don't have a good grasp of English and/or the base PCI (not even PCI-e) specifications). This means we don't have a real device_t for the function 1. That might mean either having a 'iommu_domain_add_device' that takes a raw PCI RID, or using a quirk to create fake device_t's. The fake device_t is probably a lot more work, so having a way to add an explicit PCI RID is the simplest way to go. This could then be done as a driver quirk in achi(4) for relevant Marvell controllers.

Also, you should almost never use spin mutexes. Most spin mutexes in Linux should be MTX_DEF in FreeBSD.

Reference to the Linux bug: https://bugzilla.kernel.org/show_bug.cgi?id=42679

[br]

Also here are Linux quirks for Marvell:
https://elixir.bootlin.com/linux/latest/source/drivers/pci/quirks.c#L4026

[kib]

Yes, I was aware of some AHCI controllers mis-handling RIDs. It is the duty of the driver to ensure that proper domain is created which includes all relevant contexts (not that we have an KPI for this right now).

But wait, why cannot you use iommu_set_buswide_ctx() there ?

[br]

In D26360#589183, @kib wrote:

Yes, I was aware of some AHCI controllers mis-handling RIDs. It is the duty of the driver to ensure that proper domain is created which includes all relevant contexts (not that we have an KPI for this right now).

But wait, why cannot you use iommu_set_buswide_ctx() there ?

How?

Here are RID table for the PCI devices on our N1SDP board:
0x800 XHCI
0x700 Realtek ethernet adapter
0x600 AHCI

So all the devices are on bus 0, but different slots (6, 7, 8).

[kib]

In D26360#589847, @br wrote:

In D26360#589183, @kib wrote:

Yes, I was aware of some AHCI controllers mis-handling RIDs. It is the duty of the driver to ensure that proper domain is created which includes all relevant contexts (not that we have an KPI for this right now).

But wait, why cannot you use iommu_set_buswide_ctx() there ?

How?

Here are RID table for the PCI devices on our N1SDP board:
0x800 XHCI
0x700 Realtek ethernet adapter
0x600 AHCI

So all the devices are on bus 0, but different slots (6, 7, 8).

And they are PCIe devices ?
PCIe devices must have a parent, that is switch/upstream port, to represent the link. Real PCIe is not bus, but rather a network of point-to-point links, and each link can only have one device down (or switch).

The only cases I knew, where multiple slots are filled by unrelated functionality were:

• Intel north bridge pseudo PCIe devices used for configuration of uncore. They are not PCIe at all, just Intel decided that giving access to config registers is simpler (?) through PCIe MCFG mapping
• Intel PCH hubs, which have a lot of stuff integrated and reported as slots on single bus.

But your devices are definitely not from some integrated chip, I hardly believe that Marvell AHCI and Realtek ethernet are put together on one IC. They must be physically separate chips with dedicated links to each. No ? Then I am very curious what it is.

Added later:
and in fact 6/7/8 are the bus numbers, not slots. Without sillyness like ARI, pci addresses have 3 bits for slots, and 5 bits for functions, which occupies total 1 byte for s:f part. So e.g. 0x600 is really 6:0:0 bus 6, slot 0, func 0.

In other words, buswide function should be enough.

[br]

Thanks kib@: buswide quirk works, I have updated the main diff:
https://reviews.freebsd.org/D24618

sys/dev/pci/pci.c

 static int        rt_detach(device_t dev);
 static int        rt_shutdown(device_t dev);
 static int        rt_suspend(device_t dev);
 static int        rt_resume(device_t dev);
 static void        rt_init_locked(void *priv);
 static void        rt_init(void *priv);
 static void        rt_stop_locked(void *priv);
 static void        rt_stop(void *priv);
-static void        rt_start(struct ifnet *ifp);
-static int        rt_ioctl(struct ifnet *ifp, u_long cmd, caddr_t data);
+static void        rt_start(if_t ifp);
+static int        rt_ioctl(if_t ifp, u_long cmd, caddr_t data);
 static void        rt_tx_watchdog(void *arg);
 static void        rt_intr(void *arg);
 static void        rt_rt5350_intr(void *arg);
 static void        rt_tx_coherent_intr(struct rt_softc *sc);
 static void        rt_rx_coherent_intr(struct rt_softc *sc);
 static void        rt_rx_delay_intr(struct rt_softc *sc);
 static void        rt_tx_delay_intr(struct rt_softc *sc);
 static void        rt_rx_intr(struct rt_softc *sc, int qid);

 static void        rt_sysctl_attach(struct rt_softc *sc);
 #ifdef IF_RT_PHY_SUPPORT
 void            rt_miibus_statchg(device_t);
 #endif
 #if defined(IF_RT_PHY_SUPPORT) || defined(RT_MDIO)
 static int        rt_miibus_readreg(device_t, int, int);
 static int        rt_miibus_writereg(device_t, int, int, int);
 #endif
-static int        rt_ifmedia_upd(struct ifnet *);
-static void        rt_ifmedia_sts(struct ifnet *, struct ifmediareq *);
+static int        rt_ifmedia_upd(if_t );
+static void        rt_ifmedia_sts(if_t , struct ifmediareq *);
 
 static SYSCTL_NODE(_hw, OID_AUTO, rt, CTLFLAG_RD | CTLFLAG_MPSAFE, 0,
     "RT driver parameters");
 #ifdef IF_RT_DEBUG
 static int rt_debug = 0;
 SYSCTL_INT(_hw_rt, OID_AUTO, debug, CTLFLAG_RWTUN, &rt_debug, 0,
     "RT debug level");
 #endif
reset_freng(struct rt_softc *sc)
         /* XXX hard reset kills everything so skip it ... */
         return;
 }
 
 static int
 rt_attach(device_t dev)
 {
         struct rt_softc *sc;
-        struct ifnet *ifp;
+        if_t ifp;
         int error, i;
 #ifdef FDT
         phandle_t node;
         char fdtval[32];
 #endif
 
         sc = device_get_softc(dev);
         sc->dev = dev;
#endif
 
         ifp = sc->ifp = if_alloc(IFT_ETHER);
         if (ifp == NULL) {
                 device_printf(dev, "could not if_alloc()\n");
                 error = ENOMEM;
                 goto fail;
         }
 
-        ifp->if_softc = sc;
+        if_setsoftc(ifp, sc);
         if_initname(ifp, device_get_name(sc->dev), device_get_unit(sc->dev));
-        ifp->if_flags = IFF_BROADCAST | IFF_SIMPLEX | IFF_MULTICAST;
-        ifp->if_init = rt_init;
-        ifp->if_ioctl = rt_ioctl;
-        ifp->if_start = rt_start;
+        if_setflags(ifp, IFF_BROADCAST | IFF_SIMPLEX | IFF_MULTICAST);
+        if_setinitfn(ifp, rt_init);
+        if_setioctlfn(ifp, rt_ioctl);
+        if_setstartfn(ifp, rt_start);
 #define RT_TX_QLEN      256
 
-        IFQ_SET_MAXLEN(&ifp->if_snd, RT_TX_QLEN);
-        ifp->if_snd.ifq_drv_maxlen = RT_TX_QLEN;
-        IFQ_SET_READY(&ifp->if_snd);
+        if_setsendqlen(ifp, RT_TX_QLEN);
+        if_setsendqready(ifp);
 
 #ifdef IF_RT_PHY_SUPPORT
         error = mii_attach(dev, &sc->rt_miibus, ifp, rt_ifmedia_upd,
             rt_ifmedia_sts, BMSR_DEFCAPMASK, MII_PHY_ANY, MII_OFFSET_ANY, 0);
         if (error != 0) {
                 device_printf(dev, "attaching PHYs failed\n");
                 error = ENXIO;
                 goto fail;
         }
 #else
         ifmedia_init(&sc->rt_ifmedia, 0, rt_ifmedia_upd, rt_ifmedia_sts);
         ifmedia_add(&sc->rt_ifmedia, IFM_ETHER | IFM_100_TX | IFM_FDX, 0,
             NULL);
         ifmedia_set(&sc->rt_ifmedia, IFM_ETHER | IFM_100_TX | IFM_FDX);
 
 #endif /* IF_RT_PHY_SUPPORT */
 
         ether_request_mac(dev, sc->mac_addr);
         ether_ifattach(ifp, sc->mac_addr);
 
         /*
          * Tell the upper layer(s) we support long frames.
          */
-        ifp->if_hdrlen = sizeof(struct ether_vlan_header);
-        ifp->if_capabilities |= IFCAP_VLAN_MTU;
-        ifp->if_capenable |= IFCAP_VLAN_MTU;
-        ifp->if_capabilities |= IFCAP_RXCSUM|IFCAP_TXCSUM;
-        ifp->if_capenable |= IFCAP_RXCSUM|IFCAP_TXCSUM;
+        if_setifheaderlen(ifp, sizeof(struct ether_vlan_header));
+        if_setcapabilitiesbit(ifp, IFCAP_VLAN_MTU, 0);
+        if_setcapenablebit(ifp, IFCAP_VLAN_MTU, 0);
+        if_setcapabilitiesbit(ifp, IFCAP_RXCSUM|IFCAP_TXCSUM, 0);
+        if_setcapenablebit(ifp, IFCAP_RXCSUM|IFCAP_TXCSUM, 0);
 
         /* init task queue */
         NET_TASK_INIT(&sc->rx_done_task, 0, rt_rx_done_task, sc);
         TASK_INIT(&sc->tx_done_task, 0, rt_tx_done_task, sc);
 
         sc->rx_process_limit = 100;
 
         sc->taskqueue = taskqueue_create("rt_taskq", M_NOWAIT,
fail:
 
         return (error);
 }
 
 /*
  * Set media options.
  */
 static int
-rt_ifmedia_upd(struct ifnet *ifp)
+rt_ifmedia_upd(if_t ifp)
 {
         struct rt_softc *sc;
 #ifdef IF_RT_PHY_SUPPORT
         struct mii_data *mii;
         struct mii_softc *miisc;
         int error = 0;
 
-        sc = ifp->if_softc;
+        sc = if_getsoftc(ifp);
         RT_SOFTC_LOCK(sc);
 
         mii = device_get_softc(sc->rt_miibus);
         LIST_FOREACH(miisc, &mii->mii_phys, mii_list)
                 PHY_RESET(miisc);
         error = mii_mediachg(mii);
         RT_SOFTC_UNLOCK(sc);
 
         return (error);
 
 #else /* !IF_RT_PHY_SUPPORT */
 
         struct ifmedia *ifm;
         struct ifmedia_entry *ife;
 
-        sc = ifp->if_softc;
+        sc = if_getsoftc(ifp);
         ifm = &sc->rt_ifmedia;
         ife = ifm->ifm_cur;
 
         if (IFM_TYPE(ifm->ifm_media) != IFM_ETHER)
                 return (EINVAL);
 
         if (IFM_SUBTYPE(ife->ifm_media) == IFM_AUTO) {
                 device_printf(sc->dev,
                     "AUTO is not supported for multiphy MAC");
                 return (EINVAL);
         }
 
         /*
          * Ignore everything
          */
         return (0);
 #endif /* IF_RT_PHY_SUPPORT */
 }
 
 /*
  * Report current media status.
  */
 static void
-rt_ifmedia_sts(struct ifnet *ifp, struct ifmediareq *ifmr)
+rt_ifmedia_sts(if_t ifp, struct ifmediareq *ifmr)
 {
 #ifdef IF_RT_PHY_SUPPORT
         struct rt_softc *sc;
         struct mii_data *mii;
 
-        sc = ifp->if_softc;
+        sc = if_getsoftc(ifp);
 
         RT_SOFTC_LOCK(sc);
         mii = device_get_softc(sc->rt_miibus);
         mii_pollstat(mii);
         ifmr->ifm_active = mii->mii_media_active;
         ifmr->ifm_status = mii->mii_media_status;
         ifmr->ifm_active = IFM_ETHER | IFM_100_TX | IFM_FDX;
         ifmr->ifm_status = IFM_AVALID | IFM_ACTIVE;
         RT_SOFTC_UNLOCK(sc);
 #else /* !IF_RT_PHY_SUPPORT */
 
         ifmr->ifm_status = IFM_AVALID | IFM_ACTIVE;
         ifmr->ifm_active = IFM_ETHER | IFM_100_TX | IFM_FDX;
 #endif /* IF_RT_PHY_SUPPORT */
 }
 
 static int
 rt_detach(device_t dev)
 {
         struct rt_softc *sc;
-        struct ifnet *ifp;
+        if_t ifp;
         int i;
 
         sc = device_get_softc(dev);
         ifp = sc->ifp;
 
         RT_DPRINTF(sc, RT_DEBUG_ANY, "detaching\n");
 
         RT_SOFTC_LOCK(sc);
-        ifp->if_drv_flags &= ~(IFF_DRV_RUNNING | IFF_DRV_OACTIVE);
+        if_setdrvflagbits(ifp, 0, (IFF_DRV_RUNNING | IFF_DRV_OACTIVE));
         callout_stop(&sc->tx_watchdog_ch);
         RT_SOFTC_UNLOCK(sc);
 
         taskqueue_drain(sc->taskqueue, &sc->rx_done_task);
         taskqueue_drain(sc->taskqueue, &sc->tx_done_task);
         taskqueue_drain_timeout(sc->taskqueue, &sc->periodic_task);
 
         /* free Tx and Rx rings */
rt_suspend(device_t dev)
 
         return (0);
 }
 
 static int
 rt_resume(device_t dev)
 {
         struct rt_softc *sc;
-        struct ifnet *ifp;
+        if_t ifp;
 
         sc = device_get_softc(dev);
         ifp = sc->ifp;
 
         RT_DPRINTF(sc, RT_DEBUG_ANY, "resuming\n");
 
-        if (ifp->if_flags & IFF_UP)
+        if (if_getflags(ifp) & IFF_UP)
                 rt_init(sc);
 
         return (0);
 }
 
 /*
  * rt_init_locked - Run initialization process having locked mtx.
  */
 static void
 rt_init_locked(void *priv)
 {
         struct rt_softc *sc;
-        struct ifnet *ifp;
+        if_t ifp;
 #ifdef IF_RT_PHY_SUPPORT
         struct mii_data *mii;
 #endif
         int i, ntries;
         uint32_t tmp;
 
         sc = priv;
         ifp = sc->ifp;
#endif
 
         if (rt_txrx_enable(sc) != 0)
                 goto fail;
 
 #ifdef IF_RT_PHY_SUPPORT
         if (mii) mii_mediachg(mii);
 #endif /* IF_RT_PHY_SUPPORT */
 
-        ifp->if_drv_flags &= ~IFF_DRV_OACTIVE;
-        ifp->if_drv_flags |= IFF_DRV_RUNNING;
+        if_setdrvflagbits(ifp, 0, IFF_DRV_OACTIVE);
+        if_setdrvflagbits(ifp, IFF_DRV_RUNNING, 0);
 
         sc->periodic_round = 0;
 
         taskqueue_enqueue_timeout(sc->taskqueue, &sc->periodic_task, hz / 10);
 
         return;
 
 fail:

 
 /*
  * rt_stop_locked - stop TX/RX w/ lock
  */
 static void
 rt_stop_locked(void *priv)
 {
         struct rt_softc *sc;
-        struct ifnet *ifp;
+        if_t ifp;
 
         sc = priv;
         ifp = sc->ifp;
 
         RT_DPRINTF(sc, RT_DEBUG_ANY, "stopping\n");
 
         RT_SOFTC_ASSERT_LOCKED(sc);
         sc->tx_timer = 0;
-        ifp->if_drv_flags &= ~(IFF_DRV_RUNNING | IFF_DRV_OACTIVE);
+        if_setdrvflagbits(ifp, 0, (IFF_DRV_RUNNING | IFF_DRV_OACTIVE));
         callout_stop(&sc->tx_watchdog_ch);
         RT_SOFTC_UNLOCK(sc);
         taskqueue_block(sc->taskqueue);
 
         /*
          * Sometime rt_stop_locked called from isr and we get panic
          * When found, I fix it
          */

 }
 
 /*
  * rt_tx_data - transmit packet.
  */
 static int
 rt_tx_data(struct rt_softc *sc, struct mbuf *m, int qid)
 {
-        struct ifnet *ifp;
+        if_t ifp;
         struct rt_softc_tx_ring *ring;
         struct rt_softc_tx_data *data;
         struct rt_txdesc *desc;
         struct mbuf *m_d;
         bus_dma_segment_t dma_seg[RT_SOFTC_MAX_SCATTER];
         int error, ndmasegs, ndescs, i;
 
         KASSERT(qid >= 0 && qid < RT_SOFTC_TX_RING_COUNT,
for (i = 0; i < ndmasegs; i += 2) {
                  * one doesn't use word3 at all). And so does MT7621...
                  */
 
                 if (sc->rt_chipid != RT_CHIPID_MT7621) {
                         /* Set destination */
                         if (sc->rt_chipid != RT_CHIPID_MT7620)
                             desc->dst = (TXDSCR_DST_PORT_GDMA1);
 
-                        if ((ifp->if_capenable & IFCAP_TXCSUM) != 0)
+                        if ((if_getcapenable(ifp) & IFCAP_TXCSUM) != 0)
                                 desc->dst |= (TXDSCR_IP_CSUM_GEN |
                                     TXDSCR_UDP_CSUM_GEN | TXDSCR_TCP_CSUM_GEN);
                         /* Set queue id */
                         desc->qn = qid;
                         /* No PPPoE */
                         desc->pppoe = 0;
                         /* No VLAN */
                         desc->vid = 0;
rt_tx_data(struct rt_softc *sc, struct mbuf *m, int qid)
 
         return (0);
 }
 
 /*
  * rt_start - start Transmit/Receive
  */
 static void
-rt_start(struct ifnet *ifp)
+rt_start(if_t ifp)
 {
         struct rt_softc *sc;
         struct mbuf *m;
         int qid = 0 /* XXX must check QoS priority */;
 
-        sc = ifp->if_softc;
+        sc = if_getsoftc(ifp);
 
-        if (!(ifp->if_drv_flags & IFF_DRV_RUNNING))
+        if (!(if_getdrvflags(ifp) & IFF_DRV_RUNNING))
                 return;
 
         for (;;) {
-                IFQ_DRV_DEQUEUE(&ifp->if_snd, m);
+                m = if_dequeue(ifp);
                 if (m == NULL)
                         break;
 
                 m->m_pkthdr.rcvif = NULL;
 
                 RT_SOFTC_TX_RING_LOCK(&sc->tx_ring[qid]);
 
                 if (sc->tx_ring[qid].data_queued >=
                     RT_SOFTC_TX_RING_DATA_COUNT) {
                         RT_SOFTC_TX_RING_UNLOCK(&sc->tx_ring[qid]);
 
                         RT_DPRINTF(sc, RT_DEBUG_TX,
                             "if_start: Tx ring with qid=%d is full\n", qid);
 
                         m_freem(m);
 
-                        ifp->if_drv_flags |= IFF_DRV_OACTIVE;
+                        if_setdrvflagbits(ifp, IFF_DRV_OACTIVE, 0);
                         if_inc_counter(ifp, IFCOUNTER_OERRORS, 1);
 
                         sc->tx_data_queue_full[qid]++;
 
                         break;
                 }
 
                 if (rt_tx_data(sc, m, qid) != 0) {
rt_start(if_t ifp)
         }
 }
 
 /*
  * rt_update_promisc - set/clear promiscuous mode. Unused yet, because
  * filtering done by attached Ethernet switch.
  */
 static void
-rt_update_promisc(struct ifnet *ifp)
+rt_update_promisc(if_t ifp)
 {
         struct rt_softc *sc;
 
-        sc = ifp->if_softc;
+        sc = if_getsoftc(ifp);
         printf("%s: %s promiscuous mode\n",
                 device_get_nameunit(sc->dev),
-                (ifp->if_flags & IFF_PROMISC) ? "entering" : "leaving");
+                (if_getflags(ifp) & IFF_PROMISC) ? "entering" : "leaving");
 }
 
 /*
  * rt_ioctl - ioctl handler.
  */
 static int
-rt_ioctl(struct ifnet *ifp, u_long cmd, caddr_t data)
+rt_ioctl(if_t ifp, u_long cmd, caddr_t data)
 {
         struct rt_softc *sc;
         struct ifreq *ifr;
 #ifdef IF_RT_PHY_SUPPORT
         struct mii_data *mii;
 #endif /* IF_RT_PHY_SUPPORT */
         int error, startall;
 
-        sc = ifp->if_softc;
+        sc = if_getsoftc(ifp);
         ifr = (struct ifreq *) data;
 
         error = 0;
 
         switch (cmd) {
         case SIOCSIFFLAGS:
                 startall = 0;
                 RT_SOFTC_LOCK(sc);
-                if (ifp->if_flags & IFF_UP) {
-                        if (ifp->if_drv_flags & IFF_DRV_RUNNING) {
-                                if ((ifp->if_flags ^ sc->if_flags) &
+                if (if_getflags(ifp) & IFF_UP) {
+                        if (if_getdrvflags(ifp) & IFF_DRV_RUNNING) {
+                                if ((if_getflags(ifp) ^ sc->if_flags) &
                                     IFF_PROMISC)
                                         rt_update_promisc(ifp);
                         } else {
                                 rt_init_locked(sc);
                                 startall = 1;
                         }
                 } else {
-                        if (ifp->if_drv_flags & IFF_DRV_RUNNING)
+                        if (if_getdrvflags(ifp) & IFF_DRV_RUNNING)
                                 rt_stop_locked(sc);
                 }
-                sc->if_flags = ifp->if_flags;
+                sc->if_flags = if_getflags(ifp);
                 RT_SOFTC_UNLOCK(sc);
                 break;
         case SIOCGIFMEDIA:
         case SIOCSIFMEDIA:
 #ifdef IF_RT_PHY_SUPPORT
                 mii = device_get_softc(sc->rt_miibus);
                 error = ifmedia_ioctl(ifp, ifr, &mii->mii_media, cmd);
 #else

 
 /*
  * rt_tx_watchdog - Handler of TX Watchdog
  */
 static void
 rt_tx_watchdog(void *arg)
 {
         struct rt_softc *sc;
-        struct ifnet *ifp;
+        if_t ifp;
 
         sc = arg;
         ifp = sc->ifp;
 
         if (sc->tx_timer == 0)
                 return;
 
         if (--sc->tx_timer == 0) {

 
 /*
  * rt_intr - main ISR
  */
 static void
 rt_intr(void *arg)
 {
         struct rt_softc *sc;
-        struct ifnet *ifp;
+        if_t ifp;
         uint32_t status;
 
         sc = arg;
         ifp = sc->ifp;
 
         /* acknowledge interrupts */
         status = RT_READ(sc, sc->fe_int_status);
         RT_WRITE(sc, sc->fe_int_status, status);
 
         RT_DPRINTF(sc, RT_DEBUG_INTR, "interrupt: status=0x%08x\n", status);
 
         if (status == 0xffffffff ||        /* device likely went away */
                 status == 0)            /* not for us */
                 return;
 
         sc->interrupts++;
 
-        if (!(ifp->if_drv_flags & IFF_DRV_RUNNING))
+        if (!(if_getdrvflags(ifp) & IFF_DRV_RUNNING))
                 return;
 
         if (status & CNT_PPE_AF)
                 rt_cnt_ppe_af(sc);
 
         if (status & CNT_GDM_AF)
                 rt_cnt_gdm_af(sc);
 

 
 /*
  * rt_rt5350_intr - main ISR for Ralink 5350 SoC
  */
 static void
 rt_rt5350_intr(void *arg)
 {
         struct rt_softc *sc;
-        struct ifnet *ifp;
+        if_t ifp;
         uint32_t status;
 
         sc = arg;
         ifp = sc->ifp;
 
         /* acknowledge interrupts */
         status = RT_READ(sc, sc->fe_int_status);
         RT_WRITE(sc, sc->fe_int_status, status);
 
         RT_DPRINTF(sc, RT_DEBUG_INTR, "interrupt: status=0x%08x\n", status);
 
         if (status == 0xffffffff ||     /* device likely went away */
                 status == 0)            /* not for us */
                 return;
 
         sc->interrupts++;
 
-        if (!(ifp->if_drv_flags & IFF_DRV_RUNNING))
+        if (!(if_getdrvflags(ifp) & IFF_DRV_RUNNING))
                 return;
 
         if (status & RT5350_INT_TX_COHERENT)
                 rt_tx_coherent_intr(sc);
         if (status & RT5350_INT_RX_COHERENT)
                 rt_rx_coherent_intr(sc);
         if (status & RT5350_RX_DLY_INT)
                 rt_rx_delay_intr(sc);

 
 /*
  * rt_rx_done_task - run RX task
  */
 static void
 rt_rx_done_task(void *context, int pending)
 {
         struct rt_softc *sc;
-        struct ifnet *ifp;
+        if_t ifp;
         int again;
 
         sc = context;
         ifp = sc->ifp;
 
         RT_DPRINTF(sc, RT_DEBUG_RX, "Rx done task\n");
 
-        if (!(ifp->if_drv_flags & IFF_DRV_RUNNING))
+        if (!(if_getdrvflags(ifp) & IFF_DRV_RUNNING))
                 return;
 
         sc->intr_pending_mask &= ~sc->int_rx_done_mask;
 
         again = rt_rx_eof(sc, &sc->rx_ring[0], sc->rx_process_limit);
 
         RT_SOFTC_LOCK(sc);
 

 
 /*
  * rt_tx_done_task - check for pending TX task in all queues
  */
 static void
 rt_tx_done_task(void *context, int pending)
 {
         struct rt_softc *sc;
-        struct ifnet *ifp;
+        if_t ifp;
         uint32_t intr_mask;
         int i;
 
         sc = context;
         ifp = sc->ifp;
 
         RT_DPRINTF(sc, RT_DEBUG_TX, "Tx done task\n");
 
-        if (!(ifp->if_drv_flags & IFF_DRV_RUNNING))
+        if (!(if_getdrvflags(ifp) & IFF_DRV_RUNNING))
                 return;
 
         for (i = RT_SOFTC_TX_RING_COUNT - 1; i >= 0; i--) {
                 if (sc->intr_pending_mask & (sc->int_tx_done_mask << i)) {
                         sc->intr_pending_mask &= ~(sc->int_tx_done_mask << i);
                         rt_tx_eof(sc, &sc->tx_ring[i]);
                 }
         }
 
         sc->tx_timer = 0;
 
-        ifp->if_drv_flags &= ~IFF_DRV_OACTIVE;
+        if_setdrvflagbits(ifp, 0, IFF_DRV_OACTIVE);
 
         if(sc->rt_chipid == RT_CHIPID_RT5350 ||
            sc->rt_chipid == RT_CHIPID_MT7620 ||
            sc->rt_chipid == RT_CHIPID_MT7621)
           intr_mask = (
                 RT5350_INT_TXQ3_DONE |
                 RT5350_INT_TXQ2_DONE |
                 RT5350_INT_TXQ1_DONE |
rt_tx_done_task(void *context, int pending)
         if (sc->intr_pending_mask & intr_mask) {
                 RT_DPRINTF(sc, RT_DEBUG_TX,
                     "Tx done task: scheduling again\n");
                 taskqueue_enqueue(sc->taskqueue, &sc->tx_done_task);
         }
 
         RT_SOFTC_UNLOCK(sc);
 
-        if (!IFQ_IS_EMPTY(&ifp->if_snd))
+        if (!if_sendq_empty(ifp))
                 rt_start(ifp);
 }
 
 /*
  * rt_periodic_task - run periodic task
  */
 static void
 rt_periodic_task(void *context, int pending)
 {
         struct rt_softc *sc;
-        struct ifnet *ifp;
+        if_t ifp;
 
         sc = context;
         ifp = sc->ifp;
 
         RT_DPRINTF(sc, RT_DEBUG_PERIODIC, "periodic task: round=%lu\n",
             sc->periodic_round);
 
-        if (!(ifp->if_drv_flags & IFF_DRV_RUNNING))
+        if (!(if_getdrvflags(ifp) & IFF_DRV_RUNNING))
                 return;
 
         RT_SOFTC_LOCK(sc);
         sc->periodic_round++;
         rt_update_stats(sc);
 
         if ((sc->periodic_round % 10) == 0) {
                 rt_update_raw_counters(sc);
                 rt_watchdog(sc);
         }
 
         RT_SOFTC_UNLOCK(sc);
         taskqueue_enqueue_timeout(sc->taskqueue, &sc->periodic_task, hz / 10);
 }
 
 /*
  * rt_rx_eof - check for frames that done by DMA engine and pass it into
  * network subsystem.
  */
 static int
 rt_rx_eof(struct rt_softc *sc, struct rt_softc_rx_ring *ring, int limit)
 {
-        struct ifnet *ifp;
+        if_t ifp;
 /*      struct rt_softc_rx_ring *ring; */
         struct rt_rxdesc *desc;
         struct rt_softc_rx_data *data;
         struct mbuf *m, *mnew;
         bus_dma_segment_t segs[1];
         bus_dmamap_t dma_map;
         uint32_t index, desc_flags;
         int error, nsegs, len, nframes;
RT_DPRINTF(sc, RT_DEBUG_RX,
                     "Rx frame: rxdesc flags=0x%08x\n", desc_flags);
 
                 m->m_pkthdr.rcvif = ifp;
                 /* Add 2 to fix data align, after sdp0 = addr + 2 */
                 m->m_data += 2;
                 m->m_pkthdr.len = m->m_len = len;
 
                 /* check for crc errors */
-                if ((ifp->if_capenable & IFCAP_RXCSUM) != 0) {
+                if ((if_getcapenable(ifp) & IFCAP_RXCSUM) != 0) {
                         /*check for valid checksum*/
                         if (desc_flags & (sc->csum_fail_ip|sc->csum_fail_l4)) {
                                 RT_DPRINTF(sc, RT_DEBUG_RX,
                                     "rxdesc: crc error\n");
 
                                 if_inc_counter(ifp, IFCOUNTER_IERRORS, 1);
 
-                                if (!(ifp->if_flags & IFF_PROMISC)) {
+                                if (!(if_getflags(ifp) & IFF_PROMISC)) {
                                     m_freem(m);
                                     goto skip;
                                 }
                         }
                         if ((desc_flags & sc->csum_fail_ip) == 0) {
                                 m->m_pkthdr.csum_flags |= CSUM_IP_CHECKED;
                                 m->m_pkthdr.csum_flags |= CSUM_IP_VALID;
                                 m->m_pkthdr.csum_data = 0xffff;
                         }
                         m->m_flags &= ~M_HASFCS;
                 }
 
-                (*ifp->if_input)(ifp, m);
+                if_input(ifp, m);
 skip:
                 desc->sdl0 &= ~htole16(RT_RXDESC_SDL0_DDONE);
 
                 bus_dmamap_sync(ring->desc_dma_tag, ring->desc_dma_map,
                         BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE);
 
                 ring->cur = (ring->cur + 1) % RT_SOFTC_RX_RING_DATA_COUNT;
 

 
 /*
  * rt_tx_eof - check for successful transmitted frames and mark their
  * descriptor as free.
  */
 static void
 rt_tx_eof(struct rt_softc *sc, struct rt_softc_tx_ring *ring)
 {
-        struct ifnet *ifp;
+        if_t ifp;
         struct rt_txdesc *desc;
         struct rt_softc_tx_data *data;
         uint32_t index;
         int ndescs, nframes;
 
         ifp = sc->ifp;
 
         ndescs = 0;

 }
 
 /*
  * rt_update_stats - query statistics counters and update related variables.
  */
 static void
 rt_update_stats(struct rt_softc *sc)
 {
-        struct ifnet *ifp;
+        if_t ifp;
 
         ifp = sc->ifp;
         RT_DPRINTF(sc, RT_DEBUG_STATS, "update statistic: \n");
         /* XXX do update stats here */
 }
 
 /*
  * rt_watchdog - reinit device on watchdog event.

 }
 
 /*
  * rt_txrx_enable - enable TX/RX DMA
  */
 static int
 rt_txrx_enable(struct rt_softc *sc)
 {
-        struct ifnet *ifp;
+        if_t ifp;
         uint32_t tmp;
         int ntries;
 
         ifp = sc->ifp;
 
         /* enable Tx/Rx DMA engine */
         for (ntries = 0; ntries < 200; ntries++) {
                 tmp = RT_READ(sc, sc->pdma_glo_cfg);

sys/dev/pci/pcivar.h

 static int        rt_detach(device_t dev);
 static int        rt_shutdown(device_t dev);
 static int        rt_suspend(device_t dev);
 static int        rt_resume(device_t dev);
 static void        rt_init_locked(void *priv);
 static void        rt_init(void *priv);
 static void        rt_stop_locked(void *priv);
 static void        rt_stop(void *priv);
-static void        rt_start(struct ifnet *ifp);
-static int        rt_ioctl(struct ifnet *ifp, u_long cmd, caddr_t data);
+static void        rt_start(if_t ifp);
+static int        rt_ioctl(if_t ifp, u_long cmd, caddr_t data);
 static void        rt_tx_watchdog(void *arg);
 static void        rt_intr(void *arg);
 static void        rt_rt5350_intr(void *arg);
 static void        rt_tx_coherent_intr(struct rt_softc *sc);
 static void        rt_rx_coherent_intr(struct rt_softc *sc);
 static void        rt_rx_delay_intr(struct rt_softc *sc);
 static void        rt_tx_delay_intr(struct rt_softc *sc);
 static void        rt_rx_intr(struct rt_softc *sc, int qid);

 static void        rt_sysctl_attach(struct rt_softc *sc);
 #ifdef IF_RT_PHY_SUPPORT
 void            rt_miibus_statchg(device_t);
 #endif
 #if defined(IF_RT_PHY_SUPPORT) || defined(RT_MDIO)
 static int        rt_miibus_readreg(device_t, int, int);
 static int        rt_miibus_writereg(device_t, int, int, int);
 #endif
-static int        rt_ifmedia_upd(struct ifnet *);
-static void        rt_ifmedia_sts(struct ifnet *, struct ifmediareq *);
+static int        rt_ifmedia_upd(if_t );
+static void        rt_ifmedia_sts(if_t , struct ifmediareq *);
 
 static SYSCTL_NODE(_hw, OID_AUTO, rt, CTLFLAG_RD | CTLFLAG_MPSAFE, 0,
     "RT driver parameters");
 #ifdef IF_RT_DEBUG
 static int rt_debug = 0;
 SYSCTL_INT(_hw_rt, OID_AUTO, debug, CTLFLAG_RWTUN, &rt_debug, 0,
     "RT debug level");
 #endif
reset_freng(struct rt_softc *sc)
         /* XXX hard reset kills everything so skip it ... */
         return;
 }
 
 static int
 rt_attach(device_t dev)
 {
         struct rt_softc *sc;
-        struct ifnet *ifp;
+        if_t ifp;
         int error, i;
 #ifdef FDT
         phandle_t node;
         char fdtval[32];
 #endif
 
         sc = device_get_softc(dev);
         sc->dev = dev;
#endif
 
         ifp = sc->ifp = if_alloc(IFT_ETHER);
         if (ifp == NULL) {
                 device_printf(dev, "could not if_alloc()\n");
                 error = ENOMEM;
                 goto fail;
         }
 
-        ifp->if_softc = sc;
+        if_setsoftc(ifp, sc);
         if_initname(ifp, device_get_name(sc->dev), device_get_unit(sc->dev));
-        ifp->if_flags = IFF_BROADCAST | IFF_SIMPLEX | IFF_MULTICAST;
-        ifp->if_init = rt_init;
-        ifp->if_ioctl = rt_ioctl;
-        ifp->if_start = rt_start;
+        if_setflags(ifp, IFF_BROADCAST | IFF_SIMPLEX | IFF_MULTICAST);
+        if_setinitfn(ifp, rt_init);
+        if_setioctlfn(ifp, rt_ioctl);
+        if_setstartfn(ifp, rt_start);
 #define RT_TX_QLEN      256
 
-        IFQ_SET_MAXLEN(&ifp->if_snd, RT_TX_QLEN);
-        ifp->if_snd.ifq_drv_maxlen = RT_TX_QLEN;
-        IFQ_SET_READY(&ifp->if_snd);
+        if_setsendqlen(ifp, RT_TX_QLEN);
+        if_setsendqready(ifp);
 
 #ifdef IF_RT_PHY_SUPPORT
         error = mii_attach(dev, &sc->rt_miibus, ifp, rt_ifmedia_upd,
             rt_ifmedia_sts, BMSR_DEFCAPMASK, MII_PHY_ANY, MII_OFFSET_ANY, 0);
         if (error != 0) {
                 device_printf(dev, "attaching PHYs failed\n");
                 error = ENXIO;
                 goto fail;
         }
 #else
         ifmedia_init(&sc->rt_ifmedia, 0, rt_ifmedia_upd, rt_ifmedia_sts);
         ifmedia_add(&sc->rt_ifmedia, IFM_ETHER | IFM_100_TX | IFM_FDX, 0,
             NULL);
         ifmedia_set(&sc->rt_ifmedia, IFM_ETHER | IFM_100_TX | IFM_FDX);
 
 #endif /* IF_RT_PHY_SUPPORT */
 
         ether_request_mac(dev, sc->mac_addr);
         ether_ifattach(ifp, sc->mac_addr);
 
         /*
          * Tell the upper layer(s) we support long frames.
          */
-        ifp->if_hdrlen = sizeof(struct ether_vlan_header);
-        ifp->if_capabilities |= IFCAP_VLAN_MTU;
-        ifp->if_capenable |= IFCAP_VLAN_MTU;
-        ifp->if_capabilities |= IFCAP_RXCSUM|IFCAP_TXCSUM;
-        ifp->if_capenable |= IFCAP_RXCSUM|IFCAP_TXCSUM;
+        if_setifheaderlen(ifp, sizeof(struct ether_vlan_header));
+        if_setcapabilitiesbit(ifp, IFCAP_VLAN_MTU, 0);
+        if_setcapenablebit(ifp, IFCAP_VLAN_MTU, 0);
+        if_setcapabilitiesbit(ifp, IFCAP_RXCSUM|IFCAP_TXCSUM, 0);
+        if_setcapenablebit(ifp, IFCAP_RXCSUM|IFCAP_TXCSUM, 0);
 
         /* init task queue */
         NET_TASK_INIT(&sc->rx_done_task, 0, rt_rx_done_task, sc);
         TASK_INIT(&sc->tx_done_task, 0, rt_tx_done_task, sc);
 
         sc->rx_process_limit = 100;
 
         sc->taskqueue = taskqueue_create("rt_taskq", M_NOWAIT,
fail:
 
         return (error);
 }
 
 /*
  * Set media options.
  */
 static int
-rt_ifmedia_upd(struct ifnet *ifp)
+rt_ifmedia_upd(if_t ifp)
 {
         struct rt_softc *sc;
 #ifdef IF_RT_PHY_SUPPORT
         struct mii_data *mii;
         struct mii_softc *miisc;
         int error = 0;
 
-        sc = ifp->if_softc;
+        sc = if_getsoftc(ifp);
         RT_SOFTC_LOCK(sc);
 
         mii = device_get_softc(sc->rt_miibus);
         LIST_FOREACH(miisc, &mii->mii_phys, mii_list)
                 PHY_RESET(miisc);
         error = mii_mediachg(mii);
         RT_SOFTC_UNLOCK(sc);
 
         return (error);
 
 #else /* !IF_RT_PHY_SUPPORT */
 
         struct ifmedia *ifm;
         struct ifmedia_entry *ife;
 
-        sc = ifp->if_softc;
+        sc = if_getsoftc(ifp);
         ifm = &sc->rt_ifmedia;
         ife = ifm->ifm_cur;
 
         if (IFM_TYPE(ifm->ifm_media) != IFM_ETHER)
                 return (EINVAL);
 
         if (IFM_SUBTYPE(ife->ifm_media) == IFM_AUTO) {
                 device_printf(sc->dev,
                     "AUTO is not supported for multiphy MAC");
                 return (EINVAL);
         }
 
         /*
          * Ignore everything
          */
         return (0);
 #endif /* IF_RT_PHY_SUPPORT */
 }
 
 /*
  * Report current media status.
  */
 static void
-rt_ifmedia_sts(struct ifnet *ifp, struct ifmediareq *ifmr)
+rt_ifmedia_sts(if_t ifp, struct ifmediareq *ifmr)
 {
 #ifdef IF_RT_PHY_SUPPORT
         struct rt_softc *sc;
         struct mii_data *mii;
 
-        sc = ifp->if_softc;
+        sc = if_getsoftc(ifp);
 
         RT_SOFTC_LOCK(sc);
         mii = device_get_softc(sc->rt_miibus);
         mii_pollstat(mii);
         ifmr->ifm_active = mii->mii_media_active;
         ifmr->ifm_status = mii->mii_media_status;
         ifmr->ifm_active = IFM_ETHER | IFM_100_TX | IFM_FDX;
         ifmr->ifm_status = IFM_AVALID | IFM_ACTIVE;
         RT_SOFTC_UNLOCK(sc);
 #else /* !IF_RT_PHY_SUPPORT */
 
         ifmr->ifm_status = IFM_AVALID | IFM_ACTIVE;
         ifmr->ifm_active = IFM_ETHER | IFM_100_TX | IFM_FDX;
 #endif /* IF_RT_PHY_SUPPORT */
 }
 
 static int
 rt_detach(device_t dev)
 {
         struct rt_softc *sc;
-        struct ifnet *ifp;
+        if_t ifp;
         int i;
 
         sc = device_get_softc(dev);
         ifp = sc->ifp;
 
         RT_DPRINTF(sc, RT_DEBUG_ANY, "detaching\n");
 
         RT_SOFTC_LOCK(sc);
-        ifp->if_drv_flags &= ~(IFF_DRV_RUNNING | IFF_DRV_OACTIVE);
+        if_setdrvflagbits(ifp, 0, (IFF_DRV_RUNNING | IFF_DRV_OACTIVE));
         callout_stop(&sc->tx_watchdog_ch);
         RT_SOFTC_UNLOCK(sc);
 
         taskqueue_drain(sc->taskqueue, &sc->rx_done_task);
         taskqueue_drain(sc->taskqueue, &sc->tx_done_task);
         taskqueue_drain_timeout(sc->taskqueue, &sc->periodic_task);
 
         /* free Tx and Rx rings */
rt_suspend(device_t dev)
 
         return (0);
 }
 
 static int
 rt_resume(device_t dev)
 {
         struct rt_softc *sc;
-        struct ifnet *ifp;
+        if_t ifp;
 
         sc = device_get_softc(dev);
         ifp = sc->ifp;
 
         RT_DPRINTF(sc, RT_DEBUG_ANY, "resuming\n");
 
-        if (ifp->if_flags & IFF_UP)
+        if (if_getflags(ifp) & IFF_UP)
                 rt_init(sc);
 
         return (0);
 }
 
 /*
  * rt_init_locked - Run initialization process having locked mtx.
  */
 static void
 rt_init_locked(void *priv)
 {
         struct rt_softc *sc;
-        struct ifnet *ifp;
+        if_t ifp;
 #ifdef IF_RT_PHY_SUPPORT
         struct mii_data *mii;
 #endif
         int i, ntries;
         uint32_t tmp;
 
         sc = priv;
         ifp = sc->ifp;
#endif
 
         if (rt_txrx_enable(sc) != 0)
                 goto fail;
 
 #ifdef IF_RT_PHY_SUPPORT
         if (mii) mii_mediachg(mii);
 #endif /* IF_RT_PHY_SUPPORT */
 
-        ifp->if_drv_flags &= ~IFF_DRV_OACTIVE;
-        ifp->if_drv_flags |= IFF_DRV_RUNNING;
+        if_setdrvflagbits(ifp, 0, IFF_DRV_OACTIVE);
+        if_setdrvflagbits(ifp, IFF_DRV_RUNNING, 0);
 
         sc->periodic_round = 0;
 
         taskqueue_enqueue_timeout(sc->taskqueue, &sc->periodic_task, hz / 10);
 
         return;
 
 fail:

 
 /*
  * rt_stop_locked - stop TX/RX w/ lock
  */
 static void
 rt_stop_locked(void *priv)
 {
         struct rt_softc *sc;
-        struct ifnet *ifp;
+        if_t ifp;
 
         sc = priv;
         ifp = sc->ifp;
 
         RT_DPRINTF(sc, RT_DEBUG_ANY, "stopping\n");
 
         RT_SOFTC_ASSERT_LOCKED(sc);
         sc->tx_timer = 0;
-        ifp->if_drv_flags &= ~(IFF_DRV_RUNNING | IFF_DRV_OACTIVE);
+        if_setdrvflagbits(ifp, 0, (IFF_DRV_RUNNING | IFF_DRV_OACTIVE));
         callout_stop(&sc->tx_watchdog_ch);
         RT_SOFTC_UNLOCK(sc);
         taskqueue_block(sc->taskqueue);
 
         /*
          * Sometime rt_stop_locked called from isr and we get panic
          * When found, I fix it
          */

 }
 
 /*
  * rt_tx_data - transmit packet.
  */
 static int
 rt_tx_data(struct rt_softc *sc, struct mbuf *m, int qid)
 {
-        struct ifnet *ifp;
+        if_t ifp;
         struct rt_softc_tx_ring *ring;
         struct rt_softc_tx_data *data;
         struct rt_txdesc *desc;
         struct mbuf *m_d;
         bus_dma_segment_t dma_seg[RT_SOFTC_MAX_SCATTER];
         int error, ndmasegs, ndescs, i;
 
         KASSERT(qid >= 0 && qid < RT_SOFTC_TX_RING_COUNT,
for (i = 0; i < ndmasegs; i += 2) {
                  * one doesn't use word3 at all). And so does MT7621...
                  */
 
                 if (sc->rt_chipid != RT_CHIPID_MT7621) {
                         /* Set destination */
                         if (sc->rt_chipid != RT_CHIPID_MT7620)
                             desc->dst = (TXDSCR_DST_PORT_GDMA1);
 
-                        if ((ifp->if_capenable & IFCAP_TXCSUM) != 0)
+                        if ((if_getcapenable(ifp) & IFCAP_TXCSUM) != 0)
                                 desc->dst |= (TXDSCR_IP_CSUM_GEN |
                                     TXDSCR_UDP_CSUM_GEN | TXDSCR_TCP_CSUM_GEN);
                         /* Set queue id */
                         desc->qn = qid;
                         /* No PPPoE */
                         desc->pppoe = 0;
                         /* No VLAN */
                         desc->vid = 0;
rt_tx_data(struct rt_softc *sc, struct mbuf *m, int qid)
 
         return (0);
 }
 
 /*
  * rt_start - start Transmit/Receive
  */
 static void
-rt_start(struct ifnet *ifp)
+rt_start(if_t ifp)
 {
         struct rt_softc *sc;
         struct mbuf *m;
         int qid = 0 /* XXX must check QoS priority */;
 
-        sc = ifp->if_softc;
+        sc = if_getsoftc(ifp);
 
-        if (!(ifp->if_drv_flags & IFF_DRV_RUNNING))
+        if (!(if_getdrvflags(ifp) & IFF_DRV_RUNNING))
                 return;
 
         for (;;) {
-                IFQ_DRV_DEQUEUE(&ifp->if_snd, m);
+                m = if_dequeue(ifp);
                 if (m == NULL)
                         break;
 
                 m->m_pkthdr.rcvif = NULL;
 
                 RT_SOFTC_TX_RING_LOCK(&sc->tx_ring[qid]);
 
                 if (sc->tx_ring[qid].data_queued >=
                     RT_SOFTC_TX_RING_DATA_COUNT) {
                         RT_SOFTC_TX_RING_UNLOCK(&sc->tx_ring[qid]);
 
                         RT_DPRINTF(sc, RT_DEBUG_TX,
                             "if_start: Tx ring with qid=%d is full\n", qid);
 
                         m_freem(m);
 
-                        ifp->if_drv_flags |= IFF_DRV_OACTIVE;
+                        if_setdrvflagbits(ifp, IFF_DRV_OACTIVE, 0);
                         if_inc_counter(ifp, IFCOUNTER_OERRORS, 1);
 
                         sc->tx_data_queue_full[qid]++;
 
                         break;
                 }
 
                 if (rt_tx_data(sc, m, qid) != 0) {
rt_start(if_t ifp)
         }
 }
 
 /*
  * rt_update_promisc - set/clear promiscuous mode. Unused yet, because
  * filtering done by attached Ethernet switch.
  */
 static void
-rt_update_promisc(struct ifnet *ifp)
+rt_update_promisc(if_t ifp)
 {
         struct rt_softc *sc;
 
-        sc = ifp->if_softc;
+        sc = if_getsoftc(ifp);
         printf("%s: %s promiscuous mode\n",
                 device_get_nameunit(sc->dev),
-                (ifp->if_flags & IFF_PROMISC) ? "entering" : "leaving");
+                (if_getflags(ifp) & IFF_PROMISC) ? "entering" : "leaving");
 }
 
 /*
  * rt_ioctl - ioctl handler.
  */
 static int
-rt_ioctl(struct ifnet *ifp, u_long cmd, caddr_t data)
+rt_ioctl(if_t ifp, u_long cmd, caddr_t data)
 {
         struct rt_softc *sc;
         struct ifreq *ifr;
 #ifdef IF_RT_PHY_SUPPORT
         struct mii_data *mii;
 #endif /* IF_RT_PHY_SUPPORT */
         int error, startall;
 
-        sc = ifp->if_softc;
+        sc = if_getsoftc(ifp);
         ifr = (struct ifreq *) data;
 
         error = 0;
 
         switch (cmd) {
         case SIOCSIFFLAGS:
                 startall = 0;
                 RT_SOFTC_LOCK(sc);
-                if (ifp->if_flags & IFF_UP) {
-                        if (ifp->if_drv_flags & IFF_DRV_RUNNING) {
-                                if ((ifp->if_flags ^ sc->if_flags) &
+                if (if_getflags(ifp) & IFF_UP) {
+                        if (if_getdrvflags(ifp) & IFF_DRV_RUNNING) {
+                                if ((if_getflags(ifp) ^ sc->if_flags) &
                                     IFF_PROMISC)
                                         rt_update_promisc(ifp);
                         } else {
                                 rt_init_locked(sc);
                                 startall = 1;
                         }
                 } else {
-                        if (ifp->if_drv_flags & IFF_DRV_RUNNING)
+                        if (if_getdrvflags(ifp) & IFF_DRV_RUNNING)
                                 rt_stop_locked(sc);
                 }
-                sc->if_flags = ifp->if_flags;
+                sc->if_flags = if_getflags(ifp);
                 RT_SOFTC_UNLOCK(sc);
                 break;
         case SIOCGIFMEDIA:
         case SIOCSIFMEDIA:
 #ifdef IF_RT_PHY_SUPPORT
                 mii = device_get_softc(sc->rt_miibus);
                 error = ifmedia_ioctl(ifp, ifr, &mii->mii_media, cmd);
 #else

 
 /*
  * rt_tx_watchdog - Handler of TX Watchdog
  */
 static void
 rt_tx_watchdog(void *arg)
 {
         struct rt_softc *sc;
-        struct ifnet *ifp;
+        if_t ifp;
 
         sc = arg;
         ifp = sc->ifp;
 
         if (sc->tx_timer == 0)
                 return;
 
         if (--sc->tx_timer == 0) {

 
 /*
  * rt_intr - main ISR
  */
 static void
 rt_intr(void *arg)
 {
         struct rt_softc *sc;
-        struct ifnet *ifp;
+        if_t ifp;
         uint32_t status;
 
         sc = arg;
         ifp = sc->ifp;
 
         /* acknowledge interrupts */
         status = RT_READ(sc, sc->fe_int_status);
         RT_WRITE(sc, sc->fe_int_status, status);
 
         RT_DPRINTF(sc, RT_DEBUG_INTR, "interrupt: status=0x%08x\n", status);
 
         if (status == 0xffffffff ||        /* device likely went away */
                 status == 0)            /* not for us */
                 return;
 
         sc->interrupts++;
 
-        if (!(ifp->if_drv_flags & IFF_DRV_RUNNING))
+        if (!(if_getdrvflags(ifp) & IFF_DRV_RUNNING))
                 return;
 
         if (status & CNT_PPE_AF)
                 rt_cnt_ppe_af(sc);
 
         if (status & CNT_GDM_AF)
                 rt_cnt_gdm_af(sc);
 

 
 /*
  * rt_rt5350_intr - main ISR for Ralink 5350 SoC
  */
 static void
 rt_rt5350_intr(void *arg)
 {
         struct rt_softc *sc;
-        struct ifnet *ifp;
+        if_t ifp;
         uint32_t status;
 
         sc = arg;
         ifp = sc->ifp;
 
         /* acknowledge interrupts */
         status = RT_READ(sc, sc->fe_int_status);
         RT_WRITE(sc, sc->fe_int_status, status);
 
         RT_DPRINTF(sc, RT_DEBUG_INTR, "interrupt: status=0x%08x\n", status);
 
         if (status == 0xffffffff ||     /* device likely went away */
                 status == 0)            /* not for us */
                 return;
 
         sc->interrupts++;
 
-        if (!(ifp->if_drv_flags & IFF_DRV_RUNNING))
+        if (!(if_getdrvflags(ifp) & IFF_DRV_RUNNING))
                 return;
 
         if (status & RT5350_INT_TX_COHERENT)
                 rt_tx_coherent_intr(sc);
         if (status & RT5350_INT_RX_COHERENT)
                 rt_rx_coherent_intr(sc);
         if (status & RT5350_RX_DLY_INT)
                 rt_rx_delay_intr(sc);

 
 /*
  * rt_rx_done_task - run RX task
  */
 static void
 rt_rx_done_task(void *context, int pending)
 {
         struct rt_softc *sc;
-        struct ifnet *ifp;
+        if_t ifp;
         int again;
 
         sc = context;
         ifp = sc->ifp;
 
         RT_DPRINTF(sc, RT_DEBUG_RX, "Rx done task\n");
 
-        if (!(ifp->if_drv_flags & IFF_DRV_RUNNING))
+        if (!(if_getdrvflags(ifp) & IFF_DRV_RUNNING))
                 return;
 
         sc->intr_pending_mask &= ~sc->int_rx_done_mask;
 
         again = rt_rx_eof(sc, &sc->rx_ring[0], sc->rx_process_limit);
 
         RT_SOFTC_LOCK(sc);
 

 
 /*
  * rt_tx_done_task - check for pending TX task in all queues
  */
 static void
 rt_tx_done_task(void *context, int pending)
 {
         struct rt_softc *sc;
-        struct ifnet *ifp;
+        if_t ifp;
         uint32_t intr_mask;
         int i;
 
         sc = context;
         ifp = sc->ifp;
 
         RT_DPRINTF(sc, RT_DEBUG_TX, "Tx done task\n");
 
-        if (!(ifp->if_drv_flags & IFF_DRV_RUNNING))
+        if (!(if_getdrvflags(ifp) & IFF_DRV_RUNNING))
                 return;
 
         for (i = RT_SOFTC_TX_RING_COUNT - 1; i >= 0; i--) {
                 if (sc->intr_pending_mask & (sc->int_tx_done_mask << i)) {
                         sc->intr_pending_mask &= ~(sc->int_tx_done_mask << i);
                         rt_tx_eof(sc, &sc->tx_ring[i]);
                 }
         }
 
         sc->tx_timer = 0;
 
-        ifp->if_drv_flags &= ~IFF_DRV_OACTIVE;
+        if_setdrvflagbits(ifp, 0, IFF_DRV_OACTIVE);
 
         if(sc->rt_chipid == RT_CHIPID_RT5350 ||
            sc->rt_chipid == RT_CHIPID_MT7620 ||
            sc->rt_chipid == RT_CHIPID_MT7621)
           intr_mask = (
                 RT5350_INT_TXQ3_DONE |
                 RT5350_INT_TXQ2_DONE |
                 RT5350_INT_TXQ1_DONE |
rt_tx_done_task(void *context, int pending)
         if (sc->intr_pending_mask & intr_mask) {
                 RT_DPRINTF(sc, RT_DEBUG_TX,
                     "Tx done task: scheduling again\n");
                 taskqueue_enqueue(sc->taskqueue, &sc->tx_done_task);
         }
 
         RT_SOFTC_UNLOCK(sc);
 
-        if (!IFQ_IS_EMPTY(&ifp->if_snd))
+        if (!if_sendq_empty(ifp))
                 rt_start(ifp);
 }
 
 /*
  * rt_periodic_task - run periodic task
  */
 static void
 rt_periodic_task(void *context, int pending)
 {
         struct rt_softc *sc;
-        struct ifnet *ifp;
+        if_t ifp;
 
         sc = context;
         ifp = sc->ifp;
 
         RT_DPRINTF(sc, RT_DEBUG_PERIODIC, "periodic task: round=%lu\n",
             sc->periodic_round);
 
-        if (!(ifp->if_drv_flags & IFF_DRV_RUNNING))
+        if (!(if_getdrvflags(ifp) & IFF_DRV_RUNNING))
                 return;
 
         RT_SOFTC_LOCK(sc);
         sc->periodic_round++;
         rt_update_stats(sc);
 
         if ((sc->periodic_round % 10) == 0) {
                 rt_update_raw_counters(sc);
                 rt_watchdog(sc);
         }
 
         RT_SOFTC_UNLOCK(sc);
         taskqueue_enqueue_timeout(sc->taskqueue, &sc->periodic_task, hz / 10);
 }
 
 /*
  * rt_rx_eof - check for frames that done by DMA engine and pass it into
  * network subsystem.
  */
 static int
 rt_rx_eof(struct rt_softc *sc, struct rt_softc_rx_ring *ring, int limit)
 {
-        struct ifnet *ifp;
+        if_t ifp;
 /*      struct rt_softc_rx_ring *ring; */
         struct rt_rxdesc *desc;
         struct rt_softc_rx_data *data;
         struct mbuf *m, *mnew;
         bus_dma_segment_t segs[1];
         bus_dmamap_t dma_map;
         uint32_t index, desc_flags;
         int error, nsegs, len, nframes;
RT_DPRINTF(sc, RT_DEBUG_RX,
                     "Rx frame: rxdesc flags=0x%08x\n", desc_flags);
 
                 m->m_pkthdr.rcvif = ifp;
                 /* Add 2 to fix data align, after sdp0 = addr + 2 */
                 m->m_data += 2;
                 m->m_pkthdr.len = m->m_len = len;
 
                 /* check for crc errors */
-                if ((ifp->if_capenable & IFCAP_RXCSUM) != 0) {
+                if ((if_getcapenable(ifp) & IFCAP_RXCSUM) != 0) {
                         /*check for valid checksum*/
                         if (desc_flags & (sc->csum_fail_ip|sc->csum_fail_l4)) {
                                 RT_DPRINTF(sc, RT_DEBUG_RX,
                                     "rxdesc: crc error\n");
 
                                 if_inc_counter(ifp, IFCOUNTER_IERRORS, 1);
 
-                                if (!(ifp->if_flags & IFF_PROMISC)) {
+                                if (!(if_getflags(ifp) & IFF_PROMISC)) {
                                     m_freem(m);
                                     goto skip;
                                 }
                         }
                         if ((desc_flags & sc->csum_fail_ip) == 0) {
                                 m->m_pkthdr.csum_flags |= CSUM_IP_CHECKED;
                                 m->m_pkthdr.csum_flags |= CSUM_IP_VALID;
                                 m->m_pkthdr.csum_data = 0xffff;
                         }
                         m->m_flags &= ~M_HASFCS;
                 }
 
-                (*ifp->if_input)(ifp, m);
+                if_input(ifp, m);
 skip:
                 desc->sdl0 &= ~htole16(RT_RXDESC_SDL0_DDONE);
 
                 bus_dmamap_sync(ring->desc_dma_tag, ring->desc_dma_map,
                         BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE);
 
                 ring->cur = (ring->cur + 1) % RT_SOFTC_RX_RING_DATA_COUNT;
 

 
 /*
  * rt_tx_eof - check for successful transmitted frames and mark their
  * descriptor as free.
  */
 static void
 rt_tx_eof(struct rt_softc *sc, struct rt_softc_tx_ring *ring)
 {
-        struct ifnet *ifp;
+        if_t ifp;
         struct rt_txdesc *desc;
         struct rt_softc_tx_data *data;
         uint32_t index;
         int ndescs, nframes;
 
         ifp = sc->ifp;
 
         ndescs = 0;

 }
 
 /*
  * rt_update_stats - query statistics counters and update related variables.
  */
 static void
 rt_update_stats(struct rt_softc *sc)
 {
-        struct ifnet *ifp;
+        if_t ifp;
 
         ifp = sc->ifp;
         RT_DPRINTF(sc, RT_DEBUG_STATS, "update statistic: \n");
         /* XXX do update stats here */
 }
 
 /*
  * rt_watchdog - reinit device on watchdog event.

 }
 
 /*
  * rt_txrx_enable - enable TX/RX DMA
  */
 static int
 rt_txrx_enable(struct rt_softc *sc)
 {
-        struct ifnet *ifp;
+        if_t ifp;
         uint32_t tmp;
         int ntries;
 
         ifp = sc->ifp;
 
         /* enable Tx/Rx DMA engine */
         for (ntries = 0; ntries < 200; ntries++) {
                 tmp = RT_READ(sc, sc->pdma_glo_cfg);