Index: head/share/man/man4/alc.4 =================================================================== --- head/share/man/man4/alc.4 (revision 312249) +++ head/share/man/man4/alc.4 (revision 312250) @@ -1,179 +1,181 @@ .\" Copyright (c) 2009 Pyun YongHyeon .\" All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" $FreeBSD$ .\" .Dd August 22, 2016 .Dt ALC 4 .Os .Sh NAME .Nm alc .Nd Atheros AR813x/AR815x/AR816x/AR817x Gigabit/Fast Ethernet driver .Sh SYNOPSIS To compile this driver into the kernel, place the following lines in your kernel configuration file: .Bd -ragged -offset indent .Cd "device miibus" .Cd "device alc" .Ed .Pp Alternatively, to load the driver as a module at boot time, place the following line in .Xr loader.conf 5 : .Bd -literal -offset indent if_alc_load="YES" .Ed .Sh DESCRIPTION The .Nm device driver provides support for Atheros AR813x, AR815x, AR816x and AR817x PCI Express Gigabit/Fast Ethernet controllers. .Pp All LOMs supported by the .Nm driver have TCP/UDP/IP checksum offload for transmit, TCP segmentation offload (TSO), hardware VLAN tag stripping/insertion features, Wake On Lan (WOL) and an interrupt moderation mechanism as well as a 64-bit multicast hash filter. .Pp The AR813x, AR815x, AR816x and AR817x supports Jumbo Frames (up to 9216, 6144, 9216 and 9216 bytes, respectively), which can be configured via the interface MTU setting. Selecting an MTU larger than 1500 bytes with the .Xr ifconfig 8 utility configures the adapter to receive and transmit Jumbo Frames. .Pp The .Nm driver supports the following media types: .Bl -tag -width ".Cm 10baseT/UTP" .It Cm autoselect Enable autoselection of the media type and options. The user can manually override the autoselected mode by adding media options to .Xr rc.conf 5 . .It Cm 10baseT/UTP Set 10Mbps operation. .It Cm 100baseTX Set 100Mbps (Fast Ethernet) operation. .It Cm 1000baseTX Set 1000baseTX operation over twisted pair. .El .Pp The .Nm driver supports the following media options: .Bl -tag -width ".Cm full-duplex" .It Cm full-duplex Force full duplex operation. .It Cm half-duplex Force half duplex operation. .El .Pp For more information on configuring this device, see .Xr ifconfig 8 . .Sh HARDWARE The .Nm device driver provides support for the following Ethernet controllers: .Pp .Bl -bullet -compact .It Atheros AR8131 PCI Express Gigabit Ethernet controller .It Atheros AR8132 PCI Express Fast Ethernet controller .It Atheros AR8151 v1.0 PCI Express Gigabit Ethernet controller .It Atheros AR8151 v2.0 PCI Express Gigabit Ethernet controller .It Atheros AR8152 v1.1 PCI Express Fast Ethernet controller .It Atheros AR8152 v2.0 PCI Express Fast Ethernet controller .It Atheros AR8161 PCI Express Gigabit Ethernet controller .It Atheros AR8162 PCI Express Fast Ethernet controller .It Atheros AR8171 PCI Express Gigabit Ethernet controller .It Atheros AR8172 PCI Express Fast Ethernet controller .It Killer E2200 Gigabit Ethernet controller .It Killer E2400 Gigabit Ethernet controller +.It +Killer E2500 Gigabit Ethernet controller .El .Sh LOADER TUNABLES Tunables can be set at the .Xr loader 8 prompt before booting the kernel or stored in .Xr loader.conf 5 . .Bl -tag -width "xxxxxx" .It Va hw.alc.msi_disable This tunable disables MSI support on the Ethernet hardware. The default value is 0. .It Va hw.alc.msix_disable This tunable disables MSI-X support on the Ethernet hardware. The default value is 0. .El .Sh SYSCTL VARIABLES The following variables are available as both .Xr sysctl 8 variables and .Xr loader 8 tunables: .Bl -tag -width "xxxxxx" .It Va dev.alc.%d.int_rx_mod Maximum amount of time to delay receive interrupt processing in units of 1us. The accepted range is 0 to 130000, the default is 100(100us). Value 0 completely disables the interrupt moderation. .It Va dev.alc.%d.int_tx_mod Maximum amount of time to delay transmit interrupt processing in units of 1us. The accepted range is 0 to 130000, the default is 1000(1ms). Value 0 completely disables the interrupt moderation. .It Va dev.alc.%d.process_limit Maximum amount of Rx frames to be processed in the event loop before rescheduling a taskqueue. The accepted range is 32 to 255, the default value is 64 events. The interface does not need to be brought down and up again before a change takes effect. .El .Sh SEE ALSO .Xr altq 4 , .Xr arp 4 , .Xr miibus 4 , .Xr netintro 4 , .Xr ng_ether 4 , .Xr vlan 4 , .Xr ifconfig 8 .Sh HISTORY The .Nm driver was written by .An Pyun YongHyeon Aq Mt yongari@FreeBSD.org . It first appeared in .Fx 8.0 . Index: head/sys/dev/alc/if_alc.c =================================================================== --- head/sys/dev/alc/if_alc.c (revision 312249) +++ head/sys/dev/alc/if_alc.c (revision 312250) @@ -1,4637 +1,4642 @@ /*- * Copyright (c) 2009, Pyun YongHyeon * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice unmodified, this list of conditions, and the following * disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* Driver for Atheros AR813x/AR815x PCIe Ethernet. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include /* "device miibus" required. See GENERIC if you get errors here. */ #include "miibus_if.h" #undef ALC_USE_CUSTOM_CSUM #ifdef ALC_USE_CUSTOM_CSUM #define ALC_CSUM_FEATURES (CSUM_TCP | CSUM_UDP) #else #define ALC_CSUM_FEATURES (CSUM_IP | CSUM_TCP | CSUM_UDP) #endif MODULE_DEPEND(alc, pci, 1, 1, 1); MODULE_DEPEND(alc, ether, 1, 1, 1); MODULE_DEPEND(alc, miibus, 1, 1, 1); /* Tunables. */ static int msi_disable = 0; static int msix_disable = 0; TUNABLE_INT("hw.alc.msi_disable", &msi_disable); TUNABLE_INT("hw.alc.msix_disable", &msix_disable); /* * Devices supported by this driver. */ static struct alc_ident alc_ident_table[] = { { VENDORID_ATHEROS, DEVICEID_ATHEROS_AR8131, 9 * 1024, "Atheros AR8131 PCIe Gigabit Ethernet" }, { VENDORID_ATHEROS, DEVICEID_ATHEROS_AR8132, 9 * 1024, "Atheros AR8132 PCIe Fast Ethernet" }, { VENDORID_ATHEROS, DEVICEID_ATHEROS_AR8151, 6 * 1024, "Atheros AR8151 v1.0 PCIe Gigabit Ethernet" }, { VENDORID_ATHEROS, DEVICEID_ATHEROS_AR8151_V2, 6 * 1024, "Atheros AR8151 v2.0 PCIe Gigabit Ethernet" }, { VENDORID_ATHEROS, DEVICEID_ATHEROS_AR8152_B, 6 * 1024, "Atheros AR8152 v1.1 PCIe Fast Ethernet" }, { VENDORID_ATHEROS, DEVICEID_ATHEROS_AR8152_B2, 6 * 1024, "Atheros AR8152 v2.0 PCIe Fast Ethernet" }, { VENDORID_ATHEROS, DEVICEID_ATHEROS_AR8161, 9 * 1024, "Atheros AR8161 PCIe Gigabit Ethernet" }, { VENDORID_ATHEROS, DEVICEID_ATHEROS_AR8162, 9 * 1024, "Atheros AR8162 PCIe Fast Ethernet" }, { VENDORID_ATHEROS, DEVICEID_ATHEROS_AR8171, 9 * 1024, "Atheros AR8171 PCIe Gigabit Ethernet" }, { VENDORID_ATHEROS, DEVICEID_ATHEROS_AR8172, 9 * 1024, "Atheros AR8172 PCIe Fast Ethernet" }, { VENDORID_ATHEROS, DEVICEID_ATHEROS_E2200, 9 * 1024, "Killer E2200 Gigabit Ethernet" }, { VENDORID_ATHEROS, DEVICEID_ATHEROS_E2400, 9 * 1024, "Killer E2400 Gigabit Ethernet" }, + { VENDORID_ATHEROS, DEVICEID_ATHEROS_E2500, 9 * 1024, + "Killer E2500 Gigabit Ethernet" }, { 0, 0, 0, NULL} }; static void alc_aspm(struct alc_softc *, int, int); static void alc_aspm_813x(struct alc_softc *, int); static void alc_aspm_816x(struct alc_softc *, int); static int alc_attach(device_t); static int alc_check_boundary(struct alc_softc *); static void alc_config_msi(struct alc_softc *); static int alc_detach(device_t); static void alc_disable_l0s_l1(struct alc_softc *); static int alc_dma_alloc(struct alc_softc *); static void alc_dma_free(struct alc_softc *); static void alc_dmamap_cb(void *, bus_dma_segment_t *, int, int); static void alc_dsp_fixup(struct alc_softc *, int); static int alc_encap(struct alc_softc *, struct mbuf **); static struct alc_ident * alc_find_ident(device_t); #ifndef __NO_STRICT_ALIGNMENT static struct mbuf * alc_fixup_rx(struct ifnet *, struct mbuf *); #endif static void alc_get_macaddr(struct alc_softc *); static void alc_get_macaddr_813x(struct alc_softc *); static void alc_get_macaddr_816x(struct alc_softc *); static void alc_get_macaddr_par(struct alc_softc *); static void alc_init(void *); static void alc_init_cmb(struct alc_softc *); static void alc_init_locked(struct alc_softc *); static void alc_init_rr_ring(struct alc_softc *); static int alc_init_rx_ring(struct alc_softc *); static void alc_init_smb(struct alc_softc *); static void alc_init_tx_ring(struct alc_softc *); static void alc_int_task(void *, int); static int alc_intr(void *); static int alc_ioctl(struct ifnet *, u_long, caddr_t); static void alc_mac_config(struct alc_softc *); static uint32_t alc_mii_readreg_813x(struct alc_softc *, int, int); static uint32_t alc_mii_readreg_816x(struct alc_softc *, int, int); static uint32_t alc_mii_writereg_813x(struct alc_softc *, int, int, int); static uint32_t alc_mii_writereg_816x(struct alc_softc *, int, int, int); static int alc_miibus_readreg(device_t, int, int); static void alc_miibus_statchg(device_t); static int alc_miibus_writereg(device_t, int, int, int); static uint32_t alc_miidbg_readreg(struct alc_softc *, int); static uint32_t alc_miidbg_writereg(struct alc_softc *, int, int); static uint32_t alc_miiext_readreg(struct alc_softc *, int, int); static uint32_t alc_miiext_writereg(struct alc_softc *, int, int, int); static int alc_mediachange(struct ifnet *); static int alc_mediachange_locked(struct alc_softc *); static void alc_mediastatus(struct ifnet *, struct ifmediareq *); static int alc_newbuf(struct alc_softc *, struct alc_rxdesc *); static void alc_osc_reset(struct alc_softc *); static void alc_phy_down(struct alc_softc *); static void alc_phy_reset(struct alc_softc *); static void alc_phy_reset_813x(struct alc_softc *); static void alc_phy_reset_816x(struct alc_softc *); static int alc_probe(device_t); static void alc_reset(struct alc_softc *); static int alc_resume(device_t); static void alc_rxeof(struct alc_softc *, struct rx_rdesc *); static int alc_rxintr(struct alc_softc *, int); static void alc_rxfilter(struct alc_softc *); static void alc_rxvlan(struct alc_softc *); static void alc_setlinkspeed(struct alc_softc *); static void alc_setwol(struct alc_softc *); static void alc_setwol_813x(struct alc_softc *); static void alc_setwol_816x(struct alc_softc *); static int alc_shutdown(device_t); static void alc_start(struct ifnet *); static void alc_start_locked(struct ifnet *); static void alc_start_queue(struct alc_softc *); static void alc_stats_clear(struct alc_softc *); static void alc_stats_update(struct alc_softc *); static void alc_stop(struct alc_softc *); static void alc_stop_mac(struct alc_softc *); static void alc_stop_queue(struct alc_softc *); static int alc_suspend(device_t); static void alc_sysctl_node(struct alc_softc *); static void alc_tick(void *); static void alc_txeof(struct alc_softc *); static void alc_watchdog(struct alc_softc *); static int sysctl_int_range(SYSCTL_HANDLER_ARGS, int, int); static int sysctl_hw_alc_proc_limit(SYSCTL_HANDLER_ARGS); static int sysctl_hw_alc_int_mod(SYSCTL_HANDLER_ARGS); static device_method_t alc_methods[] = { /* Device interface. */ DEVMETHOD(device_probe, alc_probe), DEVMETHOD(device_attach, alc_attach), DEVMETHOD(device_detach, alc_detach), DEVMETHOD(device_shutdown, alc_shutdown), DEVMETHOD(device_suspend, alc_suspend), DEVMETHOD(device_resume, alc_resume), /* MII interface. */ DEVMETHOD(miibus_readreg, alc_miibus_readreg), DEVMETHOD(miibus_writereg, alc_miibus_writereg), DEVMETHOD(miibus_statchg, alc_miibus_statchg), { NULL, NULL } }; static driver_t alc_driver = { "alc", alc_methods, sizeof(struct alc_softc) }; static devclass_t alc_devclass; DRIVER_MODULE(alc, pci, alc_driver, alc_devclass, 0, 0); DRIVER_MODULE(miibus, alc, miibus_driver, miibus_devclass, 0, 0); static struct resource_spec alc_res_spec_mem[] = { { SYS_RES_MEMORY, PCIR_BAR(0), RF_ACTIVE }, { -1, 0, 0 } }; static struct resource_spec alc_irq_spec_legacy[] = { { SYS_RES_IRQ, 0, RF_ACTIVE | RF_SHAREABLE }, { -1, 0, 0 } }; static struct resource_spec alc_irq_spec_msi[] = { { SYS_RES_IRQ, 1, RF_ACTIVE }, { -1, 0, 0 } }; static struct resource_spec alc_irq_spec_msix[] = { { SYS_RES_IRQ, 1, RF_ACTIVE }, { -1, 0, 0 } }; static uint32_t alc_dma_burst[] = { 128, 256, 512, 1024, 2048, 4096, 0, 0 }; static int alc_miibus_readreg(device_t dev, int phy, int reg) { struct alc_softc *sc; int v; sc = device_get_softc(dev); if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) != 0) v = alc_mii_readreg_816x(sc, phy, reg); else v = alc_mii_readreg_813x(sc, phy, reg); return (v); } static uint32_t alc_mii_readreg_813x(struct alc_softc *sc, int phy, int reg) { uint32_t v; int i; /* * For AR8132 fast ethernet controller, do not report 1000baseT * capability to mii(4). Even though AR8132 uses the same * model/revision number of F1 gigabit PHY, the PHY has no * ability to establish 1000baseT link. */ if ((sc->alc_flags & ALC_FLAG_FASTETHER) != 0 && reg == MII_EXTSR) return (0); CSR_WRITE_4(sc, ALC_MDIO, MDIO_OP_EXECUTE | MDIO_OP_READ | MDIO_SUP_PREAMBLE | MDIO_CLK_25_4 | MDIO_REG_ADDR(reg)); for (i = ALC_PHY_TIMEOUT; i > 0; i--) { DELAY(5); v = CSR_READ_4(sc, ALC_MDIO); if ((v & (MDIO_OP_EXECUTE | MDIO_OP_BUSY)) == 0) break; } if (i == 0) { device_printf(sc->alc_dev, "phy read timeout : %d\n", reg); return (0); } return ((v & MDIO_DATA_MASK) >> MDIO_DATA_SHIFT); } static uint32_t alc_mii_readreg_816x(struct alc_softc *sc, int phy, int reg) { uint32_t clk, v; int i; if ((sc->alc_flags & ALC_FLAG_LINK) != 0) clk = MDIO_CLK_25_128; else clk = MDIO_CLK_25_4; CSR_WRITE_4(sc, ALC_MDIO, MDIO_OP_EXECUTE | MDIO_OP_READ | MDIO_SUP_PREAMBLE | clk | MDIO_REG_ADDR(reg)); for (i = ALC_PHY_TIMEOUT; i > 0; i--) { DELAY(5); v = CSR_READ_4(sc, ALC_MDIO); if ((v & MDIO_OP_BUSY) == 0) break; } if (i == 0) { device_printf(sc->alc_dev, "phy read timeout : %d\n", reg); return (0); } return ((v & MDIO_DATA_MASK) >> MDIO_DATA_SHIFT); } static int alc_miibus_writereg(device_t dev, int phy, int reg, int val) { struct alc_softc *sc; int v; sc = device_get_softc(dev); if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) != 0) v = alc_mii_writereg_816x(sc, phy, reg, val); else v = alc_mii_writereg_813x(sc, phy, reg, val); return (v); } static uint32_t alc_mii_writereg_813x(struct alc_softc *sc, int phy, int reg, int val) { uint32_t v; int i; CSR_WRITE_4(sc, ALC_MDIO, MDIO_OP_EXECUTE | MDIO_OP_WRITE | (val & MDIO_DATA_MASK) << MDIO_DATA_SHIFT | MDIO_SUP_PREAMBLE | MDIO_CLK_25_4 | MDIO_REG_ADDR(reg)); for (i = ALC_PHY_TIMEOUT; i > 0; i--) { DELAY(5); v = CSR_READ_4(sc, ALC_MDIO); if ((v & (MDIO_OP_EXECUTE | MDIO_OP_BUSY)) == 0) break; } if (i == 0) device_printf(sc->alc_dev, "phy write timeout : %d\n", reg); return (0); } static uint32_t alc_mii_writereg_816x(struct alc_softc *sc, int phy, int reg, int val) { uint32_t clk, v; int i; if ((sc->alc_flags & ALC_FLAG_LINK) != 0) clk = MDIO_CLK_25_128; else clk = MDIO_CLK_25_4; CSR_WRITE_4(sc, ALC_MDIO, MDIO_OP_EXECUTE | MDIO_OP_WRITE | ((val & MDIO_DATA_MASK) << MDIO_DATA_SHIFT) | MDIO_REG_ADDR(reg) | MDIO_SUP_PREAMBLE | clk); for (i = ALC_PHY_TIMEOUT; i > 0; i--) { DELAY(5); v = CSR_READ_4(sc, ALC_MDIO); if ((v & MDIO_OP_BUSY) == 0) break; } if (i == 0) device_printf(sc->alc_dev, "phy write timeout : %d\n", reg); return (0); } static void alc_miibus_statchg(device_t dev) { struct alc_softc *sc; struct mii_data *mii; struct ifnet *ifp; uint32_t reg; sc = device_get_softc(dev); mii = device_get_softc(sc->alc_miibus); ifp = sc->alc_ifp; if (mii == NULL || ifp == NULL || (ifp->if_drv_flags & IFF_DRV_RUNNING) == 0) return; sc->alc_flags &= ~ALC_FLAG_LINK; if ((mii->mii_media_status & (IFM_ACTIVE | IFM_AVALID)) == (IFM_ACTIVE | IFM_AVALID)) { switch (IFM_SUBTYPE(mii->mii_media_active)) { case IFM_10_T: case IFM_100_TX: sc->alc_flags |= ALC_FLAG_LINK; break; case IFM_1000_T: if ((sc->alc_flags & ALC_FLAG_FASTETHER) == 0) sc->alc_flags |= ALC_FLAG_LINK; break; default: break; } } /* Stop Rx/Tx MACs. */ alc_stop_mac(sc); /* Program MACs with resolved speed/duplex/flow-control. */ if ((sc->alc_flags & ALC_FLAG_LINK) != 0) { alc_start_queue(sc); alc_mac_config(sc); /* Re-enable Tx/Rx MACs. */ reg = CSR_READ_4(sc, ALC_MAC_CFG); reg |= MAC_CFG_TX_ENB | MAC_CFG_RX_ENB; CSR_WRITE_4(sc, ALC_MAC_CFG, reg); } alc_aspm(sc, 0, IFM_SUBTYPE(mii->mii_media_active)); alc_dsp_fixup(sc, IFM_SUBTYPE(mii->mii_media_active)); } static uint32_t alc_miidbg_readreg(struct alc_softc *sc, int reg) { alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_ADDR, reg); return (alc_miibus_readreg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_DATA)); } static uint32_t alc_miidbg_writereg(struct alc_softc *sc, int reg, int val) { alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_ADDR, reg); return (alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_DATA, val)); } static uint32_t alc_miiext_readreg(struct alc_softc *sc, int devaddr, int reg) { uint32_t clk, v; int i; CSR_WRITE_4(sc, ALC_EXT_MDIO, EXT_MDIO_REG(reg) | EXT_MDIO_DEVADDR(devaddr)); if ((sc->alc_flags & ALC_FLAG_LINK) != 0) clk = MDIO_CLK_25_128; else clk = MDIO_CLK_25_4; CSR_WRITE_4(sc, ALC_MDIO, MDIO_OP_EXECUTE | MDIO_OP_READ | MDIO_SUP_PREAMBLE | clk | MDIO_MODE_EXT); for (i = ALC_PHY_TIMEOUT; i > 0; i--) { DELAY(5); v = CSR_READ_4(sc, ALC_MDIO); if ((v & MDIO_OP_BUSY) == 0) break; } if (i == 0) { device_printf(sc->alc_dev, "phy ext read timeout : %d, %d\n", devaddr, reg); return (0); } return ((v & MDIO_DATA_MASK) >> MDIO_DATA_SHIFT); } static uint32_t alc_miiext_writereg(struct alc_softc *sc, int devaddr, int reg, int val) { uint32_t clk, v; int i; CSR_WRITE_4(sc, ALC_EXT_MDIO, EXT_MDIO_REG(reg) | EXT_MDIO_DEVADDR(devaddr)); if ((sc->alc_flags & ALC_FLAG_LINK) != 0) clk = MDIO_CLK_25_128; else clk = MDIO_CLK_25_4; CSR_WRITE_4(sc, ALC_MDIO, MDIO_OP_EXECUTE | MDIO_OP_WRITE | ((val & MDIO_DATA_MASK) << MDIO_DATA_SHIFT) | MDIO_SUP_PREAMBLE | clk | MDIO_MODE_EXT); for (i = ALC_PHY_TIMEOUT; i > 0; i--) { DELAY(5); v = CSR_READ_4(sc, ALC_MDIO); if ((v & MDIO_OP_BUSY) == 0) break; } if (i == 0) device_printf(sc->alc_dev, "phy ext write timeout : %d, %d\n", devaddr, reg); return (0); } static void alc_dsp_fixup(struct alc_softc *sc, int media) { uint16_t agc, len, val; if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) != 0) return; if (AR816X_REV(sc->alc_rev) >= AR816X_REV_C0) return; /* * Vendor PHY magic. * 1000BT/AZ, wrong cable length */ if ((sc->alc_flags & ALC_FLAG_LINK) != 0) { len = alc_miiext_readreg(sc, MII_EXT_PCS, MII_EXT_CLDCTL6); len = (len >> EXT_CLDCTL6_CAB_LEN_SHIFT) & EXT_CLDCTL6_CAB_LEN_MASK; agc = alc_miidbg_readreg(sc, MII_DBG_AGC); agc = (agc >> DBG_AGC_2_VGA_SHIFT) & DBG_AGC_2_VGA_MASK; if ((media == IFM_1000_T && len > EXT_CLDCTL6_CAB_LEN_SHORT1G && agc > DBG_AGC_LONG1G_LIMT) || (media == IFM_100_TX && len > DBG_AGC_LONG100M_LIMT && agc > DBG_AGC_LONG1G_LIMT)) { alc_miidbg_writereg(sc, MII_DBG_AZ_ANADECT, DBG_AZ_ANADECT_LONG); val = alc_miiext_readreg(sc, MII_EXT_ANEG, MII_EXT_ANEG_AFE); val |= ANEG_AFEE_10BT_100M_TH; alc_miiext_writereg(sc, MII_EXT_ANEG, MII_EXT_ANEG_AFE, val); } else { alc_miidbg_writereg(sc, MII_DBG_AZ_ANADECT, DBG_AZ_ANADECT_DEFAULT); val = alc_miiext_readreg(sc, MII_EXT_ANEG, MII_EXT_ANEG_AFE); val &= ~ANEG_AFEE_10BT_100M_TH; alc_miiext_writereg(sc, MII_EXT_ANEG, MII_EXT_ANEG_AFE, val); } if ((sc->alc_flags & ALC_FLAG_LINK_WAR) != 0 && AR816X_REV(sc->alc_rev) == AR816X_REV_B0) { if (media == IFM_1000_T) { /* * Giga link threshold, raise the tolerance of * noise 50%. */ val = alc_miidbg_readreg(sc, MII_DBG_MSE20DB); val &= ~DBG_MSE20DB_TH_MASK; val |= (DBG_MSE20DB_TH_HI << DBG_MSE20DB_TH_SHIFT); alc_miidbg_writereg(sc, MII_DBG_MSE20DB, val); } else if (media == IFM_100_TX) alc_miidbg_writereg(sc, MII_DBG_MSE16DB, DBG_MSE16DB_UP); } } else { val = alc_miiext_readreg(sc, MII_EXT_ANEG, MII_EXT_ANEG_AFE); val &= ~ANEG_AFEE_10BT_100M_TH; alc_miiext_writereg(sc, MII_EXT_ANEG, MII_EXT_ANEG_AFE, val); if ((sc->alc_flags & ALC_FLAG_LINK_WAR) != 0 && AR816X_REV(sc->alc_rev) == AR816X_REV_B0) { alc_miidbg_writereg(sc, MII_DBG_MSE16DB, DBG_MSE16DB_DOWN); val = alc_miidbg_readreg(sc, MII_DBG_MSE20DB); val &= ~DBG_MSE20DB_TH_MASK; val |= (DBG_MSE20DB_TH_DEFAULT << DBG_MSE20DB_TH_SHIFT); alc_miidbg_writereg(sc, MII_DBG_MSE20DB, val); } } } static void alc_mediastatus(struct ifnet *ifp, struct ifmediareq *ifmr) { struct alc_softc *sc; struct mii_data *mii; sc = ifp->if_softc; ALC_LOCK(sc); if ((ifp->if_flags & IFF_UP) == 0) { ALC_UNLOCK(sc); return; } mii = device_get_softc(sc->alc_miibus); mii_pollstat(mii); ifmr->ifm_status = mii->mii_media_status; ifmr->ifm_active = mii->mii_media_active; ALC_UNLOCK(sc); } static int alc_mediachange(struct ifnet *ifp) { struct alc_softc *sc; int error; sc = ifp->if_softc; ALC_LOCK(sc); error = alc_mediachange_locked(sc); ALC_UNLOCK(sc); return (error); } static int alc_mediachange_locked(struct alc_softc *sc) { struct mii_data *mii; struct mii_softc *miisc; int error; ALC_LOCK_ASSERT(sc); mii = device_get_softc(sc->alc_miibus); LIST_FOREACH(miisc, &mii->mii_phys, mii_list) PHY_RESET(miisc); error = mii_mediachg(mii); return (error); } static struct alc_ident * alc_find_ident(device_t dev) { struct alc_ident *ident; uint16_t vendor, devid; vendor = pci_get_vendor(dev); devid = pci_get_device(dev); for (ident = alc_ident_table; ident->name != NULL; ident++) { if (vendor == ident->vendorid && devid == ident->deviceid) return (ident); } return (NULL); } static int alc_probe(device_t dev) { struct alc_ident *ident; ident = alc_find_ident(dev); if (ident != NULL) { device_set_desc(dev, ident->name); return (BUS_PROBE_DEFAULT); } return (ENXIO); } static void alc_get_macaddr(struct alc_softc *sc) { if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) != 0) alc_get_macaddr_816x(sc); else alc_get_macaddr_813x(sc); } static void alc_get_macaddr_813x(struct alc_softc *sc) { uint32_t opt; uint16_t val; int eeprom, i; eeprom = 0; opt = CSR_READ_4(sc, ALC_OPT_CFG); if ((CSR_READ_4(sc, ALC_MASTER_CFG) & MASTER_OTP_SEL) != 0 && (CSR_READ_4(sc, ALC_TWSI_DEBUG) & TWSI_DEBUG_DEV_EXIST) != 0) { /* * EEPROM found, let TWSI reload EEPROM configuration. * This will set ethernet address of controller. */ eeprom++; switch (sc->alc_ident->deviceid) { case DEVICEID_ATHEROS_AR8131: case DEVICEID_ATHEROS_AR8132: if ((opt & OPT_CFG_CLK_ENB) == 0) { opt |= OPT_CFG_CLK_ENB; CSR_WRITE_4(sc, ALC_OPT_CFG, opt); CSR_READ_4(sc, ALC_OPT_CFG); DELAY(1000); } break; case DEVICEID_ATHEROS_AR8151: case DEVICEID_ATHEROS_AR8151_V2: case DEVICEID_ATHEROS_AR8152_B: case DEVICEID_ATHEROS_AR8152_B2: alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_ADDR, 0x00); val = alc_miibus_readreg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_DATA); alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_DATA, val & 0xFF7F); alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_ADDR, 0x3B); val = alc_miibus_readreg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_DATA); alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_DATA, val | 0x0008); DELAY(20); break; } CSR_WRITE_4(sc, ALC_LTSSM_ID_CFG, CSR_READ_4(sc, ALC_LTSSM_ID_CFG) & ~LTSSM_ID_WRO_ENB); CSR_WRITE_4(sc, ALC_WOL_CFG, 0); CSR_READ_4(sc, ALC_WOL_CFG); CSR_WRITE_4(sc, ALC_TWSI_CFG, CSR_READ_4(sc, ALC_TWSI_CFG) | TWSI_CFG_SW_LD_START); for (i = 100; i > 0; i--) { DELAY(1000); if ((CSR_READ_4(sc, ALC_TWSI_CFG) & TWSI_CFG_SW_LD_START) == 0) break; } if (i == 0) device_printf(sc->alc_dev, "reloading EEPROM timeout!\n"); } else { if (bootverbose) device_printf(sc->alc_dev, "EEPROM not found!\n"); } if (eeprom != 0) { switch (sc->alc_ident->deviceid) { case DEVICEID_ATHEROS_AR8131: case DEVICEID_ATHEROS_AR8132: if ((opt & OPT_CFG_CLK_ENB) != 0) { opt &= ~OPT_CFG_CLK_ENB; CSR_WRITE_4(sc, ALC_OPT_CFG, opt); CSR_READ_4(sc, ALC_OPT_CFG); DELAY(1000); } break; case DEVICEID_ATHEROS_AR8151: case DEVICEID_ATHEROS_AR8151_V2: case DEVICEID_ATHEROS_AR8152_B: case DEVICEID_ATHEROS_AR8152_B2: alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_ADDR, 0x00); val = alc_miibus_readreg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_DATA); alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_DATA, val | 0x0080); alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_ADDR, 0x3B); val = alc_miibus_readreg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_DATA); alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_DATA, val & 0xFFF7); DELAY(20); break; } } alc_get_macaddr_par(sc); } static void alc_get_macaddr_816x(struct alc_softc *sc) { uint32_t reg; int i, reloaded; reloaded = 0; /* Try to reload station address via TWSI. */ for (i = 100; i > 0; i--) { reg = CSR_READ_4(sc, ALC_SLD); if ((reg & (SLD_PROGRESS | SLD_START)) == 0) break; DELAY(1000); } if (i != 0) { CSR_WRITE_4(sc, ALC_SLD, reg | SLD_START); for (i = 100; i > 0; i--) { DELAY(1000); reg = CSR_READ_4(sc, ALC_SLD); if ((reg & SLD_START) == 0) break; } if (i != 0) reloaded++; else if (bootverbose) device_printf(sc->alc_dev, "reloading station address via TWSI timed out!\n"); } /* Try to reload station address from EEPROM or FLASH. */ if (reloaded == 0) { reg = CSR_READ_4(sc, ALC_EEPROM_LD); if ((reg & (EEPROM_LD_EEPROM_EXIST | EEPROM_LD_FLASH_EXIST)) != 0) { for (i = 100; i > 0; i--) { reg = CSR_READ_4(sc, ALC_EEPROM_LD); if ((reg & (EEPROM_LD_PROGRESS | EEPROM_LD_START)) == 0) break; DELAY(1000); } if (i != 0) { CSR_WRITE_4(sc, ALC_EEPROM_LD, reg | EEPROM_LD_START); for (i = 100; i > 0; i--) { DELAY(1000); reg = CSR_READ_4(sc, ALC_EEPROM_LD); if ((reg & EEPROM_LD_START) == 0) break; } } else if (bootverbose) device_printf(sc->alc_dev, "reloading EEPROM/FLASH timed out!\n"); } } alc_get_macaddr_par(sc); } static void alc_get_macaddr_par(struct alc_softc *sc) { uint32_t ea[2]; ea[0] = CSR_READ_4(sc, ALC_PAR0); ea[1] = CSR_READ_4(sc, ALC_PAR1); sc->alc_eaddr[0] = (ea[1] >> 8) & 0xFF; sc->alc_eaddr[1] = (ea[1] >> 0) & 0xFF; sc->alc_eaddr[2] = (ea[0] >> 24) & 0xFF; sc->alc_eaddr[3] = (ea[0] >> 16) & 0xFF; sc->alc_eaddr[4] = (ea[0] >> 8) & 0xFF; sc->alc_eaddr[5] = (ea[0] >> 0) & 0xFF; } static void alc_disable_l0s_l1(struct alc_softc *sc) { uint32_t pmcfg; if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) == 0) { /* Another magic from vendor. */ pmcfg = CSR_READ_4(sc, ALC_PM_CFG); pmcfg &= ~(PM_CFG_L1_ENTRY_TIMER_MASK | PM_CFG_CLK_SWH_L1 | PM_CFG_ASPM_L0S_ENB | PM_CFG_ASPM_L1_ENB | PM_CFG_MAC_ASPM_CHK | PM_CFG_SERDES_PD_EX_L1); pmcfg |= PM_CFG_SERDES_BUDS_RX_L1_ENB | PM_CFG_SERDES_PLL_L1_ENB | PM_CFG_SERDES_L1_ENB; CSR_WRITE_4(sc, ALC_PM_CFG, pmcfg); } } static void alc_phy_reset(struct alc_softc *sc) { if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) != 0) alc_phy_reset_816x(sc); else alc_phy_reset_813x(sc); } static void alc_phy_reset_813x(struct alc_softc *sc) { uint16_t data; /* Reset magic from Linux. */ CSR_WRITE_2(sc, ALC_GPHY_CFG, GPHY_CFG_SEL_ANA_RESET); CSR_READ_2(sc, ALC_GPHY_CFG); DELAY(10 * 1000); CSR_WRITE_2(sc, ALC_GPHY_CFG, GPHY_CFG_EXT_RESET | GPHY_CFG_SEL_ANA_RESET); CSR_READ_2(sc, ALC_GPHY_CFG); DELAY(10 * 1000); /* DSP fixup, Vendor magic. */ if (sc->alc_ident->deviceid == DEVICEID_ATHEROS_AR8152_B) { alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_ADDR, 0x000A); data = alc_miibus_readreg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_DATA); alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_DATA, data & 0xDFFF); } if (sc->alc_ident->deviceid == DEVICEID_ATHEROS_AR8151 || sc->alc_ident->deviceid == DEVICEID_ATHEROS_AR8151_V2 || sc->alc_ident->deviceid == DEVICEID_ATHEROS_AR8152_B || sc->alc_ident->deviceid == DEVICEID_ATHEROS_AR8152_B2) { alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_ADDR, 0x003B); data = alc_miibus_readreg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_DATA); alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_DATA, data & 0xFFF7); DELAY(20 * 1000); } if (sc->alc_ident->deviceid == DEVICEID_ATHEROS_AR8151) { alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_ADDR, 0x0029); alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_DATA, 0x929D); } if (sc->alc_ident->deviceid == DEVICEID_ATHEROS_AR8131 || sc->alc_ident->deviceid == DEVICEID_ATHEROS_AR8132 || sc->alc_ident->deviceid == DEVICEID_ATHEROS_AR8151_V2 || sc->alc_ident->deviceid == DEVICEID_ATHEROS_AR8152_B2) { alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_ADDR, 0x0029); alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_DATA, 0xB6DD); } /* Load DSP codes, vendor magic. */ data = ANA_LOOP_SEL_10BT | ANA_EN_MASK_TB | ANA_EN_10BT_IDLE | ((1 << ANA_INTERVAL_SEL_TIMER_SHIFT) & ANA_INTERVAL_SEL_TIMER_MASK); alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_ADDR, MII_ANA_CFG18); alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_DATA, data); data = ((2 << ANA_SERDES_CDR_BW_SHIFT) & ANA_SERDES_CDR_BW_MASK) | ANA_SERDES_EN_DEEM | ANA_SERDES_SEL_HSP | ANA_SERDES_EN_PLL | ANA_SERDES_EN_LCKDT; alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_ADDR, MII_ANA_CFG5); alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_DATA, data); data = ((44 << ANA_LONG_CABLE_TH_100_SHIFT) & ANA_LONG_CABLE_TH_100_MASK) | ((33 << ANA_SHORT_CABLE_TH_100_SHIFT) & ANA_SHORT_CABLE_TH_100_SHIFT) | ANA_BP_BAD_LINK_ACCUM | ANA_BP_SMALL_BW; alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_ADDR, MII_ANA_CFG54); alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_DATA, data); data = ((11 << ANA_IECHO_ADJ_3_SHIFT) & ANA_IECHO_ADJ_3_MASK) | ((11 << ANA_IECHO_ADJ_2_SHIFT) & ANA_IECHO_ADJ_2_MASK) | ((8 << ANA_IECHO_ADJ_1_SHIFT) & ANA_IECHO_ADJ_1_MASK) | ((8 << ANA_IECHO_ADJ_0_SHIFT) & ANA_IECHO_ADJ_0_MASK); alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_ADDR, MII_ANA_CFG4); alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_DATA, data); data = ((7 & ANA_MANUL_SWICH_ON_SHIFT) & ANA_MANUL_SWICH_ON_MASK) | ANA_RESTART_CAL | ANA_MAN_ENABLE | ANA_SEL_HSP | ANA_EN_HB | ANA_OEN_125M; alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_ADDR, MII_ANA_CFG0); alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_DATA, data); DELAY(1000); /* Disable hibernation. */ alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_ADDR, 0x0029); data = alc_miibus_readreg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_DATA); data &= ~0x8000; alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_DATA, data); alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_ADDR, 0x000B); data = alc_miibus_readreg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_DATA); data &= ~0x8000; alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, ALC_MII_DBG_DATA, data); } static void alc_phy_reset_816x(struct alc_softc *sc) { uint32_t val; val = CSR_READ_4(sc, ALC_GPHY_CFG); val &= ~(GPHY_CFG_EXT_RESET | GPHY_CFG_LED_MODE | GPHY_CFG_GATE_25M_ENB | GPHY_CFG_PHY_IDDQ | GPHY_CFG_PHY_PLL_ON | GPHY_CFG_PWDOWN_HW | GPHY_CFG_100AB_ENB); val |= GPHY_CFG_SEL_ANA_RESET; #ifdef notyet val |= GPHY_CFG_HIB_PULSE | GPHY_CFG_HIB_EN | GPHY_CFG_SEL_ANA_RESET; #else /* Disable PHY hibernation. */ val &= ~(GPHY_CFG_HIB_PULSE | GPHY_CFG_HIB_EN); #endif CSR_WRITE_4(sc, ALC_GPHY_CFG, val); DELAY(10); CSR_WRITE_4(sc, ALC_GPHY_CFG, val | GPHY_CFG_EXT_RESET); DELAY(800); /* Vendor PHY magic. */ #ifdef notyet alc_miidbg_writereg(sc, MII_DBG_LEGCYPS, DBG_LEGCYPS_DEFAULT); alc_miidbg_writereg(sc, MII_DBG_SYSMODCTL, DBG_SYSMODCTL_DEFAULT); alc_miiext_writereg(sc, MII_EXT_PCS, MII_EXT_VDRVBIAS, EXT_VDRVBIAS_DEFAULT); #else /* Disable PHY hibernation. */ alc_miidbg_writereg(sc, MII_DBG_LEGCYPS, DBG_LEGCYPS_DEFAULT & ~DBG_LEGCYPS_ENB); alc_miidbg_writereg(sc, MII_DBG_HIBNEG, DBG_HIBNEG_DEFAULT & ~(DBG_HIBNEG_PSHIB_EN | DBG_HIBNEG_HIB_PULSE)); alc_miidbg_writereg(sc, MII_DBG_GREENCFG, DBG_GREENCFG_DEFAULT); #endif /* XXX Disable EEE. */ val = CSR_READ_4(sc, ALC_LPI_CTL); val &= ~LPI_CTL_ENB; CSR_WRITE_4(sc, ALC_LPI_CTL, val); alc_miiext_writereg(sc, MII_EXT_ANEG, MII_EXT_ANEG_LOCAL_EEEADV, 0); /* PHY power saving. */ alc_miidbg_writereg(sc, MII_DBG_TST10BTCFG, DBG_TST10BTCFG_DEFAULT); alc_miidbg_writereg(sc, MII_DBG_SRDSYSMOD, DBG_SRDSYSMOD_DEFAULT); alc_miidbg_writereg(sc, MII_DBG_TST100BTCFG, DBG_TST100BTCFG_DEFAULT); alc_miidbg_writereg(sc, MII_DBG_ANACTL, DBG_ANACTL_DEFAULT); val = alc_miidbg_readreg(sc, MII_DBG_GREENCFG2); val &= ~DBG_GREENCFG2_GATE_DFSE_EN; alc_miidbg_writereg(sc, MII_DBG_GREENCFG2, val); /* RTL8139C, 120m issue. */ alc_miiext_writereg(sc, MII_EXT_ANEG, MII_EXT_ANEG_NLP78, ANEG_NLP78_120M_DEFAULT); alc_miiext_writereg(sc, MII_EXT_ANEG, MII_EXT_ANEG_S3DIG10, ANEG_S3DIG10_DEFAULT); if ((sc->alc_flags & ALC_FLAG_LINK_WAR) != 0) { /* Turn off half amplitude. */ val = alc_miiext_readreg(sc, MII_EXT_PCS, MII_EXT_CLDCTL3); val |= EXT_CLDCTL3_BP_CABLE1TH_DET_GT; alc_miiext_writereg(sc, MII_EXT_PCS, MII_EXT_CLDCTL3, val); /* Turn off Green feature. */ val = alc_miidbg_readreg(sc, MII_DBG_GREENCFG2); val |= DBG_GREENCFG2_BP_GREEN; alc_miidbg_writereg(sc, MII_DBG_GREENCFG2, val); /* Turn off half bias. */ val = alc_miiext_readreg(sc, MII_EXT_PCS, MII_EXT_CLDCTL5); val |= EXT_CLDCTL5_BP_VD_HLFBIAS; alc_miiext_writereg(sc, MII_EXT_PCS, MII_EXT_CLDCTL5, val); } } static void alc_phy_down(struct alc_softc *sc) { uint32_t gphy; switch (sc->alc_ident->deviceid) { case DEVICEID_ATHEROS_AR8161: case DEVICEID_ATHEROS_E2200: case DEVICEID_ATHEROS_E2400: + case DEVICEID_ATHEROS_E2500: case DEVICEID_ATHEROS_AR8162: case DEVICEID_ATHEROS_AR8171: case DEVICEID_ATHEROS_AR8172: gphy = CSR_READ_4(sc, ALC_GPHY_CFG); gphy &= ~(GPHY_CFG_EXT_RESET | GPHY_CFG_LED_MODE | GPHY_CFG_100AB_ENB | GPHY_CFG_PHY_PLL_ON); gphy |= GPHY_CFG_HIB_EN | GPHY_CFG_HIB_PULSE | GPHY_CFG_SEL_ANA_RESET; gphy |= GPHY_CFG_PHY_IDDQ | GPHY_CFG_PWDOWN_HW; CSR_WRITE_4(sc, ALC_GPHY_CFG, gphy); break; case DEVICEID_ATHEROS_AR8151: case DEVICEID_ATHEROS_AR8151_V2: case DEVICEID_ATHEROS_AR8152_B: case DEVICEID_ATHEROS_AR8152_B2: /* * GPHY power down caused more problems on AR8151 v2.0. * When driver is reloaded after GPHY power down, * accesses to PHY/MAC registers hung the system. Only * cold boot recovered from it. I'm not sure whether * AR8151 v1.0 also requires this one though. I don't * have AR8151 v1.0 controller in hand. * The only option left is to isolate the PHY and * initiates power down the PHY which in turn saves * more power when driver is unloaded. */ alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, MII_BMCR, BMCR_ISO | BMCR_PDOWN); break; default: /* Force PHY down. */ CSR_WRITE_2(sc, ALC_GPHY_CFG, GPHY_CFG_EXT_RESET | GPHY_CFG_SEL_ANA_RESET | GPHY_CFG_PHY_IDDQ | GPHY_CFG_PWDOWN_HW); DELAY(1000); break; } } static void alc_aspm(struct alc_softc *sc, int init, int media) { if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) != 0) alc_aspm_816x(sc, init); else alc_aspm_813x(sc, media); } static void alc_aspm_813x(struct alc_softc *sc, int media) { uint32_t pmcfg; uint16_t linkcfg; if ((sc->alc_flags & ALC_FLAG_LINK) == 0) return; pmcfg = CSR_READ_4(sc, ALC_PM_CFG); if ((sc->alc_flags & (ALC_FLAG_APS | ALC_FLAG_PCIE)) == (ALC_FLAG_APS | ALC_FLAG_PCIE)) linkcfg = CSR_READ_2(sc, sc->alc_expcap + PCIER_LINK_CTL); else linkcfg = 0; pmcfg &= ~PM_CFG_SERDES_PD_EX_L1; pmcfg &= ~(PM_CFG_L1_ENTRY_TIMER_MASK | PM_CFG_LCKDET_TIMER_MASK); pmcfg |= PM_CFG_MAC_ASPM_CHK; pmcfg |= (PM_CFG_LCKDET_TIMER_DEFAULT << PM_CFG_LCKDET_TIMER_SHIFT); pmcfg &= ~(PM_CFG_ASPM_L1_ENB | PM_CFG_ASPM_L0S_ENB); if ((sc->alc_flags & ALC_FLAG_APS) != 0) { /* Disable extended sync except AR8152 B v1.0 */ linkcfg &= ~PCIEM_LINK_CTL_EXTENDED_SYNC; if (sc->alc_ident->deviceid == DEVICEID_ATHEROS_AR8152_B && sc->alc_rev == ATHEROS_AR8152_B_V10) linkcfg |= PCIEM_LINK_CTL_EXTENDED_SYNC; CSR_WRITE_2(sc, sc->alc_expcap + PCIER_LINK_CTL, linkcfg); pmcfg &= ~(PM_CFG_EN_BUFS_RX_L0S | PM_CFG_SA_DLY_ENB | PM_CFG_HOTRST); pmcfg |= (PM_CFG_L1_ENTRY_TIMER_DEFAULT << PM_CFG_L1_ENTRY_TIMER_SHIFT); pmcfg &= ~PM_CFG_PM_REQ_TIMER_MASK; pmcfg |= (PM_CFG_PM_REQ_TIMER_DEFAULT << PM_CFG_PM_REQ_TIMER_SHIFT); pmcfg |= PM_CFG_SERDES_PD_EX_L1 | PM_CFG_PCIE_RECV; } if ((sc->alc_flags & ALC_FLAG_LINK) != 0) { if ((sc->alc_flags & ALC_FLAG_L0S) != 0) pmcfg |= PM_CFG_ASPM_L0S_ENB; if ((sc->alc_flags & ALC_FLAG_L1S) != 0) pmcfg |= PM_CFG_ASPM_L1_ENB; if ((sc->alc_flags & ALC_FLAG_APS) != 0) { if (sc->alc_ident->deviceid == DEVICEID_ATHEROS_AR8152_B) pmcfg &= ~PM_CFG_ASPM_L0S_ENB; pmcfg &= ~(PM_CFG_SERDES_L1_ENB | PM_CFG_SERDES_PLL_L1_ENB | PM_CFG_SERDES_BUDS_RX_L1_ENB); pmcfg |= PM_CFG_CLK_SWH_L1; if (media == IFM_100_TX || media == IFM_1000_T) { pmcfg &= ~PM_CFG_L1_ENTRY_TIMER_MASK; switch (sc->alc_ident->deviceid) { case DEVICEID_ATHEROS_AR8152_B: pmcfg |= (7 << PM_CFG_L1_ENTRY_TIMER_SHIFT); break; case DEVICEID_ATHEROS_AR8152_B2: case DEVICEID_ATHEROS_AR8151_V2: pmcfg |= (4 << PM_CFG_L1_ENTRY_TIMER_SHIFT); break; default: pmcfg |= (15 << PM_CFG_L1_ENTRY_TIMER_SHIFT); break; } } } else { pmcfg |= PM_CFG_SERDES_L1_ENB | PM_CFG_SERDES_PLL_L1_ENB | PM_CFG_SERDES_BUDS_RX_L1_ENB; pmcfg &= ~(PM_CFG_CLK_SWH_L1 | PM_CFG_ASPM_L1_ENB | PM_CFG_ASPM_L0S_ENB); } } else { pmcfg &= ~(PM_CFG_SERDES_BUDS_RX_L1_ENB | PM_CFG_SERDES_L1_ENB | PM_CFG_SERDES_PLL_L1_ENB); pmcfg |= PM_CFG_CLK_SWH_L1; if ((sc->alc_flags & ALC_FLAG_L1S) != 0) pmcfg |= PM_CFG_ASPM_L1_ENB; } CSR_WRITE_4(sc, ALC_PM_CFG, pmcfg); } static void alc_aspm_816x(struct alc_softc *sc, int init) { uint32_t pmcfg; pmcfg = CSR_READ_4(sc, ALC_PM_CFG); pmcfg &= ~PM_CFG_L1_ENTRY_TIMER_816X_MASK; pmcfg |= PM_CFG_L1_ENTRY_TIMER_816X_DEFAULT; pmcfg &= ~PM_CFG_PM_REQ_TIMER_MASK; pmcfg |= PM_CFG_PM_REQ_TIMER_816X_DEFAULT; pmcfg &= ~PM_CFG_LCKDET_TIMER_MASK; pmcfg |= PM_CFG_LCKDET_TIMER_DEFAULT; pmcfg |= PM_CFG_SERDES_PD_EX_L1 | PM_CFG_CLK_SWH_L1 | PM_CFG_PCIE_RECV; pmcfg &= ~(PM_CFG_RX_L1_AFTER_L0S | PM_CFG_TX_L1_AFTER_L0S | PM_CFG_ASPM_L1_ENB | PM_CFG_ASPM_L0S_ENB | PM_CFG_SERDES_L1_ENB | PM_CFG_SERDES_PLL_L1_ENB | PM_CFG_SERDES_BUDS_RX_L1_ENB | PM_CFG_SA_DLY_ENB | PM_CFG_MAC_ASPM_CHK | PM_CFG_HOTRST); if (AR816X_REV(sc->alc_rev) <= AR816X_REV_A1 && (sc->alc_rev & 0x01) != 0) pmcfg |= PM_CFG_SERDES_L1_ENB | PM_CFG_SERDES_PLL_L1_ENB; if ((sc->alc_flags & ALC_FLAG_LINK) != 0) { /* Link up, enable both L0s, L1s. */ pmcfg |= PM_CFG_ASPM_L0S_ENB | PM_CFG_ASPM_L1_ENB | PM_CFG_MAC_ASPM_CHK; } else { if (init != 0) pmcfg |= PM_CFG_ASPM_L0S_ENB | PM_CFG_ASPM_L1_ENB | PM_CFG_MAC_ASPM_CHK; else if ((sc->alc_ifp->if_drv_flags & IFF_DRV_RUNNING) != 0) pmcfg |= PM_CFG_ASPM_L1_ENB | PM_CFG_MAC_ASPM_CHK; } CSR_WRITE_4(sc, ALC_PM_CFG, pmcfg); } static void alc_init_pcie(struct alc_softc *sc) { const char *aspm_state[] = { "L0s/L1", "L0s", "L1", "L0s/L1" }; uint32_t cap, ctl, val; int state; /* Clear data link and flow-control protocol error. */ val = CSR_READ_4(sc, ALC_PEX_UNC_ERR_SEV); val &= ~(PEX_UNC_ERR_SEV_DLP | PEX_UNC_ERR_SEV_FCP); CSR_WRITE_4(sc, ALC_PEX_UNC_ERR_SEV, val); if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) == 0) { CSR_WRITE_4(sc, ALC_LTSSM_ID_CFG, CSR_READ_4(sc, ALC_LTSSM_ID_CFG) & ~LTSSM_ID_WRO_ENB); CSR_WRITE_4(sc, ALC_PCIE_PHYMISC, CSR_READ_4(sc, ALC_PCIE_PHYMISC) | PCIE_PHYMISC_FORCE_RCV_DET); if (sc->alc_ident->deviceid == DEVICEID_ATHEROS_AR8152_B && sc->alc_rev == ATHEROS_AR8152_B_V10) { val = CSR_READ_4(sc, ALC_PCIE_PHYMISC2); val &= ~(PCIE_PHYMISC2_SERDES_CDR_MASK | PCIE_PHYMISC2_SERDES_TH_MASK); val |= 3 << PCIE_PHYMISC2_SERDES_CDR_SHIFT; val |= 3 << PCIE_PHYMISC2_SERDES_TH_SHIFT; CSR_WRITE_4(sc, ALC_PCIE_PHYMISC2, val); } /* Disable ASPM L0S and L1. */ cap = CSR_READ_2(sc, sc->alc_expcap + PCIER_LINK_CAP); if ((cap & PCIEM_LINK_CAP_ASPM) != 0) { ctl = CSR_READ_2(sc, sc->alc_expcap + PCIER_LINK_CTL); if ((ctl & PCIEM_LINK_CTL_RCB) != 0) sc->alc_rcb = DMA_CFG_RCB_128; if (bootverbose) device_printf(sc->alc_dev, "RCB %u bytes\n", sc->alc_rcb == DMA_CFG_RCB_64 ? 64 : 128); state = ctl & PCIEM_LINK_CTL_ASPMC; if (state & PCIEM_LINK_CTL_ASPMC_L0S) sc->alc_flags |= ALC_FLAG_L0S; if (state & PCIEM_LINK_CTL_ASPMC_L1) sc->alc_flags |= ALC_FLAG_L1S; if (bootverbose) device_printf(sc->alc_dev, "ASPM %s %s\n", aspm_state[state], state == 0 ? "disabled" : "enabled"); alc_disable_l0s_l1(sc); } else { if (bootverbose) device_printf(sc->alc_dev, "no ASPM support\n"); } } else { val = CSR_READ_4(sc, ALC_PDLL_TRNS1); val &= ~PDLL_TRNS1_D3PLLOFF_ENB; CSR_WRITE_4(sc, ALC_PDLL_TRNS1, val); val = CSR_READ_4(sc, ALC_MASTER_CFG); if (AR816X_REV(sc->alc_rev) <= AR816X_REV_A1 && (sc->alc_rev & 0x01) != 0) { if ((val & MASTER_WAKEN_25M) == 0 || (val & MASTER_CLK_SEL_DIS) == 0) { val |= MASTER_WAKEN_25M | MASTER_CLK_SEL_DIS; CSR_WRITE_4(sc, ALC_MASTER_CFG, val); } } else { if ((val & MASTER_WAKEN_25M) == 0 || (val & MASTER_CLK_SEL_DIS) != 0) { val |= MASTER_WAKEN_25M; val &= ~MASTER_CLK_SEL_DIS; CSR_WRITE_4(sc, ALC_MASTER_CFG, val); } } } alc_aspm(sc, 1, IFM_UNKNOWN); } static void alc_config_msi(struct alc_softc *sc) { uint32_t ctl, mod; if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) != 0) { /* * It seems interrupt moderation is controlled by * ALC_MSI_RETRANS_TIMER register if MSI/MSIX is active. * Driver uses RX interrupt moderation parameter to * program ALC_MSI_RETRANS_TIMER register. */ ctl = CSR_READ_4(sc, ALC_MSI_RETRANS_TIMER); ctl &= ~MSI_RETRANS_TIMER_MASK; ctl &= ~MSI_RETRANS_MASK_SEL_LINE; mod = ALC_USECS(sc->alc_int_rx_mod); if (mod == 0) mod = 1; ctl |= mod; if ((sc->alc_flags & ALC_FLAG_MSIX) != 0) CSR_WRITE_4(sc, ALC_MSI_RETRANS_TIMER, ctl | MSI_RETRANS_MASK_SEL_STD); else if ((sc->alc_flags & ALC_FLAG_MSI) != 0) CSR_WRITE_4(sc, ALC_MSI_RETRANS_TIMER, ctl | MSI_RETRANS_MASK_SEL_LINE); else CSR_WRITE_4(sc, ALC_MSI_RETRANS_TIMER, 0); } } static int alc_attach(device_t dev) { struct alc_softc *sc; struct ifnet *ifp; int base, error, i, msic, msixc; uint16_t burst; error = 0; sc = device_get_softc(dev); sc->alc_dev = dev; sc->alc_rev = pci_get_revid(dev); mtx_init(&sc->alc_mtx, device_get_nameunit(dev), MTX_NETWORK_LOCK, MTX_DEF); callout_init_mtx(&sc->alc_tick_ch, &sc->alc_mtx, 0); TASK_INIT(&sc->alc_int_task, 0, alc_int_task, sc); sc->alc_ident = alc_find_ident(dev); /* Map the device. */ pci_enable_busmaster(dev); sc->alc_res_spec = alc_res_spec_mem; sc->alc_irq_spec = alc_irq_spec_legacy; error = bus_alloc_resources(dev, sc->alc_res_spec, sc->alc_res); if (error != 0) { device_printf(dev, "cannot allocate memory resources.\n"); goto fail; } /* Set PHY address. */ sc->alc_phyaddr = ALC_PHY_ADDR; /* * One odd thing is AR8132 uses the same PHY hardware(F1 * gigabit PHY) of AR8131. So atphy(4) of AR8132 reports * the PHY supports 1000Mbps but that's not true. The PHY * used in AR8132 can't establish gigabit link even if it * shows the same PHY model/revision number of AR8131. */ switch (sc->alc_ident->deviceid) { case DEVICEID_ATHEROS_E2200: case DEVICEID_ATHEROS_E2400: + case DEVICEID_ATHEROS_E2500: sc->alc_flags |= ALC_FLAG_E2X00; /* FALLTHROUGH */ case DEVICEID_ATHEROS_AR8161: if (pci_get_subvendor(dev) == VENDORID_ATHEROS && pci_get_subdevice(dev) == 0x0091 && sc->alc_rev == 0) sc->alc_flags |= ALC_FLAG_LINK_WAR; /* FALLTHROUGH */ case DEVICEID_ATHEROS_AR8171: sc->alc_flags |= ALC_FLAG_AR816X_FAMILY; break; case DEVICEID_ATHEROS_AR8162: case DEVICEID_ATHEROS_AR8172: sc->alc_flags |= ALC_FLAG_FASTETHER | ALC_FLAG_AR816X_FAMILY; break; case DEVICEID_ATHEROS_AR8152_B: case DEVICEID_ATHEROS_AR8152_B2: sc->alc_flags |= ALC_FLAG_APS; /* FALLTHROUGH */ case DEVICEID_ATHEROS_AR8132: sc->alc_flags |= ALC_FLAG_FASTETHER; break; case DEVICEID_ATHEROS_AR8151: case DEVICEID_ATHEROS_AR8151_V2: sc->alc_flags |= ALC_FLAG_APS; /* FALLTHROUGH */ default: break; } sc->alc_flags |= ALC_FLAG_JUMBO; /* * It seems that AR813x/AR815x has silicon bug for SMB. In * addition, Atheros said that enabling SMB wouldn't improve * performance. However I think it's bad to access lots of * registers to extract MAC statistics. */ sc->alc_flags |= ALC_FLAG_SMB_BUG; /* * Don't use Tx CMB. It is known to have silicon bug. */ sc->alc_flags |= ALC_FLAG_CMB_BUG; sc->alc_chip_rev = CSR_READ_4(sc, ALC_MASTER_CFG) >> MASTER_CHIP_REV_SHIFT; if (bootverbose) { device_printf(dev, "PCI device revision : 0x%04x\n", sc->alc_rev); device_printf(dev, "Chip id/revision : 0x%04x\n", sc->alc_chip_rev); if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) != 0) device_printf(dev, "AR816x revision : 0x%x\n", AR816X_REV(sc->alc_rev)); } device_printf(dev, "%u Tx FIFO, %u Rx FIFO\n", CSR_READ_4(sc, ALC_SRAM_TX_FIFO_LEN) * 8, CSR_READ_4(sc, ALC_SRAM_RX_FIFO_LEN) * 8); /* Initialize DMA parameters. */ sc->alc_dma_rd_burst = 0; sc->alc_dma_wr_burst = 0; sc->alc_rcb = DMA_CFG_RCB_64; if (pci_find_cap(dev, PCIY_EXPRESS, &base) == 0) { sc->alc_flags |= ALC_FLAG_PCIE; sc->alc_expcap = base; burst = CSR_READ_2(sc, base + PCIER_DEVICE_CTL); sc->alc_dma_rd_burst = (burst & PCIEM_CTL_MAX_READ_REQUEST) >> 12; sc->alc_dma_wr_burst = (burst & PCIEM_CTL_MAX_PAYLOAD) >> 5; if (bootverbose) { device_printf(dev, "Read request size : %u bytes.\n", alc_dma_burst[sc->alc_dma_rd_burst]); device_printf(dev, "TLP payload size : %u bytes.\n", alc_dma_burst[sc->alc_dma_wr_burst]); } if (alc_dma_burst[sc->alc_dma_rd_burst] > 1024) sc->alc_dma_rd_burst = 3; if (alc_dma_burst[sc->alc_dma_wr_burst] > 1024) sc->alc_dma_wr_burst = 3; /* - * Force maximum payload size to 128 bytes for E2200/E2400. + * Force maximum payload size to 128 bytes for + * E2200/E2400/E2500. * Otherwise it triggers DMA write error. */ if ((sc->alc_flags & ALC_FLAG_E2X00) != 0) sc->alc_dma_wr_burst = 0; alc_init_pcie(sc); } /* Reset PHY. */ alc_phy_reset(sc); /* Reset the ethernet controller. */ alc_stop_mac(sc); alc_reset(sc); /* Allocate IRQ resources. */ msixc = pci_msix_count(dev); msic = pci_msi_count(dev); if (bootverbose) { device_printf(dev, "MSIX count : %d\n", msixc); device_printf(dev, "MSI count : %d\n", msic); } if (msixc > 1) msixc = 1; if (msic > 1) msic = 1; /* * Prefer MSIX over MSI. * AR816x controller has a silicon bug that MSI interrupt * does not assert if PCIM_CMD_INTxDIS bit of command * register is set. pci(4) was taught to handle that case. */ if (msix_disable == 0 || msi_disable == 0) { if (msix_disable == 0 && msixc > 0 && pci_alloc_msix(dev, &msixc) == 0) { if (msic == 1) { device_printf(dev, "Using %d MSIX message(s).\n", msixc); sc->alc_flags |= ALC_FLAG_MSIX; sc->alc_irq_spec = alc_irq_spec_msix; } else pci_release_msi(dev); } if (msi_disable == 0 && (sc->alc_flags & ALC_FLAG_MSIX) == 0 && msic > 0 && pci_alloc_msi(dev, &msic) == 0) { if (msic == 1) { device_printf(dev, "Using %d MSI message(s).\n", msic); sc->alc_flags |= ALC_FLAG_MSI; sc->alc_irq_spec = alc_irq_spec_msi; } else pci_release_msi(dev); } } error = bus_alloc_resources(dev, sc->alc_irq_spec, sc->alc_irq); if (error != 0) { device_printf(dev, "cannot allocate IRQ resources.\n"); goto fail; } /* Create device sysctl node. */ alc_sysctl_node(sc); if ((error = alc_dma_alloc(sc)) != 0) goto fail; /* Load station address. */ alc_get_macaddr(sc); ifp = sc->alc_ifp = if_alloc(IFT_ETHER); if (ifp == NULL) { device_printf(dev, "cannot allocate ifnet structure.\n"); error = ENXIO; goto fail; } ifp->if_softc = sc; if_initname(ifp, device_get_name(dev), device_get_unit(dev)); ifp->if_flags = IFF_BROADCAST | IFF_SIMPLEX | IFF_MULTICAST; ifp->if_ioctl = alc_ioctl; ifp->if_start = alc_start; ifp->if_init = alc_init; ifp->if_snd.ifq_drv_maxlen = ALC_TX_RING_CNT - 1; IFQ_SET_MAXLEN(&ifp->if_snd, ifp->if_snd.ifq_drv_maxlen); IFQ_SET_READY(&ifp->if_snd); ifp->if_capabilities = IFCAP_TXCSUM | IFCAP_TSO4; ifp->if_hwassist = ALC_CSUM_FEATURES | CSUM_TSO; if (pci_find_cap(dev, PCIY_PMG, &base) == 0) { ifp->if_capabilities |= IFCAP_WOL_MAGIC | IFCAP_WOL_MCAST; sc->alc_flags |= ALC_FLAG_PM; sc->alc_pmcap = base; } ifp->if_capenable = ifp->if_capabilities; /* Set up MII bus. */ error = mii_attach(dev, &sc->alc_miibus, ifp, alc_mediachange, alc_mediastatus, BMSR_DEFCAPMASK, sc->alc_phyaddr, MII_OFFSET_ANY, MIIF_DOPAUSE); if (error != 0) { device_printf(dev, "attaching PHYs failed\n"); goto fail; } ether_ifattach(ifp, sc->alc_eaddr); /* VLAN capability setup. */ ifp->if_capabilities |= IFCAP_VLAN_MTU | IFCAP_VLAN_HWTAGGING | IFCAP_VLAN_HWCSUM | IFCAP_VLAN_HWTSO; ifp->if_capenable = ifp->if_capabilities; /* * XXX * It seems enabling Tx checksum offloading makes more trouble. * Sometimes the controller does not receive any frames when * Tx checksum offloading is enabled. I'm not sure whether this * is a bug in Tx checksum offloading logic or I got broken * sample boards. To safety, don't enable Tx checksum offloading * by default but give chance to users to toggle it if they know * their controllers work without problems. * Fortunately, Tx checksum offloading for AR816x family * seems to work. */ if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) == 0) { ifp->if_capenable &= ~IFCAP_TXCSUM; ifp->if_hwassist &= ~ALC_CSUM_FEATURES; } /* Tell the upper layer(s) we support long frames. */ ifp->if_hdrlen = sizeof(struct ether_vlan_header); /* Create local taskq. */ sc->alc_tq = taskqueue_create_fast("alc_taskq", M_WAITOK, taskqueue_thread_enqueue, &sc->alc_tq); if (sc->alc_tq == NULL) { device_printf(dev, "could not create taskqueue.\n"); ether_ifdetach(ifp); error = ENXIO; goto fail; } taskqueue_start_threads(&sc->alc_tq, 1, PI_NET, "%s taskq", device_get_nameunit(sc->alc_dev)); alc_config_msi(sc); if ((sc->alc_flags & ALC_FLAG_MSIX) != 0) msic = ALC_MSIX_MESSAGES; else if ((sc->alc_flags & ALC_FLAG_MSI) != 0) msic = ALC_MSI_MESSAGES; else msic = 1; for (i = 0; i < msic; i++) { error = bus_setup_intr(dev, sc->alc_irq[i], INTR_TYPE_NET | INTR_MPSAFE, alc_intr, NULL, sc, &sc->alc_intrhand[i]); if (error != 0) break; } if (error != 0) { device_printf(dev, "could not set up interrupt handler.\n"); taskqueue_free(sc->alc_tq); sc->alc_tq = NULL; ether_ifdetach(ifp); goto fail; } fail: if (error != 0) alc_detach(dev); return (error); } static int alc_detach(device_t dev) { struct alc_softc *sc; struct ifnet *ifp; int i, msic; sc = device_get_softc(dev); ifp = sc->alc_ifp; if (device_is_attached(dev)) { ether_ifdetach(ifp); ALC_LOCK(sc); alc_stop(sc); ALC_UNLOCK(sc); callout_drain(&sc->alc_tick_ch); taskqueue_drain(sc->alc_tq, &sc->alc_int_task); } if (sc->alc_tq != NULL) { taskqueue_drain(sc->alc_tq, &sc->alc_int_task); taskqueue_free(sc->alc_tq); sc->alc_tq = NULL; } if (sc->alc_miibus != NULL) { device_delete_child(dev, sc->alc_miibus); sc->alc_miibus = NULL; } bus_generic_detach(dev); alc_dma_free(sc); if (ifp != NULL) { if_free(ifp); sc->alc_ifp = NULL; } if ((sc->alc_flags & ALC_FLAG_MSIX) != 0) msic = ALC_MSIX_MESSAGES; else if ((sc->alc_flags & ALC_FLAG_MSI) != 0) msic = ALC_MSI_MESSAGES; else msic = 1; for (i = 0; i < msic; i++) { if (sc->alc_intrhand[i] != NULL) { bus_teardown_intr(dev, sc->alc_irq[i], sc->alc_intrhand[i]); sc->alc_intrhand[i] = NULL; } } if (sc->alc_res[0] != NULL) alc_phy_down(sc); bus_release_resources(dev, sc->alc_irq_spec, sc->alc_irq); if ((sc->alc_flags & (ALC_FLAG_MSI | ALC_FLAG_MSIX)) != 0) pci_release_msi(dev); bus_release_resources(dev, sc->alc_res_spec, sc->alc_res); mtx_destroy(&sc->alc_mtx); return (0); } #define ALC_SYSCTL_STAT_ADD32(c, h, n, p, d) \ SYSCTL_ADD_UINT(c, h, OID_AUTO, n, CTLFLAG_RD, p, 0, d) #define ALC_SYSCTL_STAT_ADD64(c, h, n, p, d) \ SYSCTL_ADD_UQUAD(c, h, OID_AUTO, n, CTLFLAG_RD, p, d) static void alc_sysctl_node(struct alc_softc *sc) { struct sysctl_ctx_list *ctx; struct sysctl_oid_list *child, *parent; struct sysctl_oid *tree; struct alc_hw_stats *stats; int error; stats = &sc->alc_stats; ctx = device_get_sysctl_ctx(sc->alc_dev); child = SYSCTL_CHILDREN(device_get_sysctl_tree(sc->alc_dev)); SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "int_rx_mod", CTLTYPE_INT | CTLFLAG_RW, &sc->alc_int_rx_mod, 0, sysctl_hw_alc_int_mod, "I", "alc Rx interrupt moderation"); SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "int_tx_mod", CTLTYPE_INT | CTLFLAG_RW, &sc->alc_int_tx_mod, 0, sysctl_hw_alc_int_mod, "I", "alc Tx interrupt moderation"); /* Pull in device tunables. */ sc->alc_int_rx_mod = ALC_IM_RX_TIMER_DEFAULT; error = resource_int_value(device_get_name(sc->alc_dev), device_get_unit(sc->alc_dev), "int_rx_mod", &sc->alc_int_rx_mod); if (error == 0) { if (sc->alc_int_rx_mod < ALC_IM_TIMER_MIN || sc->alc_int_rx_mod > ALC_IM_TIMER_MAX) { device_printf(sc->alc_dev, "int_rx_mod value out of " "range; using default: %d\n", ALC_IM_RX_TIMER_DEFAULT); sc->alc_int_rx_mod = ALC_IM_RX_TIMER_DEFAULT; } } sc->alc_int_tx_mod = ALC_IM_TX_TIMER_DEFAULT; error = resource_int_value(device_get_name(sc->alc_dev), device_get_unit(sc->alc_dev), "int_tx_mod", &sc->alc_int_tx_mod); if (error == 0) { if (sc->alc_int_tx_mod < ALC_IM_TIMER_MIN || sc->alc_int_tx_mod > ALC_IM_TIMER_MAX) { device_printf(sc->alc_dev, "int_tx_mod value out of " "range; using default: %d\n", ALC_IM_TX_TIMER_DEFAULT); sc->alc_int_tx_mod = ALC_IM_TX_TIMER_DEFAULT; } } SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "process_limit", CTLTYPE_INT | CTLFLAG_RW, &sc->alc_process_limit, 0, sysctl_hw_alc_proc_limit, "I", "max number of Rx events to process"); /* Pull in device tunables. */ sc->alc_process_limit = ALC_PROC_DEFAULT; error = resource_int_value(device_get_name(sc->alc_dev), device_get_unit(sc->alc_dev), "process_limit", &sc->alc_process_limit); if (error == 0) { if (sc->alc_process_limit < ALC_PROC_MIN || sc->alc_process_limit > ALC_PROC_MAX) { device_printf(sc->alc_dev, "process_limit value out of range; " "using default: %d\n", ALC_PROC_DEFAULT); sc->alc_process_limit = ALC_PROC_DEFAULT; } } tree = SYSCTL_ADD_NODE(ctx, child, OID_AUTO, "stats", CTLFLAG_RD, NULL, "ALC statistics"); parent = SYSCTL_CHILDREN(tree); /* Rx statistics. */ tree = SYSCTL_ADD_NODE(ctx, parent, OID_AUTO, "rx", CTLFLAG_RD, NULL, "Rx MAC statistics"); child = SYSCTL_CHILDREN(tree); ALC_SYSCTL_STAT_ADD32(ctx, child, "good_frames", &stats->rx_frames, "Good frames"); ALC_SYSCTL_STAT_ADD32(ctx, child, "good_bcast_frames", &stats->rx_bcast_frames, "Good broadcast frames"); ALC_SYSCTL_STAT_ADD32(ctx, child, "good_mcast_frames", &stats->rx_mcast_frames, "Good multicast frames"); ALC_SYSCTL_STAT_ADD32(ctx, child, "pause_frames", &stats->rx_pause_frames, "Pause control frames"); ALC_SYSCTL_STAT_ADD32(ctx, child, "control_frames", &stats->rx_control_frames, "Control frames"); ALC_SYSCTL_STAT_ADD32(ctx, child, "crc_errs", &stats->rx_crcerrs, "CRC errors"); ALC_SYSCTL_STAT_ADD32(ctx, child, "len_errs", &stats->rx_lenerrs, "Frames with length mismatched"); ALC_SYSCTL_STAT_ADD64(ctx, child, "good_octets", &stats->rx_bytes, "Good octets"); ALC_SYSCTL_STAT_ADD64(ctx, child, "good_bcast_octets", &stats->rx_bcast_bytes, "Good broadcast octets"); ALC_SYSCTL_STAT_ADD64(ctx, child, "good_mcast_octets", &stats->rx_mcast_bytes, "Good multicast octets"); ALC_SYSCTL_STAT_ADD32(ctx, child, "runts", &stats->rx_runts, "Too short frames"); ALC_SYSCTL_STAT_ADD32(ctx, child, "fragments", &stats->rx_fragments, "Fragmented frames"); ALC_SYSCTL_STAT_ADD32(ctx, child, "frames_64", &stats->rx_pkts_64, "64 bytes frames"); ALC_SYSCTL_STAT_ADD32(ctx, child, "frames_65_127", &stats->rx_pkts_65_127, "65 to 127 bytes frames"); ALC_SYSCTL_STAT_ADD32(ctx, child, "frames_128_255", &stats->rx_pkts_128_255, "128 to 255 bytes frames"); ALC_SYSCTL_STAT_ADD32(ctx, child, "frames_256_511", &stats->rx_pkts_256_511, "256 to 511 bytes frames"); ALC_SYSCTL_STAT_ADD32(ctx, child, "frames_512_1023", &stats->rx_pkts_512_1023, "512 to 1023 bytes frames"); ALC_SYSCTL_STAT_ADD32(ctx, child, "frames_1024_1518", &stats->rx_pkts_1024_1518, "1024 to 1518 bytes frames"); ALC_SYSCTL_STAT_ADD32(ctx, child, "frames_1519_max", &stats->rx_pkts_1519_max, "1519 to max frames"); ALC_SYSCTL_STAT_ADD32(ctx, child, "trunc_errs", &stats->rx_pkts_truncated, "Truncated frames due to MTU size"); ALC_SYSCTL_STAT_ADD32(ctx, child, "fifo_oflows", &stats->rx_fifo_oflows, "FIFO overflows"); ALC_SYSCTL_STAT_ADD32(ctx, child, "rrs_errs", &stats->rx_rrs_errs, "Return status write-back errors"); ALC_SYSCTL_STAT_ADD32(ctx, child, "align_errs", &stats->rx_alignerrs, "Alignment errors"); ALC_SYSCTL_STAT_ADD32(ctx, child, "filtered", &stats->rx_pkts_filtered, "Frames dropped due to address filtering"); /* Tx statistics. */ tree = SYSCTL_ADD_NODE(ctx, parent, OID_AUTO, "tx", CTLFLAG_RD, NULL, "Tx MAC statistics"); child = SYSCTL_CHILDREN(tree); ALC_SYSCTL_STAT_ADD32(ctx, child, "good_frames", &stats->tx_frames, "Good frames"); ALC_SYSCTL_STAT_ADD32(ctx, child, "good_bcast_frames", &stats->tx_bcast_frames, "Good broadcast frames"); ALC_SYSCTL_STAT_ADD32(ctx, child, "good_mcast_frames", &stats->tx_mcast_frames, "Good multicast frames"); ALC_SYSCTL_STAT_ADD32(ctx, child, "pause_frames", &stats->tx_pause_frames, "Pause control frames"); ALC_SYSCTL_STAT_ADD32(ctx, child, "control_frames", &stats->tx_control_frames, "Control frames"); ALC_SYSCTL_STAT_ADD32(ctx, child, "excess_defers", &stats->tx_excess_defer, "Frames with excessive derferrals"); ALC_SYSCTL_STAT_ADD32(ctx, child, "defers", &stats->tx_excess_defer, "Frames with derferrals"); ALC_SYSCTL_STAT_ADD64(ctx, child, "good_octets", &stats->tx_bytes, "Good octets"); ALC_SYSCTL_STAT_ADD64(ctx, child, "good_bcast_octets", &stats->tx_bcast_bytes, "Good broadcast octets"); ALC_SYSCTL_STAT_ADD64(ctx, child, "good_mcast_octets", &stats->tx_mcast_bytes, "Good multicast octets"); ALC_SYSCTL_STAT_ADD32(ctx, child, "frames_64", &stats->tx_pkts_64, "64 bytes frames"); ALC_SYSCTL_STAT_ADD32(ctx, child, "frames_65_127", &stats->tx_pkts_65_127, "65 to 127 bytes frames"); ALC_SYSCTL_STAT_ADD32(ctx, child, "frames_128_255", &stats->tx_pkts_128_255, "128 to 255 bytes frames"); ALC_SYSCTL_STAT_ADD32(ctx, child, "frames_256_511", &stats->tx_pkts_256_511, "256 to 511 bytes frames"); ALC_SYSCTL_STAT_ADD32(ctx, child, "frames_512_1023", &stats->tx_pkts_512_1023, "512 to 1023 bytes frames"); ALC_SYSCTL_STAT_ADD32(ctx, child, "frames_1024_1518", &stats->tx_pkts_1024_1518, "1024 to 1518 bytes frames"); ALC_SYSCTL_STAT_ADD32(ctx, child, "frames_1519_max", &stats->tx_pkts_1519_max, "1519 to max frames"); ALC_SYSCTL_STAT_ADD32(ctx, child, "single_colls", &stats->tx_single_colls, "Single collisions"); ALC_SYSCTL_STAT_ADD32(ctx, child, "multi_colls", &stats->tx_multi_colls, "Multiple collisions"); ALC_SYSCTL_STAT_ADD32(ctx, child, "late_colls", &stats->tx_late_colls, "Late collisions"); ALC_SYSCTL_STAT_ADD32(ctx, child, "excess_colls", &stats->tx_excess_colls, "Excessive collisions"); ALC_SYSCTL_STAT_ADD32(ctx, child, "underruns", &stats->tx_underrun, "FIFO underruns"); ALC_SYSCTL_STAT_ADD32(ctx, child, "desc_underruns", &stats->tx_desc_underrun, "Descriptor write-back errors"); ALC_SYSCTL_STAT_ADD32(ctx, child, "len_errs", &stats->tx_lenerrs, "Frames with length mismatched"); ALC_SYSCTL_STAT_ADD32(ctx, child, "trunc_errs", &stats->tx_pkts_truncated, "Truncated frames due to MTU size"); } #undef ALC_SYSCTL_STAT_ADD32 #undef ALC_SYSCTL_STAT_ADD64 struct alc_dmamap_arg { bus_addr_t alc_busaddr; }; static void alc_dmamap_cb(void *arg, bus_dma_segment_t *segs, int nsegs, int error) { struct alc_dmamap_arg *ctx; if (error != 0) return; KASSERT(nsegs == 1, ("%s: %d segments returned!", __func__, nsegs)); ctx = (struct alc_dmamap_arg *)arg; ctx->alc_busaddr = segs[0].ds_addr; } /* * Normal and high Tx descriptors shares single Tx high address. * Four Rx descriptor/return rings and CMB shares the same Rx * high address. */ static int alc_check_boundary(struct alc_softc *sc) { bus_addr_t cmb_end, rx_ring_end, rr_ring_end, tx_ring_end; rx_ring_end = sc->alc_rdata.alc_rx_ring_paddr + ALC_RX_RING_SZ; rr_ring_end = sc->alc_rdata.alc_rr_ring_paddr + ALC_RR_RING_SZ; cmb_end = sc->alc_rdata.alc_cmb_paddr + ALC_CMB_SZ; tx_ring_end = sc->alc_rdata.alc_tx_ring_paddr + ALC_TX_RING_SZ; /* 4GB boundary crossing is not allowed. */ if ((ALC_ADDR_HI(rx_ring_end) != ALC_ADDR_HI(sc->alc_rdata.alc_rx_ring_paddr)) || (ALC_ADDR_HI(rr_ring_end) != ALC_ADDR_HI(sc->alc_rdata.alc_rr_ring_paddr)) || (ALC_ADDR_HI(cmb_end) != ALC_ADDR_HI(sc->alc_rdata.alc_cmb_paddr)) || (ALC_ADDR_HI(tx_ring_end) != ALC_ADDR_HI(sc->alc_rdata.alc_tx_ring_paddr))) return (EFBIG); /* * Make sure Rx return descriptor/Rx descriptor/CMB use * the same high address. */ if ((ALC_ADDR_HI(rx_ring_end) != ALC_ADDR_HI(rr_ring_end)) || (ALC_ADDR_HI(rx_ring_end) != ALC_ADDR_HI(cmb_end))) return (EFBIG); return (0); } static int alc_dma_alloc(struct alc_softc *sc) { struct alc_txdesc *txd; struct alc_rxdesc *rxd; bus_addr_t lowaddr; struct alc_dmamap_arg ctx; int error, i; lowaddr = BUS_SPACE_MAXADDR; again: /* Create parent DMA tag. */ error = bus_dma_tag_create( bus_get_dma_tag(sc->alc_dev), /* parent */ 1, 0, /* alignment, boundary */ lowaddr, /* lowaddr */ BUS_SPACE_MAXADDR, /* highaddr */ NULL, NULL, /* filter, filterarg */ BUS_SPACE_MAXSIZE_32BIT, /* maxsize */ 0, /* nsegments */ BUS_SPACE_MAXSIZE_32BIT, /* maxsegsize */ 0, /* flags */ NULL, NULL, /* lockfunc, lockarg */ &sc->alc_cdata.alc_parent_tag); if (error != 0) { device_printf(sc->alc_dev, "could not create parent DMA tag.\n"); goto fail; } /* Create DMA tag for Tx descriptor ring. */ error = bus_dma_tag_create( sc->alc_cdata.alc_parent_tag, /* parent */ ALC_TX_RING_ALIGN, 0, /* alignment, boundary */ BUS_SPACE_MAXADDR, /* lowaddr */ BUS_SPACE_MAXADDR, /* highaddr */ NULL, NULL, /* filter, filterarg */ ALC_TX_RING_SZ, /* maxsize */ 1, /* nsegments */ ALC_TX_RING_SZ, /* maxsegsize */ 0, /* flags */ NULL, NULL, /* lockfunc, lockarg */ &sc->alc_cdata.alc_tx_ring_tag); if (error != 0) { device_printf(sc->alc_dev, "could not create Tx ring DMA tag.\n"); goto fail; } /* Create DMA tag for Rx free descriptor ring. */ error = bus_dma_tag_create( sc->alc_cdata.alc_parent_tag, /* parent */ ALC_RX_RING_ALIGN, 0, /* alignment, boundary */ BUS_SPACE_MAXADDR, /* lowaddr */ BUS_SPACE_MAXADDR, /* highaddr */ NULL, NULL, /* filter, filterarg */ ALC_RX_RING_SZ, /* maxsize */ 1, /* nsegments */ ALC_RX_RING_SZ, /* maxsegsize */ 0, /* flags */ NULL, NULL, /* lockfunc, lockarg */ &sc->alc_cdata.alc_rx_ring_tag); if (error != 0) { device_printf(sc->alc_dev, "could not create Rx ring DMA tag.\n"); goto fail; } /* Create DMA tag for Rx return descriptor ring. */ error = bus_dma_tag_create( sc->alc_cdata.alc_parent_tag, /* parent */ ALC_RR_RING_ALIGN, 0, /* alignment, boundary */ BUS_SPACE_MAXADDR, /* lowaddr */ BUS_SPACE_MAXADDR, /* highaddr */ NULL, NULL, /* filter, filterarg */ ALC_RR_RING_SZ, /* maxsize */ 1, /* nsegments */ ALC_RR_RING_SZ, /* maxsegsize */ 0, /* flags */ NULL, NULL, /* lockfunc, lockarg */ &sc->alc_cdata.alc_rr_ring_tag); if (error != 0) { device_printf(sc->alc_dev, "could not create Rx return ring DMA tag.\n"); goto fail; } /* Create DMA tag for coalescing message block. */ error = bus_dma_tag_create( sc->alc_cdata.alc_parent_tag, /* parent */ ALC_CMB_ALIGN, 0, /* alignment, boundary */ BUS_SPACE_MAXADDR, /* lowaddr */ BUS_SPACE_MAXADDR, /* highaddr */ NULL, NULL, /* filter, filterarg */ ALC_CMB_SZ, /* maxsize */ 1, /* nsegments */ ALC_CMB_SZ, /* maxsegsize */ 0, /* flags */ NULL, NULL, /* lockfunc, lockarg */ &sc->alc_cdata.alc_cmb_tag); if (error != 0) { device_printf(sc->alc_dev, "could not create CMB DMA tag.\n"); goto fail; } /* Create DMA tag for status message block. */ error = bus_dma_tag_create( sc->alc_cdata.alc_parent_tag, /* parent */ ALC_SMB_ALIGN, 0, /* alignment, boundary */ BUS_SPACE_MAXADDR, /* lowaddr */ BUS_SPACE_MAXADDR, /* highaddr */ NULL, NULL, /* filter, filterarg */ ALC_SMB_SZ, /* maxsize */ 1, /* nsegments */ ALC_SMB_SZ, /* maxsegsize */ 0, /* flags */ NULL, NULL, /* lockfunc, lockarg */ &sc->alc_cdata.alc_smb_tag); if (error != 0) { device_printf(sc->alc_dev, "could not create SMB DMA tag.\n"); goto fail; } /* Allocate DMA'able memory and load the DMA map for Tx ring. */ error = bus_dmamem_alloc(sc->alc_cdata.alc_tx_ring_tag, (void **)&sc->alc_rdata.alc_tx_ring, BUS_DMA_WAITOK | BUS_DMA_ZERO | BUS_DMA_COHERENT, &sc->alc_cdata.alc_tx_ring_map); if (error != 0) { device_printf(sc->alc_dev, "could not allocate DMA'able memory for Tx ring.\n"); goto fail; } ctx.alc_busaddr = 0; error = bus_dmamap_load(sc->alc_cdata.alc_tx_ring_tag, sc->alc_cdata.alc_tx_ring_map, sc->alc_rdata.alc_tx_ring, ALC_TX_RING_SZ, alc_dmamap_cb, &ctx, 0); if (error != 0 || ctx.alc_busaddr == 0) { device_printf(sc->alc_dev, "could not load DMA'able memory for Tx ring.\n"); goto fail; } sc->alc_rdata.alc_tx_ring_paddr = ctx.alc_busaddr; /* Allocate DMA'able memory and load the DMA map for Rx ring. */ error = bus_dmamem_alloc(sc->alc_cdata.alc_rx_ring_tag, (void **)&sc->alc_rdata.alc_rx_ring, BUS_DMA_WAITOK | BUS_DMA_ZERO | BUS_DMA_COHERENT, &sc->alc_cdata.alc_rx_ring_map); if (error != 0) { device_printf(sc->alc_dev, "could not allocate DMA'able memory for Rx ring.\n"); goto fail; } ctx.alc_busaddr = 0; error = bus_dmamap_load(sc->alc_cdata.alc_rx_ring_tag, sc->alc_cdata.alc_rx_ring_map, sc->alc_rdata.alc_rx_ring, ALC_RX_RING_SZ, alc_dmamap_cb, &ctx, 0); if (error != 0 || ctx.alc_busaddr == 0) { device_printf(sc->alc_dev, "could not load DMA'able memory for Rx ring.\n"); goto fail; } sc->alc_rdata.alc_rx_ring_paddr = ctx.alc_busaddr; /* Allocate DMA'able memory and load the DMA map for Rx return ring. */ error = bus_dmamem_alloc(sc->alc_cdata.alc_rr_ring_tag, (void **)&sc->alc_rdata.alc_rr_ring, BUS_DMA_WAITOK | BUS_DMA_ZERO | BUS_DMA_COHERENT, &sc->alc_cdata.alc_rr_ring_map); if (error != 0) { device_printf(sc->alc_dev, "could not allocate DMA'able memory for Rx return ring.\n"); goto fail; } ctx.alc_busaddr = 0; error = bus_dmamap_load(sc->alc_cdata.alc_rr_ring_tag, sc->alc_cdata.alc_rr_ring_map, sc->alc_rdata.alc_rr_ring, ALC_RR_RING_SZ, alc_dmamap_cb, &ctx, 0); if (error != 0 || ctx.alc_busaddr == 0) { device_printf(sc->alc_dev, "could not load DMA'able memory for Tx ring.\n"); goto fail; } sc->alc_rdata.alc_rr_ring_paddr = ctx.alc_busaddr; /* Allocate DMA'able memory and load the DMA map for CMB. */ error = bus_dmamem_alloc(sc->alc_cdata.alc_cmb_tag, (void **)&sc->alc_rdata.alc_cmb, BUS_DMA_WAITOK | BUS_DMA_ZERO | BUS_DMA_COHERENT, &sc->alc_cdata.alc_cmb_map); if (error != 0) { device_printf(sc->alc_dev, "could not allocate DMA'able memory for CMB.\n"); goto fail; } ctx.alc_busaddr = 0; error = bus_dmamap_load(sc->alc_cdata.alc_cmb_tag, sc->alc_cdata.alc_cmb_map, sc->alc_rdata.alc_cmb, ALC_CMB_SZ, alc_dmamap_cb, &ctx, 0); if (error != 0 || ctx.alc_busaddr == 0) { device_printf(sc->alc_dev, "could not load DMA'able memory for CMB.\n"); goto fail; } sc->alc_rdata.alc_cmb_paddr = ctx.alc_busaddr; /* Allocate DMA'able memory and load the DMA map for SMB. */ error = bus_dmamem_alloc(sc->alc_cdata.alc_smb_tag, (void **)&sc->alc_rdata.alc_smb, BUS_DMA_WAITOK | BUS_DMA_ZERO | BUS_DMA_COHERENT, &sc->alc_cdata.alc_smb_map); if (error != 0) { device_printf(sc->alc_dev, "could not allocate DMA'able memory for SMB.\n"); goto fail; } ctx.alc_busaddr = 0; error = bus_dmamap_load(sc->alc_cdata.alc_smb_tag, sc->alc_cdata.alc_smb_map, sc->alc_rdata.alc_smb, ALC_SMB_SZ, alc_dmamap_cb, &ctx, 0); if (error != 0 || ctx.alc_busaddr == 0) { device_printf(sc->alc_dev, "could not load DMA'able memory for CMB.\n"); goto fail; } sc->alc_rdata.alc_smb_paddr = ctx.alc_busaddr; /* Make sure we've not crossed 4GB boundary. */ if (lowaddr != BUS_SPACE_MAXADDR_32BIT && (error = alc_check_boundary(sc)) != 0) { device_printf(sc->alc_dev, "4GB boundary crossed, " "switching to 32bit DMA addressing mode.\n"); alc_dma_free(sc); /* * Limit max allowable DMA address space to 32bit * and try again. */ lowaddr = BUS_SPACE_MAXADDR_32BIT; goto again; } /* * Create Tx buffer parent tag. * AR81[3567]x allows 64bit DMA addressing of Tx/Rx buffers * so it needs separate parent DMA tag as parent DMA address * space could be restricted to be within 32bit address space * by 4GB boundary crossing. */ error = bus_dma_tag_create( bus_get_dma_tag(sc->alc_dev), /* parent */ 1, 0, /* alignment, boundary */ BUS_SPACE_MAXADDR, /* lowaddr */ BUS_SPACE_MAXADDR, /* highaddr */ NULL, NULL, /* filter, filterarg */ BUS_SPACE_MAXSIZE_32BIT, /* maxsize */ 0, /* nsegments */ BUS_SPACE_MAXSIZE_32BIT, /* maxsegsize */ 0, /* flags */ NULL, NULL, /* lockfunc, lockarg */ &sc->alc_cdata.alc_buffer_tag); if (error != 0) { device_printf(sc->alc_dev, "could not create parent buffer DMA tag.\n"); goto fail; } /* Create DMA tag for Tx buffers. */ error = bus_dma_tag_create( sc->alc_cdata.alc_buffer_tag, /* parent */ 1, 0, /* alignment, boundary */ BUS_SPACE_MAXADDR, /* lowaddr */ BUS_SPACE_MAXADDR, /* highaddr */ NULL, NULL, /* filter, filterarg */ ALC_TSO_MAXSIZE, /* maxsize */ ALC_MAXTXSEGS, /* nsegments */ ALC_TSO_MAXSEGSIZE, /* maxsegsize */ 0, /* flags */ NULL, NULL, /* lockfunc, lockarg */ &sc->alc_cdata.alc_tx_tag); if (error != 0) { device_printf(sc->alc_dev, "could not create Tx DMA tag.\n"); goto fail; } /* Create DMA tag for Rx buffers. */ error = bus_dma_tag_create( sc->alc_cdata.alc_buffer_tag, /* parent */ ALC_RX_BUF_ALIGN, 0, /* alignment, boundary */ BUS_SPACE_MAXADDR, /* lowaddr */ BUS_SPACE_MAXADDR, /* highaddr */ NULL, NULL, /* filter, filterarg */ MCLBYTES, /* maxsize */ 1, /* nsegments */ MCLBYTES, /* maxsegsize */ 0, /* flags */ NULL, NULL, /* lockfunc, lockarg */ &sc->alc_cdata.alc_rx_tag); if (error != 0) { device_printf(sc->alc_dev, "could not create Rx DMA tag.\n"); goto fail; } /* Create DMA maps for Tx buffers. */ for (i = 0; i < ALC_TX_RING_CNT; i++) { txd = &sc->alc_cdata.alc_txdesc[i]; txd->tx_m = NULL; txd->tx_dmamap = NULL; error = bus_dmamap_create(sc->alc_cdata.alc_tx_tag, 0, &txd->tx_dmamap); if (error != 0) { device_printf(sc->alc_dev, "could not create Tx dmamap.\n"); goto fail; } } /* Create DMA maps for Rx buffers. */ if ((error = bus_dmamap_create(sc->alc_cdata.alc_rx_tag, 0, &sc->alc_cdata.alc_rx_sparemap)) != 0) { device_printf(sc->alc_dev, "could not create spare Rx dmamap.\n"); goto fail; } for (i = 0; i < ALC_RX_RING_CNT; i++) { rxd = &sc->alc_cdata.alc_rxdesc[i]; rxd->rx_m = NULL; rxd->rx_dmamap = NULL; error = bus_dmamap_create(sc->alc_cdata.alc_rx_tag, 0, &rxd->rx_dmamap); if (error != 0) { device_printf(sc->alc_dev, "could not create Rx dmamap.\n"); goto fail; } } fail: return (error); } static void alc_dma_free(struct alc_softc *sc) { struct alc_txdesc *txd; struct alc_rxdesc *rxd; int i; /* Tx buffers. */ if (sc->alc_cdata.alc_tx_tag != NULL) { for (i = 0; i < ALC_TX_RING_CNT; i++) { txd = &sc->alc_cdata.alc_txdesc[i]; if (txd->tx_dmamap != NULL) { bus_dmamap_destroy(sc->alc_cdata.alc_tx_tag, txd->tx_dmamap); txd->tx_dmamap = NULL; } } bus_dma_tag_destroy(sc->alc_cdata.alc_tx_tag); sc->alc_cdata.alc_tx_tag = NULL; } /* Rx buffers */ if (sc->alc_cdata.alc_rx_tag != NULL) { for (i = 0; i < ALC_RX_RING_CNT; i++) { rxd = &sc->alc_cdata.alc_rxdesc[i]; if (rxd->rx_dmamap != NULL) { bus_dmamap_destroy(sc->alc_cdata.alc_rx_tag, rxd->rx_dmamap); rxd->rx_dmamap = NULL; } } if (sc->alc_cdata.alc_rx_sparemap != NULL) { bus_dmamap_destroy(sc->alc_cdata.alc_rx_tag, sc->alc_cdata.alc_rx_sparemap); sc->alc_cdata.alc_rx_sparemap = NULL; } bus_dma_tag_destroy(sc->alc_cdata.alc_rx_tag); sc->alc_cdata.alc_rx_tag = NULL; } /* Tx descriptor ring. */ if (sc->alc_cdata.alc_tx_ring_tag != NULL) { if (sc->alc_rdata.alc_tx_ring_paddr != 0) bus_dmamap_unload(sc->alc_cdata.alc_tx_ring_tag, sc->alc_cdata.alc_tx_ring_map); if (sc->alc_rdata.alc_tx_ring != NULL) bus_dmamem_free(sc->alc_cdata.alc_tx_ring_tag, sc->alc_rdata.alc_tx_ring, sc->alc_cdata.alc_tx_ring_map); sc->alc_rdata.alc_tx_ring_paddr = 0; sc->alc_rdata.alc_tx_ring = NULL; bus_dma_tag_destroy(sc->alc_cdata.alc_tx_ring_tag); sc->alc_cdata.alc_tx_ring_tag = NULL; } /* Rx ring. */ if (sc->alc_cdata.alc_rx_ring_tag != NULL) { if (sc->alc_rdata.alc_rx_ring_paddr != 0) bus_dmamap_unload(sc->alc_cdata.alc_rx_ring_tag, sc->alc_cdata.alc_rx_ring_map); if (sc->alc_rdata.alc_rx_ring != NULL) bus_dmamem_free(sc->alc_cdata.alc_rx_ring_tag, sc->alc_rdata.alc_rx_ring, sc->alc_cdata.alc_rx_ring_map); sc->alc_rdata.alc_rx_ring_paddr = 0; sc->alc_rdata.alc_rx_ring = NULL; bus_dma_tag_destroy(sc->alc_cdata.alc_rx_ring_tag); sc->alc_cdata.alc_rx_ring_tag = NULL; } /* Rx return ring. */ if (sc->alc_cdata.alc_rr_ring_tag != NULL) { if (sc->alc_rdata.alc_rr_ring_paddr != 0) bus_dmamap_unload(sc->alc_cdata.alc_rr_ring_tag, sc->alc_cdata.alc_rr_ring_map); if (sc->alc_rdata.alc_rr_ring != NULL) bus_dmamem_free(sc->alc_cdata.alc_rr_ring_tag, sc->alc_rdata.alc_rr_ring, sc->alc_cdata.alc_rr_ring_map); sc->alc_rdata.alc_rr_ring_paddr = 0; sc->alc_rdata.alc_rr_ring = NULL; bus_dma_tag_destroy(sc->alc_cdata.alc_rr_ring_tag); sc->alc_cdata.alc_rr_ring_tag = NULL; } /* CMB block */ if (sc->alc_cdata.alc_cmb_tag != NULL) { if (sc->alc_rdata.alc_cmb_paddr != 0) bus_dmamap_unload(sc->alc_cdata.alc_cmb_tag, sc->alc_cdata.alc_cmb_map); if (sc->alc_rdata.alc_cmb != NULL) bus_dmamem_free(sc->alc_cdata.alc_cmb_tag, sc->alc_rdata.alc_cmb, sc->alc_cdata.alc_cmb_map); sc->alc_rdata.alc_cmb_paddr = 0; sc->alc_rdata.alc_cmb = NULL; bus_dma_tag_destroy(sc->alc_cdata.alc_cmb_tag); sc->alc_cdata.alc_cmb_tag = NULL; } /* SMB block */ if (sc->alc_cdata.alc_smb_tag != NULL) { if (sc->alc_rdata.alc_smb_paddr != 0) bus_dmamap_unload(sc->alc_cdata.alc_smb_tag, sc->alc_cdata.alc_smb_map); if (sc->alc_rdata.alc_smb != NULL) bus_dmamem_free(sc->alc_cdata.alc_smb_tag, sc->alc_rdata.alc_smb, sc->alc_cdata.alc_smb_map); sc->alc_rdata.alc_smb_paddr = 0; sc->alc_rdata.alc_smb = NULL; bus_dma_tag_destroy(sc->alc_cdata.alc_smb_tag); sc->alc_cdata.alc_smb_tag = NULL; } if (sc->alc_cdata.alc_buffer_tag != NULL) { bus_dma_tag_destroy(sc->alc_cdata.alc_buffer_tag); sc->alc_cdata.alc_buffer_tag = NULL; } if (sc->alc_cdata.alc_parent_tag != NULL) { bus_dma_tag_destroy(sc->alc_cdata.alc_parent_tag); sc->alc_cdata.alc_parent_tag = NULL; } } static int alc_shutdown(device_t dev) { return (alc_suspend(dev)); } /* * Note, this driver resets the link speed to 10/100Mbps by * restarting auto-negotiation in suspend/shutdown phase but we * don't know whether that auto-negotiation would succeed or not * as driver has no control after powering off/suspend operation. * If the renegotiation fail WOL may not work. Running at 1Gbps * will draw more power than 375mA at 3.3V which is specified in * PCI specification and that would result in complete * shutdowning power to ethernet controller. * * TODO * Save current negotiated media speed/duplex/flow-control to * softc and restore the same link again after resuming. PHY * handling such as power down/resetting to 100Mbps may be better * handled in suspend method in phy driver. */ static void alc_setlinkspeed(struct alc_softc *sc) { struct mii_data *mii; int aneg, i; mii = device_get_softc(sc->alc_miibus); mii_pollstat(mii); aneg = 0; if ((mii->mii_media_status & (IFM_ACTIVE | IFM_AVALID)) == (IFM_ACTIVE | IFM_AVALID)) { switch IFM_SUBTYPE(mii->mii_media_active) { case IFM_10_T: case IFM_100_TX: return; case IFM_1000_T: aneg++; break; default: break; } } alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, MII_100T2CR, 0); alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, MII_ANAR, ANAR_TX_FD | ANAR_TX | ANAR_10_FD | ANAR_10 | ANAR_CSMA); alc_miibus_writereg(sc->alc_dev, sc->alc_phyaddr, MII_BMCR, BMCR_RESET | BMCR_AUTOEN | BMCR_STARTNEG); DELAY(1000); if (aneg != 0) { /* * Poll link state until alc(4) get a 10/100Mbps link. */ for (i = 0; i < MII_ANEGTICKS_GIGE; i++) { mii_pollstat(mii); if ((mii->mii_media_status & (IFM_ACTIVE | IFM_AVALID)) == (IFM_ACTIVE | IFM_AVALID)) { switch (IFM_SUBTYPE( mii->mii_media_active)) { case IFM_10_T: case IFM_100_TX: alc_mac_config(sc); return; default: break; } } ALC_UNLOCK(sc); pause("alclnk", hz); ALC_LOCK(sc); } if (i == MII_ANEGTICKS_GIGE) device_printf(sc->alc_dev, "establishing a link failed, WOL may not work!"); } /* * No link, force MAC to have 100Mbps, full-duplex link. * This is the last resort and may/may not work. */ mii->mii_media_status = IFM_AVALID | IFM_ACTIVE; mii->mii_media_active = IFM_ETHER | IFM_100_TX | IFM_FDX; alc_mac_config(sc); } static void alc_setwol(struct alc_softc *sc) { if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) != 0) alc_setwol_816x(sc); else alc_setwol_813x(sc); } static void alc_setwol_813x(struct alc_softc *sc) { struct ifnet *ifp; uint32_t reg, pmcs; uint16_t pmstat; ALC_LOCK_ASSERT(sc); alc_disable_l0s_l1(sc); ifp = sc->alc_ifp; if ((sc->alc_flags & ALC_FLAG_PM) == 0) { /* Disable WOL. */ CSR_WRITE_4(sc, ALC_WOL_CFG, 0); reg = CSR_READ_4(sc, ALC_PCIE_PHYMISC); reg |= PCIE_PHYMISC_FORCE_RCV_DET; CSR_WRITE_4(sc, ALC_PCIE_PHYMISC, reg); /* Force PHY power down. */ alc_phy_down(sc); CSR_WRITE_4(sc, ALC_MASTER_CFG, CSR_READ_4(sc, ALC_MASTER_CFG) | MASTER_CLK_SEL_DIS); return; } if ((ifp->if_capenable & IFCAP_WOL) != 0) { if ((sc->alc_flags & ALC_FLAG_FASTETHER) == 0) alc_setlinkspeed(sc); CSR_WRITE_4(sc, ALC_MASTER_CFG, CSR_READ_4(sc, ALC_MASTER_CFG) & ~MASTER_CLK_SEL_DIS); } pmcs = 0; if ((ifp->if_capenable & IFCAP_WOL_MAGIC) != 0) pmcs |= WOL_CFG_MAGIC | WOL_CFG_MAGIC_ENB; CSR_WRITE_4(sc, ALC_WOL_CFG, pmcs); reg = CSR_READ_4(sc, ALC_MAC_CFG); reg &= ~(MAC_CFG_DBG | MAC_CFG_PROMISC | MAC_CFG_ALLMULTI | MAC_CFG_BCAST); if ((ifp->if_capenable & IFCAP_WOL_MCAST) != 0) reg |= MAC_CFG_ALLMULTI | MAC_CFG_BCAST; if ((ifp->if_capenable & IFCAP_WOL) != 0) reg |= MAC_CFG_RX_ENB; CSR_WRITE_4(sc, ALC_MAC_CFG, reg); reg = CSR_READ_4(sc, ALC_PCIE_PHYMISC); reg |= PCIE_PHYMISC_FORCE_RCV_DET; CSR_WRITE_4(sc, ALC_PCIE_PHYMISC, reg); if ((ifp->if_capenable & IFCAP_WOL) == 0) { /* WOL disabled, PHY power down. */ alc_phy_down(sc); CSR_WRITE_4(sc, ALC_MASTER_CFG, CSR_READ_4(sc, ALC_MASTER_CFG) | MASTER_CLK_SEL_DIS); } /* Request PME. */ pmstat = pci_read_config(sc->alc_dev, sc->alc_pmcap + PCIR_POWER_STATUS, 2); pmstat &= ~(PCIM_PSTAT_PME | PCIM_PSTAT_PMEENABLE); if ((ifp->if_capenable & IFCAP_WOL) != 0) pmstat |= PCIM_PSTAT_PME | PCIM_PSTAT_PMEENABLE; pci_write_config(sc->alc_dev, sc->alc_pmcap + PCIR_POWER_STATUS, pmstat, 2); } static void alc_setwol_816x(struct alc_softc *sc) { struct ifnet *ifp; uint32_t gphy, mac, master, pmcs, reg; uint16_t pmstat; ALC_LOCK_ASSERT(sc); ifp = sc->alc_ifp; master = CSR_READ_4(sc, ALC_MASTER_CFG); master &= ~MASTER_CLK_SEL_DIS; gphy = CSR_READ_4(sc, ALC_GPHY_CFG); gphy &= ~(GPHY_CFG_EXT_RESET | GPHY_CFG_LED_MODE | GPHY_CFG_100AB_ENB | GPHY_CFG_PHY_PLL_ON); gphy |= GPHY_CFG_HIB_EN | GPHY_CFG_HIB_PULSE | GPHY_CFG_SEL_ANA_RESET; if ((sc->alc_flags & ALC_FLAG_PM) == 0) { CSR_WRITE_4(sc, ALC_WOL_CFG, 0); gphy |= GPHY_CFG_PHY_IDDQ | GPHY_CFG_PWDOWN_HW; mac = CSR_READ_4(sc, ALC_MAC_CFG); } else { if ((ifp->if_capenable & IFCAP_WOL) != 0) { gphy |= GPHY_CFG_EXT_RESET; if ((sc->alc_flags & ALC_FLAG_FASTETHER) == 0) alc_setlinkspeed(sc); } pmcs = 0; if ((ifp->if_capenable & IFCAP_WOL_MAGIC) != 0) pmcs |= WOL_CFG_MAGIC | WOL_CFG_MAGIC_ENB; CSR_WRITE_4(sc, ALC_WOL_CFG, pmcs); mac = CSR_READ_4(sc, ALC_MAC_CFG); mac &= ~(MAC_CFG_DBG | MAC_CFG_PROMISC | MAC_CFG_ALLMULTI | MAC_CFG_BCAST); if ((ifp->if_capenable & IFCAP_WOL_MCAST) != 0) mac |= MAC_CFG_ALLMULTI | MAC_CFG_BCAST; if ((ifp->if_capenable & IFCAP_WOL) != 0) mac |= MAC_CFG_RX_ENB; alc_miiext_writereg(sc, MII_EXT_ANEG, MII_EXT_ANEG_S3DIG10, ANEG_S3DIG10_SL); } /* Enable OSC. */ reg = CSR_READ_4(sc, ALC_MISC); reg &= ~MISC_INTNLOSC_OPEN; CSR_WRITE_4(sc, ALC_MISC, reg); reg |= MISC_INTNLOSC_OPEN; CSR_WRITE_4(sc, ALC_MISC, reg); CSR_WRITE_4(sc, ALC_MASTER_CFG, master); CSR_WRITE_4(sc, ALC_MAC_CFG, mac); CSR_WRITE_4(sc, ALC_GPHY_CFG, gphy); reg = CSR_READ_4(sc, ALC_PDLL_TRNS1); reg |= PDLL_TRNS1_D3PLLOFF_ENB; CSR_WRITE_4(sc, ALC_PDLL_TRNS1, reg); if ((sc->alc_flags & ALC_FLAG_PM) != 0) { /* Request PME. */ pmstat = pci_read_config(sc->alc_dev, sc->alc_pmcap + PCIR_POWER_STATUS, 2); pmstat &= ~(PCIM_PSTAT_PME | PCIM_PSTAT_PMEENABLE); if ((ifp->if_capenable & IFCAP_WOL) != 0) pmstat |= PCIM_PSTAT_PME | PCIM_PSTAT_PMEENABLE; pci_write_config(sc->alc_dev, sc->alc_pmcap + PCIR_POWER_STATUS, pmstat, 2); } } static int alc_suspend(device_t dev) { struct alc_softc *sc; sc = device_get_softc(dev); ALC_LOCK(sc); alc_stop(sc); alc_setwol(sc); ALC_UNLOCK(sc); return (0); } static int alc_resume(device_t dev) { struct alc_softc *sc; struct ifnet *ifp; uint16_t pmstat; sc = device_get_softc(dev); ALC_LOCK(sc); if ((sc->alc_flags & ALC_FLAG_PM) != 0) { /* Disable PME and clear PME status. */ pmstat = pci_read_config(sc->alc_dev, sc->alc_pmcap + PCIR_POWER_STATUS, 2); if ((pmstat & PCIM_PSTAT_PMEENABLE) != 0) { pmstat &= ~PCIM_PSTAT_PMEENABLE; pci_write_config(sc->alc_dev, sc->alc_pmcap + PCIR_POWER_STATUS, pmstat, 2); } } /* Reset PHY. */ alc_phy_reset(sc); ifp = sc->alc_ifp; if ((ifp->if_flags & IFF_UP) != 0) { ifp->if_drv_flags &= ~IFF_DRV_RUNNING; alc_init_locked(sc); } ALC_UNLOCK(sc); return (0); } static int alc_encap(struct alc_softc *sc, struct mbuf **m_head) { struct alc_txdesc *txd, *txd_last; struct tx_desc *desc; struct mbuf *m; struct ip *ip; struct tcphdr *tcp; bus_dma_segment_t txsegs[ALC_MAXTXSEGS]; bus_dmamap_t map; uint32_t cflags, hdrlen, ip_off, poff, vtag; int error, idx, nsegs, prod; ALC_LOCK_ASSERT(sc); M_ASSERTPKTHDR((*m_head)); m = *m_head; ip = NULL; tcp = NULL; ip_off = poff = 0; if ((m->m_pkthdr.csum_flags & (ALC_CSUM_FEATURES | CSUM_TSO)) != 0) { /* * AR81[3567]x requires offset of TCP/UDP header in its * Tx descriptor to perform Tx checksum offloading. TSO * also requires TCP header offset and modification of * IP/TCP header. This kind of operation takes many CPU * cycles on FreeBSD so fast host CPU is required to get * smooth TSO performance. */ struct ether_header *eh; if (M_WRITABLE(m) == 0) { /* Get a writable copy. */ m = m_dup(*m_head, M_NOWAIT); /* Release original mbufs. */ m_freem(*m_head); if (m == NULL) { *m_head = NULL; return (ENOBUFS); } *m_head = m; } ip_off = sizeof(struct ether_header); m = m_pullup(m, ip_off); if (m == NULL) { *m_head = NULL; return (ENOBUFS); } eh = mtod(m, struct ether_header *); /* * Check if hardware VLAN insertion is off. * Additional check for LLC/SNAP frame? */ if (eh->ether_type == htons(ETHERTYPE_VLAN)) { ip_off = sizeof(struct ether_vlan_header); m = m_pullup(m, ip_off); if (m == NULL) { *m_head = NULL; return (ENOBUFS); } } m = m_pullup(m, ip_off + sizeof(struct ip)); if (m == NULL) { *m_head = NULL; return (ENOBUFS); } ip = (struct ip *)(mtod(m, char *) + ip_off); poff = ip_off + (ip->ip_hl << 2); if ((m->m_pkthdr.csum_flags & CSUM_TSO) != 0) { m = m_pullup(m, poff + sizeof(struct tcphdr)); if (m == NULL) { *m_head = NULL; return (ENOBUFS); } tcp = (struct tcphdr *)(mtod(m, char *) + poff); m = m_pullup(m, poff + (tcp->th_off << 2)); if (m == NULL) { *m_head = NULL; return (ENOBUFS); } /* * Due to strict adherence of Microsoft NDIS * Large Send specification, hardware expects * a pseudo TCP checksum inserted by upper * stack. Unfortunately the pseudo TCP * checksum that NDIS refers to does not include * TCP payload length so driver should recompute * the pseudo checksum here. Hopefully this * wouldn't be much burden on modern CPUs. * * Reset IP checksum and recompute TCP pseudo * checksum as NDIS specification said. */ ip = (struct ip *)(mtod(m, char *) + ip_off); tcp = (struct tcphdr *)(mtod(m, char *) + poff); ip->ip_sum = 0; tcp->th_sum = in_pseudo(ip->ip_src.s_addr, ip->ip_dst.s_addr, htons(IPPROTO_TCP)); } *m_head = m; } prod = sc->alc_cdata.alc_tx_prod; txd = &sc->alc_cdata.alc_txdesc[prod]; txd_last = txd; map = txd->tx_dmamap; error = bus_dmamap_load_mbuf_sg(sc->alc_cdata.alc_tx_tag, map, *m_head, txsegs, &nsegs, 0); if (error == EFBIG) { m = m_collapse(*m_head, M_NOWAIT, ALC_MAXTXSEGS); if (m == NULL) { m_freem(*m_head); *m_head = NULL; return (ENOMEM); } *m_head = m; error = bus_dmamap_load_mbuf_sg(sc->alc_cdata.alc_tx_tag, map, *m_head, txsegs, &nsegs, 0); if (error != 0) { m_freem(*m_head); *m_head = NULL; return (error); } } else if (error != 0) return (error); if (nsegs == 0) { m_freem(*m_head); *m_head = NULL; return (EIO); } /* Check descriptor overrun. */ if (sc->alc_cdata.alc_tx_cnt + nsegs >= ALC_TX_RING_CNT - 3) { bus_dmamap_unload(sc->alc_cdata.alc_tx_tag, map); return (ENOBUFS); } bus_dmamap_sync(sc->alc_cdata.alc_tx_tag, map, BUS_DMASYNC_PREWRITE); m = *m_head; cflags = TD_ETHERNET; vtag = 0; desc = NULL; idx = 0; /* Configure VLAN hardware tag insertion. */ if ((m->m_flags & M_VLANTAG) != 0) { vtag = htons(m->m_pkthdr.ether_vtag); vtag = (vtag << TD_VLAN_SHIFT) & TD_VLAN_MASK; cflags |= TD_INS_VLAN_TAG; } if ((m->m_pkthdr.csum_flags & CSUM_TSO) != 0) { /* Request TSO and set MSS. */ cflags |= TD_TSO | TD_TSO_DESCV1; cflags |= ((uint32_t)m->m_pkthdr.tso_segsz << TD_MSS_SHIFT) & TD_MSS_MASK; /* Set TCP header offset. */ cflags |= (poff << TD_TCPHDR_OFFSET_SHIFT) & TD_TCPHDR_OFFSET_MASK; /* * AR81[3567]x requires the first buffer should * only hold IP/TCP header data. Payload should * be handled in other descriptors. */ hdrlen = poff + (tcp->th_off << 2); desc = &sc->alc_rdata.alc_tx_ring[prod]; desc->len = htole32(TX_BYTES(hdrlen | vtag)); desc->flags = htole32(cflags); desc->addr = htole64(txsegs[0].ds_addr); sc->alc_cdata.alc_tx_cnt++; ALC_DESC_INC(prod, ALC_TX_RING_CNT); if (m->m_len - hdrlen > 0) { /* Handle remaining payload of the first fragment. */ desc = &sc->alc_rdata.alc_tx_ring[prod]; desc->len = htole32(TX_BYTES((m->m_len - hdrlen) | vtag)); desc->flags = htole32(cflags); desc->addr = htole64(txsegs[0].ds_addr + hdrlen); sc->alc_cdata.alc_tx_cnt++; ALC_DESC_INC(prod, ALC_TX_RING_CNT); } /* Handle remaining fragments. */ idx = 1; } else if ((m->m_pkthdr.csum_flags & ALC_CSUM_FEATURES) != 0) { /* Configure Tx checksum offload. */ #ifdef ALC_USE_CUSTOM_CSUM cflags |= TD_CUSTOM_CSUM; /* Set checksum start offset. */ cflags |= ((poff >> 1) << TD_PLOAD_OFFSET_SHIFT) & TD_PLOAD_OFFSET_MASK; /* Set checksum insertion position of TCP/UDP. */ cflags |= (((poff + m->m_pkthdr.csum_data) >> 1) << TD_CUSTOM_CSUM_OFFSET_SHIFT) & TD_CUSTOM_CSUM_OFFSET_MASK; #else if ((m->m_pkthdr.csum_flags & CSUM_IP) != 0) cflags |= TD_IPCSUM; if ((m->m_pkthdr.csum_flags & CSUM_TCP) != 0) cflags |= TD_TCPCSUM; if ((m->m_pkthdr.csum_flags & CSUM_UDP) != 0) cflags |= TD_UDPCSUM; /* Set TCP/UDP header offset. */ cflags |= (poff << TD_L4HDR_OFFSET_SHIFT) & TD_L4HDR_OFFSET_MASK; #endif } for (; idx < nsegs; idx++) { desc = &sc->alc_rdata.alc_tx_ring[prod]; desc->len = htole32(TX_BYTES(txsegs[idx].ds_len) | vtag); desc->flags = htole32(cflags); desc->addr = htole64(txsegs[idx].ds_addr); sc->alc_cdata.alc_tx_cnt++; ALC_DESC_INC(prod, ALC_TX_RING_CNT); } /* Update producer index. */ sc->alc_cdata.alc_tx_prod = prod; /* Finally set EOP on the last descriptor. */ prod = (prod + ALC_TX_RING_CNT - 1) % ALC_TX_RING_CNT; desc = &sc->alc_rdata.alc_tx_ring[prod]; desc->flags |= htole32(TD_EOP); /* Swap dmamap of the first and the last. */ txd = &sc->alc_cdata.alc_txdesc[prod]; map = txd_last->tx_dmamap; txd_last->tx_dmamap = txd->tx_dmamap; txd->tx_dmamap = map; txd->tx_m = m; return (0); } static void alc_start(struct ifnet *ifp) { struct alc_softc *sc; sc = ifp->if_softc; ALC_LOCK(sc); alc_start_locked(ifp); ALC_UNLOCK(sc); } static void alc_start_locked(struct ifnet *ifp) { struct alc_softc *sc; struct mbuf *m_head; int enq; sc = ifp->if_softc; ALC_LOCK_ASSERT(sc); /* Reclaim transmitted frames. */ if (sc->alc_cdata.alc_tx_cnt >= ALC_TX_DESC_HIWAT) alc_txeof(sc); if ((ifp->if_drv_flags & (IFF_DRV_RUNNING | IFF_DRV_OACTIVE)) != IFF_DRV_RUNNING || (sc->alc_flags & ALC_FLAG_LINK) == 0) return; for (enq = 0; !IFQ_DRV_IS_EMPTY(&ifp->if_snd); ) { IFQ_DRV_DEQUEUE(&ifp->if_snd, m_head); if (m_head == NULL) break; /* * Pack the data into the transmit ring. If we * don't have room, set the OACTIVE flag and wait * for the NIC to drain the ring. */ if (alc_encap(sc, &m_head)) { if (m_head == NULL) break; IFQ_DRV_PREPEND(&ifp->if_snd, m_head); ifp->if_drv_flags |= IFF_DRV_OACTIVE; break; } enq++; /* * If there's a BPF listener, bounce a copy of this frame * to him. */ ETHER_BPF_MTAP(ifp, m_head); } if (enq > 0) { /* Sync descriptors. */ bus_dmamap_sync(sc->alc_cdata.alc_tx_ring_tag, sc->alc_cdata.alc_tx_ring_map, BUS_DMASYNC_PREWRITE); /* Kick. Assume we're using normal Tx priority queue. */ if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) != 0) CSR_WRITE_2(sc, ALC_MBOX_TD_PRI0_PROD_IDX, (uint16_t)sc->alc_cdata.alc_tx_prod); else CSR_WRITE_4(sc, ALC_MBOX_TD_PROD_IDX, (sc->alc_cdata.alc_tx_prod << MBOX_TD_PROD_LO_IDX_SHIFT) & MBOX_TD_PROD_LO_IDX_MASK); /* Set a timeout in case the chip goes out to lunch. */ sc->alc_watchdog_timer = ALC_TX_TIMEOUT; } } static void alc_watchdog(struct alc_softc *sc) { struct ifnet *ifp; ALC_LOCK_ASSERT(sc); if (sc->alc_watchdog_timer == 0 || --sc->alc_watchdog_timer) return; ifp = sc->alc_ifp; if ((sc->alc_flags & ALC_FLAG_LINK) == 0) { if_printf(sc->alc_ifp, "watchdog timeout (lost link)\n"); if_inc_counter(ifp, IFCOUNTER_OERRORS, 1); ifp->if_drv_flags &= ~IFF_DRV_RUNNING; alc_init_locked(sc); return; } if_printf(sc->alc_ifp, "watchdog timeout -- resetting\n"); if_inc_counter(ifp, IFCOUNTER_OERRORS, 1); ifp->if_drv_flags &= ~IFF_DRV_RUNNING; alc_init_locked(sc); if (!IFQ_DRV_IS_EMPTY(&ifp->if_snd)) alc_start_locked(ifp); } static int alc_ioctl(struct ifnet *ifp, u_long cmd, caddr_t data) { struct alc_softc *sc; struct ifreq *ifr; struct mii_data *mii; int error, mask; sc = ifp->if_softc; ifr = (struct ifreq *)data; error = 0; switch (cmd) { case SIOCSIFMTU: if (ifr->ifr_mtu < ETHERMIN || ifr->ifr_mtu > (sc->alc_ident->max_framelen - sizeof(struct ether_vlan_header) - ETHER_CRC_LEN) || ((sc->alc_flags & ALC_FLAG_JUMBO) == 0 && ifr->ifr_mtu > ETHERMTU)) error = EINVAL; else if (ifp->if_mtu != ifr->ifr_mtu) { ALC_LOCK(sc); ifp->if_mtu = ifr->ifr_mtu; /* AR81[3567]x has 13 bits MSS field. */ if (ifp->if_mtu > ALC_TSO_MTU && (ifp->if_capenable & IFCAP_TSO4) != 0) { ifp->if_capenable &= ~IFCAP_TSO4; ifp->if_hwassist &= ~CSUM_TSO; VLAN_CAPABILITIES(ifp); } ALC_UNLOCK(sc); } break; case SIOCSIFFLAGS: ALC_LOCK(sc); if ((ifp->if_flags & IFF_UP) != 0) { if ((ifp->if_drv_flags & IFF_DRV_RUNNING) != 0 && ((ifp->if_flags ^ sc->alc_if_flags) & (IFF_PROMISC | IFF_ALLMULTI)) != 0) alc_rxfilter(sc); else alc_init_locked(sc); } else if ((ifp->if_drv_flags & IFF_DRV_RUNNING) != 0) alc_stop(sc); sc->alc_if_flags = ifp->if_flags; ALC_UNLOCK(sc); break; case SIOCADDMULTI: case SIOCDELMULTI: ALC_LOCK(sc); if ((ifp->if_drv_flags & IFF_DRV_RUNNING) != 0) alc_rxfilter(sc); ALC_UNLOCK(sc); break; case SIOCSIFMEDIA: case SIOCGIFMEDIA: mii = device_get_softc(sc->alc_miibus); error = ifmedia_ioctl(ifp, ifr, &mii->mii_media, cmd); break; case SIOCSIFCAP: ALC_LOCK(sc); mask = ifr->ifr_reqcap ^ ifp->if_capenable; if ((mask & IFCAP_TXCSUM) != 0 && (ifp->if_capabilities & IFCAP_TXCSUM) != 0) { ifp->if_capenable ^= IFCAP_TXCSUM; if ((ifp->if_capenable & IFCAP_TXCSUM) != 0) ifp->if_hwassist |= ALC_CSUM_FEATURES; else ifp->if_hwassist &= ~ALC_CSUM_FEATURES; } if ((mask & IFCAP_TSO4) != 0 && (ifp->if_capabilities & IFCAP_TSO4) != 0) { ifp->if_capenable ^= IFCAP_TSO4; if ((ifp->if_capenable & IFCAP_TSO4) != 0) { /* AR81[3567]x has 13 bits MSS field. */ if (ifp->if_mtu > ALC_TSO_MTU) { ifp->if_capenable &= ~IFCAP_TSO4; ifp->if_hwassist &= ~CSUM_TSO; } else ifp->if_hwassist |= CSUM_TSO; } else ifp->if_hwassist &= ~CSUM_TSO; } if ((mask & IFCAP_WOL_MCAST) != 0 && (ifp->if_capabilities & IFCAP_WOL_MCAST) != 0) ifp->if_capenable ^= IFCAP_WOL_MCAST; if ((mask & IFCAP_WOL_MAGIC) != 0 && (ifp->if_capabilities & IFCAP_WOL_MAGIC) != 0) ifp->if_capenable ^= IFCAP_WOL_MAGIC; if ((mask & IFCAP_VLAN_HWTAGGING) != 0 && (ifp->if_capabilities & IFCAP_VLAN_HWTAGGING) != 0) { ifp->if_capenable ^= IFCAP_VLAN_HWTAGGING; alc_rxvlan(sc); } if ((mask & IFCAP_VLAN_HWCSUM) != 0 && (ifp->if_capabilities & IFCAP_VLAN_HWCSUM) != 0) ifp->if_capenable ^= IFCAP_VLAN_HWCSUM; if ((mask & IFCAP_VLAN_HWTSO) != 0 && (ifp->if_capabilities & IFCAP_VLAN_HWTSO) != 0) ifp->if_capenable ^= IFCAP_VLAN_HWTSO; if ((ifp->if_capenable & IFCAP_VLAN_HWTAGGING) == 0) ifp->if_capenable &= ~(IFCAP_VLAN_HWTSO | IFCAP_VLAN_HWCSUM); ALC_UNLOCK(sc); VLAN_CAPABILITIES(ifp); break; default: error = ether_ioctl(ifp, cmd, data); break; } return (error); } static void alc_mac_config(struct alc_softc *sc) { struct mii_data *mii; uint32_t reg; ALC_LOCK_ASSERT(sc); mii = device_get_softc(sc->alc_miibus); reg = CSR_READ_4(sc, ALC_MAC_CFG); reg &= ~(MAC_CFG_FULL_DUPLEX | MAC_CFG_TX_FC | MAC_CFG_RX_FC | MAC_CFG_SPEED_MASK); if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) != 0 || sc->alc_ident->deviceid == DEVICEID_ATHEROS_AR8151 || sc->alc_ident->deviceid == DEVICEID_ATHEROS_AR8151_V2 || sc->alc_ident->deviceid == DEVICEID_ATHEROS_AR8152_B2) reg |= MAC_CFG_HASH_ALG_CRC32 | MAC_CFG_SPEED_MODE_SW; /* Reprogram MAC with resolved speed/duplex. */ switch (IFM_SUBTYPE(mii->mii_media_active)) { case IFM_10_T: case IFM_100_TX: reg |= MAC_CFG_SPEED_10_100; break; case IFM_1000_T: reg |= MAC_CFG_SPEED_1000; break; } if ((IFM_OPTIONS(mii->mii_media_active) & IFM_FDX) != 0) { reg |= MAC_CFG_FULL_DUPLEX; if ((IFM_OPTIONS(mii->mii_media_active) & IFM_ETH_TXPAUSE) != 0) reg |= MAC_CFG_TX_FC; if ((IFM_OPTIONS(mii->mii_media_active) & IFM_ETH_RXPAUSE) != 0) reg |= MAC_CFG_RX_FC; } CSR_WRITE_4(sc, ALC_MAC_CFG, reg); } static void alc_stats_clear(struct alc_softc *sc) { struct smb sb, *smb; uint32_t *reg; int i; if ((sc->alc_flags & ALC_FLAG_SMB_BUG) == 0) { bus_dmamap_sync(sc->alc_cdata.alc_smb_tag, sc->alc_cdata.alc_smb_map, BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); smb = sc->alc_rdata.alc_smb; /* Update done, clear. */ smb->updated = 0; bus_dmamap_sync(sc->alc_cdata.alc_smb_tag, sc->alc_cdata.alc_smb_map, BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); } else { for (reg = &sb.rx_frames, i = 0; reg <= &sb.rx_pkts_filtered; reg++) { CSR_READ_4(sc, ALC_RX_MIB_BASE + i); i += sizeof(uint32_t); } /* Read Tx statistics. */ for (reg = &sb.tx_frames, i = 0; reg <= &sb.tx_mcast_bytes; reg++) { CSR_READ_4(sc, ALC_TX_MIB_BASE + i); i += sizeof(uint32_t); } } } static void alc_stats_update(struct alc_softc *sc) { struct alc_hw_stats *stat; struct smb sb, *smb; struct ifnet *ifp; uint32_t *reg; int i; ALC_LOCK_ASSERT(sc); ifp = sc->alc_ifp; stat = &sc->alc_stats; if ((sc->alc_flags & ALC_FLAG_SMB_BUG) == 0) { bus_dmamap_sync(sc->alc_cdata.alc_smb_tag, sc->alc_cdata.alc_smb_map, BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); smb = sc->alc_rdata.alc_smb; if (smb->updated == 0) return; } else { smb = &sb; /* Read Rx statistics. */ for (reg = &sb.rx_frames, i = 0; reg <= &sb.rx_pkts_filtered; reg++) { *reg = CSR_READ_4(sc, ALC_RX_MIB_BASE + i); i += sizeof(uint32_t); } /* Read Tx statistics. */ for (reg = &sb.tx_frames, i = 0; reg <= &sb.tx_mcast_bytes; reg++) { *reg = CSR_READ_4(sc, ALC_TX_MIB_BASE + i); i += sizeof(uint32_t); } } /* Rx stats. */ stat->rx_frames += smb->rx_frames; stat->rx_bcast_frames += smb->rx_bcast_frames; stat->rx_mcast_frames += smb->rx_mcast_frames; stat->rx_pause_frames += smb->rx_pause_frames; stat->rx_control_frames += smb->rx_control_frames; stat->rx_crcerrs += smb->rx_crcerrs; stat->rx_lenerrs += smb->rx_lenerrs; stat->rx_bytes += smb->rx_bytes; stat->rx_runts += smb->rx_runts; stat->rx_fragments += smb->rx_fragments; stat->rx_pkts_64 += smb->rx_pkts_64; stat->rx_pkts_65_127 += smb->rx_pkts_65_127; stat->rx_pkts_128_255 += smb->rx_pkts_128_255; stat->rx_pkts_256_511 += smb->rx_pkts_256_511; stat->rx_pkts_512_1023 += smb->rx_pkts_512_1023; stat->rx_pkts_1024_1518 += smb->rx_pkts_1024_1518; stat->rx_pkts_1519_max += smb->rx_pkts_1519_max; stat->rx_pkts_truncated += smb->rx_pkts_truncated; stat->rx_fifo_oflows += smb->rx_fifo_oflows; stat->rx_rrs_errs += smb->rx_rrs_errs; stat->rx_alignerrs += smb->rx_alignerrs; stat->rx_bcast_bytes += smb->rx_bcast_bytes; stat->rx_mcast_bytes += smb->rx_mcast_bytes; stat->rx_pkts_filtered += smb->rx_pkts_filtered; /* Tx stats. */ stat->tx_frames += smb->tx_frames; stat->tx_bcast_frames += smb->tx_bcast_frames; stat->tx_mcast_frames += smb->tx_mcast_frames; stat->tx_pause_frames += smb->tx_pause_frames; stat->tx_excess_defer += smb->tx_excess_defer; stat->tx_control_frames += smb->tx_control_frames; stat->tx_deferred += smb->tx_deferred; stat->tx_bytes += smb->tx_bytes; stat->tx_pkts_64 += smb->tx_pkts_64; stat->tx_pkts_65_127 += smb->tx_pkts_65_127; stat->tx_pkts_128_255 += smb->tx_pkts_128_255; stat->tx_pkts_256_511 += smb->tx_pkts_256_511; stat->tx_pkts_512_1023 += smb->tx_pkts_512_1023; stat->tx_pkts_1024_1518 += smb->tx_pkts_1024_1518; stat->tx_pkts_1519_max += smb->tx_pkts_1519_max; stat->tx_single_colls += smb->tx_single_colls; stat->tx_multi_colls += smb->tx_multi_colls; stat->tx_late_colls += smb->tx_late_colls; stat->tx_excess_colls += smb->tx_excess_colls; stat->tx_underrun += smb->tx_underrun; stat->tx_desc_underrun += smb->tx_desc_underrun; stat->tx_lenerrs += smb->tx_lenerrs; stat->tx_pkts_truncated += smb->tx_pkts_truncated; stat->tx_bcast_bytes += smb->tx_bcast_bytes; stat->tx_mcast_bytes += smb->tx_mcast_bytes; /* Update counters in ifnet. */ if_inc_counter(ifp, IFCOUNTER_OPACKETS, smb->tx_frames); if_inc_counter(ifp, IFCOUNTER_COLLISIONS, smb->tx_single_colls + smb->tx_multi_colls * 2 + smb->tx_late_colls + smb->tx_excess_colls * HDPX_CFG_RETRY_DEFAULT); if_inc_counter(ifp, IFCOUNTER_OERRORS, smb->tx_late_colls + smb->tx_excess_colls + smb->tx_underrun + smb->tx_pkts_truncated); if_inc_counter(ifp, IFCOUNTER_IPACKETS, smb->rx_frames); if_inc_counter(ifp, IFCOUNTER_IERRORS, smb->rx_crcerrs + smb->rx_lenerrs + smb->rx_runts + smb->rx_pkts_truncated + smb->rx_fifo_oflows + smb->rx_rrs_errs + smb->rx_alignerrs); if ((sc->alc_flags & ALC_FLAG_SMB_BUG) == 0) { /* Update done, clear. */ smb->updated = 0; bus_dmamap_sync(sc->alc_cdata.alc_smb_tag, sc->alc_cdata.alc_smb_map, BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); } } static int alc_intr(void *arg) { struct alc_softc *sc; uint32_t status; sc = (struct alc_softc *)arg; status = CSR_READ_4(sc, ALC_INTR_STATUS); if ((status & ALC_INTRS) == 0) return (FILTER_STRAY); /* Disable interrupts. */ CSR_WRITE_4(sc, ALC_INTR_STATUS, INTR_DIS_INT); taskqueue_enqueue(sc->alc_tq, &sc->alc_int_task); return (FILTER_HANDLED); } static void alc_int_task(void *arg, int pending) { struct alc_softc *sc; struct ifnet *ifp; uint32_t status; int more; sc = (struct alc_softc *)arg; ifp = sc->alc_ifp; status = CSR_READ_4(sc, ALC_INTR_STATUS); ALC_LOCK(sc); if (sc->alc_morework != 0) { sc->alc_morework = 0; status |= INTR_RX_PKT; } if ((status & ALC_INTRS) == 0) goto done; /* Acknowledge interrupts but still disable interrupts. */ CSR_WRITE_4(sc, ALC_INTR_STATUS, status | INTR_DIS_INT); more = 0; if ((ifp->if_drv_flags & IFF_DRV_RUNNING) != 0) { if ((status & INTR_RX_PKT) != 0) { more = alc_rxintr(sc, sc->alc_process_limit); if (more == EAGAIN) sc->alc_morework = 1; else if (more == EIO) { ifp->if_drv_flags &= ~IFF_DRV_RUNNING; alc_init_locked(sc); ALC_UNLOCK(sc); return; } } if ((status & (INTR_DMA_RD_TO_RST | INTR_DMA_WR_TO_RST | INTR_TXQ_TO_RST)) != 0) { if ((status & INTR_DMA_RD_TO_RST) != 0) device_printf(sc->alc_dev, "DMA read error! -- resetting\n"); if ((status & INTR_DMA_WR_TO_RST) != 0) device_printf(sc->alc_dev, "DMA write error! -- resetting\n"); if ((status & INTR_TXQ_TO_RST) != 0) device_printf(sc->alc_dev, "TxQ reset! -- resetting\n"); ifp->if_drv_flags &= ~IFF_DRV_RUNNING; alc_init_locked(sc); ALC_UNLOCK(sc); return; } if ((ifp->if_drv_flags & IFF_DRV_RUNNING) != 0 && !IFQ_DRV_IS_EMPTY(&ifp->if_snd)) alc_start_locked(ifp); } if (more == EAGAIN || (CSR_READ_4(sc, ALC_INTR_STATUS) & ALC_INTRS) != 0) { ALC_UNLOCK(sc); taskqueue_enqueue(sc->alc_tq, &sc->alc_int_task); return; } done: if ((ifp->if_drv_flags & IFF_DRV_RUNNING) != 0) { /* Re-enable interrupts if we're running. */ CSR_WRITE_4(sc, ALC_INTR_STATUS, 0x7FFFFFFF); } ALC_UNLOCK(sc); } static void alc_txeof(struct alc_softc *sc) { struct ifnet *ifp; struct alc_txdesc *txd; uint32_t cons, prod; int prog; ALC_LOCK_ASSERT(sc); ifp = sc->alc_ifp; if (sc->alc_cdata.alc_tx_cnt == 0) return; bus_dmamap_sync(sc->alc_cdata.alc_tx_ring_tag, sc->alc_cdata.alc_tx_ring_map, BUS_DMASYNC_POSTWRITE); if ((sc->alc_flags & ALC_FLAG_CMB_BUG) == 0) { bus_dmamap_sync(sc->alc_cdata.alc_cmb_tag, sc->alc_cdata.alc_cmb_map, BUS_DMASYNC_POSTREAD); prod = sc->alc_rdata.alc_cmb->cons; } else { if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) != 0) prod = CSR_READ_2(sc, ALC_MBOX_TD_PRI0_CONS_IDX); else { prod = CSR_READ_4(sc, ALC_MBOX_TD_CONS_IDX); /* Assume we're using normal Tx priority queue. */ prod = (prod & MBOX_TD_CONS_LO_IDX_MASK) >> MBOX_TD_CONS_LO_IDX_SHIFT; } } cons = sc->alc_cdata.alc_tx_cons; /* * Go through our Tx list and free mbufs for those * frames which have been transmitted. */ for (prog = 0; cons != prod; prog++, ALC_DESC_INC(cons, ALC_TX_RING_CNT)) { if (sc->alc_cdata.alc_tx_cnt <= 0) break; prog++; ifp->if_drv_flags &= ~IFF_DRV_OACTIVE; sc->alc_cdata.alc_tx_cnt--; txd = &sc->alc_cdata.alc_txdesc[cons]; if (txd->tx_m != NULL) { /* Reclaim transmitted mbufs. */ bus_dmamap_sync(sc->alc_cdata.alc_tx_tag, txd->tx_dmamap, BUS_DMASYNC_POSTWRITE); bus_dmamap_unload(sc->alc_cdata.alc_tx_tag, txd->tx_dmamap); m_freem(txd->tx_m); txd->tx_m = NULL; } } if ((sc->alc_flags & ALC_FLAG_CMB_BUG) == 0) bus_dmamap_sync(sc->alc_cdata.alc_cmb_tag, sc->alc_cdata.alc_cmb_map, BUS_DMASYNC_PREREAD); sc->alc_cdata.alc_tx_cons = cons; /* * Unarm watchdog timer only when there is no pending * frames in Tx queue. */ if (sc->alc_cdata.alc_tx_cnt == 0) sc->alc_watchdog_timer = 0; } static int alc_newbuf(struct alc_softc *sc, struct alc_rxdesc *rxd) { struct mbuf *m; bus_dma_segment_t segs[1]; bus_dmamap_t map; int nsegs; m = m_getcl(M_NOWAIT, MT_DATA, M_PKTHDR); if (m == NULL) return (ENOBUFS); m->m_len = m->m_pkthdr.len = RX_BUF_SIZE_MAX; #ifndef __NO_STRICT_ALIGNMENT m_adj(m, sizeof(uint64_t)); #endif if (bus_dmamap_load_mbuf_sg(sc->alc_cdata.alc_rx_tag, sc->alc_cdata.alc_rx_sparemap, m, segs, &nsegs, 0) != 0) { m_freem(m); return (ENOBUFS); } KASSERT(nsegs == 1, ("%s: %d segments returned!", __func__, nsegs)); if (rxd->rx_m != NULL) { bus_dmamap_sync(sc->alc_cdata.alc_rx_tag, rxd->rx_dmamap, BUS_DMASYNC_POSTREAD); bus_dmamap_unload(sc->alc_cdata.alc_rx_tag, rxd->rx_dmamap); } map = rxd->rx_dmamap; rxd->rx_dmamap = sc->alc_cdata.alc_rx_sparemap; sc->alc_cdata.alc_rx_sparemap = map; bus_dmamap_sync(sc->alc_cdata.alc_rx_tag, rxd->rx_dmamap, BUS_DMASYNC_PREREAD); rxd->rx_m = m; rxd->rx_desc->addr = htole64(segs[0].ds_addr); return (0); } static int alc_rxintr(struct alc_softc *sc, int count) { struct ifnet *ifp; struct rx_rdesc *rrd; uint32_t nsegs, status; int rr_cons, prog; bus_dmamap_sync(sc->alc_cdata.alc_rr_ring_tag, sc->alc_cdata.alc_rr_ring_map, BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); bus_dmamap_sync(sc->alc_cdata.alc_rx_ring_tag, sc->alc_cdata.alc_rx_ring_map, BUS_DMASYNC_POSTWRITE); rr_cons = sc->alc_cdata.alc_rr_cons; ifp = sc->alc_ifp; for (prog = 0; (ifp->if_drv_flags & IFF_DRV_RUNNING) != 0;) { if (count-- <= 0) break; rrd = &sc->alc_rdata.alc_rr_ring[rr_cons]; status = le32toh(rrd->status); if ((status & RRD_VALID) == 0) break; nsegs = RRD_RD_CNT(le32toh(rrd->rdinfo)); if (nsegs == 0) { /* This should not happen! */ device_printf(sc->alc_dev, "unexpected segment count -- resetting\n"); return (EIO); } alc_rxeof(sc, rrd); /* Clear Rx return status. */ rrd->status = 0; ALC_DESC_INC(rr_cons, ALC_RR_RING_CNT); sc->alc_cdata.alc_rx_cons += nsegs; sc->alc_cdata.alc_rx_cons %= ALC_RR_RING_CNT; prog += nsegs; } if (prog > 0) { /* Update the consumer index. */ sc->alc_cdata.alc_rr_cons = rr_cons; /* Sync Rx return descriptors. */ bus_dmamap_sync(sc->alc_cdata.alc_rr_ring_tag, sc->alc_cdata.alc_rr_ring_map, BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); /* * Sync updated Rx descriptors such that controller see * modified buffer addresses. */ bus_dmamap_sync(sc->alc_cdata.alc_rx_ring_tag, sc->alc_cdata.alc_rx_ring_map, BUS_DMASYNC_PREWRITE); /* * Let controller know availability of new Rx buffers. * Since alc(4) use RXQ_CFG_RD_BURST_DEFAULT descriptors * it may be possible to update ALC_MBOX_RD0_PROD_IDX * only when Rx buffer pre-fetching is required. In * addition we already set ALC_RX_RD_FREE_THRESH to * RX_RD_FREE_THRESH_LO_DEFAULT descriptors. However * it still seems that pre-fetching needs more * experimentation. */ if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) != 0) CSR_WRITE_2(sc, ALC_MBOX_RD0_PROD_IDX, (uint16_t)sc->alc_cdata.alc_rx_cons); else CSR_WRITE_4(sc, ALC_MBOX_RD0_PROD_IDX, sc->alc_cdata.alc_rx_cons); } return (count > 0 ? 0 : EAGAIN); } #ifndef __NO_STRICT_ALIGNMENT static struct mbuf * alc_fixup_rx(struct ifnet *ifp, struct mbuf *m) { struct mbuf *n; int i; uint16_t *src, *dst; src = mtod(m, uint16_t *); dst = src - 3; if (m->m_next == NULL) { for (i = 0; i < (m->m_len / sizeof(uint16_t) + 1); i++) *dst++ = *src++; m->m_data -= 6; return (m); } /* * Append a new mbuf to received mbuf chain and copy ethernet * header from the mbuf chain. This can save lots of CPU * cycles for jumbo frame. */ MGETHDR(n, M_NOWAIT, MT_DATA); if (n == NULL) { if_inc_counter(ifp, IFCOUNTER_IQDROPS, 1); m_freem(m); return (NULL); } bcopy(m->m_data, n->m_data, ETHER_HDR_LEN); m->m_data += ETHER_HDR_LEN; m->m_len -= ETHER_HDR_LEN; n->m_len = ETHER_HDR_LEN; M_MOVE_PKTHDR(n, m); n->m_next = m; return (n); } #endif /* Receive a frame. */ static void alc_rxeof(struct alc_softc *sc, struct rx_rdesc *rrd) { struct alc_rxdesc *rxd; struct ifnet *ifp; struct mbuf *mp, *m; uint32_t rdinfo, status, vtag; int count, nsegs, rx_cons; ifp = sc->alc_ifp; status = le32toh(rrd->status); rdinfo = le32toh(rrd->rdinfo); rx_cons = RRD_RD_IDX(rdinfo); nsegs = RRD_RD_CNT(rdinfo); sc->alc_cdata.alc_rxlen = RRD_BYTES(status); if ((status & (RRD_ERR_SUM | RRD_ERR_LENGTH)) != 0) { /* * We want to pass the following frames to upper * layer regardless of error status of Rx return * ring. * * o IP/TCP/UDP checksum is bad. * o frame length and protocol specific length * does not match. * * Force network stack compute checksum for * errored frames. */ status |= RRD_TCP_UDPCSUM_NOK | RRD_IPCSUM_NOK; if ((status & (RRD_ERR_CRC | RRD_ERR_ALIGN | RRD_ERR_TRUNC | RRD_ERR_RUNT)) != 0) return; } for (count = 0; count < nsegs; count++, ALC_DESC_INC(rx_cons, ALC_RX_RING_CNT)) { rxd = &sc->alc_cdata.alc_rxdesc[rx_cons]; mp = rxd->rx_m; /* Add a new receive buffer to the ring. */ if (alc_newbuf(sc, rxd) != 0) { if_inc_counter(ifp, IFCOUNTER_IQDROPS, 1); /* Reuse Rx buffers. */ if (sc->alc_cdata.alc_rxhead != NULL) m_freem(sc->alc_cdata.alc_rxhead); break; } /* * Assume we've received a full sized frame. * Actual size is fixed when we encounter the end of * multi-segmented frame. */ mp->m_len = sc->alc_buf_size; /* Chain received mbufs. */ if (sc->alc_cdata.alc_rxhead == NULL) { sc->alc_cdata.alc_rxhead = mp; sc->alc_cdata.alc_rxtail = mp; } else { mp->m_flags &= ~M_PKTHDR; sc->alc_cdata.alc_rxprev_tail = sc->alc_cdata.alc_rxtail; sc->alc_cdata.alc_rxtail->m_next = mp; sc->alc_cdata.alc_rxtail = mp; } if (count == nsegs - 1) { /* Last desc. for this frame. */ m = sc->alc_cdata.alc_rxhead; m->m_flags |= M_PKTHDR; /* * It seems that L1C/L2C controller has no way * to tell hardware to strip CRC bytes. */ m->m_pkthdr.len = sc->alc_cdata.alc_rxlen - ETHER_CRC_LEN; if (nsegs > 1) { /* Set last mbuf size. */ mp->m_len = sc->alc_cdata.alc_rxlen - (nsegs - 1) * sc->alc_buf_size; /* Remove the CRC bytes in chained mbufs. */ if (mp->m_len <= ETHER_CRC_LEN) { sc->alc_cdata.alc_rxtail = sc->alc_cdata.alc_rxprev_tail; sc->alc_cdata.alc_rxtail->m_len -= (ETHER_CRC_LEN - mp->m_len); sc->alc_cdata.alc_rxtail->m_next = NULL; m_freem(mp); } else { mp->m_len -= ETHER_CRC_LEN; } } else m->m_len = m->m_pkthdr.len; m->m_pkthdr.rcvif = ifp; /* * Due to hardware bugs, Rx checksum offloading * was intentionally disabled. */ if ((ifp->if_capenable & IFCAP_VLAN_HWTAGGING) != 0 && (status & RRD_VLAN_TAG) != 0) { vtag = RRD_VLAN(le32toh(rrd->vtag)); m->m_pkthdr.ether_vtag = ntohs(vtag); m->m_flags |= M_VLANTAG; } #ifndef __NO_STRICT_ALIGNMENT m = alc_fixup_rx(ifp, m); if (m != NULL) #endif { /* Pass it on. */ ALC_UNLOCK(sc); (*ifp->if_input)(ifp, m); ALC_LOCK(sc); } } } /* Reset mbuf chains. */ ALC_RXCHAIN_RESET(sc); } static void alc_tick(void *arg) { struct alc_softc *sc; struct mii_data *mii; sc = (struct alc_softc *)arg; ALC_LOCK_ASSERT(sc); mii = device_get_softc(sc->alc_miibus); mii_tick(mii); alc_stats_update(sc); /* * alc(4) does not rely on Tx completion interrupts to reclaim * transferred buffers. Instead Tx completion interrupts are * used to hint for scheduling Tx task. So it's necessary to * release transmitted buffers by kicking Tx completion * handler. This limits the maximum reclamation delay to a hz. */ alc_txeof(sc); alc_watchdog(sc); callout_reset(&sc->alc_tick_ch, hz, alc_tick, sc); } static void alc_osc_reset(struct alc_softc *sc) { uint32_t reg; reg = CSR_READ_4(sc, ALC_MISC3); reg &= ~MISC3_25M_BY_SW; reg |= MISC3_25M_NOTO_INTNL; CSR_WRITE_4(sc, ALC_MISC3, reg); reg = CSR_READ_4(sc, ALC_MISC); if (AR816X_REV(sc->alc_rev) >= AR816X_REV_B0) { /* * Restore over-current protection default value. * This value could be reset by MAC reset. */ reg &= ~MISC_PSW_OCP_MASK; reg |= (MISC_PSW_OCP_DEFAULT << MISC_PSW_OCP_SHIFT); reg &= ~MISC_INTNLOSC_OPEN; CSR_WRITE_4(sc, ALC_MISC, reg); CSR_WRITE_4(sc, ALC_MISC, reg | MISC_INTNLOSC_OPEN); reg = CSR_READ_4(sc, ALC_MISC2); reg &= ~MISC2_CALB_START; CSR_WRITE_4(sc, ALC_MISC2, reg); CSR_WRITE_4(sc, ALC_MISC2, reg | MISC2_CALB_START); } else { reg &= ~MISC_INTNLOSC_OPEN; /* Disable isolate for revision A devices. */ if (AR816X_REV(sc->alc_rev) <= AR816X_REV_A1) reg &= ~MISC_ISO_ENB; CSR_WRITE_4(sc, ALC_MISC, reg | MISC_INTNLOSC_OPEN); CSR_WRITE_4(sc, ALC_MISC, reg); } DELAY(20); } static void alc_reset(struct alc_softc *sc) { uint32_t pmcfg, reg; int i; pmcfg = 0; if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) != 0) { /* Reset workaround. */ CSR_WRITE_4(sc, ALC_MBOX_RD0_PROD_IDX, 1); if (AR816X_REV(sc->alc_rev) <= AR816X_REV_A1 && (sc->alc_rev & 0x01) != 0) { /* Disable L0s/L1s before reset. */ pmcfg = CSR_READ_4(sc, ALC_PM_CFG); if ((pmcfg & (PM_CFG_ASPM_L0S_ENB | PM_CFG_ASPM_L1_ENB)) != 0) { pmcfg &= ~(PM_CFG_ASPM_L0S_ENB | PM_CFG_ASPM_L1_ENB); CSR_WRITE_4(sc, ALC_PM_CFG, pmcfg); } } } reg = CSR_READ_4(sc, ALC_MASTER_CFG); reg |= MASTER_OOB_DIS_OFF | MASTER_RESET; CSR_WRITE_4(sc, ALC_MASTER_CFG, reg); if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) != 0) { for (i = ALC_RESET_TIMEOUT; i > 0; i--) { DELAY(10); if (CSR_READ_4(sc, ALC_MBOX_RD0_PROD_IDX) == 0) break; } if (i == 0) device_printf(sc->alc_dev, "MAC reset timeout!\n"); } for (i = ALC_RESET_TIMEOUT; i > 0; i--) { DELAY(10); if ((CSR_READ_4(sc, ALC_MASTER_CFG) & MASTER_RESET) == 0) break; } if (i == 0) device_printf(sc->alc_dev, "master reset timeout!\n"); for (i = ALC_RESET_TIMEOUT; i > 0; i--) { reg = CSR_READ_4(sc, ALC_IDLE_STATUS); if ((reg & (IDLE_STATUS_RXMAC | IDLE_STATUS_TXMAC | IDLE_STATUS_RXQ | IDLE_STATUS_TXQ)) == 0) break; DELAY(10); } if (i == 0) device_printf(sc->alc_dev, "reset timeout(0x%08x)!\n", reg); if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) != 0) { if (AR816X_REV(sc->alc_rev) <= AR816X_REV_A1 && (sc->alc_rev & 0x01) != 0) { reg = CSR_READ_4(sc, ALC_MASTER_CFG); reg |= MASTER_CLK_SEL_DIS; CSR_WRITE_4(sc, ALC_MASTER_CFG, reg); /* Restore L0s/L1s config. */ if ((pmcfg & (PM_CFG_ASPM_L0S_ENB | PM_CFG_ASPM_L1_ENB)) != 0) CSR_WRITE_4(sc, ALC_PM_CFG, pmcfg); } alc_osc_reset(sc); reg = CSR_READ_4(sc, ALC_MISC3); reg &= ~MISC3_25M_BY_SW; reg |= MISC3_25M_NOTO_INTNL; CSR_WRITE_4(sc, ALC_MISC3, reg); reg = CSR_READ_4(sc, ALC_MISC); reg &= ~MISC_INTNLOSC_OPEN; if (AR816X_REV(sc->alc_rev) <= AR816X_REV_A1) reg &= ~MISC_ISO_ENB; CSR_WRITE_4(sc, ALC_MISC, reg); DELAY(20); } if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) != 0 || sc->alc_ident->deviceid == DEVICEID_ATHEROS_AR8152_B || sc->alc_ident->deviceid == DEVICEID_ATHEROS_AR8151_V2) CSR_WRITE_4(sc, ALC_SERDES_LOCK, CSR_READ_4(sc, ALC_SERDES_LOCK) | SERDES_MAC_CLK_SLOWDOWN | SERDES_PHY_CLK_SLOWDOWN); } static void alc_init(void *xsc) { struct alc_softc *sc; sc = (struct alc_softc *)xsc; ALC_LOCK(sc); alc_init_locked(sc); ALC_UNLOCK(sc); } static void alc_init_locked(struct alc_softc *sc) { struct ifnet *ifp; struct mii_data *mii; uint8_t eaddr[ETHER_ADDR_LEN]; bus_addr_t paddr; uint32_t reg, rxf_hi, rxf_lo; ALC_LOCK_ASSERT(sc); ifp = sc->alc_ifp; mii = device_get_softc(sc->alc_miibus); if ((ifp->if_drv_flags & IFF_DRV_RUNNING) != 0) return; /* * Cancel any pending I/O. */ alc_stop(sc); /* * Reset the chip to a known state. */ alc_reset(sc); /* Initialize Rx descriptors. */ if (alc_init_rx_ring(sc) != 0) { device_printf(sc->alc_dev, "no memory for Rx buffers.\n"); alc_stop(sc); return; } alc_init_rr_ring(sc); alc_init_tx_ring(sc); alc_init_cmb(sc); alc_init_smb(sc); /* Enable all clocks. */ if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) != 0) { CSR_WRITE_4(sc, ALC_CLK_GATING_CFG, CLK_GATING_DMAW_ENB | CLK_GATING_DMAR_ENB | CLK_GATING_TXQ_ENB | CLK_GATING_RXQ_ENB | CLK_GATING_TXMAC_ENB | CLK_GATING_RXMAC_ENB); if (AR816X_REV(sc->alc_rev) >= AR816X_REV_B0) CSR_WRITE_4(sc, ALC_IDLE_DECISN_TIMER, IDLE_DECISN_TIMER_DEFAULT_1MS); } else CSR_WRITE_4(sc, ALC_CLK_GATING_CFG, 0); /* Reprogram the station address. */ bcopy(IF_LLADDR(ifp), eaddr, ETHER_ADDR_LEN); CSR_WRITE_4(sc, ALC_PAR0, eaddr[2] << 24 | eaddr[3] << 16 | eaddr[4] << 8 | eaddr[5]); CSR_WRITE_4(sc, ALC_PAR1, eaddr[0] << 8 | eaddr[1]); /* * Clear WOL status and disable all WOL feature as WOL * would interfere Rx operation under normal environments. */ CSR_READ_4(sc, ALC_WOL_CFG); CSR_WRITE_4(sc, ALC_WOL_CFG, 0); /* Set Tx descriptor base addresses. */ paddr = sc->alc_rdata.alc_tx_ring_paddr; CSR_WRITE_4(sc, ALC_TX_BASE_ADDR_HI, ALC_ADDR_HI(paddr)); CSR_WRITE_4(sc, ALC_TDL_HEAD_ADDR_LO, ALC_ADDR_LO(paddr)); /* We don't use high priority ring. */ CSR_WRITE_4(sc, ALC_TDH_HEAD_ADDR_LO, 0); /* Set Tx descriptor counter. */ CSR_WRITE_4(sc, ALC_TD_RING_CNT, (ALC_TX_RING_CNT << TD_RING_CNT_SHIFT) & TD_RING_CNT_MASK); /* Set Rx descriptor base addresses. */ paddr = sc->alc_rdata.alc_rx_ring_paddr; CSR_WRITE_4(sc, ALC_RX_BASE_ADDR_HI, ALC_ADDR_HI(paddr)); CSR_WRITE_4(sc, ALC_RD0_HEAD_ADDR_LO, ALC_ADDR_LO(paddr)); if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) == 0) { /* We use one Rx ring. */ CSR_WRITE_4(sc, ALC_RD1_HEAD_ADDR_LO, 0); CSR_WRITE_4(sc, ALC_RD2_HEAD_ADDR_LO, 0); CSR_WRITE_4(sc, ALC_RD3_HEAD_ADDR_LO, 0); } /* Set Rx descriptor counter. */ CSR_WRITE_4(sc, ALC_RD_RING_CNT, (ALC_RX_RING_CNT << RD_RING_CNT_SHIFT) & RD_RING_CNT_MASK); /* * Let hardware split jumbo frames into alc_max_buf_sized chunks. * if it do not fit the buffer size. Rx return descriptor holds * a counter that indicates how many fragments were made by the * hardware. The buffer size should be multiple of 8 bytes. * Since hardware has limit on the size of buffer size, always * use the maximum value. * For strict-alignment architectures make sure to reduce buffer * size by 8 bytes to make room for alignment fixup. */ #ifndef __NO_STRICT_ALIGNMENT sc->alc_buf_size = RX_BUF_SIZE_MAX - sizeof(uint64_t); #else sc->alc_buf_size = RX_BUF_SIZE_MAX; #endif CSR_WRITE_4(sc, ALC_RX_BUF_SIZE, sc->alc_buf_size); paddr = sc->alc_rdata.alc_rr_ring_paddr; /* Set Rx return descriptor base addresses. */ CSR_WRITE_4(sc, ALC_RRD0_HEAD_ADDR_LO, ALC_ADDR_LO(paddr)); if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) == 0) { /* We use one Rx return ring. */ CSR_WRITE_4(sc, ALC_RRD1_HEAD_ADDR_LO, 0); CSR_WRITE_4(sc, ALC_RRD2_HEAD_ADDR_LO, 0); CSR_WRITE_4(sc, ALC_RRD3_HEAD_ADDR_LO, 0); } /* Set Rx return descriptor counter. */ CSR_WRITE_4(sc, ALC_RRD_RING_CNT, (ALC_RR_RING_CNT << RRD_RING_CNT_SHIFT) & RRD_RING_CNT_MASK); paddr = sc->alc_rdata.alc_cmb_paddr; CSR_WRITE_4(sc, ALC_CMB_BASE_ADDR_LO, ALC_ADDR_LO(paddr)); paddr = sc->alc_rdata.alc_smb_paddr; CSR_WRITE_4(sc, ALC_SMB_BASE_ADDR_HI, ALC_ADDR_HI(paddr)); CSR_WRITE_4(sc, ALC_SMB_BASE_ADDR_LO, ALC_ADDR_LO(paddr)); if (sc->alc_ident->deviceid == DEVICEID_ATHEROS_AR8152_B) { /* Reconfigure SRAM - Vendor magic. */ CSR_WRITE_4(sc, ALC_SRAM_RX_FIFO_LEN, 0x000002A0); CSR_WRITE_4(sc, ALC_SRAM_TX_FIFO_LEN, 0x00000100); CSR_WRITE_4(sc, ALC_SRAM_RX_FIFO_ADDR, 0x029F0000); CSR_WRITE_4(sc, ALC_SRAM_RD0_ADDR, 0x02BF02A0); CSR_WRITE_4(sc, ALC_SRAM_TX_FIFO_ADDR, 0x03BF02C0); CSR_WRITE_4(sc, ALC_SRAM_TD_ADDR, 0x03DF03C0); CSR_WRITE_4(sc, ALC_TXF_WATER_MARK, 0x00000000); CSR_WRITE_4(sc, ALC_RD_DMA_CFG, 0x00000000); } /* Tell hardware that we're ready to load DMA blocks. */ CSR_WRITE_4(sc, ALC_DMA_BLOCK, DMA_BLOCK_LOAD); /* Configure interrupt moderation timer. */ reg = ALC_USECS(sc->alc_int_rx_mod) << IM_TIMER_RX_SHIFT; if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) == 0) reg |= ALC_USECS(sc->alc_int_tx_mod) << IM_TIMER_TX_SHIFT; CSR_WRITE_4(sc, ALC_IM_TIMER, reg); /* * We don't want to automatic interrupt clear as task queue * for the interrupt should know interrupt status. */ reg = CSR_READ_4(sc, ALC_MASTER_CFG); reg &= ~(MASTER_IM_RX_TIMER_ENB | MASTER_IM_TX_TIMER_ENB); reg |= MASTER_SA_TIMER_ENB; if (ALC_USECS(sc->alc_int_rx_mod) != 0) reg |= MASTER_IM_RX_TIMER_ENB; if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) == 0 && ALC_USECS(sc->alc_int_tx_mod) != 0) reg |= MASTER_IM_TX_TIMER_ENB; CSR_WRITE_4(sc, ALC_MASTER_CFG, reg); /* * Disable interrupt re-trigger timer. We don't want automatic * re-triggering of un-ACKed interrupts. */ CSR_WRITE_4(sc, ALC_INTR_RETRIG_TIMER, ALC_USECS(0)); /* Configure CMB. */ if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) != 0) { CSR_WRITE_4(sc, ALC_CMB_TD_THRESH, ALC_TX_RING_CNT / 3); CSR_WRITE_4(sc, ALC_CMB_TX_TIMER, ALC_USECS(sc->alc_int_tx_mod)); } else { if ((sc->alc_flags & ALC_FLAG_CMB_BUG) == 0) { CSR_WRITE_4(sc, ALC_CMB_TD_THRESH, 4); CSR_WRITE_4(sc, ALC_CMB_TX_TIMER, ALC_USECS(5000)); } else CSR_WRITE_4(sc, ALC_CMB_TX_TIMER, ALC_USECS(0)); } /* * Hardware can be configured to issue SMB interrupt based * on programmed interval. Since there is a callout that is * invoked for every hz in driver we use that instead of * relying on periodic SMB interrupt. */ CSR_WRITE_4(sc, ALC_SMB_STAT_TIMER, ALC_USECS(0)); /* Clear MAC statistics. */ alc_stats_clear(sc); /* * Always use maximum frame size that controller can support. * Otherwise received frames that has larger frame length * than alc(4) MTU would be silently dropped in hardware. This * would make path-MTU discovery hard as sender wouldn't get * any responses from receiver. alc(4) supports * multi-fragmented frames on Rx path so it has no issue on * assembling fragmented frames. Using maximum frame size also * removes the need to reinitialize hardware when interface * MTU configuration was changed. * * Be conservative in what you do, be liberal in what you * accept from others - RFC 793. */ CSR_WRITE_4(sc, ALC_FRAME_SIZE, sc->alc_ident->max_framelen); if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) == 0) { /* Disable header split(?) */ CSR_WRITE_4(sc, ALC_HDS_CFG, 0); /* Configure IPG/IFG parameters. */ CSR_WRITE_4(sc, ALC_IPG_IFG_CFG, ((IPG_IFG_IPGT_DEFAULT << IPG_IFG_IPGT_SHIFT) & IPG_IFG_IPGT_MASK) | ((IPG_IFG_MIFG_DEFAULT << IPG_IFG_MIFG_SHIFT) & IPG_IFG_MIFG_MASK) | ((IPG_IFG_IPG1_DEFAULT << IPG_IFG_IPG1_SHIFT) & IPG_IFG_IPG1_MASK) | ((IPG_IFG_IPG2_DEFAULT << IPG_IFG_IPG2_SHIFT) & IPG_IFG_IPG2_MASK)); /* Set parameters for half-duplex media. */ CSR_WRITE_4(sc, ALC_HDPX_CFG, ((HDPX_CFG_LCOL_DEFAULT << HDPX_CFG_LCOL_SHIFT) & HDPX_CFG_LCOL_MASK) | ((HDPX_CFG_RETRY_DEFAULT << HDPX_CFG_RETRY_SHIFT) & HDPX_CFG_RETRY_MASK) | HDPX_CFG_EXC_DEF_EN | ((HDPX_CFG_ABEBT_DEFAULT << HDPX_CFG_ABEBT_SHIFT) & HDPX_CFG_ABEBT_MASK) | ((HDPX_CFG_JAMIPG_DEFAULT << HDPX_CFG_JAMIPG_SHIFT) & HDPX_CFG_JAMIPG_MASK)); } /* * Set TSO/checksum offload threshold. For frames that is * larger than this threshold, hardware wouldn't do * TSO/checksum offloading. */ reg = (sc->alc_ident->max_framelen >> TSO_OFFLOAD_THRESH_UNIT_SHIFT) & TSO_OFFLOAD_THRESH_MASK; if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) != 0) reg |= TSO_OFFLOAD_ERRLGPKT_DROP_ENB; CSR_WRITE_4(sc, ALC_TSO_OFFLOAD_THRESH, reg); /* Configure TxQ. */ reg = (alc_dma_burst[sc->alc_dma_rd_burst] << TXQ_CFG_TX_FIFO_BURST_SHIFT) & TXQ_CFG_TX_FIFO_BURST_MASK; if (sc->alc_ident->deviceid == DEVICEID_ATHEROS_AR8152_B || sc->alc_ident->deviceid == DEVICEID_ATHEROS_AR8152_B2) reg >>= 1; reg |= (TXQ_CFG_TD_BURST_DEFAULT << TXQ_CFG_TD_BURST_SHIFT) & TXQ_CFG_TD_BURST_MASK; reg |= TXQ_CFG_IP_OPTION_ENB | TXQ_CFG_8023_ENB; CSR_WRITE_4(sc, ALC_TXQ_CFG, reg | TXQ_CFG_ENHANCED_MODE); if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) != 0) { reg = (TXQ_CFG_TD_BURST_DEFAULT << HQTD_CFG_Q1_BURST_SHIFT | TXQ_CFG_TD_BURST_DEFAULT << HQTD_CFG_Q2_BURST_SHIFT | TXQ_CFG_TD_BURST_DEFAULT << HQTD_CFG_Q3_BURST_SHIFT | HQTD_CFG_BURST_ENB); CSR_WRITE_4(sc, ALC_HQTD_CFG, reg); reg = WRR_PRI_RESTRICT_NONE; reg |= (WRR_PRI_DEFAULT << WRR_PRI0_SHIFT | WRR_PRI_DEFAULT << WRR_PRI1_SHIFT | WRR_PRI_DEFAULT << WRR_PRI2_SHIFT | WRR_PRI_DEFAULT << WRR_PRI3_SHIFT); CSR_WRITE_4(sc, ALC_WRR, reg); } else { /* Configure Rx free descriptor pre-fetching. */ CSR_WRITE_4(sc, ALC_RX_RD_FREE_THRESH, ((RX_RD_FREE_THRESH_HI_DEFAULT << RX_RD_FREE_THRESH_HI_SHIFT) & RX_RD_FREE_THRESH_HI_MASK) | ((RX_RD_FREE_THRESH_LO_DEFAULT << RX_RD_FREE_THRESH_LO_SHIFT) & RX_RD_FREE_THRESH_LO_MASK)); } /* * Configure flow control parameters. * XON : 80% of Rx FIFO * XOFF : 30% of Rx FIFO */ if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) != 0) { reg = CSR_READ_4(sc, ALC_SRAM_RX_FIFO_LEN); reg &= SRAM_RX_FIFO_LEN_MASK; reg *= 8; if (reg > 8 * 1024) reg -= RX_FIFO_PAUSE_816X_RSVD; else reg -= RX_BUF_SIZE_MAX; reg /= 8; CSR_WRITE_4(sc, ALC_RX_FIFO_PAUSE_THRESH, ((reg << RX_FIFO_PAUSE_THRESH_LO_SHIFT) & RX_FIFO_PAUSE_THRESH_LO_MASK) | (((RX_FIFO_PAUSE_816X_RSVD / 8) << RX_FIFO_PAUSE_THRESH_HI_SHIFT) & RX_FIFO_PAUSE_THRESH_HI_MASK)); } else if (sc->alc_ident->deviceid == DEVICEID_ATHEROS_AR8131 || sc->alc_ident->deviceid == DEVICEID_ATHEROS_AR8132) { reg = CSR_READ_4(sc, ALC_SRAM_RX_FIFO_LEN); rxf_hi = (reg * 8) / 10; rxf_lo = (reg * 3) / 10; CSR_WRITE_4(sc, ALC_RX_FIFO_PAUSE_THRESH, ((rxf_lo << RX_FIFO_PAUSE_THRESH_LO_SHIFT) & RX_FIFO_PAUSE_THRESH_LO_MASK) | ((rxf_hi << RX_FIFO_PAUSE_THRESH_HI_SHIFT) & RX_FIFO_PAUSE_THRESH_HI_MASK)); } if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) == 0) { /* Disable RSS until I understand L1C/L2C's RSS logic. */ CSR_WRITE_4(sc, ALC_RSS_IDT_TABLE0, 0); CSR_WRITE_4(sc, ALC_RSS_CPU, 0); } /* Configure RxQ. */ reg = (RXQ_CFG_RD_BURST_DEFAULT << RXQ_CFG_RD_BURST_SHIFT) & RXQ_CFG_RD_BURST_MASK; reg |= RXQ_CFG_RSS_MODE_DIS; if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) != 0) { reg |= (RXQ_CFG_816X_IDT_TBL_SIZE_DEFAULT << RXQ_CFG_816X_IDT_TBL_SIZE_SHIFT) & RXQ_CFG_816X_IDT_TBL_SIZE_MASK; if ((sc->alc_flags & ALC_FLAG_FASTETHER) == 0) reg |= RXQ_CFG_ASPM_THROUGHPUT_LIMIT_100M; } else { if ((sc->alc_flags & ALC_FLAG_FASTETHER) == 0 && sc->alc_ident->deviceid != DEVICEID_ATHEROS_AR8151_V2) reg |= RXQ_CFG_ASPM_THROUGHPUT_LIMIT_100M; } CSR_WRITE_4(sc, ALC_RXQ_CFG, reg); /* Configure DMA parameters. */ reg = DMA_CFG_OUT_ORDER | DMA_CFG_RD_REQ_PRI; reg |= sc->alc_rcb; if ((sc->alc_flags & ALC_FLAG_CMB_BUG) == 0) reg |= DMA_CFG_CMB_ENB; if ((sc->alc_flags & ALC_FLAG_SMB_BUG) == 0) reg |= DMA_CFG_SMB_ENB; else reg |= DMA_CFG_SMB_DIS; reg |= (sc->alc_dma_rd_burst & DMA_CFG_RD_BURST_MASK) << DMA_CFG_RD_BURST_SHIFT; reg |= (sc->alc_dma_wr_burst & DMA_CFG_WR_BURST_MASK) << DMA_CFG_WR_BURST_SHIFT; reg |= (DMA_CFG_RD_DELAY_CNT_DEFAULT << DMA_CFG_RD_DELAY_CNT_SHIFT) & DMA_CFG_RD_DELAY_CNT_MASK; reg |= (DMA_CFG_WR_DELAY_CNT_DEFAULT << DMA_CFG_WR_DELAY_CNT_SHIFT) & DMA_CFG_WR_DELAY_CNT_MASK; if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) != 0) { switch (AR816X_REV(sc->alc_rev)) { case AR816X_REV_A0: case AR816X_REV_A1: reg |= DMA_CFG_RD_CHNL_SEL_2; break; case AR816X_REV_B0: /* FALLTHROUGH */ default: reg |= DMA_CFG_RD_CHNL_SEL_4; break; } } CSR_WRITE_4(sc, ALC_DMA_CFG, reg); /* * Configure Tx/Rx MACs. * - Auto-padding for short frames. * - Enable CRC generation. * Actual reconfiguration of MAC for resolved speed/duplex * is followed after detection of link establishment. * AR813x/AR815x always does checksum computation regardless * of MAC_CFG_RXCSUM_ENB bit. Also the controller is known to * have bug in protocol field in Rx return structure so * these controllers can't handle fragmented frames. Disable * Rx checksum offloading until there is a newer controller * that has sane implementation. */ reg = MAC_CFG_TX_CRC_ENB | MAC_CFG_TX_AUTO_PAD | MAC_CFG_FULL_DUPLEX | ((MAC_CFG_PREAMBLE_DEFAULT << MAC_CFG_PREAMBLE_SHIFT) & MAC_CFG_PREAMBLE_MASK); if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) != 0 || sc->alc_ident->deviceid == DEVICEID_ATHEROS_AR8151 || sc->alc_ident->deviceid == DEVICEID_ATHEROS_AR8151_V2 || sc->alc_ident->deviceid == DEVICEID_ATHEROS_AR8152_B2) reg |= MAC_CFG_HASH_ALG_CRC32 | MAC_CFG_SPEED_MODE_SW; if ((sc->alc_flags & ALC_FLAG_FASTETHER) != 0) reg |= MAC_CFG_SPEED_10_100; else reg |= MAC_CFG_SPEED_1000; CSR_WRITE_4(sc, ALC_MAC_CFG, reg); /* Set up the receive filter. */ alc_rxfilter(sc); alc_rxvlan(sc); /* Acknowledge all pending interrupts and clear it. */ CSR_WRITE_4(sc, ALC_INTR_MASK, ALC_INTRS); CSR_WRITE_4(sc, ALC_INTR_STATUS, 0xFFFFFFFF); CSR_WRITE_4(sc, ALC_INTR_STATUS, 0); ifp->if_drv_flags |= IFF_DRV_RUNNING; ifp->if_drv_flags &= ~IFF_DRV_OACTIVE; sc->alc_flags &= ~ALC_FLAG_LINK; /* Switch to the current media. */ alc_mediachange_locked(sc); callout_reset(&sc->alc_tick_ch, hz, alc_tick, sc); } static void alc_stop(struct alc_softc *sc) { struct ifnet *ifp; struct alc_txdesc *txd; struct alc_rxdesc *rxd; uint32_t reg; int i; ALC_LOCK_ASSERT(sc); /* * Mark the interface down and cancel the watchdog timer. */ ifp = sc->alc_ifp; ifp->if_drv_flags &= ~(IFF_DRV_RUNNING | IFF_DRV_OACTIVE); sc->alc_flags &= ~ALC_FLAG_LINK; callout_stop(&sc->alc_tick_ch); sc->alc_watchdog_timer = 0; alc_stats_update(sc); /* Disable interrupts. */ CSR_WRITE_4(sc, ALC_INTR_MASK, 0); CSR_WRITE_4(sc, ALC_INTR_STATUS, 0xFFFFFFFF); /* Disable DMA. */ reg = CSR_READ_4(sc, ALC_DMA_CFG); reg &= ~(DMA_CFG_CMB_ENB | DMA_CFG_SMB_ENB); reg |= DMA_CFG_SMB_DIS; CSR_WRITE_4(sc, ALC_DMA_CFG, reg); DELAY(1000); /* Stop Rx/Tx MACs. */ alc_stop_mac(sc); /* Disable interrupts which might be touched in taskq handler. */ CSR_WRITE_4(sc, ALC_INTR_STATUS, 0xFFFFFFFF); /* Disable L0s/L1s */ alc_aspm(sc, 0, IFM_UNKNOWN); /* Reclaim Rx buffers that have been processed. */ if (sc->alc_cdata.alc_rxhead != NULL) m_freem(sc->alc_cdata.alc_rxhead); ALC_RXCHAIN_RESET(sc); /* * Free Tx/Rx mbufs still in the queues. */ for (i = 0; i < ALC_RX_RING_CNT; i++) { rxd = &sc->alc_cdata.alc_rxdesc[i]; if (rxd->rx_m != NULL) { bus_dmamap_sync(sc->alc_cdata.alc_rx_tag, rxd->rx_dmamap, BUS_DMASYNC_POSTREAD); bus_dmamap_unload(sc->alc_cdata.alc_rx_tag, rxd->rx_dmamap); m_freem(rxd->rx_m); rxd->rx_m = NULL; } } for (i = 0; i < ALC_TX_RING_CNT; i++) { txd = &sc->alc_cdata.alc_txdesc[i]; if (txd->tx_m != NULL) { bus_dmamap_sync(sc->alc_cdata.alc_tx_tag, txd->tx_dmamap, BUS_DMASYNC_POSTWRITE); bus_dmamap_unload(sc->alc_cdata.alc_tx_tag, txd->tx_dmamap); m_freem(txd->tx_m); txd->tx_m = NULL; } } } static void alc_stop_mac(struct alc_softc *sc) { uint32_t reg; int i; alc_stop_queue(sc); /* Disable Rx/Tx MAC. */ reg = CSR_READ_4(sc, ALC_MAC_CFG); if ((reg & (MAC_CFG_TX_ENB | MAC_CFG_RX_ENB)) != 0) { reg &= ~(MAC_CFG_TX_ENB | MAC_CFG_RX_ENB); CSR_WRITE_4(sc, ALC_MAC_CFG, reg); } for (i = ALC_TIMEOUT; i > 0; i--) { reg = CSR_READ_4(sc, ALC_IDLE_STATUS); if ((reg & (IDLE_STATUS_RXMAC | IDLE_STATUS_TXMAC)) == 0) break; DELAY(10); } if (i == 0) device_printf(sc->alc_dev, "could not disable Rx/Tx MAC(0x%08x)!\n", reg); } static void alc_start_queue(struct alc_softc *sc) { uint32_t qcfg[] = { 0, RXQ_CFG_QUEUE0_ENB, RXQ_CFG_QUEUE0_ENB | RXQ_CFG_QUEUE1_ENB, RXQ_CFG_QUEUE0_ENB | RXQ_CFG_QUEUE1_ENB | RXQ_CFG_QUEUE2_ENB, RXQ_CFG_ENB }; uint32_t cfg; ALC_LOCK_ASSERT(sc); /* Enable RxQ. */ cfg = CSR_READ_4(sc, ALC_RXQ_CFG); if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) == 0) { cfg &= ~RXQ_CFG_ENB; cfg |= qcfg[1]; } else cfg |= RXQ_CFG_QUEUE0_ENB; CSR_WRITE_4(sc, ALC_RXQ_CFG, cfg); /* Enable TxQ. */ cfg = CSR_READ_4(sc, ALC_TXQ_CFG); cfg |= TXQ_CFG_ENB; CSR_WRITE_4(sc, ALC_TXQ_CFG, cfg); } static void alc_stop_queue(struct alc_softc *sc) { uint32_t reg; int i; /* Disable RxQ. */ reg = CSR_READ_4(sc, ALC_RXQ_CFG); if ((sc->alc_flags & ALC_FLAG_AR816X_FAMILY) == 0) { if ((reg & RXQ_CFG_ENB) != 0) { reg &= ~RXQ_CFG_ENB; CSR_WRITE_4(sc, ALC_RXQ_CFG, reg); } } else { if ((reg & RXQ_CFG_QUEUE0_ENB) != 0) { reg &= ~RXQ_CFG_QUEUE0_ENB; CSR_WRITE_4(sc, ALC_RXQ_CFG, reg); } } /* Disable TxQ. */ reg = CSR_READ_4(sc, ALC_TXQ_CFG); if ((reg & TXQ_CFG_ENB) != 0) { reg &= ~TXQ_CFG_ENB; CSR_WRITE_4(sc, ALC_TXQ_CFG, reg); } DELAY(40); for (i = ALC_TIMEOUT; i > 0; i--) { reg = CSR_READ_4(sc, ALC_IDLE_STATUS); if ((reg & (IDLE_STATUS_RXQ | IDLE_STATUS_TXQ)) == 0) break; DELAY(10); } if (i == 0) device_printf(sc->alc_dev, "could not disable RxQ/TxQ (0x%08x)!\n", reg); } static void alc_init_tx_ring(struct alc_softc *sc) { struct alc_ring_data *rd; struct alc_txdesc *txd; int i; ALC_LOCK_ASSERT(sc); sc->alc_cdata.alc_tx_prod = 0; sc->alc_cdata.alc_tx_cons = 0; sc->alc_cdata.alc_tx_cnt = 0; rd = &sc->alc_rdata; bzero(rd->alc_tx_ring, ALC_TX_RING_SZ); for (i = 0; i < ALC_TX_RING_CNT; i++) { txd = &sc->alc_cdata.alc_txdesc[i]; txd->tx_m = NULL; } bus_dmamap_sync(sc->alc_cdata.alc_tx_ring_tag, sc->alc_cdata.alc_tx_ring_map, BUS_DMASYNC_PREWRITE); } static int alc_init_rx_ring(struct alc_softc *sc) { struct alc_ring_data *rd; struct alc_rxdesc *rxd; int i; ALC_LOCK_ASSERT(sc); sc->alc_cdata.alc_rx_cons = ALC_RX_RING_CNT - 1; sc->alc_morework = 0; rd = &sc->alc_rdata; bzero(rd->alc_rx_ring, ALC_RX_RING_SZ); for (i = 0; i < ALC_RX_RING_CNT; i++) { rxd = &sc->alc_cdata.alc_rxdesc[i]; rxd->rx_m = NULL; rxd->rx_desc = &rd->alc_rx_ring[i]; if (alc_newbuf(sc, rxd) != 0) return (ENOBUFS); } /* * Since controller does not update Rx descriptors, driver * does have to read Rx descriptors back so BUS_DMASYNC_PREWRITE * is enough to ensure coherence. */ bus_dmamap_sync(sc->alc_cdata.alc_rx_ring_tag, sc->alc_cdata.alc_rx_ring_map, BUS_DMASYNC_PREWRITE); /* Let controller know availability of new Rx buffers. */ CSR_WRITE_4(sc, ALC_MBOX_RD0_PROD_IDX, sc->alc_cdata.alc_rx_cons); return (0); } static void alc_init_rr_ring(struct alc_softc *sc) { struct alc_ring_data *rd; ALC_LOCK_ASSERT(sc); sc->alc_cdata.alc_rr_cons = 0; ALC_RXCHAIN_RESET(sc); rd = &sc->alc_rdata; bzero(rd->alc_rr_ring, ALC_RR_RING_SZ); bus_dmamap_sync(sc->alc_cdata.alc_rr_ring_tag, sc->alc_cdata.alc_rr_ring_map, BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); } static void alc_init_cmb(struct alc_softc *sc) { struct alc_ring_data *rd; ALC_LOCK_ASSERT(sc); rd = &sc->alc_rdata; bzero(rd->alc_cmb, ALC_CMB_SZ); bus_dmamap_sync(sc->alc_cdata.alc_cmb_tag, sc->alc_cdata.alc_cmb_map, BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); } static void alc_init_smb(struct alc_softc *sc) { struct alc_ring_data *rd; ALC_LOCK_ASSERT(sc); rd = &sc->alc_rdata; bzero(rd->alc_smb, ALC_SMB_SZ); bus_dmamap_sync(sc->alc_cdata.alc_smb_tag, sc->alc_cdata.alc_smb_map, BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); } static void alc_rxvlan(struct alc_softc *sc) { struct ifnet *ifp; uint32_t reg; ALC_LOCK_ASSERT(sc); ifp = sc->alc_ifp; reg = CSR_READ_4(sc, ALC_MAC_CFG); if ((ifp->if_capenable & IFCAP_VLAN_HWTAGGING) != 0) reg |= MAC_CFG_VLAN_TAG_STRIP; else reg &= ~MAC_CFG_VLAN_TAG_STRIP; CSR_WRITE_4(sc, ALC_MAC_CFG, reg); } static void alc_rxfilter(struct alc_softc *sc) { struct ifnet *ifp; struct ifmultiaddr *ifma; uint32_t crc; uint32_t mchash[2]; uint32_t rxcfg; ALC_LOCK_ASSERT(sc); ifp = sc->alc_ifp; bzero(mchash, sizeof(mchash)); rxcfg = CSR_READ_4(sc, ALC_MAC_CFG); rxcfg &= ~(MAC_CFG_ALLMULTI | MAC_CFG_BCAST | MAC_CFG_PROMISC); if ((ifp->if_flags & IFF_BROADCAST) != 0) rxcfg |= MAC_CFG_BCAST; if ((ifp->if_flags & (IFF_PROMISC | IFF_ALLMULTI)) != 0) { if ((ifp->if_flags & IFF_PROMISC) != 0) rxcfg |= MAC_CFG_PROMISC; if ((ifp->if_flags & IFF_ALLMULTI) != 0) rxcfg |= MAC_CFG_ALLMULTI; mchash[0] = 0xFFFFFFFF; mchash[1] = 0xFFFFFFFF; goto chipit; } if_maddr_rlock(ifp); TAILQ_FOREACH(ifma, &sc->alc_ifp->if_multiaddrs, ifma_link) { if (ifma->ifma_addr->sa_family != AF_LINK) continue; crc = ether_crc32_be(LLADDR((struct sockaddr_dl *) ifma->ifma_addr), ETHER_ADDR_LEN); mchash[crc >> 31] |= 1 << ((crc >> 26) & 0x1f); } if_maddr_runlock(ifp); chipit: CSR_WRITE_4(sc, ALC_MAR0, mchash[0]); CSR_WRITE_4(sc, ALC_MAR1, mchash[1]); CSR_WRITE_4(sc, ALC_MAC_CFG, rxcfg); } static int sysctl_int_range(SYSCTL_HANDLER_ARGS, int low, int high) { int error, value; if (arg1 == NULL) return (EINVAL); value = *(int *)arg1; error = sysctl_handle_int(oidp, &value, 0, req); if (error || req->newptr == NULL) return (error); if (value < low || value > high) return (EINVAL); *(int *)arg1 = value; return (0); } static int sysctl_hw_alc_proc_limit(SYSCTL_HANDLER_ARGS) { return (sysctl_int_range(oidp, arg1, arg2, req, ALC_PROC_MIN, ALC_PROC_MAX)); } static int sysctl_hw_alc_int_mod(SYSCTL_HANDLER_ARGS) { return (sysctl_int_range(oidp, arg1, arg2, req, ALC_IM_TIMER_MIN, ALC_IM_TIMER_MAX)); } Index: head/sys/dev/alc/if_alcreg.h =================================================================== --- head/sys/dev/alc/if_alcreg.h (revision 312249) +++ head/sys/dev/alc/if_alcreg.h (revision 312250) @@ -1,1288 +1,1289 @@ /*- * Copyright (c) 2009, Pyun YongHyeon * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice unmodified, this list of conditions, and the following * disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ #ifndef _IF_ALCREG_H #define _IF_ALCREG_H /* * Atheros Communucations, Inc. PCI vendor ID */ #define VENDORID_ATHEROS 0x1969 /* * Atheros AR813x/AR815x device ID */ #define DEVICEID_ATHEROS_AR8131 0x1063 /* L1C */ #define DEVICEID_ATHEROS_AR8132 0x1062 /* L2C */ #define DEVICEID_ATHEROS_AR8151 0x1073 /* L1D V1.0 */ #define DEVICEID_ATHEROS_AR8151_V2 0x1083 /* L1D V2.0 */ #define DEVICEID_ATHEROS_AR8152_B 0x2060 /* L2C V1.1 */ #define DEVICEID_ATHEROS_AR8152_B2 0x2062 /* L2C V2.0 */ #define DEVICEID_ATHEROS_AR8161 0x1091 #define DEVICEID_ATHEROS_AR8162 0x1090 #define DEVICEID_ATHEROS_AR8171 0x10A1 #define DEVICEID_ATHEROS_AR8172 0x10A0 #define DEVICEID_ATHEROS_E2200 0xE091 #define DEVICEID_ATHEROS_E2400 0xE0A1 +#define DEVICEID_ATHEROS_E2500 0xE0B1 #define ATHEROS_AR8152_B_V10 0xC0 #define ATHEROS_AR8152_B_V11 0xC1 /* * Atheros AR816x/AR817x revisions */ #define AR816X_REV_A0 0 #define AR816X_REV_A1 1 #define AR816X_REV_B0 2 #define AR816X_REV_C0 3 #define AR816X_REV_SHIFT 3 #define AR816X_REV(x) ((x) >> AR816X_REV_SHIFT) /* 0x0000 - 0x02FF : PCIe configuration space */ #define ALC_PEX_UNC_ERR_SEV 0x10C #define PEX_UNC_ERR_SEV_TRN 0x00000001 #define PEX_UNC_ERR_SEV_DLP 0x00000010 #define PEX_UNC_ERR_SEV_PSN_TLP 0x00001000 #define PEX_UNC_ERR_SEV_FCP 0x00002000 #define PEX_UNC_ERR_SEV_CPL_TO 0x00004000 #define PEX_UNC_ERR_SEV_CA 0x00008000 #define PEX_UNC_ERR_SEV_UC 0x00010000 #define PEX_UNC_ERR_SEV_ROV 0x00020000 #define PEX_UNC_ERR_SEV_MLFP 0x00040000 #define PEX_UNC_ERR_SEV_ECRC 0x00080000 #define PEX_UNC_ERR_SEV_UR 0x00100000 #define ALC_EEPROM_LD 0x204 /* AR816x */ #define EEPROM_LD_START 0x00000001 #define EEPROM_LD_IDLE 0x00000010 #define EEPROM_LD_DONE 0x00000000 #define EEPROM_LD_PROGRESS 0x00000020 #define EEPROM_LD_EXIST 0x00000100 #define EEPROM_LD_EEPROM_EXIST 0x00000200 #define EEPROM_LD_FLASH_EXIST 0x00000400 #define EEPROM_LD_FLASH_END_ADDR_MASK 0x03FF0000 #define EEPROM_LD_FLASH_END_ADDR_SHIFT 16 #define ALC_TWSI_CFG 0x218 #define TWSI_CFG_SW_LD_START 0x00000800 #define TWSI_CFG_HW_LD_START 0x00001000 #define TWSI_CFG_LD_EXIST 0x00400000 #define ALC_SLD 0x218 /* AR816x */ #define SLD_START 0x00000800 #define SLD_PROGRESS 0x00001000 #define SLD_IDLE 0x00002000 #define SLD_SLVADDR_MASK 0x007F0000 #define SLD_EXIST 0x00800000 #define SLD_FREQ_MASK 0x03000000 #define SLD_FREQ_100K 0x00000000 #define SLD_FREQ_200K 0x01000000 #define SLD_FREQ_300K 0x02000000 #define SLD_FREQ_400K 0x03000000 #define ALC_PCIE_PHYMISC 0x1000 #define PCIE_PHYMISC_FORCE_RCV_DET 0x00000004 #define ALC_PCIE_PHYMISC2 0x1004 #define PCIE_PHYMISC2_SERDES_CDR_MASK 0x00030000 #define PCIE_PHYMISC2_SERDES_TH_MASK 0x000C0000 #define PCIE_PHYMISC2_SERDES_CDR_SHIFT 16 #define PCIE_PHYMISC2_SERDES_TH_SHIFT 18 #define ALC_PDLL_TRNS1 0x1104 #define PDLL_TRNS1_D3PLLOFF_ENB 0x00000800 #define ALC_TWSI_DEBUG 0x1108 #define TWSI_DEBUG_DEV_EXIST 0x20000000 #define ALC_EEPROM_CFG 0x12C0 #define EEPROM_CFG_DATA_HI_MASK 0x0000FFFF #define EEPROM_CFG_ADDR_MASK 0x03FF0000 #define EEPROM_CFG_ACK 0x40000000 #define EEPROM_CFG_RW 0x80000000 #define EEPROM_CFG_DATA_HI_SHIFT 0 #define EEPROM_CFG_ADDR_SHIFT 16 #define ALC_EEPROM_DATA_LO 0x12C4 #define ALC_OPT_CFG 0x12F0 #define OPT_CFG_CLK_ENB 0x00000002 #define ALC_PM_CFG 0x12F8 #define PM_CFG_SERDES_ENB 0x00000001 #define PM_CFG_RBER_ENB 0x00000002 #define PM_CFG_CLK_REQ_ENB 0x00000004 #define PM_CFG_ASPM_L1_ENB 0x00000008 #define PM_CFG_SERDES_L1_ENB 0x00000010 #define PM_CFG_SERDES_PLL_L1_ENB 0x00000020 #define PM_CFG_SERDES_PD_EX_L1 0x00000040 #define PM_CFG_SERDES_BUDS_RX_L1_ENB 0x00000080 #define PM_CFG_L0S_ENTRY_TIMER_MASK 0x00000F00 #define PM_CFG_RX_L1_AFTER_L0S 0x00000800 #define PM_CFG_ASPM_L0S_ENB 0x00001000 #define PM_CFG_CLK_SWH_L1 0x00002000 #define PM_CFG_CLK_PWM_VER1_1 0x00004000 #define PM_CFG_PCIE_RECV 0x00008000 #define PM_CFG_L1_ENTRY_TIMER_MASK 0x000F0000 #define PM_CFG_L1_ENTRY_TIMER_816X_MASK 0x00070000 #define PM_CFG_TX_L1_AFTER_L0S 0x00080000 #define PM_CFG_PM_REQ_TIMER_MASK 0x00F00000 #define PM_CFG_LCKDET_TIMER_MASK 0x0F000000 #define PM_CFG_EN_BUFS_RX_L0S 0x10000000 #define PM_CFG_SA_DLY_ENB 0x20000000 #define PM_CFG_MAC_ASPM_CHK 0x40000000 #define PM_CFG_HOTRST 0x80000000 #define PM_CFG_L0S_ENTRY_TIMER_SHIFT 8 #define PM_CFG_L1_ENTRY_TIMER_SHIFT 16 #define PM_CFG_PM_REQ_TIMER_SHIFT 20 #define PM_CFG_LCKDET_TIMER_SHIFT 24 #define PM_CFG_L0S_ENTRY_TIMER_DEFAULT 6 #define PM_CFG_L1_ENTRY_TIMER_DEFAULT 1 #define PM_CFG_L1_ENTRY_TIMER_816X_DEFAULT 4 #define PM_CFG_LCKDET_TIMER_DEFAULT 12 #define PM_CFG_PM_REQ_TIMER_DEFAULT 12 #define PM_CFG_PM_REQ_TIMER_816X_DEFAULT 15 #define ALC_LTSSM_ID_CFG 0x12FC #define LTSSM_ID_WRO_ENB 0x00001000 #define ALC_MASTER_CFG 0x1400 #define MASTER_RESET 0x00000001 #define MASTER_TEST_MODE_MASK 0x0000000C #define MASTER_BERT_START 0x00000010 #define MASTER_WAKEN_25M 0x00000020 #define MASTER_OOB_DIS_OFF 0x00000040 #define MASTER_SA_TIMER_ENB 0x00000080 #define MASTER_MTIMER_ENB 0x00000100 #define MASTER_MANUAL_INTR_ENB 0x00000200 #define MASTER_IM_TX_TIMER_ENB 0x00000400 #define MASTER_IM_RX_TIMER_ENB 0x00000800 #define MASTER_CLK_SEL_DIS 0x00001000 #define MASTER_CLK_SWH_MODE 0x00002000 #define MASTER_INTR_RD_CLR 0x00004000 #define MASTER_CHIP_REV_MASK 0x00FF0000 #define MASTER_CHIP_ID_MASK 0x7F000000 #define MASTER_OTP_SEL 0x80000000 #define MASTER_TEST_MODE_SHIFT 2 #define MASTER_CHIP_REV_SHIFT 16 #define MASTER_CHIP_ID_SHIFT 24 /* Number of ticks per usec for AR813x/AR815x. */ #define ALC_TICK_USECS 2 #define ALC_USECS(x) ((x) / ALC_TICK_USECS) #define ALC_MANUAL_TIMER 0x1404 #define ALC_IM_TIMER 0x1408 #define IM_TIMER_TX_MASK 0x0000FFFF #define IM_TIMER_RX_MASK 0xFFFF0000 #define IM_TIMER_TX_SHIFT 0 #define IM_TIMER_RX_SHIFT 16 #define ALC_IM_TIMER_MIN 0 #define ALC_IM_TIMER_MAX 130000 /* 130ms */ /* * 100us will ensure alc(4) wouldn't generate more than 10000 Rx * interrupts in a second. */ #define ALC_IM_RX_TIMER_DEFAULT 100 /* 100us */ /* * alc(4) does not rely on Tx completion interrupts, so set it * somewhat large value to reduce Tx completion interrupts. */ #define ALC_IM_TX_TIMER_DEFAULT 1000 /* 1ms */ #define ALC_GPHY_CFG 0x140C /* 16 bits, 32 bits on AR816x */ #define GPHY_CFG_EXT_RESET 0x0001 #define GPHY_CFG_RTL_MODE 0x0002 #define GPHY_CFG_LED_MODE 0x0004 #define GPHY_CFG_ANEG_NOW 0x0008 #define GPHY_CFG_RECV_ANEG 0x0010 #define GPHY_CFG_GATE_25M_ENB 0x0020 #define GPHY_CFG_LPW_EXIT 0x0040 #define GPHY_CFG_PHY_IDDQ 0x0080 #define GPHY_CFG_PHY_IDDQ_DIS 0x0100 #define GPHY_CFG_PCLK_SEL_DIS 0x0200 #define GPHY_CFG_HIB_EN 0x0400 #define GPHY_CFG_HIB_PULSE 0x0800 #define GPHY_CFG_SEL_ANA_RESET 0x1000 #define GPHY_CFG_PHY_PLL_ON 0x2000 #define GPHY_CFG_PWDOWN_HW 0x4000 #define GPHY_CFG_PHY_PLL_BYPASS 0x8000 #define GPHY_CFG_100AB_ENB 0x00020000 #define ALC_IDLE_STATUS 0x1410 #define IDLE_STATUS_RXMAC 0x00000001 #define IDLE_STATUS_TXMAC 0x00000002 #define IDLE_STATUS_RXQ 0x00000004 #define IDLE_STATUS_TXQ 0x00000008 #define IDLE_STATUS_DMARD 0x00000010 #define IDLE_STATUS_DMAWR 0x00000020 #define IDLE_STATUS_SMB 0x00000040 #define IDLE_STATUS_CMB 0x00000080 #define ALC_MDIO 0x1414 #define MDIO_DATA_MASK 0x0000FFFF #define MDIO_REG_ADDR_MASK 0x001F0000 #define MDIO_OP_READ 0x00200000 #define MDIO_OP_WRITE 0x00000000 #define MDIO_SUP_PREAMBLE 0x00400000 #define MDIO_OP_EXECUTE 0x00800000 #define MDIO_CLK_25_4 0x00000000 #define MDIO_CLK_25_6 0x02000000 #define MDIO_CLK_25_8 0x03000000 #define MDIO_CLK_25_10 0x04000000 #define MDIO_CLK_25_14 0x05000000 #define MDIO_CLK_25_20 0x06000000 #define MDIO_CLK_25_128 0x07000000 #define MDIO_OP_BUSY 0x08000000 #define MDIO_AP_ENB 0x10000000 #define MDIO_MODE_EXT 0x40000000 #define MDIO_DATA_SHIFT 0 #define MDIO_REG_ADDR_SHIFT 16 #define MDIO_REG_ADDR(x) \ (((x) << MDIO_REG_ADDR_SHIFT) & MDIO_REG_ADDR_MASK) /* Default PHY address. */ #define ALC_PHY_ADDR 0 #define ALC_PHY_STATUS 0x1418 #define PHY_STATUS_RECV_ENB 0x00000001 #define PHY_STATUS_GENERAL_MASK 0x0000FFFF #define PHY_STATUS_OE_PWSP_MASK 0x07FF0000 #define PHY_STATUS_LPW_STATE 0x80000000 #define PHY_STATIS_OE_PWSP_SHIFT 16 /* Packet memory BIST. */ #define ALC_BIST0 0x141C #define BIST0_ENB 0x00000001 #define BIST0_SRAM_FAIL 0x00000002 #define BIST0_FUSE_FLAG 0x00000004 /* PCIe retry buffer BIST. */ #define ALC_BIST1 0x1420 #define BIST1_ENB 0x00000001 #define BIST1_SRAM_FAIL 0x00000002 #define BIST1_FUSE_FLAG 0x00000004 #define ALC_SERDES_LOCK 0x1424 #define SERDES_LOCK_DET 0x00000001 #define SERDES_LOCK_DET_ENB 0x00000002 #define SERDES_MAC_CLK_SLOWDOWN 0x00020000 #define SERDES_PHY_CLK_SLOWDOWN 0x00040000 #define ALC_LPI_CTL 0x1440 #define LPI_CTL_ENB 0x00000001 #define ALC_EXT_MDIO 0x1448 #define EXT_MDIO_REG_MASK 0x0000FFFF #define EXT_MDIO_DEVADDR_MASK 0x001F0000 #define EXT_MDIO_REG_SHIFT 0 #define EXT_MDIO_DEVADDR_SHIFT 16 #define EXT_MDIO_REG(x) \ (((x) << EXT_MDIO_REG_SHIFT) & EXT_MDIO_REG_MASK) #define EXT_MDIO_DEVADDR(x) \ (((x) << EXT_MDIO_DEVADDR_SHIFT) & EXT_MDIO_DEVADDR_MASK) #define ALC_IDLE_DECISN_TIMER 0x1474 #define IDLE_DECISN_TIMER_DEFAULT_1MS 0x400 #define ALC_MAC_CFG 0x1480 #define MAC_CFG_TX_ENB 0x00000001 #define MAC_CFG_RX_ENB 0x00000002 #define MAC_CFG_TX_FC 0x00000004 #define MAC_CFG_RX_FC 0x00000008 #define MAC_CFG_LOOP 0x00000010 #define MAC_CFG_FULL_DUPLEX 0x00000020 #define MAC_CFG_TX_CRC_ENB 0x00000040 #define MAC_CFG_TX_AUTO_PAD 0x00000080 #define MAC_CFG_TX_LENCHK 0x00000100 #define MAC_CFG_RX_JUMBO_ENB 0x00000200 #define MAC_CFG_PREAMBLE_MASK 0x00003C00 #define MAC_CFG_VLAN_TAG_STRIP 0x00004000 #define MAC_CFG_PROMISC 0x00008000 #define MAC_CFG_TX_PAUSE 0x00010000 #define MAC_CFG_SCNT 0x00020000 #define MAC_CFG_SYNC_RST_TX 0x00040000 #define MAC_CFG_SIM_RST_TX 0x00080000 #define MAC_CFG_SPEED_MASK 0x00300000 #define MAC_CFG_SPEED_10_100 0x00100000 #define MAC_CFG_SPEED_1000 0x00200000 #define MAC_CFG_DBG_TX_BACKOFF 0x00400000 #define MAC_CFG_TX_JUMBO_ENB 0x00800000 #define MAC_CFG_RXCSUM_ENB 0x01000000 #define MAC_CFG_ALLMULTI 0x02000000 #define MAC_CFG_BCAST 0x04000000 #define MAC_CFG_DBG 0x08000000 #define MAC_CFG_SINGLE_PAUSE_ENB 0x10000000 #define MAC_CFG_HASH_ALG_CRC32 0x20000000 #define MAC_CFG_SPEED_MODE_SW 0x40000000 #define MAC_CFG_FAST_PAUSE 0x80000000 #define MAC_CFG_PREAMBLE_SHIFT 10 #define MAC_CFG_PREAMBLE_DEFAULT 7 #define ALC_IPG_IFG_CFG 0x1484 #define IPG_IFG_IPGT_MASK 0x0000007F #define IPG_IFG_MIFG_MASK 0x0000FF00 #define IPG_IFG_IPG1_MASK 0x007F0000 #define IPG_IFG_IPG2_MASK 0x7F000000 #define IPG_IFG_IPGT_SHIFT 0 #define IPG_IFG_IPGT_DEFAULT 0x60 #define IPG_IFG_MIFG_SHIFT 8 #define IPG_IFG_MIFG_DEFAULT 0x50 #define IPG_IFG_IPG1_SHIFT 16 #define IPG_IFG_IPG1_DEFAULT 0x40 #define IPG_IFG_IPG2_SHIFT 24 #define IPG_IFG_IPG2_DEFAULT 0x60 /* Station address. */ #define ALC_PAR0 0x1488 #define ALC_PAR1 0x148C /* 64bit multicast hash register. */ #define ALC_MAR0 0x1490 #define ALC_MAR1 0x1494 /* half-duplex parameter configuration. */ #define ALC_HDPX_CFG 0x1498 #define HDPX_CFG_LCOL_MASK 0x000003FF #define HDPX_CFG_RETRY_MASK 0x0000F000 #define HDPX_CFG_EXC_DEF_EN 0x00010000 #define HDPX_CFG_NO_BACK_C 0x00020000 #define HDPX_CFG_NO_BACK_P 0x00040000 #define HDPX_CFG_ABEBE 0x00080000 #define HDPX_CFG_ABEBT_MASK 0x00F00000 #define HDPX_CFG_JAMIPG_MASK 0x0F000000 #define HDPX_CFG_LCOL_SHIFT 0 #define HDPX_CFG_LCOL_DEFAULT 0x37 #define HDPX_CFG_RETRY_SHIFT 12 #define HDPX_CFG_RETRY_DEFAULT 0x0F #define HDPX_CFG_ABEBT_SHIFT 20 #define HDPX_CFG_ABEBT_DEFAULT 0x0A #define HDPX_CFG_JAMIPG_SHIFT 24 #define HDPX_CFG_JAMIPG_DEFAULT 0x07 #define ALC_FRAME_SIZE 0x149C #define ALC_WOL_CFG 0x14A0 #define WOL_CFG_PATTERN 0x00000001 #define WOL_CFG_PATTERN_ENB 0x00000002 #define WOL_CFG_MAGIC 0x00000004 #define WOL_CFG_MAGIC_ENB 0x00000008 #define WOL_CFG_LINK_CHG 0x00000010 #define WOL_CFG_LINK_CHG_ENB 0x00000020 #define WOL_CFG_PATTERN_DET 0x00000100 #define WOL_CFG_MAGIC_DET 0x00000200 #define WOL_CFG_LINK_CHG_DET 0x00000400 #define WOL_CFG_CLK_SWITCH_ENB 0x00008000 #define WOL_CFG_PATTERN0 0x00010000 #define WOL_CFG_PATTERN1 0x00020000 #define WOL_CFG_PATTERN2 0x00040000 #define WOL_CFG_PATTERN3 0x00080000 #define WOL_CFG_PATTERN4 0x00100000 #define WOL_CFG_PATTERN5 0x00200000 #define WOL_CFG_PATTERN6 0x00400000 /* WOL pattern length. */ #define ALC_PATTERN_CFG0 0x14A4 #define PATTERN_CFG_0_LEN_MASK 0x0000007F #define PATTERN_CFG_1_LEN_MASK 0x00007F00 #define PATTERN_CFG_2_LEN_MASK 0x007F0000 #define PATTERN_CFG_3_LEN_MASK 0x7F000000 #define ALC_PATTERN_CFG1 0x14A8 #define PATTERN_CFG_4_LEN_MASK 0x0000007F #define PATTERN_CFG_5_LEN_MASK 0x00007F00 #define PATTERN_CFG_6_LEN_MASK 0x007F0000 /* RSS */ #define ALC_RSS_KEY0 0x14B0 #define ALC_RSS_KEY1 0x14B4 #define ALC_RSS_KEY2 0x14B8 #define ALC_RSS_KEY3 0x14BC #define ALC_RSS_KEY4 0x14C0 #define ALC_RSS_KEY5 0x14C4 #define ALC_RSS_KEY6 0x14C8 #define ALC_RSS_KEY7 0x14CC #define ALC_RSS_KEY8 0x14D0 #define ALC_RSS_KEY9 0x14D4 #define ALC_RSS_IDT_TABLE0 0x14E0 #define ALC_TD_PRI2_HEAD_ADDR_LO 0x14E0 /* AR816x */ #define ALC_RSS_IDT_TABLE1 0x14E4 #define ALC_TD_PRI3_HEAD_ADDR_LO 0x14E4 /* AR816x */ #define ALC_RSS_IDT_TABLE2 0x14E8 #define ALC_RSS_IDT_TABLE3 0x14EC #define ALC_RSS_IDT_TABLE4 0x14F0 #define ALC_RSS_IDT_TABLE5 0x14F4 #define ALC_RSS_IDT_TABLE6 0x14F8 #define ALC_RSS_IDT_TABLE7 0x14FC #define ALC_SRAM_RD0_ADDR 0x1500 #define ALC_SRAM_RD1_ADDR 0x1504 #define ALC_SRAM_RD2_ADDR 0x1508 #define ALC_SRAM_RD3_ADDR 0x150C #define RD_HEAD_ADDR_MASK 0x000003FF #define RD_TAIL_ADDR_MASK 0x03FF0000 #define RD_HEAD_ADDR_SHIFT 0 #define RD_TAIL_ADDR_SHIFT 16 #define ALC_RD_NIC_LEN0 0x1510 /* 8 bytes unit */ #define RD_NIC_LEN_MASK 0x000003FF #define ALC_RD_NIC_LEN1 0x1514 #define ALC_SRAM_TD_ADDR 0x1518 #define TD_HEAD_ADDR_MASK 0x000003FF #define TD_TAIL_ADDR_MASK 0x03FF0000 #define TD_HEAD_ADDR_SHIFT 0 #define TD_TAIL_ADDR_SHIFT 16 #define ALC_SRAM_TD_LEN 0x151C /* 8 bytes unit */ #define SRAM_TD_LEN_MASK 0x000003FF #define ALC_SRAM_RX_FIFO_ADDR 0x1520 #define ALC_SRAM_RX_FIFO_LEN 0x1524 #define SRAM_RX_FIFO_LEN_MASK 0x00000FFF #define SRAM_RX_FIFO_LEN_SHIFT 0 #define ALC_SRAM_TX_FIFO_ADDR 0x1528 #define ALC_SRAM_TX_FIFO_LEN 0x152C #define ALC_SRAM_TCPH_ADDR 0x1530 #define SRAM_TCPH_ADDR_MASK 0x00000FFF #define SRAM_PATH_ADDR_MASK 0x0FFF0000 #define SRAM_TCPH_ADDR_SHIFT 0 #define SRAM_PKTH_ADDR_SHIFT 16 #define ALC_DMA_BLOCK 0x1534 #define DMA_BLOCK_LOAD 0x00000001 #define ALC_RX_BASE_ADDR_HI 0x1540 #define ALC_TX_BASE_ADDR_HI 0x1544 #define ALC_SMB_BASE_ADDR_HI 0x1548 #define ALC_SMB_BASE_ADDR_LO 0x154C #define ALC_RD0_HEAD_ADDR_LO 0x1550 #define ALC_RD1_HEAD_ADDR_LO 0x1554 #define ALC_RD2_HEAD_ADDR_LO 0x1558 #define ALC_RD3_HEAD_ADDR_LO 0x155C #define ALC_RD_RING_CNT 0x1560 #define RD_RING_CNT_MASK 0x00000FFF #define RD_RING_CNT_SHIFT 0 #define ALC_RX_BUF_SIZE 0x1564 #define RX_BUF_SIZE_MASK 0x0000FFFF /* * If larger buffer size than 1536 is specified the controller * will be locked up. This is hardware limitation. */ #define RX_BUF_SIZE_MAX 1536 #define ALC_RRD0_HEAD_ADDR_LO 0x1568 #define ALC_RRD1_HEAD_ADDR_LO 0x156C #define ALC_RRD2_HEAD_ADDR_LO 0x1570 #define ALC_RRD3_HEAD_ADDR_LO 0x1574 #define ALC_RRD_RING_CNT 0x1578 #define RRD_RING_CNT_MASK 0x00000FFF #define RRD_RING_CNT_SHIFT 0 #define ALC_TDH_HEAD_ADDR_LO 0x157C #define ALC_TD_PRI1_HEAD_ADDR_LO 0x157C /* AR816x */ #define ALC_TDL_HEAD_ADDR_LO 0x1580 #define ALC_TD_PRI0_HEAD_ADDR_LO 0x1580 /* AR816x */ #define ALC_TD_RING_CNT 0x1584 #define TD_RING_CNT_MASK 0x0000FFFF #define TD_RING_CNT_SHIFT 0 #define ALC_CMB_BASE_ADDR_LO 0x1588 #define ALC_TXQ_CFG 0x1590 #define TXQ_CFG_TD_BURST_MASK 0x0000000F #define TXQ_CFG_IP_OPTION_ENB 0x00000010 #define TXQ_CFG_ENB 0x00000020 #define TXQ_CFG_ENHANCED_MODE 0x00000040 #define TXQ_CFG_8023_ENB 0x00000080 #define TXQ_CFG_TX_FIFO_BURST_MASK 0xFFFF0000 #define TXQ_CFG_TD_BURST_SHIFT 0 #define TXQ_CFG_TD_BURST_DEFAULT 5 #define TXQ_CFG_TX_FIFO_BURST_SHIFT 16 #define ALC_TSO_OFFLOAD_THRESH 0x1594 /* 8 bytes unit */ #define TSO_OFFLOAD_THRESH_MASK 0x000007FF #define TSO_OFFLOAD_ERRLGPKT_DROP_ENB 0x00000800 #define TSO_OFFLOAD_THRESH_SHIFT 0 #define TSO_OFFLOAD_THRESH_UNIT 8 #define TSO_OFFLOAD_THRESH_UNIT_SHIFT 3 #define ALC_TXF_WATER_MARK 0x1598 /* 8 bytes unit */ #define TXF_WATER_MARK_HI_MASK 0x00000FFF #define TXF_WATER_MARK_LO_MASK 0x0FFF0000 #define TXF_WATER_MARK_BURST_ENB 0x80000000 #define TXF_WATER_MARK_LO_SHIFT 0 #define TXF_WATER_MARK_HI_SHIFT 16 #define ALC_THROUGHPUT_MON 0x159C #define THROUGHPUT_MON_RATE_MASK 0x00000003 #define THROUGHPUT_MON_ENB 0x00000080 #define THROUGHPUT_MON_RATE_SHIFT 0 #define ALC_RXQ_CFG 0x15A0 #define RXQ_CFG_ASPM_THROUGHPUT_LIMIT_MASK 0x00000003 #define RXQ_CFG_ASPM_THROUGHPUT_LIMIT_NONE 0x00000000 #define RXQ_CFG_ASPM_THROUGHPUT_LIMIT_1M 0x00000001 #define RXQ_CFG_ASPM_THROUGHPUT_LIMIT_10M 0x00000002 #define RXQ_CFG_ASPM_THROUGHPUT_LIMIT_100M 0x00000003 #define RXQ_CFG_QUEUE1_ENB 0x00000010 #define RXQ_CFG_QUEUE2_ENB 0x00000020 #define RXQ_CFG_QUEUE3_ENB 0x00000040 #define RXQ_CFG_IPV6_CSUM_ENB 0x00000080 #define RXQ_CFG_RSS_HASH_TBL_LEN_MASK 0x0000FF00 #define RXQ_CFG_RSS_HASH_IPV4 0x00010000 #define RXQ_CFG_RSS_HASH_IPV4_TCP 0x00020000 #define RXQ_CFG_RSS_HASH_IPV6 0x00040000 #define RXQ_CFG_RSS_HASH_IPV6_TCP 0x00080000 #define RXQ_CFG_RD_BURST_MASK 0x03F00000 #define RXQ_CFG_RSS_MODE_DIS 0x00000000 #define RXQ_CFG_RSS_MODE_SQSINT 0x04000000 #define RXQ_CFG_RSS_MODE_MQUESINT 0x08000000 #define RXQ_CFG_RSS_MODE_MQUEMINT 0x0C000000 #define RXQ_CFG_NIP_QUEUE_SEL_TBL 0x10000000 #define RXQ_CFG_RSS_HASH_ENB 0x20000000 #define RXQ_CFG_CUT_THROUGH_ENB 0x40000000 #define RXQ_CFG_QUEUE0_ENB 0x80000000 #define RXQ_CFG_RSS_HASH_TBL_LEN_SHIFT 8 #define RXQ_CFG_RD_BURST_DEFAULT 8 #define RXQ_CFG_RD_BURST_SHIFT 20 #define RXQ_CFG_ENB \ (RXQ_CFG_QUEUE0_ENB | RXQ_CFG_QUEUE1_ENB | \ RXQ_CFG_QUEUE2_ENB | RXQ_CFG_QUEUE3_ENB) /* AR816x specific bits */ #define RXQ_CFG_816X_RSS_HASH_IPV4 0x00000004 #define RXQ_CFG_816X_RSS_HASH_IPV4_TCP 0x00000008 #define RXQ_CFG_816X_RSS_HASH_IPV6 0x00000010 #define RXQ_CFG_816X_RSS_HASH_IPV6_TCP 0x00000020 #define RXQ_CFG_816X_RSS_HASH_MASK 0x0000003C #define RXQ_CFG_816X_IPV6_PARSE_ENB 0x00000080 #define RXQ_CFG_816X_IDT_TBL_SIZE_MASK 0x0001FF00 #define RXQ_CFG_816X_IDT_TBL_SIZE_SHIFT 8 #define RXQ_CFG_816X_IDT_TBL_SIZE_DEFAULT 0x100 #define ALC_RX_RD_FREE_THRESH 0x15A4 /* 8 bytes unit. */ #define RX_RD_FREE_THRESH_HI_MASK 0x0000003F #define RX_RD_FREE_THRESH_LO_MASK 0x00000FC0 #define RX_RD_FREE_THRESH_HI_SHIFT 0 #define RX_RD_FREE_THRESH_LO_SHIFT 6 #define RX_RD_FREE_THRESH_HI_DEFAULT 16 #define RX_RD_FREE_THRESH_LO_DEFAULT 8 #define ALC_RX_FIFO_PAUSE_THRESH 0x15A8 #define RX_FIFO_PAUSE_THRESH_LO_MASK 0x00000FFF #define RX_FIFO_PAUSE_THRESH_HI_MASK 0x0FFF0000 #define RX_FIFO_PAUSE_THRESH_LO_SHIFT 0 #define RX_FIFO_PAUSE_THRESH_HI_SHIFT 16 /* * Size = tx-packet(1522) + IPG(12) + SOF(8) + 64(Pause) + IPG(12) + SOF(8) + * rx-packet(1522) + delay-of-link(64) * = 3212. */ #define RX_FIFO_PAUSE_816X_RSVD 3212 #define ALC_RD_DMA_CFG 0x15AC #define RD_DMA_CFG_THRESH_MASK 0x00000FFF /* 8 bytes unit */ #define RD_DMA_CFG_TIMER_MASK 0xFFFF0000 #define RD_DMA_CFG_THRESH_SHIFT 0 #define RD_DMA_CFG_TIMER_SHIFT 16 #define RD_DMA_CFG_THRESH_DEFAULT 0x100 #define RD_DMA_CFG_TIMER_DEFAULT 0 #define RD_DMA_CFG_TICK_USECS 8 #define ALC_RD_DMA_CFG_USECS(x) ((x) / RD_DMA_CFG_TICK_USECS) #define ALC_RSS_HASH_VALUE 0x15B0 #define ALC_RSS_HASH_FLAG 0x15B4 #define ALC_RSS_CPU 0x15B8 #define ALC_DMA_CFG 0x15C0 #define DMA_CFG_IN_ORDER 0x00000001 #define DMA_CFG_ENH_ORDER 0x00000002 #define DMA_CFG_OUT_ORDER 0x00000004 #define DMA_CFG_RCB_64 0x00000000 #define DMA_CFG_RCB_128 0x00000008 #define DMA_CFG_PEND_AUTO_RST 0x00000008 #define DMA_CFG_RD_BURST_128 0x00000000 #define DMA_CFG_RD_BURST_256 0x00000010 #define DMA_CFG_RD_BURST_512 0x00000020 #define DMA_CFG_RD_BURST_1024 0x00000030 #define DMA_CFG_RD_BURST_2048 0x00000040 #define DMA_CFG_RD_BURST_4096 0x00000050 #define DMA_CFG_WR_BURST_128 0x00000000 #define DMA_CFG_WR_BURST_256 0x00000080 #define DMA_CFG_WR_BURST_512 0x00000100 #define DMA_CFG_WR_BURST_1024 0x00000180 #define DMA_CFG_WR_BURST_2048 0x00000200 #define DMA_CFG_WR_BURST_4096 0x00000280 #define DMA_CFG_RD_REQ_PRI 0x00000400 #define DMA_CFG_RD_DELAY_CNT_MASK 0x0000F800 #define DMA_CFG_WR_DELAY_CNT_MASK 0x000F0000 #define DMA_CFG_CMB_ENB 0x00100000 #define DMA_CFG_SMB_ENB 0x00200000 #define DMA_CFG_CMB_NOW 0x00400000 #define DMA_CFG_SMB_DIS 0x01000000 #define DMA_CFG_RD_CHNL_SEL_MASK 0x0C000000 #define DMA_CFG_RD_CHNL_SEL_1 0x00000000 #define DMA_CFG_RD_CHNL_SEL_2 0x04000000 #define DMA_CFG_RD_CHNL_SEL_3 0x08000000 #define DMA_CFG_RD_CHNL_SEL_4 0x0C000000 #define DMA_CFG_WSRAM_RDCTL 0x10000000 #define DMA_CFG_RD_PEND_CLR 0x20000000 #define DMA_CFG_WR_PEND_CLR 0x40000000 #define DMA_CFG_SMB_NOW 0x80000000 #define DMA_CFG_RD_BURST_MASK 0x07 #define DMA_CFG_RD_BURST_SHIFT 4 #define DMA_CFG_WR_BURST_MASK 0x07 #define DMA_CFG_WR_BURST_SHIFT 7 #define DMA_CFG_RD_DELAY_CNT_SHIFT 11 #define DMA_CFG_WR_DELAY_CNT_SHIFT 16 #define DMA_CFG_RD_DELAY_CNT_DEFAULT 15 #define DMA_CFG_WR_DELAY_CNT_DEFAULT 4 #define ALC_SMB_STAT_TIMER 0x15C4 #define SMB_STAT_TIMER_MASK 0x00FFFFFF #define SMB_STAT_TIMER_SHIFT 0 #define ALC_CMB_TD_THRESH 0x15C8 #define CMB_TD_THRESH_MASK 0x0000FFFF #define CMB_TD_THRESH_SHIFT 0 #define ALC_CMB_TX_TIMER 0x15CC #define CMB_TX_TIMER_MASK 0x0000FFFF #define CMB_TX_TIMER_SHIFT 0 #define ALC_MSI_MAP_TBL1 0x15D0 #define ALC_MSI_ID_MAP 0x15D4 #define ALC_MSI_MAP_TBL2 0x15D8 #define ALC_MBOX_RD0_PROD_IDX 0x15E0 #define ALC_MBOX_RD1_PROD_IDX 0x15E4 #define ALC_MBOX_RD2_PROD_IDX 0x15E8 #define ALC_MBOX_RD3_PROD_IDX 0x15EC #define ALC_MBOX_RD_PROD_MASK 0x0000FFFF #define MBOX_RD_PROD_SHIFT 0 #define ALC_MBOX_TD_PROD_IDX 0x15F0 #define MBOX_TD_PROD_HI_IDX_MASK 0x0000FFFF #define MBOX_TD_PROD_LO_IDX_MASK 0xFFFF0000 #define MBOX_TD_PROD_HI_IDX_SHIFT 0 #define MBOX_TD_PROD_LO_IDX_SHIFT 16 #define ALC_MBOX_TD_PRI1_PROD_IDX 0x15F0 /* 16 bits AR816x */ #define ALC_MBOX_TD_PRI0_PROD_IDX 0x15F2 /* 16 bits AR816x */ #define ALC_MBOX_TD_CONS_IDX 0x15F4 #define MBOX_TD_CONS_HI_IDX_MASK 0x0000FFFF #define MBOX_TD_CONS_LO_IDX_MASK 0xFFFF0000 #define MBOX_TD_CONS_HI_IDX_SHIFT 0 #define MBOX_TD_CONS_LO_IDX_SHIFT 16 #define ALC_MBOX_TD_PRI1_CONS_IDX 0x15F4 /* 16 bits AR816x */ #define ALC_MBOX_TD_PRI0_CONS_IDX 0x15F6 /* 16 bits AR816x */ #define ALC_MBOX_RD01_CONS_IDX 0x15F8 #define MBOX_RD0_CONS_IDX_MASK 0x0000FFFF #define MBOX_RD1_CONS_IDX_MASK 0xFFFF0000 #define MBOX_RD0_CONS_IDX_SHIFT 0 #define MBOX_RD1_CONS_IDX_SHIFT 16 #define ALC_MBOX_RD23_CONS_IDX 0x15FC #define MBOX_RD2_CONS_IDX_MASK 0x0000FFFF #define MBOX_RD3_CONS_IDX_MASK 0xFFFF0000 #define MBOX_RD2_CONS_IDX_SHIFT 0 #define MBOX_RD3_CONS_IDX_SHIFT 16 #define ALC_INTR_STATUS 0x1600 #define INTR_SMB 0x00000001 #define INTR_TIMER 0x00000002 #define INTR_MANUAL_TIMER 0x00000004 #define INTR_RX_FIFO_OFLOW 0x00000008 #define INTR_RD0_UNDERRUN 0x00000010 #define INTR_RD1_UNDERRUN 0x00000020 #define INTR_RD2_UNDERRUN 0x00000040 #define INTR_RD3_UNDERRUN 0x00000080 #define INTR_TX_FIFO_UNDERRUN 0x00000100 #define INTR_DMA_RD_TO_RST 0x00000200 #define INTR_DMA_WR_TO_RST 0x00000400 #define INTR_TX_CREDIT 0x00000800 #define INTR_GPHY 0x00001000 #define INTR_GPHY_LOW_PW 0x00002000 #define INTR_TXQ_TO_RST 0x00004000 #define INTR_TX_PKT0 0x00008000 #define INTR_RX_PKT0 0x00010000 #define INTR_RX_PKT1 0x00020000 #define INTR_RX_PKT2 0x00040000 #define INTR_RX_PKT3 0x00080000 #define INTR_MAC_RX 0x00100000 #define INTR_MAC_TX 0x00200000 #define INTR_UNDERRUN 0x00400000 #define INTR_FRAME_ERROR 0x00800000 #define INTR_FRAME_OK 0x01000000 #define INTR_CSUM_ERROR 0x02000000 #define INTR_PHY_LINK_DOWN 0x04000000 #define INTR_DIS_INT 0x80000000 /* INTR status for AR816x/AR817x 4 TX queues, 8 RX queues */ #define INTR_TX_PKT1 0x00000020 #define INTR_TX_PKT2 0x00000040 #define INTR_TX_PKT3 0x00000080 #define INTR_RX_PKT4 0x08000000 #define INTR_RX_PKT5 0x10000000 #define INTR_RX_PKT6 0x20000000 #define INTR_RX_PKT7 0x40000000 /* Interrupt Mask Register */ #define ALC_INTR_MASK 0x1604 #ifdef notyet #define INTR_RX_PKT \ (INTR_RX_PKT0 | INTR_RX_PKT1 | INTR_RX_PKT2 | \ INTR_RX_PKT3) #define INTR_RD_UNDERRUN \ (INTR_RD0_UNDERRUN | INTR_RD1_UNDERRUN | \ INTR_RD2_UNDERRUN | INTR_RD3_UNDERRUN) #else #define INTR_TX_PKT INTR_TX_PKT0 #define INTR_RX_PKT INTR_RX_PKT0 #define INTR_RD_UNDERRUN INTR_RD0_UNDERRUN #endif #define ALC_INTRS \ (INTR_DMA_RD_TO_RST | INTR_DMA_WR_TO_RST | \ INTR_TXQ_TO_RST | INTR_RX_PKT | INTR_TX_PKT | \ INTR_RX_FIFO_OFLOW | INTR_RD_UNDERRUN | \ INTR_TX_FIFO_UNDERRUN) #define ALC_INTR_RETRIG_TIMER 0x1608 #define INTR_RETRIG_TIMER_MASK 0x0000FFFF #define INTR_RETRIG_TIMER_SHIFT 0 #define ALC_HDS_CFG 0x160C #define HDS_CFG_ENB 0x00000001 #define HDS_CFG_BACKFILLSIZE_MASK 0x000FFF00 #define HDS_CFG_MAX_HDRSIZE_MASK 0xFFF00000 #define HDS_CFG_BACKFILLSIZE_SHIFT 8 #define HDS_CFG_MAX_HDRSIZE_SHIFT 20 #define ALC_MBOX_TD_PRI3_PROD_IDX 0x1618 /* 16 bits AR816x */ #define ALC_MBOX_TD_PRI2_PROD_IDX 0x161A /* 16 bits AR816x */ #define ALC_MBOX_TD_PRI3_CONS_IDX 0x161C /* 16 bits AR816x */ #define ALC_MBOX_TD_PRI2_CONS_IDX 0x161E /* 16 bits AR816x */ /* AR813x/AR815x registers for MAC statistics */ #define ALC_RX_MIB_BASE 0x1700 #define ALC_TX_MIB_BASE 0x1760 #define ALC_DRV 0x1804 /* AR816x */ #define DRV_ASPM_SPD10LMT_1M 0x00000000 #define DRV_ASPM_SPD10LMT_10M 0x00000001 #define DRV_ASPM_SPD10LMT_100M 0x00000002 #define DRV_ASPM_SPD10LMT_NO 0x00000003 #define DRV_ASPM_SPD10LMT_MASK 0x00000003 #define DRV_ASPM_SPD100LMT_1M 0x00000000 #define DRV_ASPM_SPD100LMT_10M 0x00000004 #define DRV_ASPM_SPD100LMT_100M 0x00000008 #define DRV_ASPM_SPD100LMT_NO 0x0000000C #define DRV_ASPM_SPD100LMT_MASK 0x0000000C #define DRV_ASPM_SPD1000LMT_100M 0x00000000 #define DRV_ASPM_SPD1000LMT_NO 0x00000010 #define DRV_ASPM_SPD1000LMT_1M 0x00000020 #define DRV_ASPM_SPD1000LMT_10M 0x00000030 #define DRV_ASPM_SPD1000LMT_MASK 0x00000000 #define DRV_WOLCAP_BIOS_EN 0x00000100 #define DRV_WOLMAGIC_EN 0x00000200 #define DRV_WOLLINKUP_EN 0x00000400 #define DRV_WOLPATTERN_EN 0x00000800 #define DRV_AZ_EN 0x00001000 #define DRV_WOLS5_BIOS_EN 0x00010000 #define DRV_WOLS5_EN 0x00020000 #define DRV_DISABLE 0x00040000 #define DRV_PHY_MASK 0x1FE00000 #define DRV_PHY_EEE 0x00200000 #define DRV_PHY_APAUSE 0x00400000 #define DRV_PHY_PAUSE 0x00800000 #define DRV_PHY_DUPLEX 0x01000000 #define DRV_PHY_10 0x02000000 #define DRV_PHY_100 0x04000000 #define DRV_PHY_1000 0x08000000 #define DRV_PHY_AUTO 0x10000000 #define DRV_PHY_SHIFT 21 #define ALC_CLK_GATING_CFG 0x1814 #define CLK_GATING_DMAW_ENB 0x0001 #define CLK_GATING_DMAR_ENB 0x0002 #define CLK_GATING_TXQ_ENB 0x0004 #define CLK_GATING_RXQ_ENB 0x0008 #define CLK_GATING_TXMAC_ENB 0x0010 #define CLK_GATING_RXMAC_ENB 0x0020 #define ALC_DEBUG_DATA0 0x1900 #define ALC_DEBUG_DATA1 0x1904 #define ALC_MSI_RETRANS_TIMER 0x1920 #define MSI_RETRANS_TIMER_MASK 0x0000FFFF #define MSI_RETRANS_MASK_SEL_STD 0x00000000 #define MSI_RETRANS_MASK_SEL_LINE 0x00010000 #define MSI_RETRANS_TIMER_SHIFT 0 #define ALC_WRR 0x1938 #define WRR_PRI0_MASK 0x0000001F #define WRR_PRI1_MASK 0x00001F00 #define WRR_PRI2_MASK 0x001F0000 #define WRR_PRI3_MASK 0x1F000000 #define WRR_PRI_RESTRICT_MASK 0x60000000 #define WRR_PRI_RESTRICT_ALL 0x00000000 #define WRR_PRI_RESTRICT_HI 0x20000000 #define WRR_PRI_RESTRICT_HI2 0x40000000 #define WRR_PRI_RESTRICT_NONE 0x60000000 #define WRR_PRI0_SHIFT 0 #define WRR_PRI1_SHIFT 8 #define WRR_PRI2_SHIFT 16 #define WRR_PRI3_SHIFT 24 #define WRR_PRI_DEFAULT 4 #define WRR_PRI_RESTRICT_SHIFT 29 #define ALC_HQTD_CFG 0x193C #define HQTD_CFG_Q1_BURST_MASK 0x0000000F #define HQTD_CFG_Q2_BURST_MASK 0x000000F0 #define HQTD_CFG_Q3_BURST_MASK 0x00000F00 #define HQTD_CFG_BURST_ENB 0x80000000 #define HQTD_CFG_Q1_BURST_SHIFT 0 #define HQTD_CFG_Q2_BURST_SHIFT 4 #define HQTD_CFG_Q3_BURST_SHIFT 8 #define ALC_MISC 0x19C0 #define MISC_INTNLOSC_OPEN 0x00000008 #define MISC_ISO_ENB 0x00001000 #define MISC_PSW_OCP_MASK 0x00E00000 #define MISC_PSW_OCP_SHIFT 21 #define MISC_PSW_OCP_DEFAULT 7 #define ALC_MISC2 0x19C8 #define MISC2_CALB_START 0x00000001 #define ALC_MISC3 0x19CC #define MISC3_25M_NOTO_INTNL 0x00000001 #define MISC3_25M_BY_SW 0x00000002 #define ALC_MII_DBG_ADDR 0x1D #define ALC_MII_DBG_DATA 0x1E #define MII_ANA_CFG0 0x00 #define ANA_RESTART_CAL 0x0001 #define ANA_MANUL_SWICH_ON_MASK 0x001E #define ANA_MAN_ENABLE 0x0020 #define ANA_SEL_HSP 0x0040 #define ANA_EN_HB 0x0080 #define ANA_EN_HBIAS 0x0100 #define ANA_OEN_125M 0x0200 #define ANA_EN_LCKDT 0x0400 #define ANA_LCKDT_PHY 0x0800 #define ANA_AFE_MODE 0x1000 #define ANA_VCO_SLOW 0x2000 #define ANA_VCO_FAST 0x4000 #define ANA_SEL_CLK125M_DSP 0x8000 #define ANA_MANUL_SWICH_ON_SHIFT 1 #define MII_DBG_ANACTL 0x00 #define DBG_ANACTL_DEFAULT 0x02EF #define MII_ANA_CFG4 0x04 #define ANA_IECHO_ADJ_MASK 0x0F #define ANA_IECHO_ADJ_3_MASK 0x000F #define ANA_IECHO_ADJ_2_MASK 0x00F0 #define ANA_IECHO_ADJ_1_MASK 0x0F00 #define ANA_IECHO_ADJ_0_MASK 0xF000 #define ANA_IECHO_ADJ_3_SHIFT 0 #define ANA_IECHO_ADJ_2_SHIFT 4 #define ANA_IECHO_ADJ_1_SHIFT 8 #define ANA_IECHO_ADJ_0_SHIFT 12 #define MII_DBG_SYSMODCTL 0x04 #define DBG_SYSMODCTL_DEFAULT 0xBB8B #define MII_ANA_CFG5 0x05 #define ANA_SERDES_CDR_BW_MASK 0x0003 #define ANA_MS_PAD_DBG 0x0004 #define ANA_SPEEDUP_DBG 0x0008 #define ANA_SERDES_TH_LOS_MASK 0x0030 #define ANA_SERDES_EN_DEEM 0x0040 #define ANA_SERDES_TXELECIDLE 0x0080 #define ANA_SERDES_BEACON 0x0100 #define ANA_SERDES_HALFTXDR 0x0200 #define ANA_SERDES_SEL_HSP 0x0400 #define ANA_SERDES_EN_PLL 0x0800 #define ANA_SERDES_EN 0x1000 #define ANA_SERDES_EN_LCKDT 0x2000 #define ANA_SERDES_CDR_BW_SHIFT 0 #define ANA_SERDES_TH_LOS_SHIFT 4 #define MII_DBG_SRDSYSMOD 0x05 #define DBG_SRDSYSMOD_DEFAULT 0x2C46 #define MII_ANA_CFG11 0x0B #define ANA_PS_HIB_EN 0x8000 #define MII_DBG_HIBNEG 0x0B #define DBG_HIBNEG_HIB_PULSE 0x1000 #define DBG_HIBNEG_PSHIB_EN 0x8000 #define DBG_HIBNEG_DEFAULT 0xBC40 #define MII_ANA_CFG18 0x12 #define ANA_TEST_MODE_10BT_01MASK 0x0003 #define ANA_LOOP_SEL_10BT 0x0004 #define ANA_RGMII_MODE_SW 0x0008 #define ANA_EN_LONGECABLE 0x0010 #define ANA_TEST_MODE_10BT_2 0x0020 #define ANA_EN_10BT_IDLE 0x0400 #define ANA_EN_MASK_TB 0x0800 #define ANA_TRIGGER_SEL_TIMER_MASK 0x3000 #define ANA_INTERVAL_SEL_TIMER_MASK 0xC000 #define ANA_TEST_MODE_10BT_01SHIFT 0 #define ANA_TRIGGER_SEL_TIMER_SHIFT 12 #define ANA_INTERVAL_SEL_TIMER_SHIFT 14 #define MII_DBG_TST10BTCFG 0x12 #define DBG_TST10BTCFG_DEFAULT 0x4C04 #define MII_DBG_AZ_ANADECT 0x15 #define DBG_AZ_ANADECT_DEFAULT 0x3220 #define DBG_AZ_ANADECT_LONG 0x3210 #define MII_DBG_MSE16DB 0x18 #define DBG_MSE16DB_UP 0x05EA #define DBG_MSE16DB_DOWN 0x02EA #define MII_DBG_MSE20DB 0x1C #define DBG_MSE20DB_TH_MASK 0x01FC #define DBG_MSE20DB_TH_DEFAULT 0x2E #define DBG_MSE20DB_TH_HI 0x54 #define DBG_MSE20DB_TH_SHIFT 2 #define MII_DBG_AGC 0x23 #define DBG_AGC_2_VGA_MASK 0x3F00 #define DBG_AGC_2_VGA_SHIFT 8 #define DBG_AGC_LONG1G_LIMT 40 #define DBG_AGC_LONG100M_LIMT 44 #define MII_ANA_CFG41 0x29 #define ANA_TOP_PS_EN 0x8000 #define MII_DBG_LEGCYPS 0x29 #define DBG_LEGCYPS_ENB 0x8000 #define DBG_LEGCYPS_DEFAULT 0x129D #define MII_ANA_CFG54 0x36 #define ANA_LONG_CABLE_TH_100_MASK 0x003F #define ANA_DESERVED 0x0040 #define ANA_EN_LIT_CH 0x0080 #define ANA_SHORT_CABLE_TH_100_MASK 0x3F00 #define ANA_BP_BAD_LINK_ACCUM 0x4000 #define ANA_BP_SMALL_BW 0x8000 #define ANA_LONG_CABLE_TH_100_SHIFT 0 #define ANA_SHORT_CABLE_TH_100_SHIFT 8 #define MII_DBG_TST100BTCFG 0x36 #define DBG_TST100BTCFG_DEFAULT 0xE12C #define MII_DBG_GREENCFG 0x3B #define DBG_GREENCFG_DEFAULT 0x7078 #define MII_DBG_GREENCFG2 0x3D #define DBG_GREENCFG2_GATE_DFSE_EN 0x0080 #define DBG_GREENCFG2_BP_GREEN 0x8000 /* Device addr 3 */ #define MII_EXT_PCS 3 #define MII_EXT_CLDCTL3 0x8003 #define EXT_CLDCTL3_BP_CABLE1TH_DET_GT 0x8000 #define MII_EXT_CLDCTL5 0x8005 #define EXT_CLDCTL5_BP_VD_HLFBIAS 0x4000 #define MII_EXT_CLDCTL6 0x8006 #define EXT_CLDCTL6_CAB_LEN_MASK 0x00FF #define EXT_CLDCTL6_CAB_LEN_SHIFT 0 #define EXT_CLDCTL6_CAB_LEN_SHORT1G 116 #define EXT_CLDCTL6_CAB_LEN_SHORT100M 152 #define MII_EXT_VDRVBIAS 0x8062 #define EXT_VDRVBIAS_DEFAULT 3 /* Device addr 7 */ #define MII_EXT_ANEG 7 #define MII_EXT_ANEG_LOCAL_EEEADV 0x3C #define ANEG_LOCA_EEEADV_100BT 0x0002 #define ANEG_LOCA_EEEADV_1000BT 0x0004 #define MII_EXT_ANEG_AFE 0x801A #define ANEG_AFEE_10BT_100M_TH 0x0040 #define MII_EXT_ANEG_S3DIG10 0x8023 #define ANEG_S3DIG10_SL 0x0001 #define ANEG_S3DIG10_DEFAULT 0 #define MII_EXT_ANEG_NLP78 0x8027 #define ANEG_NLP78_120M_DEFAULT 0x8A05 /* Statistics counters collected by the MAC. */ struct smb { /* Rx stats. */ uint32_t rx_frames; uint32_t rx_bcast_frames; uint32_t rx_mcast_frames; uint32_t rx_pause_frames; uint32_t rx_control_frames; uint32_t rx_crcerrs; uint32_t rx_lenerrs; uint32_t rx_bytes; uint32_t rx_runts; uint32_t rx_fragments; uint32_t rx_pkts_64; uint32_t rx_pkts_65_127; uint32_t rx_pkts_128_255; uint32_t rx_pkts_256_511; uint32_t rx_pkts_512_1023; uint32_t rx_pkts_1024_1518; uint32_t rx_pkts_1519_max; uint32_t rx_pkts_truncated; uint32_t rx_fifo_oflows; uint32_t rx_rrs_errs; uint32_t rx_alignerrs; uint32_t rx_bcast_bytes; uint32_t rx_mcast_bytes; uint32_t rx_pkts_filtered; /* Tx stats. */ uint32_t tx_frames; uint32_t tx_bcast_frames; uint32_t tx_mcast_frames; uint32_t tx_pause_frames; uint32_t tx_excess_defer; uint32_t tx_control_frames; uint32_t tx_deferred; uint32_t tx_bytes; uint32_t tx_pkts_64; uint32_t tx_pkts_65_127; uint32_t tx_pkts_128_255; uint32_t tx_pkts_256_511; uint32_t tx_pkts_512_1023; uint32_t tx_pkts_1024_1518; uint32_t tx_pkts_1519_max; uint32_t tx_single_colls; uint32_t tx_multi_colls; uint32_t tx_late_colls; uint32_t tx_excess_colls; uint32_t tx_underrun; uint32_t tx_desc_underrun; uint32_t tx_lenerrs; uint32_t tx_pkts_truncated; uint32_t tx_bcast_bytes; uint32_t tx_mcast_bytes; uint32_t updated; }; /* CMB(Coalesing message block) */ struct cmb { uint32_t cons; }; /* Rx free descriptor */ struct rx_desc { uint64_t addr; }; /* Rx return descriptor */ struct rx_rdesc { uint32_t rdinfo; #define RRD_CSUM_MASK 0x0000FFFF #define RRD_RD_CNT_MASK 0x000F0000 #define RRD_RD_IDX_MASK 0xFFF00000 #define RRD_CSUM_SHIFT 0 #define RRD_RD_CNT_SHIFT 16 #define RRD_RD_IDX_SHIFT 20 #define RRD_CSUM(x) \ (((x) & RRD_CSUM_MASK) >> RRD_CSUM_SHIFT) #define RRD_RD_CNT(x) \ (((x) & RRD_RD_CNT_MASK) >> RRD_RD_CNT_SHIFT) #define RRD_RD_IDX(x) \ (((x) & RRD_RD_IDX_MASK) >> RRD_RD_IDX_SHIFT) uint32_t rss; uint32_t vtag; #define RRD_VLAN_MASK 0x0000FFFF #define RRD_HEAD_LEN_MASK 0x00FF0000 #define RRD_HDS_MASK 0x03000000 #define RRD_HDS_NONE 0x00000000 #define RRD_HDS_HEAD 0x01000000 #define RRD_HDS_DATA 0x02000000 #define RRD_CPU_MASK 0x0C000000 #define RRD_HASH_FLAG_MASK 0xF0000000 #define RRD_VLAN_SHIFT 0 #define RRD_HEAD_LEN_SHIFT 16 #define RRD_HDS_SHIFT 24 #define RRD_CPU_SHIFT 26 #define RRD_HASH_FLAG_SHIFT 28 #define RRD_VLAN(x) \ (((x) & RRD_VLAN_MASK) >> RRD_VLAN_SHIFT) #define RRD_HEAD_LEN(x) \ (((x) & RRD_HEAD_LEN_MASK) >> RRD_HEAD_LEN_SHIFT) #define RRD_CPU(x) \ (((x) & RRD_CPU_MASK) >> RRD_CPU_SHIFT) uint32_t status; #define RRD_LEN_MASK 0x00003FFF #define RRD_LEN_SHIFT 0 #define RRD_TCP_UDPCSUM_NOK 0x00004000 #define RRD_IPCSUM_NOK 0x00008000 #define RRD_VLAN_TAG 0x00010000 #define RRD_PROTO_MASK 0x000E0000 #define RRD_PROTO_IPV4 0x00020000 #define RRD_PROTO_IPV6 0x000C0000 #define RRD_ERR_SUM 0x00100000 #define RRD_ERR_CRC 0x00200000 #define RRD_ERR_ALIGN 0x00400000 #define RRD_ERR_TRUNC 0x00800000 #define RRD_ERR_RUNT 0x01000000 #define RRD_ERR_ICMP 0x02000000 #define RRD_BCAST 0x04000000 #define RRD_MCAST 0x08000000 #define RRD_SNAP_LLC 0x10000000 #define RRD_ETHER 0x00000000 #define RRD_FIFO_FULL 0x20000000 #define RRD_ERR_LENGTH 0x40000000 #define RRD_VALID 0x80000000 #define RRD_BYTES(x) \ (((x) & RRD_LEN_MASK) >> RRD_LEN_SHIFT) #define RRD_IPV4(x) \ (((x) & RRD_PROTO_MASK) == RRD_PROTO_IPV4) }; /* Tx descriptor */ struct tx_desc { uint32_t len; #define TD_BUFLEN_MASK 0x00003FFF #define TD_VLAN_MASK 0xFFFF0000 #define TD_BUFLEN_SHIFT 0 #define TX_BYTES(x) \ (((x) << TD_BUFLEN_SHIFT) & TD_BUFLEN_MASK) #define TD_VLAN_SHIFT 16 uint32_t flags; #define TD_L4HDR_OFFSET_MASK 0x000000FF /* byte unit */ #define TD_TCPHDR_OFFSET_MASK 0x000000FF /* byte unit */ #define TD_PLOAD_OFFSET_MASK 0x000000FF /* 2 bytes unit */ #define TD_CUSTOM_CSUM 0x00000100 #define TD_IPCSUM 0x00000200 #define TD_TCPCSUM 0x00000400 #define TD_UDPCSUM 0x00000800 #define TD_TSO 0x00001000 #define TD_TSO_DESCV1 0x00000000 #define TD_TSO_DESCV2 0x00002000 #define TD_CON_VLAN_TAG 0x00004000 #define TD_INS_VLAN_TAG 0x00008000 #define TD_IPV4_DESCV2 0x00010000 #define TD_LLC_SNAP 0x00020000 #define TD_ETHERNET 0x00000000 #define TD_CUSTOM_CSUM_OFFSET_MASK 0x03FC0000 /* 2 bytes unit */ #define TD_CUSTOM_CSUM_EVEN_PAD 0x40000000 #define TD_MSS_MASK 0x7FFC0000 #define TD_EOP 0x80000000 #define TD_L4HDR_OFFSET_SHIFT 0 #define TD_TCPHDR_OFFSET_SHIFT 0 #define TD_PLOAD_OFFSET_SHIFT 0 #define TD_CUSTOM_CSUM_OFFSET_SHIFT 18 #define TD_MSS_SHIFT 18 uint64_t addr; }; #endif /* _IF_ALCREG_H */ Index: head/sys/dev/pci/pci.c =================================================================== --- head/sys/dev/pci/pci.c (revision 312249) +++ head/sys/dev/pci/pci.c (revision 312250) @@ -1,6107 +1,6108 @@ /*- * Copyright (c) 1997, Stefan Esser * Copyright (c) 2000, Michael Smith * Copyright (c) 2000, BSDi * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice unmodified, this list of conditions, and the following * disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include "opt_bus.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #if defined(__i386__) || defined(__amd64__) || defined(__powerpc__) #include #endif #include #include #include #include #ifdef PCI_IOV #include #include #endif #include #include #include #include #include "pcib_if.h" #include "pci_if.h" #define PCIR_IS_BIOS(cfg, reg) \ (((cfg)->hdrtype == PCIM_HDRTYPE_NORMAL && reg == PCIR_BIOS) || \ ((cfg)->hdrtype == PCIM_HDRTYPE_BRIDGE && reg == PCIR_BIOS_1)) static int pci_has_quirk(uint32_t devid, int quirk); static pci_addr_t pci_mapbase(uint64_t mapreg); static const char *pci_maptype(uint64_t mapreg); static int pci_maprange(uint64_t mapreg); static pci_addr_t pci_rombase(uint64_t mapreg); static int pci_romsize(uint64_t testval); static void pci_fixancient(pcicfgregs *cfg); static int pci_printf(pcicfgregs *cfg, const char *fmt, ...); static int pci_porten(device_t dev); static int pci_memen(device_t dev); static void pci_assign_interrupt(device_t bus, device_t dev, int force_route); static int pci_add_map(device_t bus, device_t dev, int reg, struct resource_list *rl, int force, int prefetch); static int pci_probe(device_t dev); static int pci_attach(device_t dev); static int pci_detach(device_t dev); static void pci_load_vendor_data(void); static int pci_describe_parse_line(char **ptr, int *vendor, int *device, char **desc); static char *pci_describe_device(device_t dev); static int pci_modevent(module_t mod, int what, void *arg); static void pci_hdrtypedata(device_t pcib, int b, int s, int f, pcicfgregs *cfg); static void pci_read_cap(device_t pcib, pcicfgregs *cfg); static int pci_read_vpd_reg(device_t pcib, pcicfgregs *cfg, int reg, uint32_t *data); #if 0 static int pci_write_vpd_reg(device_t pcib, pcicfgregs *cfg, int reg, uint32_t data); #endif static void pci_read_vpd(device_t pcib, pcicfgregs *cfg); static void pci_mask_msix(device_t dev, u_int index); static void pci_unmask_msix(device_t dev, u_int index); static int pci_msi_blacklisted(void); static int pci_msix_blacklisted(void); static void pci_resume_msi(device_t dev); static void pci_resume_msix(device_t dev); static int pci_remap_intr_method(device_t bus, device_t dev, u_int irq); static int pci_get_id_method(device_t dev, device_t child, enum pci_id_type type, uintptr_t *rid); static struct pci_devinfo * pci_fill_devinfo(device_t pcib, device_t bus, int d, int b, int s, int f, uint16_t vid, uint16_t did); static device_method_t pci_methods[] = { /* Device interface */ DEVMETHOD(device_probe, pci_probe), DEVMETHOD(device_attach, pci_attach), DEVMETHOD(device_detach, pci_detach), DEVMETHOD(device_shutdown, bus_generic_shutdown), DEVMETHOD(device_suspend, bus_generic_suspend), DEVMETHOD(device_resume, pci_resume), /* Bus interface */ DEVMETHOD(bus_print_child, pci_print_child), DEVMETHOD(bus_probe_nomatch, pci_probe_nomatch), DEVMETHOD(bus_read_ivar, pci_read_ivar), DEVMETHOD(bus_write_ivar, pci_write_ivar), DEVMETHOD(bus_driver_added, pci_driver_added), DEVMETHOD(bus_setup_intr, pci_setup_intr), DEVMETHOD(bus_teardown_intr, pci_teardown_intr), DEVMETHOD(bus_get_dma_tag, pci_get_dma_tag), DEVMETHOD(bus_get_resource_list,pci_get_resource_list), DEVMETHOD(bus_set_resource, bus_generic_rl_set_resource), DEVMETHOD(bus_get_resource, bus_generic_rl_get_resource), DEVMETHOD(bus_delete_resource, pci_delete_resource), DEVMETHOD(bus_alloc_resource, pci_alloc_resource), DEVMETHOD(bus_adjust_resource, bus_generic_adjust_resource), DEVMETHOD(bus_release_resource, pci_release_resource), DEVMETHOD(bus_activate_resource, pci_activate_resource), DEVMETHOD(bus_deactivate_resource, pci_deactivate_resource), DEVMETHOD(bus_child_deleted, pci_child_deleted), DEVMETHOD(bus_child_detached, pci_child_detached), DEVMETHOD(bus_child_pnpinfo_str, pci_child_pnpinfo_str_method), DEVMETHOD(bus_child_location_str, pci_child_location_str_method), DEVMETHOD(bus_remap_intr, pci_remap_intr_method), DEVMETHOD(bus_suspend_child, pci_suspend_child), DEVMETHOD(bus_resume_child, pci_resume_child), DEVMETHOD(bus_rescan, pci_rescan_method), /* PCI interface */ DEVMETHOD(pci_read_config, pci_read_config_method), DEVMETHOD(pci_write_config, pci_write_config_method), DEVMETHOD(pci_enable_busmaster, pci_enable_busmaster_method), DEVMETHOD(pci_disable_busmaster, pci_disable_busmaster_method), DEVMETHOD(pci_enable_io, pci_enable_io_method), DEVMETHOD(pci_disable_io, pci_disable_io_method), DEVMETHOD(pci_get_vpd_ident, pci_get_vpd_ident_method), DEVMETHOD(pci_get_vpd_readonly, pci_get_vpd_readonly_method), DEVMETHOD(pci_get_powerstate, pci_get_powerstate_method), DEVMETHOD(pci_set_powerstate, pci_set_powerstate_method), DEVMETHOD(pci_assign_interrupt, pci_assign_interrupt_method), DEVMETHOD(pci_find_cap, pci_find_cap_method), DEVMETHOD(pci_find_extcap, pci_find_extcap_method), DEVMETHOD(pci_find_htcap, pci_find_htcap_method), DEVMETHOD(pci_alloc_msi, pci_alloc_msi_method), DEVMETHOD(pci_alloc_msix, pci_alloc_msix_method), DEVMETHOD(pci_enable_msi, pci_enable_msi_method), DEVMETHOD(pci_enable_msix, pci_enable_msix_method), DEVMETHOD(pci_disable_msi, pci_disable_msi_method), DEVMETHOD(pci_remap_msix, pci_remap_msix_method), DEVMETHOD(pci_release_msi, pci_release_msi_method), DEVMETHOD(pci_msi_count, pci_msi_count_method), DEVMETHOD(pci_msix_count, pci_msix_count_method), DEVMETHOD(pci_msix_pba_bar, pci_msix_pba_bar_method), DEVMETHOD(pci_msix_table_bar, pci_msix_table_bar_method), DEVMETHOD(pci_get_id, pci_get_id_method), DEVMETHOD(pci_alloc_devinfo, pci_alloc_devinfo_method), DEVMETHOD(pci_child_added, pci_child_added_method), #ifdef PCI_IOV DEVMETHOD(pci_iov_attach, pci_iov_attach_method), DEVMETHOD(pci_iov_detach, pci_iov_detach_method), DEVMETHOD(pci_create_iov_child, pci_create_iov_child_method), #endif DEVMETHOD_END }; DEFINE_CLASS_0(pci, pci_driver, pci_methods, sizeof(struct pci_softc)); static devclass_t pci_devclass; DRIVER_MODULE(pci, pcib, pci_driver, pci_devclass, pci_modevent, NULL); MODULE_VERSION(pci, 1); static char *pci_vendordata; static size_t pci_vendordata_size; struct pci_quirk { uint32_t devid; /* Vendor/device of the card */ int type; #define PCI_QUIRK_MAP_REG 1 /* PCI map register in weird place */ #define PCI_QUIRK_DISABLE_MSI 2 /* Neither MSI nor MSI-X work */ #define PCI_QUIRK_ENABLE_MSI_VM 3 /* Older chipset in VM where MSI works */ #define PCI_QUIRK_UNMAP_REG 4 /* Ignore PCI map register */ #define PCI_QUIRK_DISABLE_MSIX 5 /* MSI-X doesn't work */ #define PCI_QUIRK_MSI_INTX_BUG 6 /* PCIM_CMD_INTxDIS disables MSI */ int arg1; int arg2; }; static const struct pci_quirk pci_quirks[] = { /* The Intel 82371AB and 82443MX have a map register at offset 0x90. */ { 0x71138086, PCI_QUIRK_MAP_REG, 0x90, 0 }, { 0x719b8086, PCI_QUIRK_MAP_REG, 0x90, 0 }, /* As does the Serverworks OSB4 (the SMBus mapping register) */ { 0x02001166, PCI_QUIRK_MAP_REG, 0x90, 0 }, /* * MSI doesn't work with the ServerWorks CNB20-HE Host Bridge * or the CMIC-SL (AKA ServerWorks GC_LE). */ { 0x00141166, PCI_QUIRK_DISABLE_MSI, 0, 0 }, { 0x00171166, PCI_QUIRK_DISABLE_MSI, 0, 0 }, /* * MSI doesn't work on earlier Intel chipsets including * E7500, E7501, E7505, 845, 865, 875/E7210, and 855. */ { 0x25408086, PCI_QUIRK_DISABLE_MSI, 0, 0 }, { 0x254c8086, PCI_QUIRK_DISABLE_MSI, 0, 0 }, { 0x25508086, PCI_QUIRK_DISABLE_MSI, 0, 0 }, { 0x25608086, PCI_QUIRK_DISABLE_MSI, 0, 0 }, { 0x25708086, PCI_QUIRK_DISABLE_MSI, 0, 0 }, { 0x25788086, PCI_QUIRK_DISABLE_MSI, 0, 0 }, { 0x35808086, PCI_QUIRK_DISABLE_MSI, 0, 0 }, /* * MSI doesn't work with devices behind the AMD 8131 HT-PCIX * bridge. */ { 0x74501022, PCI_QUIRK_DISABLE_MSI, 0, 0 }, /* * MSI-X allocation doesn't work properly for devices passed through * by VMware up to at least ESXi 5.1. */ { 0x079015ad, PCI_QUIRK_DISABLE_MSIX, 0, 0 }, /* PCI/PCI-X */ { 0x07a015ad, PCI_QUIRK_DISABLE_MSIX, 0, 0 }, /* PCIe */ /* * Some virtualization environments emulate an older chipset * but support MSI just fine. QEMU uses the Intel 82440. */ { 0x12378086, PCI_QUIRK_ENABLE_MSI_VM, 0, 0 }, /* * HPET MMIO base address may appear in Bar1 for AMD SB600 SMBus * controller depending on SoftPciRst register (PM_IO 0x55 [7]). * It prevents us from attaching hpet(4) when the bit is unset. * Note this quirk only affects SB600 revision A13 and earlier. * For SB600 A21 and later, firmware must set the bit to hide it. * For SB700 and later, it is unused and hardcoded to zero. */ { 0x43851002, PCI_QUIRK_UNMAP_REG, 0x14, 0 }, /* - * Atheros AR8161/AR8162/E2200/E2400 Ethernet controllers have a - * bug that MSI interrupt does not assert if PCIM_CMD_INTxDIS bit + * Atheros AR8161/AR8162/E2200/E2400/E2500 Ethernet controllers have + * a bug that MSI interrupt does not assert if PCIM_CMD_INTxDIS bit * of the command register is set. */ { 0x10911969, PCI_QUIRK_MSI_INTX_BUG, 0, 0 }, { 0xE0911969, PCI_QUIRK_MSI_INTX_BUG, 0, 0 }, { 0xE0A11969, PCI_QUIRK_MSI_INTX_BUG, 0, 0 }, + { 0xE0B11969, PCI_QUIRK_MSI_INTX_BUG, 0, 0 }, { 0x10901969, PCI_QUIRK_MSI_INTX_BUG, 0, 0 }, /* * Broadcom BCM5714(S)/BCM5715(S)/BCM5780(S) Ethernet MACs don't * issue MSI interrupts with PCIM_CMD_INTxDIS set either. */ { 0x166814e4, PCI_QUIRK_MSI_INTX_BUG, 0, 0 }, /* BCM5714 */ { 0x166914e4, PCI_QUIRK_MSI_INTX_BUG, 0, 0 }, /* BCM5714S */ { 0x166a14e4, PCI_QUIRK_MSI_INTX_BUG, 0, 0 }, /* BCM5780 */ { 0x166b14e4, PCI_QUIRK_MSI_INTX_BUG, 0, 0 }, /* BCM5780S */ { 0x167814e4, PCI_QUIRK_MSI_INTX_BUG, 0, 0 }, /* BCM5715 */ { 0x167914e4, PCI_QUIRK_MSI_INTX_BUG, 0, 0 }, /* BCM5715S */ { 0 } }; /* map register information */ #define PCI_MAPMEM 0x01 /* memory map */ #define PCI_MAPMEMP 0x02 /* prefetchable memory map */ #define PCI_MAPPORT 0x04 /* port map */ struct devlist pci_devq; uint32_t pci_generation; uint32_t pci_numdevs = 0; static int pcie_chipset, pcix_chipset; /* sysctl vars */ SYSCTL_NODE(_hw, OID_AUTO, pci, CTLFLAG_RD, 0, "PCI bus tuning parameters"); static int pci_enable_io_modes = 1; SYSCTL_INT(_hw_pci, OID_AUTO, enable_io_modes, CTLFLAG_RWTUN, &pci_enable_io_modes, 1, "Enable I/O and memory bits in the config register. Some BIOSes do not\n\ enable these bits correctly. We'd like to do this all the time, but there\n\ are some peripherals that this causes problems with."); static int pci_do_realloc_bars = 0; SYSCTL_INT(_hw_pci, OID_AUTO, realloc_bars, CTLFLAG_RWTUN, &pci_do_realloc_bars, 0, "Attempt to allocate a new range for any BARs whose original " "firmware-assigned ranges fail to allocate during the initial device scan."); static int pci_do_power_nodriver = 0; SYSCTL_INT(_hw_pci, OID_AUTO, do_power_nodriver, CTLFLAG_RWTUN, &pci_do_power_nodriver, 0, "Place a function into D3 state when no driver attaches to it. 0 means\n\ disable. 1 means conservatively place devices into D3 state. 2 means\n\ aggressively place devices into D3 state. 3 means put absolutely everything\n\ in D3 state."); int pci_do_power_resume = 1; SYSCTL_INT(_hw_pci, OID_AUTO, do_power_resume, CTLFLAG_RWTUN, &pci_do_power_resume, 1, "Transition from D3 -> D0 on resume."); int pci_do_power_suspend = 1; SYSCTL_INT(_hw_pci, OID_AUTO, do_power_suspend, CTLFLAG_RWTUN, &pci_do_power_suspend, 1, "Transition from D0 -> D3 on suspend."); static int pci_do_msi = 1; SYSCTL_INT(_hw_pci, OID_AUTO, enable_msi, CTLFLAG_RWTUN, &pci_do_msi, 1, "Enable support for MSI interrupts"); static int pci_do_msix = 1; SYSCTL_INT(_hw_pci, OID_AUTO, enable_msix, CTLFLAG_RWTUN, &pci_do_msix, 1, "Enable support for MSI-X interrupts"); static int pci_msix_rewrite_table = 0; SYSCTL_INT(_hw_pci, OID_AUTO, msix_rewrite_table, CTLFLAG_RWTUN, &pci_msix_rewrite_table, 0, "Rewrite entire MSI-X table when updating MSI-X entries"); static int pci_honor_msi_blacklist = 1; SYSCTL_INT(_hw_pci, OID_AUTO, honor_msi_blacklist, CTLFLAG_RDTUN, &pci_honor_msi_blacklist, 1, "Honor chipset blacklist for MSI/MSI-X"); #if defined(__i386__) || defined(__amd64__) static int pci_usb_takeover = 1; #else static int pci_usb_takeover = 0; #endif SYSCTL_INT(_hw_pci, OID_AUTO, usb_early_takeover, CTLFLAG_RDTUN, &pci_usb_takeover, 1, "Enable early takeover of USB controllers.\n\ Disable this if you depend on BIOS emulation of USB devices, that is\n\ you use USB devices (like keyboard or mouse) but do not load USB drivers"); static int pci_clear_bars; SYSCTL_INT(_hw_pci, OID_AUTO, clear_bars, CTLFLAG_RDTUN, &pci_clear_bars, 0, "Ignore firmware-assigned resources for BARs."); #if defined(NEW_PCIB) && defined(PCI_RES_BUS) static int pci_clear_buses; SYSCTL_INT(_hw_pci, OID_AUTO, clear_buses, CTLFLAG_RDTUN, &pci_clear_buses, 0, "Ignore firmware-assigned bus numbers."); #endif static int pci_enable_ari = 1; SYSCTL_INT(_hw_pci, OID_AUTO, enable_ari, CTLFLAG_RDTUN, &pci_enable_ari, 0, "Enable support for PCIe Alternative RID Interpretation"); static int pci_has_quirk(uint32_t devid, int quirk) { const struct pci_quirk *q; for (q = &pci_quirks[0]; q->devid; q++) { if (q->devid == devid && q->type == quirk) return (1); } return (0); } /* Find a device_t by bus/slot/function in domain 0 */ device_t pci_find_bsf(uint8_t bus, uint8_t slot, uint8_t func) { return (pci_find_dbsf(0, bus, slot, func)); } /* Find a device_t by domain/bus/slot/function */ device_t pci_find_dbsf(uint32_t domain, uint8_t bus, uint8_t slot, uint8_t func) { struct pci_devinfo *dinfo; STAILQ_FOREACH(dinfo, &pci_devq, pci_links) { if ((dinfo->cfg.domain == domain) && (dinfo->cfg.bus == bus) && (dinfo->cfg.slot == slot) && (dinfo->cfg.func == func)) { return (dinfo->cfg.dev); } } return (NULL); } /* Find a device_t by vendor/device ID */ device_t pci_find_device(uint16_t vendor, uint16_t device) { struct pci_devinfo *dinfo; STAILQ_FOREACH(dinfo, &pci_devq, pci_links) { if ((dinfo->cfg.vendor == vendor) && (dinfo->cfg.device == device)) { return (dinfo->cfg.dev); } } return (NULL); } device_t pci_find_class(uint8_t class, uint8_t subclass) { struct pci_devinfo *dinfo; STAILQ_FOREACH(dinfo, &pci_devq, pci_links) { if (dinfo->cfg.baseclass == class && dinfo->cfg.subclass == subclass) { return (dinfo->cfg.dev); } } return (NULL); } static int pci_printf(pcicfgregs *cfg, const char *fmt, ...) { va_list ap; int retval; retval = printf("pci%d:%d:%d:%d: ", cfg->domain, cfg->bus, cfg->slot, cfg->func); va_start(ap, fmt); retval += vprintf(fmt, ap); va_end(ap); return (retval); } /* return base address of memory or port map */ static pci_addr_t pci_mapbase(uint64_t mapreg) { if (PCI_BAR_MEM(mapreg)) return (mapreg & PCIM_BAR_MEM_BASE); else return (mapreg & PCIM_BAR_IO_BASE); } /* return map type of memory or port map */ static const char * pci_maptype(uint64_t mapreg) { if (PCI_BAR_IO(mapreg)) return ("I/O Port"); if (mapreg & PCIM_BAR_MEM_PREFETCH) return ("Prefetchable Memory"); return ("Memory"); } /* return log2 of map size decoded for memory or port map */ int pci_mapsize(uint64_t testval) { int ln2size; testval = pci_mapbase(testval); ln2size = 0; if (testval != 0) { while ((testval & 1) == 0) { ln2size++; testval >>= 1; } } return (ln2size); } /* return base address of device ROM */ static pci_addr_t pci_rombase(uint64_t mapreg) { return (mapreg & PCIM_BIOS_ADDR_MASK); } /* return log2 of map size decided for device ROM */ static int pci_romsize(uint64_t testval) { int ln2size; testval = pci_rombase(testval); ln2size = 0; if (testval != 0) { while ((testval & 1) == 0) { ln2size++; testval >>= 1; } } return (ln2size); } /* return log2 of address range supported by map register */ static int pci_maprange(uint64_t mapreg) { int ln2range = 0; if (PCI_BAR_IO(mapreg)) ln2range = 32; else switch (mapreg & PCIM_BAR_MEM_TYPE) { case PCIM_BAR_MEM_32: ln2range = 32; break; case PCIM_BAR_MEM_1MB: ln2range = 20; break; case PCIM_BAR_MEM_64: ln2range = 64; break; } return (ln2range); } /* adjust some values from PCI 1.0 devices to match 2.0 standards ... */ static void pci_fixancient(pcicfgregs *cfg) { if ((cfg->hdrtype & PCIM_HDRTYPE) != PCIM_HDRTYPE_NORMAL) return; /* PCI to PCI bridges use header type 1 */ if (cfg->baseclass == PCIC_BRIDGE && cfg->subclass == PCIS_BRIDGE_PCI) cfg->hdrtype = PCIM_HDRTYPE_BRIDGE; } /* extract header type specific config data */ static void pci_hdrtypedata(device_t pcib, int b, int s, int f, pcicfgregs *cfg) { #define REG(n, w) PCIB_READ_CONFIG(pcib, b, s, f, n, w) switch (cfg->hdrtype & PCIM_HDRTYPE) { case PCIM_HDRTYPE_NORMAL: cfg->subvendor = REG(PCIR_SUBVEND_0, 2); cfg->subdevice = REG(PCIR_SUBDEV_0, 2); cfg->mingnt = REG(PCIR_MINGNT, 1); cfg->maxlat = REG(PCIR_MAXLAT, 1); cfg->nummaps = PCI_MAXMAPS_0; break; case PCIM_HDRTYPE_BRIDGE: cfg->bridge.br_seclat = REG(PCIR_SECLAT_1, 1); cfg->bridge.br_subbus = REG(PCIR_SUBBUS_1, 1); cfg->bridge.br_secbus = REG(PCIR_SECBUS_1, 1); cfg->bridge.br_pribus = REG(PCIR_PRIBUS_1, 1); cfg->bridge.br_control = REG(PCIR_BRIDGECTL_1, 2); cfg->nummaps = PCI_MAXMAPS_1; break; case PCIM_HDRTYPE_CARDBUS: cfg->bridge.br_seclat = REG(PCIR_SECLAT_2, 1); cfg->bridge.br_subbus = REG(PCIR_SUBBUS_2, 1); cfg->bridge.br_secbus = REG(PCIR_SECBUS_2, 1); cfg->bridge.br_pribus = REG(PCIR_PRIBUS_2, 1); cfg->bridge.br_control = REG(PCIR_BRIDGECTL_2, 2); cfg->subvendor = REG(PCIR_SUBVEND_2, 2); cfg->subdevice = REG(PCIR_SUBDEV_2, 2); cfg->nummaps = PCI_MAXMAPS_2; break; } #undef REG } /* read configuration header into pcicfgregs structure */ struct pci_devinfo * pci_read_device(device_t pcib, device_t bus, int d, int b, int s, int f) { #define REG(n, w) PCIB_READ_CONFIG(pcib, b, s, f, n, w) uint16_t vid, did; vid = REG(PCIR_VENDOR, 2); did = REG(PCIR_DEVICE, 2); if (vid != 0xffff) return (pci_fill_devinfo(pcib, bus, d, b, s, f, vid, did)); return (NULL); } struct pci_devinfo * pci_alloc_devinfo_method(device_t dev) { return (malloc(sizeof(struct pci_devinfo), M_DEVBUF, M_WAITOK | M_ZERO)); } static struct pci_devinfo * pci_fill_devinfo(device_t pcib, device_t bus, int d, int b, int s, int f, uint16_t vid, uint16_t did) { struct pci_devinfo *devlist_entry; pcicfgregs *cfg; devlist_entry = PCI_ALLOC_DEVINFO(bus); cfg = &devlist_entry->cfg; cfg->domain = d; cfg->bus = b; cfg->slot = s; cfg->func = f; cfg->vendor = vid; cfg->device = did; cfg->cmdreg = REG(PCIR_COMMAND, 2); cfg->statreg = REG(PCIR_STATUS, 2); cfg->baseclass = REG(PCIR_CLASS, 1); cfg->subclass = REG(PCIR_SUBCLASS, 1); cfg->progif = REG(PCIR_PROGIF, 1); cfg->revid = REG(PCIR_REVID, 1); cfg->hdrtype = REG(PCIR_HDRTYPE, 1); cfg->cachelnsz = REG(PCIR_CACHELNSZ, 1); cfg->lattimer = REG(PCIR_LATTIMER, 1); cfg->intpin = REG(PCIR_INTPIN, 1); cfg->intline = REG(PCIR_INTLINE, 1); cfg->mfdev = (cfg->hdrtype & PCIM_MFDEV) != 0; cfg->hdrtype &= ~PCIM_MFDEV; STAILQ_INIT(&cfg->maps); cfg->iov = NULL; pci_fixancient(cfg); pci_hdrtypedata(pcib, b, s, f, cfg); if (REG(PCIR_STATUS, 2) & PCIM_STATUS_CAPPRESENT) pci_read_cap(pcib, cfg); STAILQ_INSERT_TAIL(&pci_devq, devlist_entry, pci_links); devlist_entry->conf.pc_sel.pc_domain = cfg->domain; devlist_entry->conf.pc_sel.pc_bus = cfg->bus; devlist_entry->conf.pc_sel.pc_dev = cfg->slot; devlist_entry->conf.pc_sel.pc_func = cfg->func; devlist_entry->conf.pc_hdr = cfg->hdrtype; devlist_entry->conf.pc_subvendor = cfg->subvendor; devlist_entry->conf.pc_subdevice = cfg->subdevice; devlist_entry->conf.pc_vendor = cfg->vendor; devlist_entry->conf.pc_device = cfg->device; devlist_entry->conf.pc_class = cfg->baseclass; devlist_entry->conf.pc_subclass = cfg->subclass; devlist_entry->conf.pc_progif = cfg->progif; devlist_entry->conf.pc_revid = cfg->revid; pci_numdevs++; pci_generation++; return (devlist_entry); } #undef REG static void pci_ea_fill_info(device_t pcib, pcicfgregs *cfg) { #define REG(n, w) PCIB_READ_CONFIG(pcib, cfg->bus, cfg->slot, cfg->func, \ cfg->ea.ea_location + (n), w) int num_ent; int ptr; int a, b; uint32_t val; int ent_size; uint32_t dw[4]; uint64_t base, max_offset; struct pci_ea_entry *eae; if (cfg->ea.ea_location == 0) return; STAILQ_INIT(&cfg->ea.ea_entries); /* Determine the number of entries */ num_ent = REG(PCIR_EA_NUM_ENT, 2); num_ent &= PCIM_EA_NUM_ENT_MASK; /* Find the first entry to care of */ ptr = PCIR_EA_FIRST_ENT; /* Skip DWORD 2 for type 1 functions */ if ((cfg->hdrtype & PCIM_HDRTYPE) == PCIM_HDRTYPE_BRIDGE) ptr += 4; for (a = 0; a < num_ent; a++) { eae = malloc(sizeof(*eae), M_DEVBUF, M_WAITOK | M_ZERO); eae->eae_cfg_offset = cfg->ea.ea_location + ptr; /* Read a number of dwords in the entry */ val = REG(ptr, 4); ptr += 4; ent_size = (val & PCIM_EA_ES); for (b = 0; b < ent_size; b++) { dw[b] = REG(ptr, 4); ptr += 4; } eae->eae_flags = val; eae->eae_bei = (PCIM_EA_BEI & val) >> PCIM_EA_BEI_OFFSET; base = dw[0] & PCIM_EA_FIELD_MASK; max_offset = dw[1] | ~PCIM_EA_FIELD_MASK; b = 2; if (((dw[0] & PCIM_EA_IS_64) != 0) && (b < ent_size)) { base |= (uint64_t)dw[b] << 32UL; b++; } if (((dw[1] & PCIM_EA_IS_64) != 0) && (b < ent_size)) { max_offset |= (uint64_t)dw[b] << 32UL; b++; } eae->eae_base = base; eae->eae_max_offset = max_offset; STAILQ_INSERT_TAIL(&cfg->ea.ea_entries, eae, eae_link); if (bootverbose) { printf("PCI(EA) dev %04x:%04x, bei %d, flags #%x, base #%jx, max_offset #%jx\n", cfg->vendor, cfg->device, eae->eae_bei, eae->eae_flags, (uintmax_t)eae->eae_base, (uintmax_t)eae->eae_max_offset); } } } #undef REG static void pci_read_cap(device_t pcib, pcicfgregs *cfg) { #define REG(n, w) PCIB_READ_CONFIG(pcib, cfg->bus, cfg->slot, cfg->func, n, w) #define WREG(n, v, w) PCIB_WRITE_CONFIG(pcib, cfg->bus, cfg->slot, cfg->func, n, v, w) #if defined(__i386__) || defined(__amd64__) || defined(__powerpc__) uint64_t addr; #endif uint32_t val; int ptr, nextptr, ptrptr; switch (cfg->hdrtype & PCIM_HDRTYPE) { case PCIM_HDRTYPE_NORMAL: case PCIM_HDRTYPE_BRIDGE: ptrptr = PCIR_CAP_PTR; break; case PCIM_HDRTYPE_CARDBUS: ptrptr = PCIR_CAP_PTR_2; /* cardbus capabilities ptr */ break; default: return; /* no extended capabilities support */ } nextptr = REG(ptrptr, 1); /* sanity check? */ /* * Read capability entries. */ while (nextptr != 0) { /* Sanity check */ if (nextptr > 255) { printf("illegal PCI extended capability offset %d\n", nextptr); return; } /* Find the next entry */ ptr = nextptr; nextptr = REG(ptr + PCICAP_NEXTPTR, 1); /* Process this entry */ switch (REG(ptr + PCICAP_ID, 1)) { case PCIY_PMG: /* PCI power management */ if (cfg->pp.pp_cap == 0) { cfg->pp.pp_cap = REG(ptr + PCIR_POWER_CAP, 2); cfg->pp.pp_status = ptr + PCIR_POWER_STATUS; cfg->pp.pp_bse = ptr + PCIR_POWER_BSE; if ((nextptr - ptr) > PCIR_POWER_DATA) cfg->pp.pp_data = ptr + PCIR_POWER_DATA; } break; case PCIY_HT: /* HyperTransport */ /* Determine HT-specific capability type. */ val = REG(ptr + PCIR_HT_COMMAND, 2); if ((val & 0xe000) == PCIM_HTCAP_SLAVE) cfg->ht.ht_slave = ptr; #if defined(__i386__) || defined(__amd64__) || defined(__powerpc__) switch (val & PCIM_HTCMD_CAP_MASK) { case PCIM_HTCAP_MSI_MAPPING: if (!(val & PCIM_HTCMD_MSI_FIXED)) { /* Sanity check the mapping window. */ addr = REG(ptr + PCIR_HTMSI_ADDRESS_HI, 4); addr <<= 32; addr |= REG(ptr + PCIR_HTMSI_ADDRESS_LO, 4); if (addr != MSI_INTEL_ADDR_BASE) device_printf(pcib, "HT device at pci%d:%d:%d:%d has non-default MSI window 0x%llx\n", cfg->domain, cfg->bus, cfg->slot, cfg->func, (long long)addr); } else addr = MSI_INTEL_ADDR_BASE; cfg->ht.ht_msimap = ptr; cfg->ht.ht_msictrl = val; cfg->ht.ht_msiaddr = addr; break; } #endif break; case PCIY_MSI: /* PCI MSI */ cfg->msi.msi_location = ptr; cfg->msi.msi_ctrl = REG(ptr + PCIR_MSI_CTRL, 2); cfg->msi.msi_msgnum = 1 << ((cfg->msi.msi_ctrl & PCIM_MSICTRL_MMC_MASK)>>1); break; case PCIY_MSIX: /* PCI MSI-X */ cfg->msix.msix_location = ptr; cfg->msix.msix_ctrl = REG(ptr + PCIR_MSIX_CTRL, 2); cfg->msix.msix_msgnum = (cfg->msix.msix_ctrl & PCIM_MSIXCTRL_TABLE_SIZE) + 1; val = REG(ptr + PCIR_MSIX_TABLE, 4); cfg->msix.msix_table_bar = PCIR_BAR(val & PCIM_MSIX_BIR_MASK); cfg->msix.msix_table_offset = val & ~PCIM_MSIX_BIR_MASK; val = REG(ptr + PCIR_MSIX_PBA, 4); cfg->msix.msix_pba_bar = PCIR_BAR(val & PCIM_MSIX_BIR_MASK); cfg->msix.msix_pba_offset = val & ~PCIM_MSIX_BIR_MASK; break; case PCIY_VPD: /* PCI Vital Product Data */ cfg->vpd.vpd_reg = ptr; break; case PCIY_SUBVENDOR: /* Should always be true. */ if ((cfg->hdrtype & PCIM_HDRTYPE) == PCIM_HDRTYPE_BRIDGE) { val = REG(ptr + PCIR_SUBVENDCAP_ID, 4); cfg->subvendor = val & 0xffff; cfg->subdevice = val >> 16; } break; case PCIY_PCIX: /* PCI-X */ /* * Assume we have a PCI-X chipset if we have * at least one PCI-PCI bridge with a PCI-X * capability. Note that some systems with * PCI-express or HT chipsets might match on * this check as well. */ if ((cfg->hdrtype & PCIM_HDRTYPE) == PCIM_HDRTYPE_BRIDGE) pcix_chipset = 1; cfg->pcix.pcix_location = ptr; break; case PCIY_EXPRESS: /* PCI-express */ /* * Assume we have a PCI-express chipset if we have * at least one PCI-express device. */ pcie_chipset = 1; cfg->pcie.pcie_location = ptr; val = REG(ptr + PCIER_FLAGS, 2); cfg->pcie.pcie_type = val & PCIEM_FLAGS_TYPE; break; case PCIY_EA: /* Enhanced Allocation */ cfg->ea.ea_location = ptr; pci_ea_fill_info(pcib, cfg); break; default: break; } } #if defined(__powerpc__) /* * Enable the MSI mapping window for all HyperTransport * slaves. PCI-PCI bridges have their windows enabled via * PCIB_MAP_MSI(). */ if (cfg->ht.ht_slave != 0 && cfg->ht.ht_msimap != 0 && !(cfg->ht.ht_msictrl & PCIM_HTCMD_MSI_ENABLE)) { device_printf(pcib, "Enabling MSI window for HyperTransport slave at pci%d:%d:%d:%d\n", cfg->domain, cfg->bus, cfg->slot, cfg->func); cfg->ht.ht_msictrl |= PCIM_HTCMD_MSI_ENABLE; WREG(cfg->ht.ht_msimap + PCIR_HT_COMMAND, cfg->ht.ht_msictrl, 2); } #endif /* REG and WREG use carry through to next functions */ } /* * PCI Vital Product Data */ #define PCI_VPD_TIMEOUT 1000000 static int pci_read_vpd_reg(device_t pcib, pcicfgregs *cfg, int reg, uint32_t *data) { int count = PCI_VPD_TIMEOUT; KASSERT((reg & 3) == 0, ("VPD register must by 4 byte aligned")); WREG(cfg->vpd.vpd_reg + PCIR_VPD_ADDR, reg, 2); while ((REG(cfg->vpd.vpd_reg + PCIR_VPD_ADDR, 2) & 0x8000) != 0x8000) { if (--count < 0) return (ENXIO); DELAY(1); /* limit looping */ } *data = (REG(cfg->vpd.vpd_reg + PCIR_VPD_DATA, 4)); return (0); } #if 0 static int pci_write_vpd_reg(device_t pcib, pcicfgregs *cfg, int reg, uint32_t data) { int count = PCI_VPD_TIMEOUT; KASSERT((reg & 3) == 0, ("VPD register must by 4 byte aligned")); WREG(cfg->vpd.vpd_reg + PCIR_VPD_DATA, data, 4); WREG(cfg->vpd.vpd_reg + PCIR_VPD_ADDR, reg | 0x8000, 2); while ((REG(cfg->vpd.vpd_reg + PCIR_VPD_ADDR, 2) & 0x8000) == 0x8000) { if (--count < 0) return (ENXIO); DELAY(1); /* limit looping */ } return (0); } #endif #undef PCI_VPD_TIMEOUT struct vpd_readstate { device_t pcib; pcicfgregs *cfg; uint32_t val; int bytesinval; int off; uint8_t cksum; }; static int vpd_nextbyte(struct vpd_readstate *vrs, uint8_t *data) { uint32_t reg; uint8_t byte; if (vrs->bytesinval == 0) { if (pci_read_vpd_reg(vrs->pcib, vrs->cfg, vrs->off, ®)) return (ENXIO); vrs->val = le32toh(reg); vrs->off += 4; byte = vrs->val & 0xff; vrs->bytesinval = 3; } else { vrs->val = vrs->val >> 8; byte = vrs->val & 0xff; vrs->bytesinval--; } vrs->cksum += byte; *data = byte; return (0); } static void pci_read_vpd(device_t pcib, pcicfgregs *cfg) { struct vpd_readstate vrs; int state; int name; int remain; int i; int alloc, off; /* alloc/off for RO/W arrays */ int cksumvalid; int dflen; uint8_t byte; uint8_t byte2; /* init vpd reader */ vrs.bytesinval = 0; vrs.off = 0; vrs.pcib = pcib; vrs.cfg = cfg; vrs.cksum = 0; state = 0; name = remain = i = 0; /* shut up stupid gcc */ alloc = off = 0; /* shut up stupid gcc */ dflen = 0; /* shut up stupid gcc */ cksumvalid = -1; while (state >= 0) { if (vpd_nextbyte(&vrs, &byte)) { state = -2; break; } #if 0 printf("vpd: val: %#x, off: %d, bytesinval: %d, byte: %#hhx, " \ "state: %d, remain: %d, name: %#x, i: %d\n", vrs.val, vrs.off, vrs.bytesinval, byte, state, remain, name, i); #endif switch (state) { case 0: /* item name */ if (byte & 0x80) { if (vpd_nextbyte(&vrs, &byte2)) { state = -2; break; } remain = byte2; if (vpd_nextbyte(&vrs, &byte2)) { state = -2; break; } remain |= byte2 << 8; if (remain > (0x7f*4 - vrs.off)) { state = -1; pci_printf(cfg, "invalid VPD data, remain %#x\n", remain); } name = byte & 0x7f; } else { remain = byte & 0x7; name = (byte >> 3) & 0xf; } switch (name) { case 0x2: /* String */ cfg->vpd.vpd_ident = malloc(remain + 1, M_DEVBUF, M_WAITOK); i = 0; state = 1; break; case 0xf: /* End */ state = -1; break; case 0x10: /* VPD-R */ alloc = 8; off = 0; cfg->vpd.vpd_ros = malloc(alloc * sizeof(*cfg->vpd.vpd_ros), M_DEVBUF, M_WAITOK | M_ZERO); state = 2; break; case 0x11: /* VPD-W */ alloc = 8; off = 0; cfg->vpd.vpd_w = malloc(alloc * sizeof(*cfg->vpd.vpd_w), M_DEVBUF, M_WAITOK | M_ZERO); state = 5; break; default: /* Invalid data, abort */ state = -1; break; } break; case 1: /* Identifier String */ cfg->vpd.vpd_ident[i++] = byte; remain--; if (remain == 0) { cfg->vpd.vpd_ident[i] = '\0'; state = 0; } break; case 2: /* VPD-R Keyword Header */ if (off == alloc) { cfg->vpd.vpd_ros = reallocf(cfg->vpd.vpd_ros, (alloc *= 2) * sizeof(*cfg->vpd.vpd_ros), M_DEVBUF, M_WAITOK | M_ZERO); } cfg->vpd.vpd_ros[off].keyword[0] = byte; if (vpd_nextbyte(&vrs, &byte2)) { state = -2; break; } cfg->vpd.vpd_ros[off].keyword[1] = byte2; if (vpd_nextbyte(&vrs, &byte2)) { state = -2; break; } cfg->vpd.vpd_ros[off].len = dflen = byte2; if (dflen == 0 && strncmp(cfg->vpd.vpd_ros[off].keyword, "RV", 2) == 0) { /* * if this happens, we can't trust the rest * of the VPD. */ pci_printf(cfg, "bad keyword length: %d\n", dflen); cksumvalid = 0; state = -1; break; } else if (dflen == 0) { cfg->vpd.vpd_ros[off].value = malloc(1 * sizeof(*cfg->vpd.vpd_ros[off].value), M_DEVBUF, M_WAITOK); cfg->vpd.vpd_ros[off].value[0] = '\x00'; } else cfg->vpd.vpd_ros[off].value = malloc( (dflen + 1) * sizeof(*cfg->vpd.vpd_ros[off].value), M_DEVBUF, M_WAITOK); remain -= 3; i = 0; /* keep in sync w/ state 3's transistions */ if (dflen == 0 && remain == 0) state = 0; else if (dflen == 0) state = 2; else state = 3; break; case 3: /* VPD-R Keyword Value */ cfg->vpd.vpd_ros[off].value[i++] = byte; if (strncmp(cfg->vpd.vpd_ros[off].keyword, "RV", 2) == 0 && cksumvalid == -1) { if (vrs.cksum == 0) cksumvalid = 1; else { if (bootverbose) pci_printf(cfg, "bad VPD cksum, remain %hhu\n", vrs.cksum); cksumvalid = 0; state = -1; break; } } dflen--; remain--; /* keep in sync w/ state 2's transistions */ if (dflen == 0) cfg->vpd.vpd_ros[off++].value[i++] = '\0'; if (dflen == 0 && remain == 0) { cfg->vpd.vpd_rocnt = off; cfg->vpd.vpd_ros = reallocf(cfg->vpd.vpd_ros, off * sizeof(*cfg->vpd.vpd_ros), M_DEVBUF, M_WAITOK | M_ZERO); state = 0; } else if (dflen == 0) state = 2; break; case 4: remain--; if (remain == 0) state = 0; break; case 5: /* VPD-W Keyword Header */ if (off == alloc) { cfg->vpd.vpd_w = reallocf(cfg->vpd.vpd_w, (alloc *= 2) * sizeof(*cfg->vpd.vpd_w), M_DEVBUF, M_WAITOK | M_ZERO); } cfg->vpd.vpd_w[off].keyword[0] = byte; if (vpd_nextbyte(&vrs, &byte2)) { state = -2; break; } cfg->vpd.vpd_w[off].keyword[1] = byte2; if (vpd_nextbyte(&vrs, &byte2)) { state = -2; break; } cfg->vpd.vpd_w[off].len = dflen = byte2; cfg->vpd.vpd_w[off].start = vrs.off - vrs.bytesinval; cfg->vpd.vpd_w[off].value = malloc((dflen + 1) * sizeof(*cfg->vpd.vpd_w[off].value), M_DEVBUF, M_WAITOK); remain -= 3; i = 0; /* keep in sync w/ state 6's transistions */ if (dflen == 0 && remain == 0) state = 0; else if (dflen == 0) state = 5; else state = 6; break; case 6: /* VPD-W Keyword Value */ cfg->vpd.vpd_w[off].value[i++] = byte; dflen--; remain--; /* keep in sync w/ state 5's transistions */ if (dflen == 0) cfg->vpd.vpd_w[off++].value[i++] = '\0'; if (dflen == 0 && remain == 0) { cfg->vpd.vpd_wcnt = off; cfg->vpd.vpd_w = reallocf(cfg->vpd.vpd_w, off * sizeof(*cfg->vpd.vpd_w), M_DEVBUF, M_WAITOK | M_ZERO); state = 0; } else if (dflen == 0) state = 5; break; default: pci_printf(cfg, "invalid state: %d\n", state); state = -1; break; } } if (cksumvalid == 0 || state < -1) { /* read-only data bad, clean up */ if (cfg->vpd.vpd_ros != NULL) { for (off = 0; cfg->vpd.vpd_ros[off].value; off++) free(cfg->vpd.vpd_ros[off].value, M_DEVBUF); free(cfg->vpd.vpd_ros, M_DEVBUF); cfg->vpd.vpd_ros = NULL; } } if (state < -1) { /* I/O error, clean up */ pci_printf(cfg, "failed to read VPD data.\n"); if (cfg->vpd.vpd_ident != NULL) { free(cfg->vpd.vpd_ident, M_DEVBUF); cfg->vpd.vpd_ident = NULL; } if (cfg->vpd.vpd_w != NULL) { for (off = 0; cfg->vpd.vpd_w[off].value; off++) free(cfg->vpd.vpd_w[off].value, M_DEVBUF); free(cfg->vpd.vpd_w, M_DEVBUF); cfg->vpd.vpd_w = NULL; } } cfg->vpd.vpd_cached = 1; #undef REG #undef WREG } int pci_get_vpd_ident_method(device_t dev, device_t child, const char **identptr) { struct pci_devinfo *dinfo = device_get_ivars(child); pcicfgregs *cfg = &dinfo->cfg; if (!cfg->vpd.vpd_cached && cfg->vpd.vpd_reg != 0) pci_read_vpd(device_get_parent(dev), cfg); *identptr = cfg->vpd.vpd_ident; if (*identptr == NULL) return (ENXIO); return (0); } int pci_get_vpd_readonly_method(device_t dev, device_t child, const char *kw, const char **vptr) { struct pci_devinfo *dinfo = device_get_ivars(child); pcicfgregs *cfg = &dinfo->cfg; int i; if (!cfg->vpd.vpd_cached && cfg->vpd.vpd_reg != 0) pci_read_vpd(device_get_parent(dev), cfg); for (i = 0; i < cfg->vpd.vpd_rocnt; i++) if (memcmp(kw, cfg->vpd.vpd_ros[i].keyword, sizeof(cfg->vpd.vpd_ros[i].keyword)) == 0) { *vptr = cfg->vpd.vpd_ros[i].value; return (0); } *vptr = NULL; return (ENXIO); } struct pcicfg_vpd * pci_fetch_vpd_list(device_t dev) { struct pci_devinfo *dinfo = device_get_ivars(dev); pcicfgregs *cfg = &dinfo->cfg; if (!cfg->vpd.vpd_cached && cfg->vpd.vpd_reg != 0) pci_read_vpd(device_get_parent(device_get_parent(dev)), cfg); return (&cfg->vpd); } /* * Find the requested HyperTransport capability and return the offset * in configuration space via the pointer provided. The function * returns 0 on success and an error code otherwise. */ int pci_find_htcap_method(device_t dev, device_t child, int capability, int *capreg) { int ptr, error; uint16_t val; error = pci_find_cap(child, PCIY_HT, &ptr); if (error) return (error); /* * Traverse the capabilities list checking each HT capability * to see if it matches the requested HT capability. */ while (ptr != 0) { val = pci_read_config(child, ptr + PCIR_HT_COMMAND, 2); if (capability == PCIM_HTCAP_SLAVE || capability == PCIM_HTCAP_HOST) val &= 0xe000; else val &= PCIM_HTCMD_CAP_MASK; if (val == capability) { if (capreg != NULL) *capreg = ptr; return (0); } /* Skip to the next HT capability. */ while (ptr != 0) { ptr = pci_read_config(child, ptr + PCICAP_NEXTPTR, 1); if (pci_read_config(child, ptr + PCICAP_ID, 1) == PCIY_HT) break; } } return (ENOENT); } /* * Find the requested capability and return the offset in * configuration space via the pointer provided. The function returns * 0 on success and an error code otherwise. */ int pci_find_cap_method(device_t dev, device_t child, int capability, int *capreg) { struct pci_devinfo *dinfo = device_get_ivars(child); pcicfgregs *cfg = &dinfo->cfg; u_int32_t status; u_int8_t ptr; /* * Check the CAP_LIST bit of the PCI status register first. */ status = pci_read_config(child, PCIR_STATUS, 2); if (!(status & PCIM_STATUS_CAPPRESENT)) return (ENXIO); /* * Determine the start pointer of the capabilities list. */ switch (cfg->hdrtype & PCIM_HDRTYPE) { case PCIM_HDRTYPE_NORMAL: case PCIM_HDRTYPE_BRIDGE: ptr = PCIR_CAP_PTR; break; case PCIM_HDRTYPE_CARDBUS: ptr = PCIR_CAP_PTR_2; break; default: /* XXX: panic? */ return (ENXIO); /* no extended capabilities support */ } ptr = pci_read_config(child, ptr, 1); /* * Traverse the capabilities list. */ while (ptr != 0) { if (pci_read_config(child, ptr + PCICAP_ID, 1) == capability) { if (capreg != NULL) *capreg = ptr; return (0); } ptr = pci_read_config(child, ptr + PCICAP_NEXTPTR, 1); } return (ENOENT); } /* * Find the requested extended capability and return the offset in * configuration space via the pointer provided. The function returns * 0 on success and an error code otherwise. */ int pci_find_extcap_method(device_t dev, device_t child, int capability, int *capreg) { struct pci_devinfo *dinfo = device_get_ivars(child); pcicfgregs *cfg = &dinfo->cfg; uint32_t ecap; uint16_t ptr; /* Only supported for PCI-express devices. */ if (cfg->pcie.pcie_location == 0) return (ENXIO); ptr = PCIR_EXTCAP; ecap = pci_read_config(child, ptr, 4); if (ecap == 0xffffffff || ecap == 0) return (ENOENT); for (;;) { if (PCI_EXTCAP_ID(ecap) == capability) { if (capreg != NULL) *capreg = ptr; return (0); } ptr = PCI_EXTCAP_NEXTPTR(ecap); if (ptr == 0) break; ecap = pci_read_config(child, ptr, 4); } return (ENOENT); } /* * Support for MSI-X message interrupts. */ static void pci_write_msix_entry(device_t dev, u_int index, uint64_t address, uint32_t data) { struct pci_devinfo *dinfo = device_get_ivars(dev); struct pcicfg_msix *msix = &dinfo->cfg.msix; uint32_t offset; KASSERT(msix->msix_table_len > index, ("bogus index")); offset = msix->msix_table_offset + index * 16; bus_write_4(msix->msix_table_res, offset, address & 0xffffffff); bus_write_4(msix->msix_table_res, offset + 4, address >> 32); bus_write_4(msix->msix_table_res, offset + 8, data); } void pci_enable_msix_method(device_t dev, device_t child, u_int index, uint64_t address, uint32_t data) { if (pci_msix_rewrite_table) { struct pci_devinfo *dinfo = device_get_ivars(child); struct pcicfg_msix *msix = &dinfo->cfg.msix; /* * Some VM hosts require MSIX to be disabled in the * control register before updating the MSIX table * entries are allowed. It is not enough to only * disable MSIX while updating a single entry. MSIX * must be disabled while updating all entries in the * table. */ pci_write_config(child, msix->msix_location + PCIR_MSIX_CTRL, msix->msix_ctrl & ~PCIM_MSIXCTRL_MSIX_ENABLE, 2); pci_resume_msix(child); } else pci_write_msix_entry(child, index, address, data); /* Enable MSI -> HT mapping. */ pci_ht_map_msi(child, address); } void pci_mask_msix(device_t dev, u_int index) { struct pci_devinfo *dinfo = device_get_ivars(dev); struct pcicfg_msix *msix = &dinfo->cfg.msix; uint32_t offset, val; KASSERT(msix->msix_msgnum > index, ("bogus index")); offset = msix->msix_table_offset + index * 16 + 12; val = bus_read_4(msix->msix_table_res, offset); if (!(val & PCIM_MSIX_VCTRL_MASK)) { val |= PCIM_MSIX_VCTRL_MASK; bus_write_4(msix->msix_table_res, offset, val); } } void pci_unmask_msix(device_t dev, u_int index) { struct pci_devinfo *dinfo = device_get_ivars(dev); struct pcicfg_msix *msix = &dinfo->cfg.msix; uint32_t offset, val; KASSERT(msix->msix_table_len > index, ("bogus index")); offset = msix->msix_table_offset + index * 16 + 12; val = bus_read_4(msix->msix_table_res, offset); if (val & PCIM_MSIX_VCTRL_MASK) { val &= ~PCIM_MSIX_VCTRL_MASK; bus_write_4(msix->msix_table_res, offset, val); } } int pci_pending_msix(device_t dev, u_int index) { struct pci_devinfo *dinfo = device_get_ivars(dev); struct pcicfg_msix *msix = &dinfo->cfg.msix; uint32_t offset, bit; KASSERT(msix->msix_table_len > index, ("bogus index")); offset = msix->msix_pba_offset + (index / 32) * 4; bit = 1 << index % 32; return (bus_read_4(msix->msix_pba_res, offset) & bit); } /* * Restore MSI-X registers and table during resume. If MSI-X is * enabled then walk the virtual table to restore the actual MSI-X * table. */ static void pci_resume_msix(device_t dev) { struct pci_devinfo *dinfo = device_get_ivars(dev); struct pcicfg_msix *msix = &dinfo->cfg.msix; struct msix_table_entry *mte; struct msix_vector *mv; int i; if (msix->msix_alloc > 0) { /* First, mask all vectors. */ for (i = 0; i < msix->msix_msgnum; i++) pci_mask_msix(dev, i); /* Second, program any messages with at least one handler. */ for (i = 0; i < msix->msix_table_len; i++) { mte = &msix->msix_table[i]; if (mte->mte_vector == 0 || mte->mte_handlers == 0) continue; mv = &msix->msix_vectors[mte->mte_vector - 1]; pci_write_msix_entry(dev, i, mv->mv_address, mv->mv_data); pci_unmask_msix(dev, i); } } pci_write_config(dev, msix->msix_location + PCIR_MSIX_CTRL, msix->msix_ctrl, 2); } /* * Attempt to allocate *count MSI-X messages. The actual number allocated is * returned in *count. After this function returns, each message will be * available to the driver as SYS_RES_IRQ resources starting at rid 1. */ int pci_alloc_msix_method(device_t dev, device_t child, int *count) { struct pci_devinfo *dinfo = device_get_ivars(child); pcicfgregs *cfg = &dinfo->cfg; struct resource_list_entry *rle; int actual, error, i, irq, max; /* Don't let count == 0 get us into trouble. */ if (*count == 0) return (EINVAL); /* If rid 0 is allocated, then fail. */ rle = resource_list_find(&dinfo->resources, SYS_RES_IRQ, 0); if (rle != NULL && rle->res != NULL) return (ENXIO); /* Already have allocated messages? */ if (cfg->msi.msi_alloc != 0 || cfg->msix.msix_alloc != 0) return (ENXIO); /* If MSI-X is blacklisted for this system, fail. */ if (pci_msix_blacklisted()) return (ENXIO); /* MSI-X capability present? */ if (cfg->msix.msix_location == 0 || !pci_do_msix) return (ENODEV); /* Make sure the appropriate BARs are mapped. */ rle = resource_list_find(&dinfo->resources, SYS_RES_MEMORY, cfg->msix.msix_table_bar); if (rle == NULL || rle->res == NULL || !(rman_get_flags(rle->res) & RF_ACTIVE)) return (ENXIO); cfg->msix.msix_table_res = rle->res; if (cfg->msix.msix_pba_bar != cfg->msix.msix_table_bar) { rle = resource_list_find(&dinfo->resources, SYS_RES_MEMORY, cfg->msix.msix_pba_bar); if (rle == NULL || rle->res == NULL || !(rman_get_flags(rle->res) & RF_ACTIVE)) return (ENXIO); } cfg->msix.msix_pba_res = rle->res; if (bootverbose) device_printf(child, "attempting to allocate %d MSI-X vectors (%d supported)\n", *count, cfg->msix.msix_msgnum); max = min(*count, cfg->msix.msix_msgnum); for (i = 0; i < max; i++) { /* Allocate a message. */ error = PCIB_ALLOC_MSIX(device_get_parent(dev), child, &irq); if (error) { if (i == 0) return (error); break; } resource_list_add(&dinfo->resources, SYS_RES_IRQ, i + 1, irq, irq, 1); } actual = i; if (bootverbose) { rle = resource_list_find(&dinfo->resources, SYS_RES_IRQ, 1); if (actual == 1) device_printf(child, "using IRQ %ju for MSI-X\n", rle->start); else { int run; /* * Be fancy and try to print contiguous runs of * IRQ values as ranges. 'irq' is the previous IRQ. * 'run' is true if we are in a range. */ device_printf(child, "using IRQs %ju", rle->start); irq = rle->start; run = 0; for (i = 1; i < actual; i++) { rle = resource_list_find(&dinfo->resources, SYS_RES_IRQ, i + 1); /* Still in a run? */ if (rle->start == irq + 1) { run = 1; irq++; continue; } /* Finish previous range. */ if (run) { printf("-%d", irq); run = 0; } /* Start new range. */ printf(",%ju", rle->start); irq = rle->start; } /* Unfinished range? */ if (run) printf("-%d", irq); printf(" for MSI-X\n"); } } /* Mask all vectors. */ for (i = 0; i < cfg->msix.msix_msgnum; i++) pci_mask_msix(child, i); /* Allocate and initialize vector data and virtual table. */ cfg->msix.msix_vectors = malloc(sizeof(struct msix_vector) * actual, M_DEVBUF, M_WAITOK | M_ZERO); cfg->msix.msix_table = malloc(sizeof(struct msix_table_entry) * actual, M_DEVBUF, M_WAITOK | M_ZERO); for (i = 0; i < actual; i++) { rle = resource_list_find(&dinfo->resources, SYS_RES_IRQ, i + 1); cfg->msix.msix_vectors[i].mv_irq = rle->start; cfg->msix.msix_table[i].mte_vector = i + 1; } /* Update control register to enable MSI-X. */ cfg->msix.msix_ctrl |= PCIM_MSIXCTRL_MSIX_ENABLE; pci_write_config(child, cfg->msix.msix_location + PCIR_MSIX_CTRL, cfg->msix.msix_ctrl, 2); /* Update counts of alloc'd messages. */ cfg->msix.msix_alloc = actual; cfg->msix.msix_table_len = actual; *count = actual; return (0); } /* * By default, pci_alloc_msix() will assign the allocated IRQ * resources consecutively to the first N messages in the MSI-X table. * However, device drivers may want to use different layouts if they * either receive fewer messages than they asked for, or they wish to * populate the MSI-X table sparsely. This method allows the driver * to specify what layout it wants. It must be called after a * successful pci_alloc_msix() but before any of the associated * SYS_RES_IRQ resources are allocated via bus_alloc_resource(). * * The 'vectors' array contains 'count' message vectors. The array * maps directly to the MSI-X table in that index 0 in the array * specifies the vector for the first message in the MSI-X table, etc. * The vector value in each array index can either be 0 to indicate * that no vector should be assigned to a message slot, or it can be a * number from 1 to N (where N is the count returned from a * succcessful call to pci_alloc_msix()) to indicate which message * vector (IRQ) to be used for the corresponding message. * * On successful return, each message with a non-zero vector will have * an associated SYS_RES_IRQ whose rid is equal to the array index + * 1. Additionally, if any of the IRQs allocated via the previous * call to pci_alloc_msix() are not used in the mapping, those IRQs * will be freed back to the system automatically. * * For example, suppose a driver has a MSI-X table with 6 messages and * asks for 6 messages, but pci_alloc_msix() only returns a count of * 3. Call the three vectors allocated by pci_alloc_msix() A, B, and * C. After the call to pci_alloc_msix(), the device will be setup to * have an MSI-X table of ABC--- (where - means no vector assigned). * If the driver then passes a vector array of { 1, 0, 1, 2, 0, 2 }, * then the MSI-X table will look like A-AB-B, and the 'C' vector will * be freed back to the system. This device will also have valid * SYS_RES_IRQ rids of 1, 3, 4, and 6. * * In any case, the SYS_RES_IRQ rid X will always map to the message * at MSI-X table index X - 1 and will only be valid if a vector is * assigned to that table entry. */ int pci_remap_msix_method(device_t dev, device_t child, int count, const u_int *vectors) { struct pci_devinfo *dinfo = device_get_ivars(child); struct pcicfg_msix *msix = &dinfo->cfg.msix; struct resource_list_entry *rle; int i, irq, j, *used; /* * Have to have at least one message in the table but the * table can't be bigger than the actual MSI-X table in the * device. */ if (count == 0 || count > msix->msix_msgnum) return (EINVAL); /* Sanity check the vectors. */ for (i = 0; i < count; i++) if (vectors[i] > msix->msix_alloc) return (EINVAL); /* * Make sure there aren't any holes in the vectors to be used. * It's a big pain to support it, and it doesn't really make * sense anyway. Also, at least one vector must be used. */ used = malloc(sizeof(int) * msix->msix_alloc, M_DEVBUF, M_WAITOK | M_ZERO); for (i = 0; i < count; i++) if (vectors[i] != 0) used[vectors[i] - 1] = 1; for (i = 0; i < msix->msix_alloc - 1; i++) if (used[i] == 0 && used[i + 1] == 1) { free(used, M_DEVBUF); return (EINVAL); } if (used[0] != 1) { free(used, M_DEVBUF); return (EINVAL); } /* Make sure none of the resources are allocated. */ for (i = 0; i < msix->msix_table_len; i++) { if (msix->msix_table[i].mte_vector == 0) continue; if (msix->msix_table[i].mte_handlers > 0) { free(used, M_DEVBUF); return (EBUSY); } rle = resource_list_find(&dinfo->resources, SYS_RES_IRQ, i + 1); KASSERT(rle != NULL, ("missing resource")); if (rle->res != NULL) { free(used, M_DEVBUF); return (EBUSY); } } /* Free the existing resource list entries. */ for (i = 0; i < msix->msix_table_len; i++) { if (msix->msix_table[i].mte_vector == 0) continue; resource_list_delete(&dinfo->resources, SYS_RES_IRQ, i + 1); } /* * Build the new virtual table keeping track of which vectors are * used. */ free(msix->msix_table, M_DEVBUF); msix->msix_table = malloc(sizeof(struct msix_table_entry) * count, M_DEVBUF, M_WAITOK | M_ZERO); for (i = 0; i < count; i++) msix->msix_table[i].mte_vector = vectors[i]; msix->msix_table_len = count; /* Free any unused IRQs and resize the vectors array if necessary. */ j = msix->msix_alloc - 1; if (used[j] == 0) { struct msix_vector *vec; while (used[j] == 0) { PCIB_RELEASE_MSIX(device_get_parent(dev), child, msix->msix_vectors[j].mv_irq); j--; } vec = malloc(sizeof(struct msix_vector) * (j + 1), M_DEVBUF, M_WAITOK); bcopy(msix->msix_vectors, vec, sizeof(struct msix_vector) * (j + 1)); free(msix->msix_vectors, M_DEVBUF); msix->msix_vectors = vec; msix->msix_alloc = j + 1; } free(used, M_DEVBUF); /* Map the IRQs onto the rids. */ for (i = 0; i < count; i++) { if (vectors[i] == 0) continue; irq = msix->msix_vectors[vectors[i] - 1].mv_irq; resource_list_add(&dinfo->resources, SYS_RES_IRQ, i + 1, irq, irq, 1); } if (bootverbose) { device_printf(child, "Remapped MSI-X IRQs as: "); for (i = 0; i < count; i++) { if (i != 0) printf(", "); if (vectors[i] == 0) printf("---"); else printf("%d", msix->msix_vectors[vectors[i] - 1].mv_irq); } printf("\n"); } return (0); } static int pci_release_msix(device_t dev, device_t child) { struct pci_devinfo *dinfo = device_get_ivars(child); struct pcicfg_msix *msix = &dinfo->cfg.msix; struct resource_list_entry *rle; int i; /* Do we have any messages to release? */ if (msix->msix_alloc == 0) return (ENODEV); /* Make sure none of the resources are allocated. */ for (i = 0; i < msix->msix_table_len; i++) { if (msix->msix_table[i].mte_vector == 0) continue; if (msix->msix_table[i].mte_handlers > 0) return (EBUSY); rle = resource_list_find(&dinfo->resources, SYS_RES_IRQ, i + 1); KASSERT(rle != NULL, ("missing resource")); if (rle->res != NULL) return (EBUSY); } /* Update control register to disable MSI-X. */ msix->msix_ctrl &= ~PCIM_MSIXCTRL_MSIX_ENABLE; pci_write_config(child, msix->msix_location + PCIR_MSIX_CTRL, msix->msix_ctrl, 2); /* Free the resource list entries. */ for (i = 0; i < msix->msix_table_len; i++) { if (msix->msix_table[i].mte_vector == 0) continue; resource_list_delete(&dinfo->resources, SYS_RES_IRQ, i + 1); } free(msix->msix_table, M_DEVBUF); msix->msix_table_len = 0; /* Release the IRQs. */ for (i = 0; i < msix->msix_alloc; i++) PCIB_RELEASE_MSIX(device_get_parent(dev), child, msix->msix_vectors[i].mv_irq); free(msix->msix_vectors, M_DEVBUF); msix->msix_alloc = 0; return (0); } /* * Return the max supported MSI-X messages this device supports. * Basically, assuming the MD code can alloc messages, this function * should return the maximum value that pci_alloc_msix() can return. * Thus, it is subject to the tunables, etc. */ int pci_msix_count_method(device_t dev, device_t child) { struct pci_devinfo *dinfo = device_get_ivars(child); struct pcicfg_msix *msix = &dinfo->cfg.msix; if (pci_do_msix && msix->msix_location != 0) return (msix->msix_msgnum); return (0); } int pci_msix_pba_bar_method(device_t dev, device_t child) { struct pci_devinfo *dinfo = device_get_ivars(child); struct pcicfg_msix *msix = &dinfo->cfg.msix; if (pci_do_msix && msix->msix_location != 0) return (msix->msix_pba_bar); return (-1); } int pci_msix_table_bar_method(device_t dev, device_t child) { struct pci_devinfo *dinfo = device_get_ivars(child); struct pcicfg_msix *msix = &dinfo->cfg.msix; if (pci_do_msix && msix->msix_location != 0) return (msix->msix_table_bar); return (-1); } /* * HyperTransport MSI mapping control */ void pci_ht_map_msi(device_t dev, uint64_t addr) { struct pci_devinfo *dinfo = device_get_ivars(dev); struct pcicfg_ht *ht = &dinfo->cfg.ht; if (!ht->ht_msimap) return; if (addr && !(ht->ht_msictrl & PCIM_HTCMD_MSI_ENABLE) && ht->ht_msiaddr >> 20 == addr >> 20) { /* Enable MSI -> HT mapping. */ ht->ht_msictrl |= PCIM_HTCMD_MSI_ENABLE; pci_write_config(dev, ht->ht_msimap + PCIR_HT_COMMAND, ht->ht_msictrl, 2); } if (!addr && ht->ht_msictrl & PCIM_HTCMD_MSI_ENABLE) { /* Disable MSI -> HT mapping. */ ht->ht_msictrl &= ~PCIM_HTCMD_MSI_ENABLE; pci_write_config(dev, ht->ht_msimap + PCIR_HT_COMMAND, ht->ht_msictrl, 2); } } int pci_get_max_payload(device_t dev) { struct pci_devinfo *dinfo = device_get_ivars(dev); int cap; uint16_t val; cap = dinfo->cfg.pcie.pcie_location; if (cap == 0) return (0); val = pci_read_config(dev, cap + PCIER_DEVICE_CTL, 2); val &= PCIEM_CTL_MAX_PAYLOAD; val >>= 5; return (1 << (val + 7)); } int pci_get_max_read_req(device_t dev) { struct pci_devinfo *dinfo = device_get_ivars(dev); int cap; uint16_t val; cap = dinfo->cfg.pcie.pcie_location; if (cap == 0) return (0); val = pci_read_config(dev, cap + PCIER_DEVICE_CTL, 2); val &= PCIEM_CTL_MAX_READ_REQUEST; val >>= 12; return (1 << (val + 7)); } int pci_set_max_read_req(device_t dev, int size) { struct pci_devinfo *dinfo = device_get_ivars(dev); int cap; uint16_t val; cap = dinfo->cfg.pcie.pcie_location; if (cap == 0) return (0); if (size < 128) size = 128; if (size > 4096) size = 4096; size = (1 << (fls(size) - 1)); val = pci_read_config(dev, cap + PCIER_DEVICE_CTL, 2); val &= ~PCIEM_CTL_MAX_READ_REQUEST; val |= (fls(size) - 8) << 12; pci_write_config(dev, cap + PCIER_DEVICE_CTL, val, 2); return (size); } uint32_t pcie_read_config(device_t dev, int reg, int width) { struct pci_devinfo *dinfo = device_get_ivars(dev); int cap; cap = dinfo->cfg.pcie.pcie_location; if (cap == 0) { if (width == 2) return (0xffff); return (0xffffffff); } return (pci_read_config(dev, cap + reg, width)); } void pcie_write_config(device_t dev, int reg, uint32_t value, int width) { struct pci_devinfo *dinfo = device_get_ivars(dev); int cap; cap = dinfo->cfg.pcie.pcie_location; if (cap == 0) return; pci_write_config(dev, cap + reg, value, width); } /* * Adjusts a PCI-e capability register by clearing the bits in mask * and setting the bits in (value & mask). Bits not set in mask are * not adjusted. * * Returns the old value on success or all ones on failure. */ uint32_t pcie_adjust_config(device_t dev, int reg, uint32_t mask, uint32_t value, int width) { struct pci_devinfo *dinfo = device_get_ivars(dev); uint32_t old, new; int cap; cap = dinfo->cfg.pcie.pcie_location; if (cap == 0) { if (width == 2) return (0xffff); return (0xffffffff); } old = pci_read_config(dev, cap + reg, width); new = old & ~mask; new |= (value & mask); pci_write_config(dev, cap + reg, new, width); return (old); } /* * Support for MSI message signalled interrupts. */ void pci_enable_msi_method(device_t dev, device_t child, uint64_t address, uint16_t data) { struct pci_devinfo *dinfo = device_get_ivars(child); struct pcicfg_msi *msi = &dinfo->cfg.msi; /* Write data and address values. */ pci_write_config(child, msi->msi_location + PCIR_MSI_ADDR, address & 0xffffffff, 4); if (msi->msi_ctrl & PCIM_MSICTRL_64BIT) { pci_write_config(child, msi->msi_location + PCIR_MSI_ADDR_HIGH, address >> 32, 4); pci_write_config(child, msi->msi_location + PCIR_MSI_DATA_64BIT, data, 2); } else pci_write_config(child, msi->msi_location + PCIR_MSI_DATA, data, 2); /* Enable MSI in the control register. */ msi->msi_ctrl |= PCIM_MSICTRL_MSI_ENABLE; pci_write_config(child, msi->msi_location + PCIR_MSI_CTRL, msi->msi_ctrl, 2); /* Enable MSI -> HT mapping. */ pci_ht_map_msi(child, address); } void pci_disable_msi_method(device_t dev, device_t child) { struct pci_devinfo *dinfo = device_get_ivars(child); struct pcicfg_msi *msi = &dinfo->cfg.msi; /* Disable MSI -> HT mapping. */ pci_ht_map_msi(child, 0); /* Disable MSI in the control register. */ msi->msi_ctrl &= ~PCIM_MSICTRL_MSI_ENABLE; pci_write_config(child, msi->msi_location + PCIR_MSI_CTRL, msi->msi_ctrl, 2); } /* * Restore MSI registers during resume. If MSI is enabled then * restore the data and address registers in addition to the control * register. */ static void pci_resume_msi(device_t dev) { struct pci_devinfo *dinfo = device_get_ivars(dev); struct pcicfg_msi *msi = &dinfo->cfg.msi; uint64_t address; uint16_t data; if (msi->msi_ctrl & PCIM_MSICTRL_MSI_ENABLE) { address = msi->msi_addr; data = msi->msi_data; pci_write_config(dev, msi->msi_location + PCIR_MSI_ADDR, address & 0xffffffff, 4); if (msi->msi_ctrl & PCIM_MSICTRL_64BIT) { pci_write_config(dev, msi->msi_location + PCIR_MSI_ADDR_HIGH, address >> 32, 4); pci_write_config(dev, msi->msi_location + PCIR_MSI_DATA_64BIT, data, 2); } else pci_write_config(dev, msi->msi_location + PCIR_MSI_DATA, data, 2); } pci_write_config(dev, msi->msi_location + PCIR_MSI_CTRL, msi->msi_ctrl, 2); } static int pci_remap_intr_method(device_t bus, device_t dev, u_int irq) { struct pci_devinfo *dinfo = device_get_ivars(dev); pcicfgregs *cfg = &dinfo->cfg; struct resource_list_entry *rle; struct msix_table_entry *mte; struct msix_vector *mv; uint64_t addr; uint32_t data; int error, i, j; /* * Handle MSI first. We try to find this IRQ among our list * of MSI IRQs. If we find it, we request updated address and * data registers and apply the results. */ if (cfg->msi.msi_alloc > 0) { /* If we don't have any active handlers, nothing to do. */ if (cfg->msi.msi_handlers == 0) return (0); for (i = 0; i < cfg->msi.msi_alloc; i++) { rle = resource_list_find(&dinfo->resources, SYS_RES_IRQ, i + 1); if (rle->start == irq) { error = PCIB_MAP_MSI(device_get_parent(bus), dev, irq, &addr, &data); if (error) return (error); pci_disable_msi(dev); dinfo->cfg.msi.msi_addr = addr; dinfo->cfg.msi.msi_data = data; pci_enable_msi(dev, addr, data); return (0); } } return (ENOENT); } /* * For MSI-X, we check to see if we have this IRQ. If we do, * we request the updated mapping info. If that works, we go * through all the slots that use this IRQ and update them. */ if (cfg->msix.msix_alloc > 0) { for (i = 0; i < cfg->msix.msix_alloc; i++) { mv = &cfg->msix.msix_vectors[i]; if (mv->mv_irq == irq) { error = PCIB_MAP_MSI(device_get_parent(bus), dev, irq, &addr, &data); if (error) return (error); mv->mv_address = addr; mv->mv_data = data; for (j = 0; j < cfg->msix.msix_table_len; j++) { mte = &cfg->msix.msix_table[j]; if (mte->mte_vector != i + 1) continue; if (mte->mte_handlers == 0) continue; pci_mask_msix(dev, j); pci_enable_msix(dev, j, addr, data); pci_unmask_msix(dev, j); } } } return (ENOENT); } return (ENOENT); } /* * Returns true if the specified device is blacklisted because MSI * doesn't work. */ int pci_msi_device_blacklisted(device_t dev) { if (!pci_honor_msi_blacklist) return (0); return (pci_has_quirk(pci_get_devid(dev), PCI_QUIRK_DISABLE_MSI)); } /* * Determine if MSI is blacklisted globally on this system. Currently, * we just check for blacklisted chipsets as represented by the * host-PCI bridge at device 0:0:0. In the future, it may become * necessary to check other system attributes, such as the kenv values * that give the motherboard manufacturer and model number. */ static int pci_msi_blacklisted(void) { device_t dev; if (!pci_honor_msi_blacklist) return (0); /* Blacklist all non-PCI-express and non-PCI-X chipsets. */ if (!(pcie_chipset || pcix_chipset)) { if (vm_guest != VM_GUEST_NO) { /* * Whitelist older chipsets in virtual * machines known to support MSI. */ dev = pci_find_bsf(0, 0, 0); if (dev != NULL) return (!pci_has_quirk(pci_get_devid(dev), PCI_QUIRK_ENABLE_MSI_VM)); } return (1); } dev = pci_find_bsf(0, 0, 0); if (dev != NULL) return (pci_msi_device_blacklisted(dev)); return (0); } /* * Returns true if the specified device is blacklisted because MSI-X * doesn't work. Note that this assumes that if MSI doesn't work, * MSI-X doesn't either. */ int pci_msix_device_blacklisted(device_t dev) { if (!pci_honor_msi_blacklist) return (0); if (pci_has_quirk(pci_get_devid(dev), PCI_QUIRK_DISABLE_MSIX)) return (1); return (pci_msi_device_blacklisted(dev)); } /* * Determine if MSI-X is blacklisted globally on this system. If MSI * is blacklisted, assume that MSI-X is as well. Check for additional * chipsets where MSI works but MSI-X does not. */ static int pci_msix_blacklisted(void) { device_t dev; if (!pci_honor_msi_blacklist) return (0); dev = pci_find_bsf(0, 0, 0); if (dev != NULL && pci_has_quirk(pci_get_devid(dev), PCI_QUIRK_DISABLE_MSIX)) return (1); return (pci_msi_blacklisted()); } /* * Attempt to allocate *count MSI messages. The actual number allocated is * returned in *count. After this function returns, each message will be * available to the driver as SYS_RES_IRQ resources starting at a rid 1. */ int pci_alloc_msi_method(device_t dev, device_t child, int *count) { struct pci_devinfo *dinfo = device_get_ivars(child); pcicfgregs *cfg = &dinfo->cfg; struct resource_list_entry *rle; int actual, error, i, irqs[32]; uint16_t ctrl; /* Don't let count == 0 get us into trouble. */ if (*count == 0) return (EINVAL); /* If rid 0 is allocated, then fail. */ rle = resource_list_find(&dinfo->resources, SYS_RES_IRQ, 0); if (rle != NULL && rle->res != NULL) return (ENXIO); /* Already have allocated messages? */ if (cfg->msi.msi_alloc != 0 || cfg->msix.msix_alloc != 0) return (ENXIO); /* If MSI is blacklisted for this system, fail. */ if (pci_msi_blacklisted()) return (ENXIO); /* MSI capability present? */ if (cfg->msi.msi_location == 0 || !pci_do_msi) return (ENODEV); if (bootverbose) device_printf(child, "attempting to allocate %d MSI vectors (%d supported)\n", *count, cfg->msi.msi_msgnum); /* Don't ask for more than the device supports. */ actual = min(*count, cfg->msi.msi_msgnum); /* Don't ask for more than 32 messages. */ actual = min(actual, 32); /* MSI requires power of 2 number of messages. */ if (!powerof2(actual)) return (EINVAL); for (;;) { /* Try to allocate N messages. */ error = PCIB_ALLOC_MSI(device_get_parent(dev), child, actual, actual, irqs); if (error == 0) break; if (actual == 1) return (error); /* Try N / 2. */ actual >>= 1; } /* * We now have N actual messages mapped onto SYS_RES_IRQ * resources in the irqs[] array, so add new resources * starting at rid 1. */ for (i = 0; i < actual; i++) resource_list_add(&dinfo->resources, SYS_RES_IRQ, i + 1, irqs[i], irqs[i], 1); if (bootverbose) { if (actual == 1) device_printf(child, "using IRQ %d for MSI\n", irqs[0]); else { int run; /* * Be fancy and try to print contiguous runs * of IRQ values as ranges. 'run' is true if * we are in a range. */ device_printf(child, "using IRQs %d", irqs[0]); run = 0; for (i = 1; i < actual; i++) { /* Still in a run? */ if (irqs[i] == irqs[i - 1] + 1) { run = 1; continue; } /* Finish previous range. */ if (run) { printf("-%d", irqs[i - 1]); run = 0; } /* Start new range. */ printf(",%d", irqs[i]); } /* Unfinished range? */ if (run) printf("-%d", irqs[actual - 1]); printf(" for MSI\n"); } } /* Update control register with actual count. */ ctrl = cfg->msi.msi_ctrl; ctrl &= ~PCIM_MSICTRL_MME_MASK; ctrl |= (ffs(actual) - 1) << 4; cfg->msi.msi_ctrl = ctrl; pci_write_config(child, cfg->msi.msi_location + PCIR_MSI_CTRL, ctrl, 2); /* Update counts of alloc'd messages. */ cfg->msi.msi_alloc = actual; cfg->msi.msi_handlers = 0; *count = actual; return (0); } /* Release the MSI messages associated with this device. */ int pci_release_msi_method(device_t dev, device_t child) { struct pci_devinfo *dinfo = device_get_ivars(child); struct pcicfg_msi *msi = &dinfo->cfg.msi; struct resource_list_entry *rle; int error, i, irqs[32]; /* Try MSI-X first. */ error = pci_release_msix(dev, child); if (error != ENODEV) return (error); /* Do we have any messages to release? */ if (msi->msi_alloc == 0) return (ENODEV); KASSERT(msi->msi_alloc <= 32, ("more than 32 alloc'd messages")); /* Make sure none of the resources are allocated. */ if (msi->msi_handlers > 0) return (EBUSY); for (i = 0; i < msi->msi_alloc; i++) { rle = resource_list_find(&dinfo->resources, SYS_RES_IRQ, i + 1); KASSERT(rle != NULL, ("missing MSI resource")); if (rle->res != NULL) return (EBUSY); irqs[i] = rle->start; } /* Update control register with 0 count. */ KASSERT(!(msi->msi_ctrl & PCIM_MSICTRL_MSI_ENABLE), ("%s: MSI still enabled", __func__)); msi->msi_ctrl &= ~PCIM_MSICTRL_MME_MASK; pci_write_config(child, msi->msi_location + PCIR_MSI_CTRL, msi->msi_ctrl, 2); /* Release the messages. */ PCIB_RELEASE_MSI(device_get_parent(dev), child, msi->msi_alloc, irqs); for (i = 0; i < msi->msi_alloc; i++) resource_list_delete(&dinfo->resources, SYS_RES_IRQ, i + 1); /* Update alloc count. */ msi->msi_alloc = 0; msi->msi_addr = 0; msi->msi_data = 0; return (0); } /* * Return the max supported MSI messages this device supports. * Basically, assuming the MD code can alloc messages, this function * should return the maximum value that pci_alloc_msi() can return. * Thus, it is subject to the tunables, etc. */ int pci_msi_count_method(device_t dev, device_t child) { struct pci_devinfo *dinfo = device_get_ivars(child); struct pcicfg_msi *msi = &dinfo->cfg.msi; if (pci_do_msi && msi->msi_location != 0) return (msi->msi_msgnum); return (0); } /* free pcicfgregs structure and all depending data structures */ int pci_freecfg(struct pci_devinfo *dinfo) { struct devlist *devlist_head; struct pci_map *pm, *next; int i; devlist_head = &pci_devq; if (dinfo->cfg.vpd.vpd_reg) { free(dinfo->cfg.vpd.vpd_ident, M_DEVBUF); for (i = 0; i < dinfo->cfg.vpd.vpd_rocnt; i++) free(dinfo->cfg.vpd.vpd_ros[i].value, M_DEVBUF); free(dinfo->cfg.vpd.vpd_ros, M_DEVBUF); for (i = 0; i < dinfo->cfg.vpd.vpd_wcnt; i++) free(dinfo->cfg.vpd.vpd_w[i].value, M_DEVBUF); free(dinfo->cfg.vpd.vpd_w, M_DEVBUF); } STAILQ_FOREACH_SAFE(pm, &dinfo->cfg.maps, pm_link, next) { free(pm, M_DEVBUF); } STAILQ_REMOVE(devlist_head, dinfo, pci_devinfo, pci_links); free(dinfo, M_DEVBUF); /* increment the generation count */ pci_generation++; /* we're losing one device */ pci_numdevs--; return (0); } /* * PCI power manangement */ int pci_set_powerstate_method(device_t dev, device_t child, int state) { struct pci_devinfo *dinfo = device_get_ivars(child); pcicfgregs *cfg = &dinfo->cfg; uint16_t status; int oldstate, highest, delay; if (cfg->pp.pp_cap == 0) return (EOPNOTSUPP); /* * Optimize a no state change request away. While it would be OK to * write to the hardware in theory, some devices have shown odd * behavior when going from D3 -> D3. */ oldstate = pci_get_powerstate(child); if (oldstate == state) return (0); /* * The PCI power management specification states that after a state * transition between PCI power states, system software must * guarantee a minimal delay before the function accesses the device. * Compute the worst case delay that we need to guarantee before we * access the device. Many devices will be responsive much more * quickly than this delay, but there are some that don't respond * instantly to state changes. Transitions to/from D3 state require * 10ms, while D2 requires 200us, and D0/1 require none. The delay * is done below with DELAY rather than a sleeper function because * this function can be called from contexts where we cannot sleep. */ highest = (oldstate > state) ? oldstate : state; if (highest == PCI_POWERSTATE_D3) delay = 10000; else if (highest == PCI_POWERSTATE_D2) delay = 200; else delay = 0; status = PCI_READ_CONFIG(dev, child, cfg->pp.pp_status, 2) & ~PCIM_PSTAT_DMASK; switch (state) { case PCI_POWERSTATE_D0: status |= PCIM_PSTAT_D0; break; case PCI_POWERSTATE_D1: if ((cfg->pp.pp_cap & PCIM_PCAP_D1SUPP) == 0) return (EOPNOTSUPP); status |= PCIM_PSTAT_D1; break; case PCI_POWERSTATE_D2: if ((cfg->pp.pp_cap & PCIM_PCAP_D2SUPP) == 0) return (EOPNOTSUPP); status |= PCIM_PSTAT_D2; break; case PCI_POWERSTATE_D3: status |= PCIM_PSTAT_D3; break; default: return (EINVAL); } if (bootverbose) pci_printf(cfg, "Transition from D%d to D%d\n", oldstate, state); PCI_WRITE_CONFIG(dev, child, cfg->pp.pp_status, status, 2); if (delay) DELAY(delay); return (0); } int pci_get_powerstate_method(device_t dev, device_t child) { struct pci_devinfo *dinfo = device_get_ivars(child); pcicfgregs *cfg = &dinfo->cfg; uint16_t status; int result; if (cfg->pp.pp_cap != 0) { status = PCI_READ_CONFIG(dev, child, cfg->pp.pp_status, 2); switch (status & PCIM_PSTAT_DMASK) { case PCIM_PSTAT_D0: result = PCI_POWERSTATE_D0; break; case PCIM_PSTAT_D1: result = PCI_POWERSTATE_D1; break; case PCIM_PSTAT_D2: result = PCI_POWERSTATE_D2; break; case PCIM_PSTAT_D3: result = PCI_POWERSTATE_D3; break; default: result = PCI_POWERSTATE_UNKNOWN; break; } } else { /* No support, device is always at D0 */ result = PCI_POWERSTATE_D0; } return (result); } /* * Some convenience functions for PCI device drivers. */ static __inline void pci_set_command_bit(device_t dev, device_t child, uint16_t bit) { uint16_t command; command = PCI_READ_CONFIG(dev, child, PCIR_COMMAND, 2); command |= bit; PCI_WRITE_CONFIG(dev, child, PCIR_COMMAND, command, 2); } static __inline void pci_clear_command_bit(device_t dev, device_t child, uint16_t bit) { uint16_t command; command = PCI_READ_CONFIG(dev, child, PCIR_COMMAND, 2); command &= ~bit; PCI_WRITE_CONFIG(dev, child, PCIR_COMMAND, command, 2); } int pci_enable_busmaster_method(device_t dev, device_t child) { pci_set_command_bit(dev, child, PCIM_CMD_BUSMASTEREN); return (0); } int pci_disable_busmaster_method(device_t dev, device_t child) { pci_clear_command_bit(dev, child, PCIM_CMD_BUSMASTEREN); return (0); } int pci_enable_io_method(device_t dev, device_t child, int space) { uint16_t bit; switch(space) { case SYS_RES_IOPORT: bit = PCIM_CMD_PORTEN; break; case SYS_RES_MEMORY: bit = PCIM_CMD_MEMEN; break; default: return (EINVAL); } pci_set_command_bit(dev, child, bit); return (0); } int pci_disable_io_method(device_t dev, device_t child, int space) { uint16_t bit; switch(space) { case SYS_RES_IOPORT: bit = PCIM_CMD_PORTEN; break; case SYS_RES_MEMORY: bit = PCIM_CMD_MEMEN; break; default: return (EINVAL); } pci_clear_command_bit(dev, child, bit); return (0); } /* * New style pci driver. Parent device is either a pci-host-bridge or a * pci-pci-bridge. Both kinds are represented by instances of pcib. */ void pci_print_verbose(struct pci_devinfo *dinfo) { if (bootverbose) { pcicfgregs *cfg = &dinfo->cfg; printf("found->\tvendor=0x%04x, dev=0x%04x, revid=0x%02x\n", cfg->vendor, cfg->device, cfg->revid); printf("\tdomain=%d, bus=%d, slot=%d, func=%d\n", cfg->domain, cfg->bus, cfg->slot, cfg->func); printf("\tclass=%02x-%02x-%02x, hdrtype=0x%02x, mfdev=%d\n", cfg->baseclass, cfg->subclass, cfg->progif, cfg->hdrtype, cfg->mfdev); printf("\tcmdreg=0x%04x, statreg=0x%04x, cachelnsz=%d (dwords)\n", cfg->cmdreg, cfg->statreg, cfg->cachelnsz); printf("\tlattimer=0x%02x (%d ns), mingnt=0x%02x (%d ns), maxlat=0x%02x (%d ns)\n", cfg->lattimer, cfg->lattimer * 30, cfg->mingnt, cfg->mingnt * 250, cfg->maxlat, cfg->maxlat * 250); if (cfg->intpin > 0) printf("\tintpin=%c, irq=%d\n", cfg->intpin +'a' -1, cfg->intline); if (cfg->pp.pp_cap) { uint16_t status; status = pci_read_config(cfg->dev, cfg->pp.pp_status, 2); printf("\tpowerspec %d supports D0%s%s D3 current D%d\n", cfg->pp.pp_cap & PCIM_PCAP_SPEC, cfg->pp.pp_cap & PCIM_PCAP_D1SUPP ? " D1" : "", cfg->pp.pp_cap & PCIM_PCAP_D2SUPP ? " D2" : "", status & PCIM_PSTAT_DMASK); } if (cfg->msi.msi_location) { int ctrl; ctrl = cfg->msi.msi_ctrl; printf("\tMSI supports %d message%s%s%s\n", cfg->msi.msi_msgnum, (cfg->msi.msi_msgnum == 1) ? "" : "s", (ctrl & PCIM_MSICTRL_64BIT) ? ", 64 bit" : "", (ctrl & PCIM_MSICTRL_VECTOR) ? ", vector masks":""); } if (cfg->msix.msix_location) { printf("\tMSI-X supports %d message%s ", cfg->msix.msix_msgnum, (cfg->msix.msix_msgnum == 1) ? "" : "s"); if (cfg->msix.msix_table_bar == cfg->msix.msix_pba_bar) printf("in map 0x%x\n", cfg->msix.msix_table_bar); else printf("in maps 0x%x and 0x%x\n", cfg->msix.msix_table_bar, cfg->msix.msix_pba_bar); } } } static int pci_porten(device_t dev) { return (pci_read_config(dev, PCIR_COMMAND, 2) & PCIM_CMD_PORTEN) != 0; } static int pci_memen(device_t dev) { return (pci_read_config(dev, PCIR_COMMAND, 2) & PCIM_CMD_MEMEN) != 0; } void pci_read_bar(device_t dev, int reg, pci_addr_t *mapp, pci_addr_t *testvalp, int *bar64) { struct pci_devinfo *dinfo; pci_addr_t map, testval; int ln2range; uint16_t cmd; /* * The device ROM BAR is special. It is always a 32-bit * memory BAR. Bit 0 is special and should not be set when * sizing the BAR. */ dinfo = device_get_ivars(dev); if (PCIR_IS_BIOS(&dinfo->cfg, reg)) { map = pci_read_config(dev, reg, 4); pci_write_config(dev, reg, 0xfffffffe, 4); testval = pci_read_config(dev, reg, 4); pci_write_config(dev, reg, map, 4); *mapp = map; *testvalp = testval; if (bar64 != NULL) *bar64 = 0; return; } map = pci_read_config(dev, reg, 4); ln2range = pci_maprange(map); if (ln2range == 64) map |= (pci_addr_t)pci_read_config(dev, reg + 4, 4) << 32; /* * Disable decoding via the command register before * determining the BAR's length since we will be placing it in * a weird state. */ cmd = pci_read_config(dev, PCIR_COMMAND, 2); pci_write_config(dev, PCIR_COMMAND, cmd & ~(PCI_BAR_MEM(map) ? PCIM_CMD_MEMEN : PCIM_CMD_PORTEN), 2); /* * Determine the BAR's length by writing all 1's. The bottom * log_2(size) bits of the BAR will stick as 0 when we read * the value back. */ pci_write_config(dev, reg, 0xffffffff, 4); testval = pci_read_config(dev, reg, 4); if (ln2range == 64) { pci_write_config(dev, reg + 4, 0xffffffff, 4); testval |= (pci_addr_t)pci_read_config(dev, reg + 4, 4) << 32; } /* * Restore the original value of the BAR. We may have reprogrammed * the BAR of the low-level console device and when booting verbose, * we need the console device addressable. */ pci_write_config(dev, reg, map, 4); if (ln2range == 64) pci_write_config(dev, reg + 4, map >> 32, 4); pci_write_config(dev, PCIR_COMMAND, cmd, 2); *mapp = map; *testvalp = testval; if (bar64 != NULL) *bar64 = (ln2range == 64); } static void pci_write_bar(device_t dev, struct pci_map *pm, pci_addr_t base) { struct pci_devinfo *dinfo; int ln2range; /* The device ROM BAR is always a 32-bit memory BAR. */ dinfo = device_get_ivars(dev); if (PCIR_IS_BIOS(&dinfo->cfg, pm->pm_reg)) ln2range = 32; else ln2range = pci_maprange(pm->pm_value); pci_write_config(dev, pm->pm_reg, base, 4); if (ln2range == 64) pci_write_config(dev, pm->pm_reg + 4, base >> 32, 4); pm->pm_value = pci_read_config(dev, pm->pm_reg, 4); if (ln2range == 64) pm->pm_value |= (pci_addr_t)pci_read_config(dev, pm->pm_reg + 4, 4) << 32; } struct pci_map * pci_find_bar(device_t dev, int reg) { struct pci_devinfo *dinfo; struct pci_map *pm; dinfo = device_get_ivars(dev); STAILQ_FOREACH(pm, &dinfo->cfg.maps, pm_link) { if (pm->pm_reg == reg) return (pm); } return (NULL); } int pci_bar_enabled(device_t dev, struct pci_map *pm) { struct pci_devinfo *dinfo; uint16_t cmd; dinfo = device_get_ivars(dev); if (PCIR_IS_BIOS(&dinfo->cfg, pm->pm_reg) && !(pm->pm_value & PCIM_BIOS_ENABLE)) return (0); cmd = pci_read_config(dev, PCIR_COMMAND, 2); if (PCIR_IS_BIOS(&dinfo->cfg, pm->pm_reg) || PCI_BAR_MEM(pm->pm_value)) return ((cmd & PCIM_CMD_MEMEN) != 0); else return ((cmd & PCIM_CMD_PORTEN) != 0); } struct pci_map * pci_add_bar(device_t dev, int reg, pci_addr_t value, pci_addr_t size) { struct pci_devinfo *dinfo; struct pci_map *pm, *prev; dinfo = device_get_ivars(dev); pm = malloc(sizeof(*pm), M_DEVBUF, M_WAITOK | M_ZERO); pm->pm_reg = reg; pm->pm_value = value; pm->pm_size = size; STAILQ_FOREACH(prev, &dinfo->cfg.maps, pm_link) { KASSERT(prev->pm_reg != pm->pm_reg, ("duplicate map %02x", reg)); if (STAILQ_NEXT(prev, pm_link) == NULL || STAILQ_NEXT(prev, pm_link)->pm_reg > pm->pm_reg) break; } if (prev != NULL) STAILQ_INSERT_AFTER(&dinfo->cfg.maps, prev, pm, pm_link); else STAILQ_INSERT_TAIL(&dinfo->cfg.maps, pm, pm_link); return (pm); } static void pci_restore_bars(device_t dev) { struct pci_devinfo *dinfo; struct pci_map *pm; int ln2range; dinfo = device_get_ivars(dev); STAILQ_FOREACH(pm, &dinfo->cfg.maps, pm_link) { if (PCIR_IS_BIOS(&dinfo->cfg, pm->pm_reg)) ln2range = 32; else ln2range = pci_maprange(pm->pm_value); pci_write_config(dev, pm->pm_reg, pm->pm_value, 4); if (ln2range == 64) pci_write_config(dev, pm->pm_reg + 4, pm->pm_value >> 32, 4); } } /* * Add a resource based on a pci map register. Return 1 if the map * register is a 32bit map register or 2 if it is a 64bit register. */ static int pci_add_map(device_t bus, device_t dev, int reg, struct resource_list *rl, int force, int prefetch) { struct pci_map *pm; pci_addr_t base, map, testval; pci_addr_t start, end, count; int barlen, basezero, flags, maprange, mapsize, type; uint16_t cmd; struct resource *res; /* * The BAR may already exist if the device is a CardBus card * whose CIS is stored in this BAR. */ pm = pci_find_bar(dev, reg); if (pm != NULL) { maprange = pci_maprange(pm->pm_value); barlen = maprange == 64 ? 2 : 1; return (barlen); } pci_read_bar(dev, reg, &map, &testval, NULL); if (PCI_BAR_MEM(map)) { type = SYS_RES_MEMORY; if (map & PCIM_BAR_MEM_PREFETCH) prefetch = 1; } else type = SYS_RES_IOPORT; mapsize = pci_mapsize(testval); base = pci_mapbase(map); #ifdef __PCI_BAR_ZERO_VALID basezero = 0; #else basezero = base == 0; #endif maprange = pci_maprange(map); barlen = maprange == 64 ? 2 : 1; /* * For I/O registers, if bottom bit is set, and the next bit up * isn't clear, we know we have a BAR that doesn't conform to the * spec, so ignore it. Also, sanity check the size of the data * areas to the type of memory involved. Memory must be at least * 16 bytes in size, while I/O ranges must be at least 4. */ if (PCI_BAR_IO(testval) && (testval & PCIM_BAR_IO_RESERVED) != 0) return (barlen); if ((type == SYS_RES_MEMORY && mapsize < 4) || (type == SYS_RES_IOPORT && mapsize < 2)) return (barlen); /* Save a record of this BAR. */ pm = pci_add_bar(dev, reg, map, mapsize); if (bootverbose) { printf("\tmap[%02x]: type %s, range %2d, base %#jx, size %2d", reg, pci_maptype(map), maprange, (uintmax_t)base, mapsize); if (type == SYS_RES_IOPORT && !pci_porten(dev)) printf(", port disabled\n"); else if (type == SYS_RES_MEMORY && !pci_memen(dev)) printf(", memory disabled\n"); else printf(", enabled\n"); } /* * If base is 0, then we have problems if this architecture does * not allow that. It is best to ignore such entries for the * moment. These will be allocated later if the driver specifically * requests them. However, some removable buses look better when * all resources are allocated, so allow '0' to be overriden. * * Similarly treat maps whose values is the same as the test value * read back. These maps have had all f's written to them by the * BIOS in an attempt to disable the resources. */ if (!force && (basezero || map == testval)) return (barlen); if ((u_long)base != base) { device_printf(bus, "pci%d:%d:%d:%d bar %#x too many address bits", pci_get_domain(dev), pci_get_bus(dev), pci_get_slot(dev), pci_get_function(dev), reg); return (barlen); } /* * This code theoretically does the right thing, but has * undesirable side effects in some cases where peripherals * respond oddly to having these bits enabled. Let the user * be able to turn them off (since pci_enable_io_modes is 1 by * default). */ if (pci_enable_io_modes) { /* Turn on resources that have been left off by a lazy BIOS */ if (type == SYS_RES_IOPORT && !pci_porten(dev)) { cmd = pci_read_config(dev, PCIR_COMMAND, 2); cmd |= PCIM_CMD_PORTEN; pci_write_config(dev, PCIR_COMMAND, cmd, 2); } if (type == SYS_RES_MEMORY && !pci_memen(dev)) { cmd = pci_read_config(dev, PCIR_COMMAND, 2); cmd |= PCIM_CMD_MEMEN; pci_write_config(dev, PCIR_COMMAND, cmd, 2); } } else { if (type == SYS_RES_IOPORT && !pci_porten(dev)) return (barlen); if (type == SYS_RES_MEMORY && !pci_memen(dev)) return (barlen); } count = (pci_addr_t)1 << mapsize; flags = RF_ALIGNMENT_LOG2(mapsize); if (prefetch) flags |= RF_PREFETCHABLE; if (basezero || base == pci_mapbase(testval) || pci_clear_bars) { start = 0; /* Let the parent decide. */ end = ~0; } else { start = base; end = base + count - 1; } resource_list_add(rl, type, reg, start, end, count); /* * Try to allocate the resource for this BAR from our parent * so that this resource range is already reserved. The * driver for this device will later inherit this resource in * pci_alloc_resource(). */ res = resource_list_reserve(rl, bus, dev, type, ®, start, end, count, flags); if (pci_do_realloc_bars && res == NULL && (start != 0 || end != ~0)) { /* * If the allocation fails, try to allocate a resource for * this BAR using any available range. The firmware felt * it was important enough to assign a resource, so don't * disable decoding if we can help it. */ resource_list_delete(rl, type, reg); resource_list_add(rl, type, reg, 0, ~0, count); res = resource_list_reserve(rl, bus, dev, type, ®, 0, ~0, count, flags); } if (res == NULL) { /* * If the allocation fails, delete the resource list entry * and disable decoding for this device. * * If the driver requests this resource in the future, * pci_reserve_map() will try to allocate a fresh * resource range. */ resource_list_delete(rl, type, reg); pci_disable_io(dev, type); if (bootverbose) device_printf(bus, "pci%d:%d:%d:%d bar %#x failed to allocate\n", pci_get_domain(dev), pci_get_bus(dev), pci_get_slot(dev), pci_get_function(dev), reg); } else { start = rman_get_start(res); pci_write_bar(dev, pm, start); } return (barlen); } /* * For ATA devices we need to decide early what addressing mode to use. * Legacy demands that the primary and secondary ATA ports sits on the * same addresses that old ISA hardware did. This dictates that we use * those addresses and ignore the BAR's if we cannot set PCI native * addressing mode. */ static void pci_ata_maps(device_t bus, device_t dev, struct resource_list *rl, int force, uint32_t prefetchmask) { int rid, type, progif; #if 0 /* if this device supports PCI native addressing use it */ progif = pci_read_config(dev, PCIR_PROGIF, 1); if ((progif & 0x8a) == 0x8a) { if (pci_mapbase(pci_read_config(dev, PCIR_BAR(0), 4)) && pci_mapbase(pci_read_config(dev, PCIR_BAR(2), 4))) { printf("Trying ATA native PCI addressing mode\n"); pci_write_config(dev, PCIR_PROGIF, progif | 0x05, 1); } } #endif progif = pci_read_config(dev, PCIR_PROGIF, 1); type = SYS_RES_IOPORT; if (progif & PCIP_STORAGE_IDE_MODEPRIM) { pci_add_map(bus, dev, PCIR_BAR(0), rl, force, prefetchmask & (1 << 0)); pci_add_map(bus, dev, PCIR_BAR(1), rl, force, prefetchmask & (1 << 1)); } else { rid = PCIR_BAR(0); resource_list_add(rl, type, rid, 0x1f0, 0x1f7, 8); (void)resource_list_reserve(rl, bus, dev, type, &rid, 0x1f0, 0x1f7, 8, 0); rid = PCIR_BAR(1); resource_list_add(rl, type, rid, 0x3f6, 0x3f6, 1); (void)resource_list_reserve(rl, bus, dev, type, &rid, 0x3f6, 0x3f6, 1, 0); } if (progif & PCIP_STORAGE_IDE_MODESEC) { pci_add_map(bus, dev, PCIR_BAR(2), rl, force, prefetchmask & (1 << 2)); pci_add_map(bus, dev, PCIR_BAR(3), rl, force, prefetchmask & (1 << 3)); } else { rid = PCIR_BAR(2); resource_list_add(rl, type, rid, 0x170, 0x177, 8); (void)resource_list_reserve(rl, bus, dev, type, &rid, 0x170, 0x177, 8, 0); rid = PCIR_BAR(3); resource_list_add(rl, type, rid, 0x376, 0x376, 1); (void)resource_list_reserve(rl, bus, dev, type, &rid, 0x376, 0x376, 1, 0); } pci_add_map(bus, dev, PCIR_BAR(4), rl, force, prefetchmask & (1 << 4)); pci_add_map(bus, dev, PCIR_BAR(5), rl, force, prefetchmask & (1 << 5)); } static void pci_assign_interrupt(device_t bus, device_t dev, int force_route) { struct pci_devinfo *dinfo = device_get_ivars(dev); pcicfgregs *cfg = &dinfo->cfg; char tunable_name[64]; int irq; /* Has to have an intpin to have an interrupt. */ if (cfg->intpin == 0) return; /* Let the user override the IRQ with a tunable. */ irq = PCI_INVALID_IRQ; snprintf(tunable_name, sizeof(tunable_name), "hw.pci%d.%d.%d.INT%c.irq", cfg->domain, cfg->bus, cfg->slot, cfg->intpin + 'A' - 1); if (TUNABLE_INT_FETCH(tunable_name, &irq) && (irq >= 255 || irq <= 0)) irq = PCI_INVALID_IRQ; /* * If we didn't get an IRQ via the tunable, then we either use the * IRQ value in the intline register or we ask the bus to route an * interrupt for us. If force_route is true, then we only use the * value in the intline register if the bus was unable to assign an * IRQ. */ if (!PCI_INTERRUPT_VALID(irq)) { if (!PCI_INTERRUPT_VALID(cfg->intline) || force_route) irq = PCI_ASSIGN_INTERRUPT(bus, dev); if (!PCI_INTERRUPT_VALID(irq)) irq = cfg->intline; } /* If after all that we don't have an IRQ, just bail. */ if (!PCI_INTERRUPT_VALID(irq)) return; /* Update the config register if it changed. */ if (irq != cfg->intline) { cfg->intline = irq; pci_write_config(dev, PCIR_INTLINE, irq, 1); } /* Add this IRQ as rid 0 interrupt resource. */ resource_list_add(&dinfo->resources, SYS_RES_IRQ, 0, irq, irq, 1); } /* Perform early OHCI takeover from SMM. */ static void ohci_early_takeover(device_t self) { struct resource *res; uint32_t ctl; int rid; int i; rid = PCIR_BAR(0); res = bus_alloc_resource_any(self, SYS_RES_MEMORY, &rid, RF_ACTIVE); if (res == NULL) return; ctl = bus_read_4(res, OHCI_CONTROL); if (ctl & OHCI_IR) { if (bootverbose) printf("ohci early: " "SMM active, request owner change\n"); bus_write_4(res, OHCI_COMMAND_STATUS, OHCI_OCR); for (i = 0; (i < 100) && (ctl & OHCI_IR); i++) { DELAY(1000); ctl = bus_read_4(res, OHCI_CONTROL); } if (ctl & OHCI_IR) { if (bootverbose) printf("ohci early: " "SMM does not respond, resetting\n"); bus_write_4(res, OHCI_CONTROL, OHCI_HCFS_RESET); } /* Disable interrupts */ bus_write_4(res, OHCI_INTERRUPT_DISABLE, OHCI_ALL_INTRS); } bus_release_resource(self, SYS_RES_MEMORY, rid, res); } /* Perform early UHCI takeover from SMM. */ static void uhci_early_takeover(device_t self) { struct resource *res; int rid; /* * Set the PIRQD enable bit and switch off all the others. We don't * want legacy support to interfere with us XXX Does this also mean * that the BIOS won't touch the keyboard anymore if it is connected * to the ports of the root hub? */ pci_write_config(self, PCI_LEGSUP, PCI_LEGSUP_USBPIRQDEN, 2); /* Disable interrupts */ rid = PCI_UHCI_BASE_REG; res = bus_alloc_resource_any(self, SYS_RES_IOPORT, &rid, RF_ACTIVE); if (res != NULL) { bus_write_2(res, UHCI_INTR, 0); bus_release_resource(self, SYS_RES_IOPORT, rid, res); } } /* Perform early EHCI takeover from SMM. */ static void ehci_early_takeover(device_t self) { struct resource *res; uint32_t cparams; uint32_t eec; uint8_t eecp; uint8_t bios_sem; uint8_t offs; int rid; int i; rid = PCIR_BAR(0); res = bus_alloc_resource_any(self, SYS_RES_MEMORY, &rid, RF_ACTIVE); if (res == NULL) return; cparams = bus_read_4(res, EHCI_HCCPARAMS); /* Synchronise with the BIOS if it owns the controller. */ for (eecp = EHCI_HCC_EECP(cparams); eecp != 0; eecp = EHCI_EECP_NEXT(eec)) { eec = pci_read_config(self, eecp, 4); if (EHCI_EECP_ID(eec) != EHCI_EC_LEGSUP) { continue; } bios_sem = pci_read_config(self, eecp + EHCI_LEGSUP_BIOS_SEM, 1); if (bios_sem == 0) { continue; } if (bootverbose) printf("ehci early: " "SMM active, request owner change\n"); pci_write_config(self, eecp + EHCI_LEGSUP_OS_SEM, 1, 1); for (i = 0; (i < 100) && (bios_sem != 0); i++) { DELAY(1000); bios_sem = pci_read_config(self, eecp + EHCI_LEGSUP_BIOS_SEM, 1); } if (bios_sem != 0) { if (bootverbose) printf("ehci early: " "SMM does not respond\n"); } /* Disable interrupts */ offs = EHCI_CAPLENGTH(bus_read_4(res, EHCI_CAPLEN_HCIVERSION)); bus_write_4(res, offs + EHCI_USBINTR, 0); } bus_release_resource(self, SYS_RES_MEMORY, rid, res); } /* Perform early XHCI takeover from SMM. */ static void xhci_early_takeover(device_t self) { struct resource *res; uint32_t cparams; uint32_t eec; uint8_t eecp; uint8_t bios_sem; uint8_t offs; int rid; int i; rid = PCIR_BAR(0); res = bus_alloc_resource_any(self, SYS_RES_MEMORY, &rid, RF_ACTIVE); if (res == NULL) return; cparams = bus_read_4(res, XHCI_HCSPARAMS0); eec = -1; /* Synchronise with the BIOS if it owns the controller. */ for (eecp = XHCI_HCS0_XECP(cparams) << 2; eecp != 0 && XHCI_XECP_NEXT(eec); eecp += XHCI_XECP_NEXT(eec) << 2) { eec = bus_read_4(res, eecp); if (XHCI_XECP_ID(eec) != XHCI_ID_USB_LEGACY) continue; bios_sem = bus_read_1(res, eecp + XHCI_XECP_BIOS_SEM); if (bios_sem == 0) continue; if (bootverbose) printf("xhci early: " "SMM active, request owner change\n"); bus_write_1(res, eecp + XHCI_XECP_OS_SEM, 1); /* wait a maximum of 5 second */ for (i = 0; (i < 5000) && (bios_sem != 0); i++) { DELAY(1000); bios_sem = bus_read_1(res, eecp + XHCI_XECP_BIOS_SEM); } if (bios_sem != 0) { if (bootverbose) printf("xhci early: " "SMM does not respond\n"); } /* Disable interrupts */ offs = bus_read_1(res, XHCI_CAPLENGTH); bus_write_4(res, offs + XHCI_USBCMD, 0); bus_read_4(res, offs + XHCI_USBSTS); } bus_release_resource(self, SYS_RES_MEMORY, rid, res); } #if defined(NEW_PCIB) && defined(PCI_RES_BUS) static void pci_reserve_secbus(device_t bus, device_t dev, pcicfgregs *cfg, struct resource_list *rl) { struct resource *res; char *cp; rman_res_t start, end, count; int rid, sec_bus, sec_reg, sub_bus, sub_reg, sup_bus; switch (cfg->hdrtype & PCIM_HDRTYPE) { case PCIM_HDRTYPE_BRIDGE: sec_reg = PCIR_SECBUS_1; sub_reg = PCIR_SUBBUS_1; break; case PCIM_HDRTYPE_CARDBUS: sec_reg = PCIR_SECBUS_2; sub_reg = PCIR_SUBBUS_2; break; default: return; } /* * If the existing bus range is valid, attempt to reserve it * from our parent. If this fails for any reason, clear the * secbus and subbus registers. * * XXX: Should we reset sub_bus to sec_bus if it is < sec_bus? * This would at least preserve the existing sec_bus if it is * valid. */ sec_bus = PCI_READ_CONFIG(bus, dev, sec_reg, 1); sub_bus = PCI_READ_CONFIG(bus, dev, sub_reg, 1); /* Quirk handling. */ switch (pci_get_devid(dev)) { case 0x12258086: /* Intel 82454KX/GX (Orion) */ sup_bus = pci_read_config(dev, 0x41, 1); if (sup_bus != 0xff) { sec_bus = sup_bus + 1; sub_bus = sup_bus + 1; PCI_WRITE_CONFIG(bus, dev, sec_reg, sec_bus, 1); PCI_WRITE_CONFIG(bus, dev, sub_reg, sub_bus, 1); } break; case 0x00dd10de: /* Compaq R3000 BIOS sets wrong subordinate bus number. */ if ((cp = kern_getenv("smbios.planar.maker")) == NULL) break; if (strncmp(cp, "Compal", 6) != 0) { freeenv(cp); break; } freeenv(cp); if ((cp = kern_getenv("smbios.planar.product")) == NULL) break; if (strncmp(cp, "08A0", 4) != 0) { freeenv(cp); break; } freeenv(cp); if (sub_bus < 0xa) { sub_bus = 0xa; PCI_WRITE_CONFIG(bus, dev, sub_reg, sub_bus, 1); } break; } if (bootverbose) printf("\tsecbus=%d, subbus=%d\n", sec_bus, sub_bus); if (sec_bus > 0 && sub_bus >= sec_bus) { start = sec_bus; end = sub_bus; count = end - start + 1; resource_list_add(rl, PCI_RES_BUS, 0, 0, ~0, count); /* * If requested, clear secondary bus registers in * bridge devices to force a complete renumbering * rather than reserving the existing range. However, * preserve the existing size. */ if (pci_clear_buses) goto clear; rid = 0; res = resource_list_reserve(rl, bus, dev, PCI_RES_BUS, &rid, start, end, count, 0); if (res != NULL) return; if (bootverbose) device_printf(bus, "pci%d:%d:%d:%d secbus failed to allocate\n", pci_get_domain(dev), pci_get_bus(dev), pci_get_slot(dev), pci_get_function(dev)); } clear: PCI_WRITE_CONFIG(bus, dev, sec_reg, 0, 1); PCI_WRITE_CONFIG(bus, dev, sub_reg, 0, 1); } static struct resource * pci_alloc_secbus(device_t dev, device_t child, int *rid, rman_res_t start, rman_res_t end, rman_res_t count, u_int flags) { struct pci_devinfo *dinfo; pcicfgregs *cfg; struct resource_list *rl; struct resource *res; int sec_reg, sub_reg; dinfo = device_get_ivars(child); cfg = &dinfo->cfg; rl = &dinfo->resources; switch (cfg->hdrtype & PCIM_HDRTYPE) { case PCIM_HDRTYPE_BRIDGE: sec_reg = PCIR_SECBUS_1; sub_reg = PCIR_SUBBUS_1; break; case PCIM_HDRTYPE_CARDBUS: sec_reg = PCIR_SECBUS_2; sub_reg = PCIR_SUBBUS_2; break; default: return (NULL); } if (*rid != 0) return (NULL); if (resource_list_find(rl, PCI_RES_BUS, *rid) == NULL) resource_list_add(rl, PCI_RES_BUS, *rid, start, end, count); if (!resource_list_reserved(rl, PCI_RES_BUS, *rid)) { res = resource_list_reserve(rl, dev, child, PCI_RES_BUS, rid, start, end, count, flags & ~RF_ACTIVE); if (res == NULL) { resource_list_delete(rl, PCI_RES_BUS, *rid); device_printf(child, "allocating %ju bus%s failed\n", count, count == 1 ? "" : "es"); return (NULL); } if (bootverbose) device_printf(child, "Lazy allocation of %ju bus%s at %ju\n", count, count == 1 ? "" : "es", rman_get_start(res)); PCI_WRITE_CONFIG(dev, child, sec_reg, rman_get_start(res), 1); PCI_WRITE_CONFIG(dev, child, sub_reg, rman_get_end(res), 1); } return (resource_list_alloc(rl, dev, child, PCI_RES_BUS, rid, start, end, count, flags)); } #endif static int pci_ea_bei_to_rid(device_t dev, int bei) { #ifdef PCI_IOV struct pci_devinfo *dinfo; int iov_pos; struct pcicfg_iov *iov; dinfo = device_get_ivars(dev); iov = dinfo->cfg.iov; if (iov != NULL) iov_pos = iov->iov_pos; else iov_pos = 0; #endif /* Check if matches BAR */ if ((bei >= PCIM_EA_BEI_BAR_0) && (bei <= PCIM_EA_BEI_BAR_5)) return (PCIR_BAR(bei)); /* Check ROM */ if (bei == PCIM_EA_BEI_ROM) return (PCIR_BIOS); #ifdef PCI_IOV /* Check if matches VF_BAR */ if ((iov != NULL) && (bei >= PCIM_EA_BEI_VF_BAR_0) && (bei <= PCIM_EA_BEI_VF_BAR_5)) return (PCIR_SRIOV_BAR(bei - PCIM_EA_BEI_VF_BAR_0) + iov_pos); #endif return (-1); } int pci_ea_is_enabled(device_t dev, int rid) { struct pci_ea_entry *ea; struct pci_devinfo *dinfo; dinfo = device_get_ivars(dev); STAILQ_FOREACH(ea, &dinfo->cfg.ea.ea_entries, eae_link) { if (pci_ea_bei_to_rid(dev, ea->eae_bei) == rid) return ((ea->eae_flags & PCIM_EA_ENABLE) > 0); } return (0); } void pci_add_resources_ea(device_t bus, device_t dev, int alloc_iov) { struct pci_ea_entry *ea; struct pci_devinfo *dinfo; pci_addr_t start, end, count; struct resource_list *rl; int type, flags, rid; struct resource *res; uint32_t tmp; #ifdef PCI_IOV struct pcicfg_iov *iov; #endif dinfo = device_get_ivars(dev); rl = &dinfo->resources; flags = 0; #ifdef PCI_IOV iov = dinfo->cfg.iov; #endif if (dinfo->cfg.ea.ea_location == 0) return; STAILQ_FOREACH(ea, &dinfo->cfg.ea.ea_entries, eae_link) { /* * TODO: Ignore EA-BAR if is not enabled. * Currently the EA implementation supports * only situation, where EA structure contains * predefined entries. In case they are not enabled * leave them unallocated and proceed with * a legacy-BAR mechanism. */ if ((ea->eae_flags & PCIM_EA_ENABLE) == 0) continue; switch ((ea->eae_flags & PCIM_EA_PP) >> PCIM_EA_PP_OFFSET) { case PCIM_EA_P_MEM_PREFETCH: case PCIM_EA_P_VF_MEM_PREFETCH: flags = RF_PREFETCHABLE; /* FALLTHROUGH */ case PCIM_EA_P_VF_MEM: case PCIM_EA_P_MEM: type = SYS_RES_MEMORY; break; case PCIM_EA_P_IO: type = SYS_RES_IOPORT; break; default: continue; } if (alloc_iov != 0) { #ifdef PCI_IOV /* Allocating IOV, confirm BEI matches */ if ((ea->eae_bei < PCIM_EA_BEI_VF_BAR_0) || (ea->eae_bei > PCIM_EA_BEI_VF_BAR_5)) continue; #else continue; #endif } else { /* Allocating BAR, confirm BEI matches */ if (((ea->eae_bei < PCIM_EA_BEI_BAR_0) || (ea->eae_bei > PCIM_EA_BEI_BAR_5)) && (ea->eae_bei != PCIM_EA_BEI_ROM)) continue; } rid = pci_ea_bei_to_rid(dev, ea->eae_bei); if (rid < 0) continue; /* Skip resources already allocated by EA */ if ((resource_list_find(rl, SYS_RES_MEMORY, rid) != NULL) || (resource_list_find(rl, SYS_RES_IOPORT, rid) != NULL)) continue; start = ea->eae_base; count = ea->eae_max_offset + 1; #ifdef PCI_IOV if (iov != NULL) count = count * iov->iov_num_vfs; #endif end = start + count - 1; if (count == 0) continue; resource_list_add(rl, type, rid, start, end, count); res = resource_list_reserve(rl, bus, dev, type, &rid, start, end, count, flags); if (res == NULL) { resource_list_delete(rl, type, rid); /* * Failed to allocate using EA, disable entry. * Another attempt to allocation will be performed * further, but this time using legacy BAR registers */ tmp = pci_read_config(dev, ea->eae_cfg_offset, 4); tmp &= ~PCIM_EA_ENABLE; pci_write_config(dev, ea->eae_cfg_offset, tmp, 4); /* * Disabling entry might fail in case it is hardwired. * Read flags again to match current status. */ ea->eae_flags = pci_read_config(dev, ea->eae_cfg_offset, 4); continue; } /* As per specification, fill BAR with zeros */ pci_write_config(dev, rid, 0, 4); } } void pci_add_resources(device_t bus, device_t dev, int force, uint32_t prefetchmask) { struct pci_devinfo *dinfo; pcicfgregs *cfg; struct resource_list *rl; const struct pci_quirk *q; uint32_t devid; int i; dinfo = device_get_ivars(dev); cfg = &dinfo->cfg; rl = &dinfo->resources; devid = (cfg->device << 16) | cfg->vendor; /* Allocate resources using Enhanced Allocation */ pci_add_resources_ea(bus, dev, 0); /* ATA devices needs special map treatment */ if ((pci_get_class(dev) == PCIC_STORAGE) && (pci_get_subclass(dev) == PCIS_STORAGE_IDE) && ((pci_get_progif(dev) & PCIP_STORAGE_IDE_MASTERDEV) || (!pci_read_config(dev, PCIR_BAR(0), 4) && !pci_read_config(dev, PCIR_BAR(2), 4))) ) pci_ata_maps(bus, dev, rl, force, prefetchmask); else for (i = 0; i < cfg->nummaps;) { /* Skip resources already managed by EA */ if ((resource_list_find(rl, SYS_RES_MEMORY, PCIR_BAR(i)) != NULL) || (resource_list_find(rl, SYS_RES_IOPORT, PCIR_BAR(i)) != NULL) || pci_ea_is_enabled(dev, PCIR_BAR(i))) { i++; continue; } /* * Skip quirked resources. */ for (q = &pci_quirks[0]; q->devid != 0; q++) if (q->devid == devid && q->type == PCI_QUIRK_UNMAP_REG && q->arg1 == PCIR_BAR(i)) break; if (q->devid != 0) { i++; continue; } i += pci_add_map(bus, dev, PCIR_BAR(i), rl, force, prefetchmask & (1 << i)); } /* * Add additional, quirked resources. */ for (q = &pci_quirks[0]; q->devid != 0; q++) if (q->devid == devid && q->type == PCI_QUIRK_MAP_REG) pci_add_map(bus, dev, q->arg1, rl, force, 0); if (cfg->intpin > 0 && PCI_INTERRUPT_VALID(cfg->intline)) { #ifdef __PCI_REROUTE_INTERRUPT /* * Try to re-route interrupts. Sometimes the BIOS or * firmware may leave bogus values in these registers. * If the re-route fails, then just stick with what we * have. */ pci_assign_interrupt(bus, dev, 1); #else pci_assign_interrupt(bus, dev, 0); #endif } if (pci_usb_takeover && pci_get_class(dev) == PCIC_SERIALBUS && pci_get_subclass(dev) == PCIS_SERIALBUS_USB) { if (pci_get_progif(dev) == PCIP_SERIALBUS_USB_XHCI) xhci_early_takeover(dev); else if (pci_get_progif(dev) == PCIP_SERIALBUS_USB_EHCI) ehci_early_takeover(dev); else if (pci_get_progif(dev) == PCIP_SERIALBUS_USB_OHCI) ohci_early_takeover(dev); else if (pci_get_progif(dev) == PCIP_SERIALBUS_USB_UHCI) uhci_early_takeover(dev); } #if defined(NEW_PCIB) && defined(PCI_RES_BUS) /* * Reserve resources for secondary bus ranges behind bridge * devices. */ pci_reserve_secbus(bus, dev, cfg, rl); #endif } static struct pci_devinfo * pci_identify_function(device_t pcib, device_t dev, int domain, int busno, int slot, int func) { struct pci_devinfo *dinfo; dinfo = pci_read_device(pcib, dev, domain, busno, slot, func); if (dinfo != NULL) pci_add_child(dev, dinfo); return (dinfo); } void pci_add_children(device_t dev, int domain, int busno) { #define REG(n, w) PCIB_READ_CONFIG(pcib, busno, s, f, n, w) device_t pcib = device_get_parent(dev); struct pci_devinfo *dinfo; int maxslots; int s, f, pcifunchigh; uint8_t hdrtype; int first_func; /* * Try to detect a device at slot 0, function 0. If it exists, try to * enable ARI. We must enable ARI before detecting the rest of the * functions on this bus as ARI changes the set of slots and functions * that are legal on this bus. */ dinfo = pci_identify_function(pcib, dev, domain, busno, 0, 0); if (dinfo != NULL && pci_enable_ari) PCIB_TRY_ENABLE_ARI(pcib, dinfo->cfg.dev); /* * Start looking for new devices on slot 0 at function 1 because we * just identified the device at slot 0, function 0. */ first_func = 1; maxslots = PCIB_MAXSLOTS(pcib); for (s = 0; s <= maxslots; s++, first_func = 0) { pcifunchigh = 0; f = 0; DELAY(1); hdrtype = REG(PCIR_HDRTYPE, 1); if ((hdrtype & PCIM_HDRTYPE) > PCI_MAXHDRTYPE) continue; if (hdrtype & PCIM_MFDEV) pcifunchigh = PCIB_MAXFUNCS(pcib); for (f = first_func; f <= pcifunchigh; f++) pci_identify_function(pcib, dev, domain, busno, s, f); } #undef REG } int pci_rescan_method(device_t dev) { #define REG(n, w) PCIB_READ_CONFIG(pcib, busno, s, f, n, w) device_t pcib = device_get_parent(dev); struct pci_softc *sc; device_t child, *devlist, *unchanged; int devcount, error, i, j, maxslots, oldcount; int busno, domain, s, f, pcifunchigh; uint8_t hdrtype; /* No need to check for ARI on a rescan. */ error = device_get_children(dev, &devlist, &devcount); if (error) return (error); if (devcount != 0) { unchanged = malloc(devcount * sizeof(device_t), M_TEMP, M_NOWAIT | M_ZERO); if (unchanged == NULL) { free(devlist, M_TEMP); return (ENOMEM); } } else unchanged = NULL; sc = device_get_softc(dev); domain = pcib_get_domain(dev); busno = pcib_get_bus(dev); maxslots = PCIB_MAXSLOTS(pcib); for (s = 0; s <= maxslots; s++) { /* If function 0 is not present, skip to the next slot. */ f = 0; if (REG(PCIR_VENDOR, 2) == 0xffff) continue; pcifunchigh = 0; hdrtype = REG(PCIR_HDRTYPE, 1); if ((hdrtype & PCIM_HDRTYPE) > PCI_MAXHDRTYPE) continue; if (hdrtype & PCIM_MFDEV) pcifunchigh = PCIB_MAXFUNCS(pcib); for (f = 0; f <= pcifunchigh; f++) { if (REG(PCIR_VENDOR, 2) == 0xffff) continue; /* * Found a valid function. Check if a * device_t for this device already exists. */ for (i = 0; i < devcount; i++) { child = devlist[i]; if (child == NULL) continue; if (pci_get_slot(child) == s && pci_get_function(child) == f) { unchanged[i] = child; goto next_func; } } pci_identify_function(pcib, dev, domain, busno, s, f); next_func:; } } /* Remove devices that are no longer present. */ for (i = 0; i < devcount; i++) { if (unchanged[i] != NULL) continue; device_delete_child(dev, devlist[i]); } free(devlist, M_TEMP); oldcount = devcount; /* Try to attach the devices just added. */ error = device_get_children(dev, &devlist, &devcount); if (error) { free(unchanged, M_TEMP); return (error); } for (i = 0; i < devcount; i++) { for (j = 0; j < oldcount; j++) { if (devlist[i] == unchanged[j]) goto next_device; } device_probe_and_attach(devlist[i]); next_device:; } free(unchanged, M_TEMP); free(devlist, M_TEMP); return (0); #undef REG } #ifdef PCI_IOV device_t pci_add_iov_child(device_t bus, device_t pf, uint16_t rid, uint16_t vid, uint16_t did) { struct pci_devinfo *pf_dinfo, *vf_dinfo; device_t pcib; int busno, slot, func; pf_dinfo = device_get_ivars(pf); pcib = device_get_parent(bus); PCIB_DECODE_RID(pcib, rid, &busno, &slot, &func); vf_dinfo = pci_fill_devinfo(pcib, bus, pci_get_domain(pcib), busno, slot, func, vid, did); vf_dinfo->cfg.flags |= PCICFG_VF; pci_add_child(bus, vf_dinfo); return (vf_dinfo->cfg.dev); } device_t pci_create_iov_child_method(device_t bus, device_t pf, uint16_t rid, uint16_t vid, uint16_t did) { return (pci_add_iov_child(bus, pf, rid, vid, did)); } #endif void pci_add_child(device_t bus, struct pci_devinfo *dinfo) { dinfo->cfg.dev = device_add_child(bus, NULL, -1); device_set_ivars(dinfo->cfg.dev, dinfo); resource_list_init(&dinfo->resources); pci_cfg_save(dinfo->cfg.dev, dinfo, 0); pci_cfg_restore(dinfo->cfg.dev, dinfo); pci_print_verbose(dinfo); pci_add_resources(bus, dinfo->cfg.dev, 0, 0); pci_child_added(dinfo->cfg.dev); EVENTHANDLER_INVOKE(pci_add_device, dinfo->cfg.dev); } void pci_child_added_method(device_t dev, device_t child) { } static int pci_probe(device_t dev) { device_set_desc(dev, "PCI bus"); /* Allow other subclasses to override this driver. */ return (BUS_PROBE_GENERIC); } int pci_attach_common(device_t dev) { struct pci_softc *sc; int busno, domain; #ifdef PCI_DMA_BOUNDARY int error, tag_valid; #endif #ifdef PCI_RES_BUS int rid; #endif sc = device_get_softc(dev); domain = pcib_get_domain(dev); busno = pcib_get_bus(dev); #ifdef PCI_RES_BUS rid = 0; sc->sc_bus = bus_alloc_resource(dev, PCI_RES_BUS, &rid, busno, busno, 1, 0); if (sc->sc_bus == NULL) { device_printf(dev, "failed to allocate bus number\n"); return (ENXIO); } #endif if (bootverbose) device_printf(dev, "domain=%d, physical bus=%d\n", domain, busno); #ifdef PCI_DMA_BOUNDARY tag_valid = 0; if (device_get_devclass(device_get_parent(device_get_parent(dev))) != devclass_find("pci")) { error = bus_dma_tag_create(bus_get_dma_tag(dev), 1, PCI_DMA_BOUNDARY, BUS_SPACE_MAXADDR, BUS_SPACE_MAXADDR, NULL, NULL, BUS_SPACE_MAXSIZE, BUS_SPACE_UNRESTRICTED, BUS_SPACE_MAXSIZE, 0, NULL, NULL, &sc->sc_dma_tag); if (error) device_printf(dev, "Failed to create DMA tag: %d\n", error); else tag_valid = 1; } if (!tag_valid) #endif sc->sc_dma_tag = bus_get_dma_tag(dev); return (0); } static int pci_attach(device_t dev) { int busno, domain, error; error = pci_attach_common(dev); if (error) return (error); /* * Since there can be multiple independently numbered PCI * buses on systems with multiple PCI domains, we can't use * the unit number to decide which bus we are probing. We ask * the parent pcib what our domain and bus numbers are. */ domain = pcib_get_domain(dev); busno = pcib_get_bus(dev); pci_add_children(dev, domain, busno); return (bus_generic_attach(dev)); } static int pci_detach(device_t dev) { #ifdef PCI_RES_BUS struct pci_softc *sc; #endif int error; error = bus_generic_detach(dev); if (error) return (error); #ifdef PCI_RES_BUS sc = device_get_softc(dev); error = bus_release_resource(dev, PCI_RES_BUS, 0, sc->sc_bus); if (error) return (error); #endif return (device_delete_children(dev)); } static void pci_set_power_child(device_t dev, device_t child, int state) { device_t pcib; int dstate; /* * Set the device to the given state. If the firmware suggests * a different power state, use it instead. If power management * is not present, the firmware is responsible for managing * device power. Skip children who aren't attached since they * are handled separately. */ pcib = device_get_parent(dev); dstate = state; if (device_is_attached(child) && PCIB_POWER_FOR_SLEEP(pcib, child, &dstate) == 0) pci_set_powerstate(child, dstate); } int pci_suspend_child(device_t dev, device_t child) { struct pci_devinfo *dinfo; int error; dinfo = device_get_ivars(child); /* * Save the PCI configuration space for the child and set the * device in the appropriate power state for this sleep state. */ pci_cfg_save(child, dinfo, 0); /* Suspend devices before potentially powering them down. */ error = bus_generic_suspend_child(dev, child); if (error) return (error); if (pci_do_power_suspend) pci_set_power_child(dev, child, PCI_POWERSTATE_D3); return (0); } int pci_resume_child(device_t dev, device_t child) { struct pci_devinfo *dinfo; if (pci_do_power_resume) pci_set_power_child(dev, child, PCI_POWERSTATE_D0); dinfo = device_get_ivars(child); pci_cfg_restore(child, dinfo); if (!device_is_attached(child)) pci_cfg_save(child, dinfo, 1); bus_generic_resume_child(dev, child); return (0); } int pci_resume(device_t dev) { device_t child, *devlist; int error, i, numdevs; if ((error = device_get_children(dev, &devlist, &numdevs)) != 0) return (error); /* * Resume critical devices first, then everything else later. */ for (i = 0; i < numdevs; i++) { child = devlist[i]; switch (pci_get_class(child)) { case PCIC_DISPLAY: case PCIC_MEMORY: case PCIC_BRIDGE: case PCIC_BASEPERIPH: BUS_RESUME_CHILD(dev, child); break; } } for (i = 0; i < numdevs; i++) { child = devlist[i]; switch (pci_get_class(child)) { case PCIC_DISPLAY: case PCIC_MEMORY: case PCIC_BRIDGE: case PCIC_BASEPERIPH: break; default: BUS_RESUME_CHILD(dev, child); } } free(devlist, M_TEMP); return (0); } static void pci_load_vendor_data(void) { caddr_t data; void *ptr; size_t sz; data = preload_search_by_type("pci_vendor_data"); if (data != NULL) { ptr = preload_fetch_addr(data); sz = preload_fetch_size(data); if (ptr != NULL && sz != 0) { pci_vendordata = ptr; pci_vendordata_size = sz; /* terminate the database */ pci_vendordata[pci_vendordata_size] = '\n'; } } } void pci_driver_added(device_t dev, driver_t *driver) { int numdevs; device_t *devlist; device_t child; struct pci_devinfo *dinfo; int i; if (bootverbose) device_printf(dev, "driver added\n"); DEVICE_IDENTIFY(driver, dev); if (device_get_children(dev, &devlist, &numdevs) != 0) return; for (i = 0; i < numdevs; i++) { child = devlist[i]; if (device_get_state(child) != DS_NOTPRESENT) continue; dinfo = device_get_ivars(child); pci_print_verbose(dinfo); if (bootverbose) pci_printf(&dinfo->cfg, "reprobing on driver added\n"); pci_cfg_restore(child, dinfo); if (device_probe_and_attach(child) != 0) pci_child_detached(dev, child); } free(devlist, M_TEMP); } int pci_setup_intr(device_t dev, device_t child, struct resource *irq, int flags, driver_filter_t *filter, driver_intr_t *intr, void *arg, void **cookiep) { struct pci_devinfo *dinfo; struct msix_table_entry *mte; struct msix_vector *mv; uint64_t addr; uint32_t data; void *cookie; int error, rid; error = bus_generic_setup_intr(dev, child, irq, flags, filter, intr, arg, &cookie); if (error) return (error); /* If this is not a direct child, just bail out. */ if (device_get_parent(child) != dev) { *cookiep = cookie; return(0); } rid = rman_get_rid(irq); if (rid == 0) { /* Make sure that INTx is enabled */ pci_clear_command_bit(dev, child, PCIM_CMD_INTxDIS); } else { /* * Check to see if the interrupt is MSI or MSI-X. * Ask our parent to map the MSI and give * us the address and data register values. * If we fail for some reason, teardown the * interrupt handler. */ dinfo = device_get_ivars(child); if (dinfo->cfg.msi.msi_alloc > 0) { if (dinfo->cfg.msi.msi_addr == 0) { KASSERT(dinfo->cfg.msi.msi_handlers == 0, ("MSI has handlers, but vectors not mapped")); error = PCIB_MAP_MSI(device_get_parent(dev), child, rman_get_start(irq), &addr, &data); if (error) goto bad; dinfo->cfg.msi.msi_addr = addr; dinfo->cfg.msi.msi_data = data; } if (dinfo->cfg.msi.msi_handlers == 0) pci_enable_msi(child, dinfo->cfg.msi.msi_addr, dinfo->cfg.msi.msi_data); dinfo->cfg.msi.msi_handlers++; } else { KASSERT(dinfo->cfg.msix.msix_alloc > 0, ("No MSI or MSI-X interrupts allocated")); KASSERT(rid <= dinfo->cfg.msix.msix_table_len, ("MSI-X index too high")); mte = &dinfo->cfg.msix.msix_table[rid - 1]; KASSERT(mte->mte_vector != 0, ("no message vector")); mv = &dinfo->cfg.msix.msix_vectors[mte->mte_vector - 1]; KASSERT(mv->mv_irq == rman_get_start(irq), ("IRQ mismatch")); if (mv->mv_address == 0) { KASSERT(mte->mte_handlers == 0, ("MSI-X table entry has handlers, but vector not mapped")); error = PCIB_MAP_MSI(device_get_parent(dev), child, rman_get_start(irq), &addr, &data); if (error) goto bad; mv->mv_address = addr; mv->mv_data = data; } /* * The MSIX table entry must be made valid by * incrementing the mte_handlers before * calling pci_enable_msix() and * pci_resume_msix(). Else the MSIX rewrite * table quirk will not work as expected. */ mte->mte_handlers++; if (mte->mte_handlers == 1) { pci_enable_msix(child, rid - 1, mv->mv_address, mv->mv_data); pci_unmask_msix(child, rid - 1); } } /* * Make sure that INTx is disabled if we are using MSI/MSI-X, * unless the device is affected by PCI_QUIRK_MSI_INTX_BUG, * in which case we "enable" INTx so MSI/MSI-X actually works. */ if (!pci_has_quirk(pci_get_devid(child), PCI_QUIRK_MSI_INTX_BUG)) pci_set_command_bit(dev, child, PCIM_CMD_INTxDIS); else pci_clear_command_bit(dev, child, PCIM_CMD_INTxDIS); bad: if (error) { (void)bus_generic_teardown_intr(dev, child, irq, cookie); return (error); } } *cookiep = cookie; return (0); } int pci_teardown_intr(device_t dev, device_t child, struct resource *irq, void *cookie) { struct msix_table_entry *mte; struct resource_list_entry *rle; struct pci_devinfo *dinfo; int error, rid; if (irq == NULL || !(rman_get_flags(irq) & RF_ACTIVE)) return (EINVAL); /* If this isn't a direct child, just bail out */ if (device_get_parent(child) != dev) return(bus_generic_teardown_intr(dev, child, irq, cookie)); rid = rman_get_rid(irq); if (rid == 0) { /* Mask INTx */ pci_set_command_bit(dev, child, PCIM_CMD_INTxDIS); } else { /* * Check to see if the interrupt is MSI or MSI-X. If so, * decrement the appropriate handlers count and mask the * MSI-X message, or disable MSI messages if the count * drops to 0. */ dinfo = device_get_ivars(child); rle = resource_list_find(&dinfo->resources, SYS_RES_IRQ, rid); if (rle->res != irq) return (EINVAL); if (dinfo->cfg.msi.msi_alloc > 0) { KASSERT(rid <= dinfo->cfg.msi.msi_alloc, ("MSI-X index too high")); if (dinfo->cfg.msi.msi_handlers == 0) return (EINVAL); dinfo->cfg.msi.msi_handlers--; if (dinfo->cfg.msi.msi_handlers == 0) pci_disable_msi(child); } else { KASSERT(dinfo->cfg.msix.msix_alloc > 0, ("No MSI or MSI-X interrupts allocated")); KASSERT(rid <= dinfo->cfg.msix.msix_table_len, ("MSI-X index too high")); mte = &dinfo->cfg.msix.msix_table[rid - 1]; if (mte->mte_handlers == 0) return (EINVAL); mte->mte_handlers--; if (mte->mte_handlers == 0) pci_mask_msix(child, rid - 1); } } error = bus_generic_teardown_intr(dev, child, irq, cookie); if (rid > 0) KASSERT(error == 0, ("%s: generic teardown failed for MSI/MSI-X", __func__)); return (error); } int pci_print_child(device_t dev, device_t child) { struct pci_devinfo *dinfo; struct resource_list *rl; int retval = 0; dinfo = device_get_ivars(child); rl = &dinfo->resources; retval += bus_print_child_header(dev, child); retval += resource_list_print_type(rl, "port", SYS_RES_IOPORT, "%#jx"); retval += resource_list_print_type(rl, "mem", SYS_RES_MEMORY, "%#jx"); retval += resource_list_print_type(rl, "irq", SYS_RES_IRQ, "%jd"); if (device_get_flags(dev)) retval += printf(" flags %#x", device_get_flags(dev)); retval += printf(" at device %d.%d", pci_get_slot(child), pci_get_function(child)); retval += bus_print_child_domain(dev, child); retval += bus_print_child_footer(dev, child); return (retval); } static const struct { int class; int subclass; int report; /* 0 = bootverbose, 1 = always */ const char *desc; } pci_nomatch_tab[] = { {PCIC_OLD, -1, 1, "old"}, {PCIC_OLD, PCIS_OLD_NONVGA, 1, "non-VGA display device"}, {PCIC_OLD, PCIS_OLD_VGA, 1, "VGA-compatible display device"}, {PCIC_STORAGE, -1, 1, "mass storage"}, {PCIC_STORAGE, PCIS_STORAGE_SCSI, 1, "SCSI"}, {PCIC_STORAGE, PCIS_STORAGE_IDE, 1, "ATA"}, {PCIC_STORAGE, PCIS_STORAGE_FLOPPY, 1, "floppy disk"}, {PCIC_STORAGE, PCIS_STORAGE_IPI, 1, "IPI"}, {PCIC_STORAGE, PCIS_STORAGE_RAID, 1, "RAID"}, {PCIC_STORAGE, PCIS_STORAGE_ATA_ADMA, 1, "ATA (ADMA)"}, {PCIC_STORAGE, PCIS_STORAGE_SATA, 1, "SATA"}, {PCIC_STORAGE, PCIS_STORAGE_SAS, 1, "SAS"}, {PCIC_STORAGE, PCIS_STORAGE_NVM, 1, "NVM"}, {PCIC_NETWORK, -1, 1, "network"}, {PCIC_NETWORK, PCIS_NETWORK_ETHERNET, 1, "ethernet"}, {PCIC_NETWORK, PCIS_NETWORK_TOKENRING, 1, "token ring"}, {PCIC_NETWORK, PCIS_NETWORK_FDDI, 1, "fddi"}, {PCIC_NETWORK, PCIS_NETWORK_ATM, 1, "ATM"}, {PCIC_NETWORK, PCIS_NETWORK_ISDN, 1, "ISDN"}, {PCIC_DISPLAY, -1, 1, "display"}, {PCIC_DISPLAY, PCIS_DISPLAY_VGA, 1, "VGA"}, {PCIC_DISPLAY, PCIS_DISPLAY_XGA, 1, "XGA"}, {PCIC_DISPLAY, PCIS_DISPLAY_3D, 1, "3D"}, {PCIC_MULTIMEDIA, -1, 1, "multimedia"}, {PCIC_MULTIMEDIA, PCIS_MULTIMEDIA_VIDEO, 1, "video"}, {PCIC_MULTIMEDIA, PCIS_MULTIMEDIA_AUDIO, 1, "audio"}, {PCIC_MULTIMEDIA, PCIS_MULTIMEDIA_TELE, 1, "telephony"}, {PCIC_MULTIMEDIA, PCIS_MULTIMEDIA_HDA, 1, "HDA"}, {PCIC_MEMORY, -1, 1, "memory"}, {PCIC_MEMORY, PCIS_MEMORY_RAM, 1, "RAM"}, {PCIC_MEMORY, PCIS_MEMORY_FLASH, 1, "flash"}, {PCIC_BRIDGE, -1, 1, "bridge"}, {PCIC_BRIDGE, PCIS_BRIDGE_HOST, 1, "HOST-PCI"}, {PCIC_BRIDGE, PCIS_BRIDGE_ISA, 1, "PCI-ISA"}, {PCIC_BRIDGE, PCIS_BRIDGE_EISA, 1, "PCI-EISA"}, {PCIC_BRIDGE, PCIS_BRIDGE_MCA, 1, "PCI-MCA"}, {PCIC_BRIDGE, PCIS_BRIDGE_PCI, 1, "PCI-PCI"}, {PCIC_BRIDGE, PCIS_BRIDGE_PCMCIA, 1, "PCI-PCMCIA"}, {PCIC_BRIDGE, PCIS_BRIDGE_NUBUS, 1, "PCI-NuBus"}, {PCIC_BRIDGE, PCIS_BRIDGE_CARDBUS, 1, "PCI-CardBus"}, {PCIC_BRIDGE, PCIS_BRIDGE_RACEWAY, 1, "PCI-RACEway"}, {PCIC_SIMPLECOMM, -1, 1, "simple comms"}, {PCIC_SIMPLECOMM, PCIS_SIMPLECOMM_UART, 1, "UART"}, /* could detect 16550 */ {PCIC_SIMPLECOMM, PCIS_SIMPLECOMM_PAR, 1, "parallel port"}, {PCIC_SIMPLECOMM, PCIS_SIMPLECOMM_MULSER, 1, "multiport serial"}, {PCIC_SIMPLECOMM, PCIS_SIMPLECOMM_MODEM, 1, "generic modem"}, {PCIC_BASEPERIPH, -1, 0, "base peripheral"}, {PCIC_BASEPERIPH, PCIS_BASEPERIPH_PIC, 1, "interrupt controller"}, {PCIC_BASEPERIPH, PCIS_BASEPERIPH_DMA, 1, "DMA controller"}, {PCIC_BASEPERIPH, PCIS_BASEPERIPH_TIMER, 1, "timer"}, {PCIC_BASEPERIPH, PCIS_BASEPERIPH_RTC, 1, "realtime clock"}, {PCIC_BASEPERIPH, PCIS_BASEPERIPH_PCIHOT, 1, "PCI hot-plug controller"}, {PCIC_BASEPERIPH, PCIS_BASEPERIPH_SDHC, 1, "SD host controller"}, {PCIC_BASEPERIPH, PCIS_BASEPERIPH_IOMMU, 1, "IOMMU"}, {PCIC_INPUTDEV, -1, 1, "input device"}, {PCIC_INPUTDEV, PCIS_INPUTDEV_KEYBOARD, 1, "keyboard"}, {PCIC_INPUTDEV, PCIS_INPUTDEV_DIGITIZER,1, "digitizer"}, {PCIC_INPUTDEV, PCIS_INPUTDEV_MOUSE, 1, "mouse"}, {PCIC_INPUTDEV, PCIS_INPUTDEV_SCANNER, 1, "scanner"}, {PCIC_INPUTDEV, PCIS_INPUTDEV_GAMEPORT, 1, "gameport"}, {PCIC_DOCKING, -1, 1, "docking station"}, {PCIC_PROCESSOR, -1, 1, "processor"}, {PCIC_SERIALBUS, -1, 1, "serial bus"}, {PCIC_SERIALBUS, PCIS_SERIALBUS_FW, 1, "FireWire"}, {PCIC_SERIALBUS, PCIS_SERIALBUS_ACCESS, 1, "AccessBus"}, {PCIC_SERIALBUS, PCIS_SERIALBUS_SSA, 1, "SSA"}, {PCIC_SERIALBUS, PCIS_SERIALBUS_USB, 1, "USB"}, {PCIC_SERIALBUS, PCIS_SERIALBUS_FC, 1, "Fibre Channel"}, {PCIC_SERIALBUS, PCIS_SERIALBUS_SMBUS, 0, "SMBus"}, {PCIC_WIRELESS, -1, 1, "wireless controller"}, {PCIC_WIRELESS, PCIS_WIRELESS_IRDA, 1, "iRDA"}, {PCIC_WIRELESS, PCIS_WIRELESS_IR, 1, "IR"}, {PCIC_WIRELESS, PCIS_WIRELESS_RF, 1, "RF"}, {PCIC_INTELLIIO, -1, 1, "intelligent I/O controller"}, {PCIC_INTELLIIO, PCIS_INTELLIIO_I2O, 1, "I2O"}, {PCIC_SATCOM, -1, 1, "satellite communication"}, {PCIC_SATCOM, PCIS_SATCOM_TV, 1, "sat TV"}, {PCIC_SATCOM, PCIS_SATCOM_AUDIO, 1, "sat audio"}, {PCIC_SATCOM, PCIS_SATCOM_VOICE, 1, "sat voice"}, {PCIC_SATCOM, PCIS_SATCOM_DATA, 1, "sat data"}, {PCIC_CRYPTO, -1, 1, "encrypt/decrypt"}, {PCIC_CRYPTO, PCIS_CRYPTO_NETCOMP, 1, "network/computer crypto"}, {PCIC_CRYPTO, PCIS_CRYPTO_ENTERTAIN, 1, "entertainment crypto"}, {PCIC_DASP, -1, 0, "dasp"}, {PCIC_DASP, PCIS_DASP_DPIO, 1, "DPIO module"}, {PCIC_DASP, PCIS_DASP_PERFCNTRS, 1, "performance counters"}, {PCIC_DASP, PCIS_DASP_COMM_SYNC, 1, "communication synchronizer"}, {PCIC_DASP, PCIS_DASP_MGMT_CARD, 1, "signal processing management"}, {0, 0, 0, NULL} }; void pci_probe_nomatch(device_t dev, device_t child) { int i, report; const char *cp, *scp; char *device; /* * Look for a listing for this device in a loaded device database. */ report = 1; if ((device = pci_describe_device(child)) != NULL) { device_printf(dev, "<%s>", device); free(device, M_DEVBUF); } else { /* * Scan the class/subclass descriptions for a general * description. */ cp = "unknown"; scp = NULL; for (i = 0; pci_nomatch_tab[i].desc != NULL; i++) { if (pci_nomatch_tab[i].class == pci_get_class(child)) { if (pci_nomatch_tab[i].subclass == -1) { cp = pci_nomatch_tab[i].desc; report = pci_nomatch_tab[i].report; } else if (pci_nomatch_tab[i].subclass == pci_get_subclass(child)) { scp = pci_nomatch_tab[i].desc; report = pci_nomatch_tab[i].report; } } } if (report || bootverbose) { device_printf(dev, "<%s%s%s>", cp ? cp : "", ((cp != NULL) && (scp != NULL)) ? ", " : "", scp ? scp : ""); } } if (report || bootverbose) { printf(" at device %d.%d (no driver attached)\n", pci_get_slot(child), pci_get_function(child)); } pci_cfg_save(child, device_get_ivars(child), 1); } void pci_child_detached(device_t dev, device_t child) { struct pci_devinfo *dinfo; struct resource_list *rl; dinfo = device_get_ivars(child); rl = &dinfo->resources; /* * Have to deallocate IRQs before releasing any MSI messages and * have to release MSI messages before deallocating any memory * BARs. */ if (resource_list_release_active(rl, dev, child, SYS_RES_IRQ) != 0) pci_printf(&dinfo->cfg, "Device leaked IRQ resources\n"); if (dinfo->cfg.msi.msi_alloc != 0 || dinfo->cfg.msix.msix_alloc != 0) { pci_printf(&dinfo->cfg, "Device leaked MSI vectors\n"); (void)pci_release_msi(child); } if (resource_list_release_active(rl, dev, child, SYS_RES_MEMORY) != 0) pci_printf(&dinfo->cfg, "Device leaked memory resources\n"); if (resource_list_release_active(rl, dev, child, SYS_RES_IOPORT) != 0) pci_printf(&dinfo->cfg, "Device leaked I/O resources\n"); #ifdef PCI_RES_BUS if (resource_list_release_active(rl, dev, child, PCI_RES_BUS) != 0) pci_printf(&dinfo->cfg, "Device leaked PCI bus numbers\n"); #endif pci_cfg_save(child, dinfo, 1); } /* * Parse the PCI device database, if loaded, and return a pointer to a * description of the device. * * The database is flat text formatted as follows: * * Any line not in a valid format is ignored. * Lines are terminated with newline '\n' characters. * * A VENDOR line consists of the 4 digit (hex) vendor code, a TAB, then * the vendor name. * * A DEVICE line is entered immediately below the corresponding VENDOR ID. * - devices cannot be listed without a corresponding VENDOR line. * A DEVICE line consists of a TAB, the 4 digit (hex) device code, * another TAB, then the device name. */ /* * Assuming (ptr) points to the beginning of a line in the database, * return the vendor or device and description of the next entry. * The value of (vendor) or (device) inappropriate for the entry type * is set to -1. Returns nonzero at the end of the database. * * Note that this is slightly unrobust in the face of corrupt data; * we attempt to safeguard against this by spamming the end of the * database with a newline when we initialise. */ static int pci_describe_parse_line(char **ptr, int *vendor, int *device, char **desc) { char *cp = *ptr; int left; *device = -1; *vendor = -1; **desc = '\0'; for (;;) { left = pci_vendordata_size - (cp - pci_vendordata); if (left <= 0) { *ptr = cp; return(1); } /* vendor entry? */ if (*cp != '\t' && sscanf(cp, "%x\t%80[^\n]", vendor, *desc) == 2) break; /* device entry? */ if (*cp == '\t' && sscanf(cp, "%x\t%80[^\n]", device, *desc) == 2) break; /* skip to next line */ while (*cp != '\n' && left > 0) { cp++; left--; } if (*cp == '\n') { cp++; left--; } } /* skip to next line */ while (*cp != '\n' && left > 0) { cp++; left--; } if (*cp == '\n' && left > 0) cp++; *ptr = cp; return(0); } static char * pci_describe_device(device_t dev) { int vendor, device; char *desc, *vp, *dp, *line; desc = vp = dp = NULL; /* * If we have no vendor data, we can't do anything. */ if (pci_vendordata == NULL) goto out; /* * Scan the vendor data looking for this device */ line = pci_vendordata; if ((vp = malloc(80, M_DEVBUF, M_NOWAIT)) == NULL) goto out; for (;;) { if (pci_describe_parse_line(&line, &vendor, &device, &vp)) goto out; if (vendor == pci_get_vendor(dev)) break; } if ((dp = malloc(80, M_DEVBUF, M_NOWAIT)) == NULL) goto out; for (;;) { if (pci_describe_parse_line(&line, &vendor, &device, &dp)) { *dp = 0; break; } if (vendor != -1) { *dp = 0; break; } if (device == pci_get_device(dev)) break; } if (dp[0] == '\0') snprintf(dp, 80, "0x%x", pci_get_device(dev)); if ((desc = malloc(strlen(vp) + strlen(dp) + 3, M_DEVBUF, M_NOWAIT)) != NULL) sprintf(desc, "%s, %s", vp, dp); out: if (vp != NULL) free(vp, M_DEVBUF); if (dp != NULL) free(dp, M_DEVBUF); return(desc); } int pci_read_ivar(device_t dev, device_t child, int which, uintptr_t *result) { struct pci_devinfo *dinfo; pcicfgregs *cfg; dinfo = device_get_ivars(child); cfg = &dinfo->cfg; switch (which) { case PCI_IVAR_ETHADDR: /* * The generic accessor doesn't deal with failure, so * we set the return value, then return an error. */ *((uint8_t **) result) = NULL; return (EINVAL); case PCI_IVAR_SUBVENDOR: *result = cfg->subvendor; break; case PCI_IVAR_SUBDEVICE: *result = cfg->subdevice; break; case PCI_IVAR_VENDOR: *result = cfg->vendor; break; case PCI_IVAR_DEVICE: *result = cfg->device; break; case PCI_IVAR_DEVID: *result = (cfg->device << 16) | cfg->vendor; break; case PCI_IVAR_CLASS: *result = cfg->baseclass; break; case PCI_IVAR_SUBCLASS: *result = cfg->subclass; break; case PCI_IVAR_PROGIF: *result = cfg->progif; break; case PCI_IVAR_REVID: *result = cfg->revid; break; case PCI_IVAR_INTPIN: *result = cfg->intpin; break; case PCI_IVAR_IRQ: *result = cfg->intline; break; case PCI_IVAR_DOMAIN: *result = cfg->domain; break; case PCI_IVAR_BUS: *result = cfg->bus; break; case PCI_IVAR_SLOT: *result = cfg->slot; break; case PCI_IVAR_FUNCTION: *result = cfg->func; break; case PCI_IVAR_CMDREG: *result = cfg->cmdreg; break; case PCI_IVAR_CACHELNSZ: *result = cfg->cachelnsz; break; case PCI_IVAR_MINGNT: if (cfg->hdrtype != PCIM_HDRTYPE_NORMAL) { *result = -1; return (EINVAL); } *result = cfg->mingnt; break; case PCI_IVAR_MAXLAT: if (cfg->hdrtype != PCIM_HDRTYPE_NORMAL) { *result = -1; return (EINVAL); } *result = cfg->maxlat; break; case PCI_IVAR_LATTIMER: *result = cfg->lattimer; break; default: return (ENOENT); } return (0); } int pci_write_ivar(device_t dev, device_t child, int which, uintptr_t value) { struct pci_devinfo *dinfo; dinfo = device_get_ivars(child); switch (which) { case PCI_IVAR_INTPIN: dinfo->cfg.intpin = value; return (0); case PCI_IVAR_ETHADDR: case PCI_IVAR_SUBVENDOR: case PCI_IVAR_SUBDEVICE: case PCI_IVAR_VENDOR: case PCI_IVAR_DEVICE: case PCI_IVAR_DEVID: case PCI_IVAR_CLASS: case PCI_IVAR_SUBCLASS: case PCI_IVAR_PROGIF: case PCI_IVAR_REVID: case PCI_IVAR_IRQ: case PCI_IVAR_DOMAIN: case PCI_IVAR_BUS: case PCI_IVAR_SLOT: case PCI_IVAR_FUNCTION: return (EINVAL); /* disallow for now */ default: return (ENOENT); } } #include "opt_ddb.h" #ifdef DDB #include #include /* * List resources based on pci map registers, used for within ddb */ DB_SHOW_COMMAND(pciregs, db_pci_dump) { struct pci_devinfo *dinfo; struct devlist *devlist_head; struct pci_conf *p; const char *name; int i, error, none_count; none_count = 0; /* get the head of the device queue */ devlist_head = &pci_devq; /* * Go through the list of devices and print out devices */ for (error = 0, i = 0, dinfo = STAILQ_FIRST(devlist_head); (dinfo != NULL) && (error == 0) && (i < pci_numdevs) && !db_pager_quit; dinfo = STAILQ_NEXT(dinfo, pci_links), i++) { /* Populate pd_name and pd_unit */ name = NULL; if (dinfo->cfg.dev) name = device_get_name(dinfo->cfg.dev); p = &dinfo->conf; db_printf("%s%d@pci%d:%d:%d:%d:\tclass=0x%06x card=0x%08x " "chip=0x%08x rev=0x%02x hdr=0x%02x\n", (name && *name) ? name : "none", (name && *name) ? (int)device_get_unit(dinfo->cfg.dev) : none_count++, p->pc_sel.pc_domain, p->pc_sel.pc_bus, p->pc_sel.pc_dev, p->pc_sel.pc_func, (p->pc_class << 16) | (p->pc_subclass << 8) | p->pc_progif, (p->pc_subdevice << 16) | p->pc_subvendor, (p->pc_device << 16) | p->pc_vendor, p->pc_revid, p->pc_hdr); } } #endif /* DDB */ static struct resource * pci_reserve_map(device_t dev, device_t child, int type, int *rid, rman_res_t start, rman_res_t end, rman_res_t count, u_int num, u_int flags) { struct pci_devinfo *dinfo = device_get_ivars(child); struct resource_list *rl = &dinfo->resources; struct resource *res; struct pci_map *pm; uint16_t cmd; pci_addr_t map, testval; int mapsize; res = NULL; /* If rid is managed by EA, ignore it */ if (pci_ea_is_enabled(child, *rid)) goto out; pm = pci_find_bar(child, *rid); if (pm != NULL) { /* This is a BAR that we failed to allocate earlier. */ mapsize = pm->pm_size; map = pm->pm_value; } else { /* * Weed out the bogons, and figure out how large the * BAR/map is. BARs that read back 0 here are bogus * and unimplemented. Note: atapci in legacy mode are * special and handled elsewhere in the code. If you * have a atapci device in legacy mode and it fails * here, that other code is broken. */ pci_read_bar(child, *rid, &map, &testval, NULL); /* * Determine the size of the BAR and ignore BARs with a size * of 0. Device ROM BARs use a different mask value. */ if (PCIR_IS_BIOS(&dinfo->cfg, *rid)) mapsize = pci_romsize(testval); else mapsize = pci_mapsize(testval); if (mapsize == 0) goto out; pm = pci_add_bar(child, *rid, map, mapsize); } if (PCI_BAR_MEM(map) || PCIR_IS_BIOS(&dinfo->cfg, *rid)) { if (type != SYS_RES_MEMORY) { if (bootverbose) device_printf(dev, "child %s requested type %d for rid %#x," " but the BAR says it is an memio\n", device_get_nameunit(child), type, *rid); goto out; } } else { if (type != SYS_RES_IOPORT) { if (bootverbose) device_printf(dev, "child %s requested type %d for rid %#x," " but the BAR says it is an ioport\n", device_get_nameunit(child), type, *rid); goto out; } } /* * For real BARs, we need to override the size that * the driver requests, because that's what the BAR * actually uses and we would otherwise have a * situation where we might allocate the excess to * another driver, which won't work. */ count = ((pci_addr_t)1 << mapsize) * num; if (RF_ALIGNMENT(flags) < mapsize) flags = (flags & ~RF_ALIGNMENT_MASK) | RF_ALIGNMENT_LOG2(mapsize); if (PCI_BAR_MEM(map) && (map & PCIM_BAR_MEM_PREFETCH)) flags |= RF_PREFETCHABLE; /* * Allocate enough resource, and then write back the * appropriate BAR for that resource. */ resource_list_add(rl, type, *rid, start, end, count); res = resource_list_reserve(rl, dev, child, type, rid, start, end, count, flags & ~RF_ACTIVE); if (res == NULL) { resource_list_delete(rl, type, *rid); device_printf(child, "%#jx bytes of rid %#x res %d failed (%#jx, %#jx).\n", count, *rid, type, start, end); goto out; } if (bootverbose) device_printf(child, "Lazy allocation of %#jx bytes rid %#x type %d at %#jx\n", count, *rid, type, rman_get_start(res)); /* Disable decoding via the CMD register before updating the BAR */ cmd = pci_read_config(child, PCIR_COMMAND, 2); pci_write_config(child, PCIR_COMMAND, cmd & ~(PCI_BAR_MEM(map) ? PCIM_CMD_MEMEN : PCIM_CMD_PORTEN), 2); map = rman_get_start(res); pci_write_bar(child, pm, map); /* Restore the original value of the CMD register */ pci_write_config(child, PCIR_COMMAND, cmd, 2); out: return (res); } struct resource * pci_alloc_multi_resource(device_t dev, device_t child, int type, int *rid, rman_res_t start, rman_res_t end, rman_res_t count, u_long num, u_int flags) { struct pci_devinfo *dinfo; struct resource_list *rl; struct resource_list_entry *rle; struct resource *res; pcicfgregs *cfg; /* * Perform lazy resource allocation */ dinfo = device_get_ivars(child); rl = &dinfo->resources; cfg = &dinfo->cfg; switch (type) { #if defined(NEW_PCIB) && defined(PCI_RES_BUS) case PCI_RES_BUS: return (pci_alloc_secbus(dev, child, rid, start, end, count, flags)); #endif case SYS_RES_IRQ: /* * Can't alloc legacy interrupt once MSI messages have * been allocated. */ if (*rid == 0 && (cfg->msi.msi_alloc > 0 || cfg->msix.msix_alloc > 0)) return (NULL); /* * If the child device doesn't have an interrupt * routed and is deserving of an interrupt, try to * assign it one. */ if (*rid == 0 && !PCI_INTERRUPT_VALID(cfg->intline) && (cfg->intpin != 0)) pci_assign_interrupt(dev, child, 0); break; case SYS_RES_IOPORT: case SYS_RES_MEMORY: #ifdef NEW_PCIB /* * PCI-PCI bridge I/O window resources are not BARs. * For those allocations just pass the request up the * tree. */ if (cfg->hdrtype == PCIM_HDRTYPE_BRIDGE) { switch (*rid) { case PCIR_IOBASEL_1: case PCIR_MEMBASE_1: case PCIR_PMBASEL_1: /* * XXX: Should we bother creating a resource * list entry? */ return (bus_generic_alloc_resource(dev, child, type, rid, start, end, count, flags)); } } #endif /* Reserve resources for this BAR if needed. */ rle = resource_list_find(rl, type, *rid); if (rle == NULL) { res = pci_reserve_map(dev, child, type, rid, start, end, count, num, flags); if (res == NULL) return (NULL); } } return (resource_list_alloc(rl, dev, child, type, rid, start, end, count, flags)); } struct resource * pci_alloc_resource(device_t dev, device_t child, int type, int *rid, rman_res_t start, rman_res_t end, rman_res_t count, u_int flags) { #ifdef PCI_IOV struct pci_devinfo *dinfo; #endif if (device_get_parent(child) != dev) return (BUS_ALLOC_RESOURCE(device_get_parent(dev), child, type, rid, start, end, count, flags)); #ifdef PCI_IOV dinfo = device_get_ivars(child); if (dinfo->cfg.flags & PCICFG_VF) { switch (type) { /* VFs can't have I/O BARs. */ case SYS_RES_IOPORT: return (NULL); case SYS_RES_MEMORY: return (pci_vf_alloc_mem_resource(dev, child, rid, start, end, count, flags)); } /* Fall through for other types of resource allocations. */ } #endif return (pci_alloc_multi_resource(dev, child, type, rid, start, end, count, 1, flags)); } int pci_release_resource(device_t dev, device_t child, int type, int rid, struct resource *r) { struct pci_devinfo *dinfo; struct resource_list *rl; pcicfgregs *cfg; if (device_get_parent(child) != dev) return (BUS_RELEASE_RESOURCE(device_get_parent(dev), child, type, rid, r)); dinfo = device_get_ivars(child); cfg = &dinfo->cfg; #ifdef PCI_IOV if (dinfo->cfg.flags & PCICFG_VF) { switch (type) { /* VFs can't have I/O BARs. */ case SYS_RES_IOPORT: return (EDOOFUS); case SYS_RES_MEMORY: return (pci_vf_release_mem_resource(dev, child, rid, r)); } /* Fall through for other types of resource allocations. */ } #endif #ifdef NEW_PCIB /* * PCI-PCI bridge I/O window resources are not BARs. For * those allocations just pass the request up the tree. */ if (cfg->hdrtype == PCIM_HDRTYPE_BRIDGE && (type == SYS_RES_IOPORT || type == SYS_RES_MEMORY)) { switch (rid) { case PCIR_IOBASEL_1: case PCIR_MEMBASE_1: case PCIR_PMBASEL_1: return (bus_generic_release_resource(dev, child, type, rid, r)); } } #endif rl = &dinfo->resources; return (resource_list_release(rl, dev, child, type, rid, r)); } int pci_activate_resource(device_t dev, device_t child, int type, int rid, struct resource *r) { struct pci_devinfo *dinfo; int error; error = bus_generic_activate_resource(dev, child, type, rid, r); if (error) return (error); /* Enable decoding in the command register when activating BARs. */ if (device_get_parent(child) == dev) { /* Device ROMs need their decoding explicitly enabled. */ dinfo = device_get_ivars(child); if (type == SYS_RES_MEMORY && PCIR_IS_BIOS(&dinfo->cfg, rid)) pci_write_bar(child, pci_find_bar(child, rid), rman_get_start(r) | PCIM_BIOS_ENABLE); switch (type) { case SYS_RES_IOPORT: case SYS_RES_MEMORY: error = PCI_ENABLE_IO(dev, child, type); break; } } return (error); } int pci_deactivate_resource(device_t dev, device_t child, int type, int rid, struct resource *r) { struct pci_devinfo *dinfo; int error; error = bus_generic_deactivate_resource(dev, child, type, rid, r); if (error) return (error); /* Disable decoding for device ROMs. */ if (device_get_parent(child) == dev) { dinfo = device_get_ivars(child); if (type == SYS_RES_MEMORY && PCIR_IS_BIOS(&dinfo->cfg, rid)) pci_write_bar(child, pci_find_bar(child, rid), rman_get_start(r)); } return (0); } void pci_child_deleted(device_t dev, device_t child) { struct resource_list_entry *rle; struct resource_list *rl; struct pci_devinfo *dinfo; dinfo = device_get_ivars(child); rl = &dinfo->resources; EVENTHANDLER_INVOKE(pci_delete_device, child); /* Turn off access to resources we're about to free */ if (bus_child_present(child) != 0) { pci_write_config(child, PCIR_COMMAND, pci_read_config(child, PCIR_COMMAND, 2) & ~(PCIM_CMD_MEMEN | PCIM_CMD_PORTEN), 2); pci_disable_busmaster(child); } /* Free all allocated resources */ STAILQ_FOREACH(rle, rl, link) { if (rle->res) { if (rman_get_flags(rle->res) & RF_ACTIVE || resource_list_busy(rl, rle->type, rle->rid)) { pci_printf(&dinfo->cfg, "Resource still owned, oops. " "(type=%d, rid=%d, addr=%lx)\n", rle->type, rle->rid, rman_get_start(rle->res)); bus_release_resource(child, rle->type, rle->rid, rle->res); } resource_list_unreserve(rl, dev, child, rle->type, rle->rid); } } resource_list_free(rl); pci_freecfg(dinfo); } void pci_delete_resource(device_t dev, device_t child, int type, int rid) { struct pci_devinfo *dinfo; struct resource_list *rl; struct resource_list_entry *rle; if (device_get_parent(child) != dev) return; dinfo = device_get_ivars(child); rl = &dinfo->resources; rle = resource_list_find(rl, type, rid); if (rle == NULL) return; if (rle->res) { if (rman_get_flags(rle->res) & RF_ACTIVE || resource_list_busy(rl, type, rid)) { device_printf(dev, "delete_resource: " "Resource still owned by child, oops. " "(type=%d, rid=%d, addr=%jx)\n", type, rid, rman_get_start(rle->res)); return; } resource_list_unreserve(rl, dev, child, type, rid); } resource_list_delete(rl, type, rid); } struct resource_list * pci_get_resource_list (device_t dev, device_t child) { struct pci_devinfo *dinfo = device_get_ivars(child); return (&dinfo->resources); } bus_dma_tag_t pci_get_dma_tag(device_t bus, device_t dev) { struct pci_softc *sc = device_get_softc(bus); return (sc->sc_dma_tag); } uint32_t pci_read_config_method(device_t dev, device_t child, int reg, int width) { struct pci_devinfo *dinfo = device_get_ivars(child); pcicfgregs *cfg = &dinfo->cfg; #ifdef PCI_IOV /* * SR-IOV VFs don't implement the VID or DID registers, so we have to * emulate them here. */ if (cfg->flags & PCICFG_VF) { if (reg == PCIR_VENDOR) { switch (width) { case 4: return (cfg->device << 16 | cfg->vendor); case 2: return (cfg->vendor); case 1: return (cfg->vendor & 0xff); default: return (0xffffffff); } } else if (reg == PCIR_DEVICE) { switch (width) { /* Note that an unaligned 4-byte read is an error. */ case 2: return (cfg->device); case 1: return (cfg->device & 0xff); default: return (0xffffffff); } } } #endif return (PCIB_READ_CONFIG(device_get_parent(dev), cfg->bus, cfg->slot, cfg->func, reg, width)); } void pci_write_config_method(device_t dev, device_t child, int reg, uint32_t val, int width) { struct pci_devinfo *dinfo = device_get_ivars(child); pcicfgregs *cfg = &dinfo->cfg; PCIB_WRITE_CONFIG(device_get_parent(dev), cfg->bus, cfg->slot, cfg->func, reg, val, width); } int pci_child_location_str_method(device_t dev, device_t child, char *buf, size_t buflen) { snprintf(buf, buflen, "slot=%d function=%d dbsf=pci%d:%d:%d:%d", pci_get_slot(child), pci_get_function(child), pci_get_domain(child), pci_get_bus(child), pci_get_slot(child), pci_get_function(child)); return (0); } int pci_child_pnpinfo_str_method(device_t dev, device_t child, char *buf, size_t buflen) { struct pci_devinfo *dinfo; pcicfgregs *cfg; dinfo = device_get_ivars(child); cfg = &dinfo->cfg; snprintf(buf, buflen, "vendor=0x%04x device=0x%04x subvendor=0x%04x " "subdevice=0x%04x class=0x%02x%02x%02x", cfg->vendor, cfg->device, cfg->subvendor, cfg->subdevice, cfg->baseclass, cfg->subclass, cfg->progif); return (0); } int pci_assign_interrupt_method(device_t dev, device_t child) { struct pci_devinfo *dinfo = device_get_ivars(child); pcicfgregs *cfg = &dinfo->cfg; return (PCIB_ROUTE_INTERRUPT(device_get_parent(dev), child, cfg->intpin)); } static void pci_lookup(void *arg, const char *name, device_t *dev) { long val; char *end; int domain, bus, slot, func; if (*dev != NULL) return; /* * Accept pciconf-style selectors of either pciD:B:S:F or * pciB:S:F. In the latter case, the domain is assumed to * be zero. */ if (strncmp(name, "pci", 3) != 0) return; val = strtol(name + 3, &end, 10); if (val < 0 || val > INT_MAX || *end != ':') return; domain = val; val = strtol(end + 1, &end, 10); if (val < 0 || val > INT_MAX || *end != ':') return; bus = val; val = strtol(end + 1, &end, 10); if (val < 0 || val > INT_MAX) return; slot = val; if (*end == ':') { val = strtol(end + 1, &end, 10); if (val < 0 || val > INT_MAX || *end != '\0') return; func = val; } else if (*end == '\0') { func = slot; slot = bus; bus = domain; domain = 0; } else return; if (domain > PCI_DOMAINMAX || bus > PCI_BUSMAX || slot > PCI_SLOTMAX || func > PCIE_ARI_FUNCMAX || (slot != 0 && func > PCI_FUNCMAX)) return; *dev = pci_find_dbsf(domain, bus, slot, func); } static int pci_modevent(module_t mod, int what, void *arg) { static struct cdev *pci_cdev; static eventhandler_tag tag; switch (what) { case MOD_LOAD: STAILQ_INIT(&pci_devq); pci_generation = 0; pci_cdev = make_dev(&pcicdev, 0, UID_ROOT, GID_WHEEL, 0644, "pci"); pci_load_vendor_data(); tag = EVENTHANDLER_REGISTER(dev_lookup, pci_lookup, NULL, 1000); break; case MOD_UNLOAD: if (tag != NULL) EVENTHANDLER_DEREGISTER(dev_lookup, tag); destroy_dev(pci_cdev); break; } return (0); } static void pci_cfg_restore_pcie(device_t dev, struct pci_devinfo *dinfo) { #define WREG(n, v) pci_write_config(dev, pos + (n), (v), 2) struct pcicfg_pcie *cfg; int version, pos; cfg = &dinfo->cfg.pcie; pos = cfg->pcie_location; version = cfg->pcie_flags & PCIEM_FLAGS_VERSION; WREG(PCIER_DEVICE_CTL, cfg->pcie_device_ctl); if (version > 1 || cfg->pcie_type == PCIEM_TYPE_ROOT_PORT || cfg->pcie_type == PCIEM_TYPE_ENDPOINT || cfg->pcie_type == PCIEM_TYPE_LEGACY_ENDPOINT) WREG(PCIER_LINK_CTL, cfg->pcie_link_ctl); if (version > 1 || (cfg->pcie_type == PCIEM_TYPE_ROOT_PORT || (cfg->pcie_type == PCIEM_TYPE_DOWNSTREAM_PORT && (cfg->pcie_flags & PCIEM_FLAGS_SLOT)))) WREG(PCIER_SLOT_CTL, cfg->pcie_slot_ctl); if (version > 1 || cfg->pcie_type == PCIEM_TYPE_ROOT_PORT || cfg->pcie_type == PCIEM_TYPE_ROOT_EC) WREG(PCIER_ROOT_CTL, cfg->pcie_root_ctl); if (version > 1) { WREG(PCIER_DEVICE_CTL2, cfg->pcie_device_ctl2); WREG(PCIER_LINK_CTL2, cfg->pcie_link_ctl2); WREG(PCIER_SLOT_CTL2, cfg->pcie_slot_ctl2); } #undef WREG } static void pci_cfg_restore_pcix(device_t dev, struct pci_devinfo *dinfo) { pci_write_config(dev, dinfo->cfg.pcix.pcix_location + PCIXR_COMMAND, dinfo->cfg.pcix.pcix_command, 2); } void pci_cfg_restore(device_t dev, struct pci_devinfo *dinfo) { /* * Restore the device to full power mode. We must do this * before we restore the registers because moving from D3 to * D0 will cause the chip's BARs and some other registers to * be reset to some unknown power on reset values. Cut down * the noise on boot by doing nothing if we are already in * state D0. */ if (pci_get_powerstate(dev) != PCI_POWERSTATE_D0) pci_set_powerstate(dev, PCI_POWERSTATE_D0); pci_write_config(dev, PCIR_COMMAND, dinfo->cfg.cmdreg, 2); pci_write_config(dev, PCIR_INTLINE, dinfo->cfg.intline, 1); pci_write_config(dev, PCIR_INTPIN, dinfo->cfg.intpin, 1); pci_write_config(dev, PCIR_CACHELNSZ, dinfo->cfg.cachelnsz, 1); pci_write_config(dev, PCIR_LATTIMER, dinfo->cfg.lattimer, 1); pci_write_config(dev, PCIR_PROGIF, dinfo->cfg.progif, 1); pci_write_config(dev, PCIR_REVID, dinfo->cfg.revid, 1); switch (dinfo->cfg.hdrtype & PCIM_HDRTYPE) { case PCIM_HDRTYPE_NORMAL: pci_write_config(dev, PCIR_MINGNT, dinfo->cfg.mingnt, 1); pci_write_config(dev, PCIR_MAXLAT, dinfo->cfg.maxlat, 1); break; case PCIM_HDRTYPE_BRIDGE: pci_write_config(dev, PCIR_SECLAT_1, dinfo->cfg.bridge.br_seclat, 1); pci_write_config(dev, PCIR_SUBBUS_1, dinfo->cfg.bridge.br_subbus, 1); pci_write_config(dev, PCIR_SECBUS_1, dinfo->cfg.bridge.br_secbus, 1); pci_write_config(dev, PCIR_PRIBUS_1, dinfo->cfg.bridge.br_pribus, 1); pci_write_config(dev, PCIR_BRIDGECTL_1, dinfo->cfg.bridge.br_control, 2); break; case PCIM_HDRTYPE_CARDBUS: pci_write_config(dev, PCIR_SECLAT_2, dinfo->cfg.bridge.br_seclat, 1); pci_write_config(dev, PCIR_SUBBUS_2, dinfo->cfg.bridge.br_subbus, 1); pci_write_config(dev, PCIR_SECBUS_2, dinfo->cfg.bridge.br_secbus, 1); pci_write_config(dev, PCIR_PRIBUS_2, dinfo->cfg.bridge.br_pribus, 1); pci_write_config(dev, PCIR_BRIDGECTL_2, dinfo->cfg.bridge.br_control, 2); break; } pci_restore_bars(dev); /* * Restore extended capabilities for PCI-Express and PCI-X */ if (dinfo->cfg.pcie.pcie_location != 0) pci_cfg_restore_pcie(dev, dinfo); if (dinfo->cfg.pcix.pcix_location != 0) pci_cfg_restore_pcix(dev, dinfo); /* Restore MSI and MSI-X configurations if they are present. */ if (dinfo->cfg.msi.msi_location != 0) pci_resume_msi(dev); if (dinfo->cfg.msix.msix_location != 0) pci_resume_msix(dev); #ifdef PCI_IOV if (dinfo->cfg.iov != NULL) pci_iov_cfg_restore(dev, dinfo); #endif } static void pci_cfg_save_pcie(device_t dev, struct pci_devinfo *dinfo) { #define RREG(n) pci_read_config(dev, pos + (n), 2) struct pcicfg_pcie *cfg; int version, pos; cfg = &dinfo->cfg.pcie; pos = cfg->pcie_location; cfg->pcie_flags = RREG(PCIER_FLAGS); version = cfg->pcie_flags & PCIEM_FLAGS_VERSION; cfg->pcie_device_ctl = RREG(PCIER_DEVICE_CTL); if (version > 1 || cfg->pcie_type == PCIEM_TYPE_ROOT_PORT || cfg->pcie_type == PCIEM_TYPE_ENDPOINT || cfg->pcie_type == PCIEM_TYPE_LEGACY_ENDPOINT) cfg->pcie_link_ctl = RREG(PCIER_LINK_CTL); if (version > 1 || (cfg->pcie_type == PCIEM_TYPE_ROOT_PORT || (cfg->pcie_type == PCIEM_TYPE_DOWNSTREAM_PORT && (cfg->pcie_flags & PCIEM_FLAGS_SLOT)))) cfg->pcie_slot_ctl = RREG(PCIER_SLOT_CTL); if (version > 1 || cfg->pcie_type == PCIEM_TYPE_ROOT_PORT || cfg->pcie_type == PCIEM_TYPE_ROOT_EC) cfg->pcie_root_ctl = RREG(PCIER_ROOT_CTL); if (version > 1) { cfg->pcie_device_ctl2 = RREG(PCIER_DEVICE_CTL2); cfg->pcie_link_ctl2 = RREG(PCIER_LINK_CTL2); cfg->pcie_slot_ctl2 = RREG(PCIER_SLOT_CTL2); } #undef RREG } static void pci_cfg_save_pcix(device_t dev, struct pci_devinfo *dinfo) { dinfo->cfg.pcix.pcix_command = pci_read_config(dev, dinfo->cfg.pcix.pcix_location + PCIXR_COMMAND, 2); } void pci_cfg_save(device_t dev, struct pci_devinfo *dinfo, int setstate) { uint32_t cls; int ps; /* * Some drivers apparently write to these registers w/o updating our * cached copy. No harm happens if we update the copy, so do so here * so we can restore them. The COMMAND register is modified by the * bus w/o updating the cache. This should represent the normally * writable portion of the 'defined' part of type 0/1/2 headers. */ dinfo->cfg.vendor = pci_read_config(dev, PCIR_VENDOR, 2); dinfo->cfg.device = pci_read_config(dev, PCIR_DEVICE, 2); dinfo->cfg.cmdreg = pci_read_config(dev, PCIR_COMMAND, 2); dinfo->cfg.intline = pci_read_config(dev, PCIR_INTLINE, 1); dinfo->cfg.intpin = pci_read_config(dev, PCIR_INTPIN, 1); dinfo->cfg.cachelnsz = pci_read_config(dev, PCIR_CACHELNSZ, 1); dinfo->cfg.lattimer = pci_read_config(dev, PCIR_LATTIMER, 1); dinfo->cfg.baseclass = pci_read_config(dev, PCIR_CLASS, 1); dinfo->cfg.subclass = pci_read_config(dev, PCIR_SUBCLASS, 1); dinfo->cfg.progif = pci_read_config(dev, PCIR_PROGIF, 1); dinfo->cfg.revid = pci_read_config(dev, PCIR_REVID, 1); switch (dinfo->cfg.hdrtype & PCIM_HDRTYPE) { case PCIM_HDRTYPE_NORMAL: dinfo->cfg.subvendor = pci_read_config(dev, PCIR_SUBVEND_0, 2); dinfo->cfg.subdevice = pci_read_config(dev, PCIR_SUBDEV_0, 2); dinfo->cfg.mingnt = pci_read_config(dev, PCIR_MINGNT, 1); dinfo->cfg.maxlat = pci_read_config(dev, PCIR_MAXLAT, 1); break; case PCIM_HDRTYPE_BRIDGE: dinfo->cfg.bridge.br_seclat = pci_read_config(dev, PCIR_SECLAT_1, 1); dinfo->cfg.bridge.br_subbus = pci_read_config(dev, PCIR_SUBBUS_1, 1); dinfo->cfg.bridge.br_secbus = pci_read_config(dev, PCIR_SECBUS_1, 1); dinfo->cfg.bridge.br_pribus = pci_read_config(dev, PCIR_PRIBUS_1, 1); dinfo->cfg.bridge.br_control = pci_read_config(dev, PCIR_BRIDGECTL_1, 2); break; case PCIM_HDRTYPE_CARDBUS: dinfo->cfg.bridge.br_seclat = pci_read_config(dev, PCIR_SECLAT_2, 1); dinfo->cfg.bridge.br_subbus = pci_read_config(dev, PCIR_SUBBUS_2, 1); dinfo->cfg.bridge.br_secbus = pci_read_config(dev, PCIR_SECBUS_2, 1); dinfo->cfg.bridge.br_pribus = pci_read_config(dev, PCIR_PRIBUS_2, 1); dinfo->cfg.bridge.br_control = pci_read_config(dev, PCIR_BRIDGECTL_2, 2); dinfo->cfg.subvendor = pci_read_config(dev, PCIR_SUBVEND_2, 2); dinfo->cfg.subdevice = pci_read_config(dev, PCIR_SUBDEV_2, 2); break; } if (dinfo->cfg.pcie.pcie_location != 0) pci_cfg_save_pcie(dev, dinfo); if (dinfo->cfg.pcix.pcix_location != 0) pci_cfg_save_pcix(dev, dinfo); #ifdef PCI_IOV if (dinfo->cfg.iov != NULL) pci_iov_cfg_save(dev, dinfo); #endif /* * don't set the state for display devices, base peripherals and * memory devices since bad things happen when they are powered down. * We should (a) have drivers that can easily detach and (b) use * generic drivers for these devices so that some device actually * attaches. We need to make sure that when we implement (a) we don't * power the device down on a reattach. */ cls = pci_get_class(dev); if (!setstate) return; switch (pci_do_power_nodriver) { case 0: /* NO powerdown at all */ return; case 1: /* Conservative about what to power down */ if (cls == PCIC_STORAGE) return; /*FALLTHROUGH*/ case 2: /* Aggressive about what to power down */ if (cls == PCIC_DISPLAY || cls == PCIC_MEMORY || cls == PCIC_BASEPERIPH) return; /*FALLTHROUGH*/ case 3: /* Power down everything */ break; } /* * PCI spec says we can only go into D3 state from D0 state. * Transition from D[12] into D0 before going to D3 state. */ ps = pci_get_powerstate(dev); if (ps != PCI_POWERSTATE_D0 && ps != PCI_POWERSTATE_D3) pci_set_powerstate(dev, PCI_POWERSTATE_D0); if (pci_get_powerstate(dev) != PCI_POWERSTATE_D3) pci_set_powerstate(dev, PCI_POWERSTATE_D3); } /* Wrapper APIs suitable for device driver use. */ void pci_save_state(device_t dev) { struct pci_devinfo *dinfo; dinfo = device_get_ivars(dev); pci_cfg_save(dev, dinfo, 0); } void pci_restore_state(device_t dev) { struct pci_devinfo *dinfo; dinfo = device_get_ivars(dev); pci_cfg_restore(dev, dinfo); } static int pci_get_id_method(device_t dev, device_t child, enum pci_id_type type, uintptr_t *id) { return (PCIB_GET_ID(device_get_parent(dev), child, type, id)); } /* Find the upstream port of a given PCI device in a root complex. */ device_t pci_find_pcie_root_port(device_t dev) { struct pci_devinfo *dinfo; devclass_t pci_class; device_t pcib, bus; pci_class = devclass_find("pci"); KASSERT(device_get_devclass(device_get_parent(dev)) == pci_class, ("%s: non-pci device %s", __func__, device_get_nameunit(dev))); /* * Walk the bridge hierarchy until we find a PCI-e root * port or a non-PCI device. */ for (;;) { bus = device_get_parent(dev); KASSERT(bus != NULL, ("%s: null parent of %s", __func__, device_get_nameunit(dev))); pcib = device_get_parent(bus); KASSERT(pcib != NULL, ("%s: null bridge of %s", __func__, device_get_nameunit(bus))); /* * pcib's parent must be a PCI bus for this to be a * PCI-PCI bridge. */ if (device_get_devclass(device_get_parent(pcib)) != pci_class) return (NULL); dinfo = device_get_ivars(pcib); if (dinfo->cfg.pcie.pcie_location != 0 && dinfo->cfg.pcie.pcie_type == PCIEM_TYPE_ROOT_PORT) return (pcib); dev = pcib; } } /* * Wait for pending transactions to complete on a PCI-express function. * * The maximum delay is specified in milliseconds in max_delay. Note * that this function may sleep. * * Returns true if the function is idle and false if the timeout is * exceeded. If dev is not a PCI-express function, this returns true. */ bool pcie_wait_for_pending_transactions(device_t dev, u_int max_delay) { struct pci_devinfo *dinfo = device_get_ivars(dev); uint16_t sta; int cap; cap = dinfo->cfg.pcie.pcie_location; if (cap == 0) return (true); sta = pci_read_config(dev, cap + PCIER_DEVICE_STA, 2); while (sta & PCIEM_STA_TRANSACTION_PND) { if (max_delay == 0) return (false); /* Poll once every 100 milliseconds up to the timeout. */ if (max_delay > 100) { pause_sbt("pcietp", 100 * SBT_1MS, 0, C_HARDCLOCK); max_delay -= 100; } else { pause_sbt("pcietp", max_delay * SBT_1MS, 0, C_HARDCLOCK); max_delay = 0; } sta = pci_read_config(dev, cap + PCIER_DEVICE_STA, 2); } return (true); } /* * Determine the maximum Completion Timeout in microseconds. * * For non-PCI-express functions this returns 0. */ int pcie_get_max_completion_timeout(device_t dev) { struct pci_devinfo *dinfo = device_get_ivars(dev); int cap; cap = dinfo->cfg.pcie.pcie_location; if (cap == 0) return (0); /* * Functions using the 1.x spec use the default timeout range of * 50 microseconds to 50 milliseconds. Functions that do not * support programmable timeouts also use this range. */ if ((dinfo->cfg.pcie.pcie_flags & PCIEM_FLAGS_VERSION) < 2 || (pci_read_config(dev, cap + PCIER_DEVICE_CAP2, 4) & PCIEM_CAP2_COMP_TIMO_RANGES) == 0) return (50 * 1000); switch (pci_read_config(dev, cap + PCIER_DEVICE_CTL2, 2) & PCIEM_CTL2_COMP_TIMO_VAL) { case PCIEM_CTL2_COMP_TIMO_100US: return (100); case PCIEM_CTL2_COMP_TIMO_10MS: return (10 * 1000); case PCIEM_CTL2_COMP_TIMO_55MS: return (55 * 1000); case PCIEM_CTL2_COMP_TIMO_210MS: return (210 * 1000); case PCIEM_CTL2_COMP_TIMO_900MS: return (900 * 1000); case PCIEM_CTL2_COMP_TIMO_3500MS: return (3500 * 1000); case PCIEM_CTL2_COMP_TIMO_13S: return (13 * 1000 * 1000); case PCIEM_CTL2_COMP_TIMO_64S: return (64 * 1000 * 1000); default: return (50 * 1000); } } /* * Perform a Function Level Reset (FLR) on a device. * * This function first waits for any pending transactions to complete * within the timeout specified by max_delay. If transactions are * still pending, the function will return false without attempting a * reset. * * If dev is not a PCI-express function or does not support FLR, this * function returns false. * * Note that no registers are saved or restored. The caller is * responsible for saving and restoring any registers including * PCI-standard registers via pci_save_state() and * pci_restore_state(). */ bool pcie_flr(device_t dev, u_int max_delay, bool force) { struct pci_devinfo *dinfo = device_get_ivars(dev); uint16_t cmd, ctl; int compl_delay; int cap; cap = dinfo->cfg.pcie.pcie_location; if (cap == 0) return (false); if (!(pci_read_config(dev, cap + PCIER_DEVICE_CAP, 4) & PCIEM_CAP_FLR)) return (false); /* * Disable busmastering to prevent generation of new * transactions while waiting for the device to go idle. If * the idle timeout fails, the command register is restored * which will re-enable busmastering. */ cmd = pci_read_config(dev, PCIR_COMMAND, 2); pci_write_config(dev, PCIR_COMMAND, cmd & ~(PCIM_CMD_BUSMASTEREN), 2); if (!pcie_wait_for_pending_transactions(dev, max_delay)) { if (!force) { pci_write_config(dev, PCIR_COMMAND, cmd, 2); return (false); } pci_printf(&dinfo->cfg, "Resetting with transactions pending after %d ms\n", max_delay); /* * Extend the post-FLR delay to cover the maximum * Completion Timeout delay of anything in flight * during the FLR delay. Enforce a minimum delay of * at least 10ms. */ compl_delay = pcie_get_max_completion_timeout(dev) / 1000; if (compl_delay < 10) compl_delay = 10; } else compl_delay = 0; /* Initiate the reset. */ ctl = pci_read_config(dev, cap + PCIER_DEVICE_CTL, 2); pci_write_config(dev, cap + PCIER_DEVICE_CTL, ctl | PCIEM_CTL_INITIATE_FLR, 2); /* Wait for 100ms. */ pause_sbt("pcieflr", (100 + compl_delay) * SBT_1MS, 0, C_HARDCLOCK); if (pci_read_config(dev, cap + PCIER_DEVICE_STA, 2) & PCIEM_STA_TRANSACTION_PND) pci_printf(&dinfo->cfg, "Transactions pending after FLR!\n"); return (true); }