Index: stable/8/share/man/man4/ahci.4 =================================================================== --- stable/8/share/man/man4/ahci.4 (revision 243119) +++ stable/8/share/man/man4/ahci.4 (revision 243120) @@ -1,174 +1,174 @@ .\" Copyright (c) 2009 Alexander Motin .\" All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" 3. The name of the author may not be used to endorse or promote products .\" derived from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR .\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES .\" OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. .\" IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, .\" INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT .\" NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF .\" THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .\" .\" $FreeBSD$ .\" .Dd June 18, 2012 .Dt AHCI 4 .Os .Sh NAME .Nm ahci .Nd Serial ATA Advanced Host Controller Interface driver .Sh SYNOPSIS To compile this driver into the kernel, place the following lines in your kernel configuration file: .Bd -ragged -offset indent .Cd "device pci" .Cd "device scbus" .Cd "device ahci" .Ed .Pp Alternatively, to load the driver as a module at boot time, place the following line in .Xr loader.conf 5 : .Bd -literal -offset indent ahci_load="YES" .Ed .Pp The following tunables are settable from the .Xr loader 8 : .Bl -ohang .It Va hint.ahci. Ns Ar X Ns Va .msi controls Message Signaled Interrupts (MSI) usage by the specified controller. .Bl -tag -compact .It 0 MSI disabled; .It 1 single MSI vector used, if supported (default); .It 2 multiple MSI vectors used, if supported; .El .It Va hint.ahci. Ns Ar X Ns Va .ccc controls Command Completion Coalescing (CCC) usage by the specified controller. Non-zero value enables CCC and defines maximum time (in ms), request can wait for interrupt, if there are some more requests present on controller queue. CCC reduces number of context switches on systems with many parallel requests, but it can decrease disk performance on some workloads due to additional command latency. .It Va hint.ahcich. Ns Ar X Ns Va .pm_level controls SATA interface Power Management for the specified channel, allowing some power to be saved at the cost of additional command latency. Possible values: .Bl -tag -compact .It 0 interface Power Management is disabled (default); .It 1 device is allowed to initiate PM state change, host is passive; .It 2 host initiates PARTIAL PM state transition every time port becomes idle; .It 3 host initiates SLUMBER PM state transition every time port becomes idle. .It 4 driver initiates PARTIAL PM state transition 1ms after port becomes idle; .It 5 driver initiates SLUMBER PM state transition 125ms after port becomes idle. .El Some controllers, such as ICH8, do not implement modes 2 and 3 with NCQ used. Because of artificial entering latency, performance degradation in modes 4 and 5 is much smaller then in modes 2 and 3. .Pp Note that interface Power Management complicates device presence detection. A manual bus reset/rescan may be needed after device hot-plug, unless hardware implements Cold Presence Detection. .It Va hint.ahcich. Ns Ar X Ns Va .sata_rev setting to nonzero value limits maximum SATA revision (speed). Values 1, 2 and 3 are respectively 1.5, 3 and 6Gbps. .It Va hw.ahci.force setting to nonzero value forces driver attach to some known AHCI-capable chips even if they are configured for legacy IDE emulation. Default is 1. .El .Sh DESCRIPTION This driver provides the .Xr CAM 4 subsystem with native access to the .Tn SATA ports of AHCI-compatible controllers. Each SATA port found is represented to CAM as a separate bus with one target, or, if HBA supports Port Multipliers, 16 targets. Most of the bus-management details are handled by the SATA-specific transport of CAM. Connected ATA disks are handled by the ATA protocol disk peripheral driver .Xr ada 4 . ATAPI devices are handled by the SCSI protocol peripheral drivers .Xr cd 4 , .Xr da 4 , .Xr sa 4 , etc. .Pp Driver features include support for Serial ATA and ATAPI devices, Port Multipliers (including FIS-based switching, when supported), hardware command queues (up to 32 commands per port), Native Command Queuing, SATA interface Power Management, device hot-plug and Message Signaled Interrupts. .Pp Driver supports "LED" enclosure management messages, defined by the AHCI. When supported by hardware, it allows to control per-port activity, locate and fault LEDs via the .Xr led 4 API for localization and status reporting purposes. Supporting AHCI controllers may transmit that information to the backplane controllers via SGPIO interface. Backplane controllers interpret received statuses in some way (IBPI standard) to report them using present indicators. .Pp AHCI hardware is also supported by ataahci driver from .Xr ata 4 subsystem. If both drivers are loaded at the same time, this one will be given precedence as the more functional of the two. .Sh HARDWARE The .Nm driver supports AHCI compatible controllers having PCI class 1 (mass storage), subclass 6 (SATA) and programming interface 1 (AHCI). .Pp Also, in cooperation with atamarvell and atajmicron drivers of ata(4), it supports AHCI part of legacy-PATA + AHCI-SATA combined controllers, -such as JMicron JMB36x and Marvell 88SX61xx. +such as JMicron JMB36x and Marvell 88SE61xx. .Sh FILES .Bl -tag -width /dev/led/ahcich*.locate .It Pa /dev/led/ahcich*.act activity LED device nodes .It Pa /dev/led/ahcich*.fault fault LED device nodes .It Pa /dev/led/ahcich*.locate locate LED device nodes .El .Sh SEE ALSO .Xr ada 4 , .Xr ata 4 , .Xr cam 4 , .Xr cd 4 , .Xr da 4 , .Xr sa 4 .Sh HISTORY The .Nm driver first appeared in .Fx 8.0 . .Sh AUTHORS .An Alexander Motin Aq mav@FreeBSD.org . Index: stable/8/share/man/man4/ata.4 =================================================================== --- stable/8/share/man/man4/ata.4 (revision 243119) +++ stable/8/share/man/man4/ata.4 (revision 243120) @@ -1,286 +1,286 @@ .\" .\" Copyright (c) 2003 Søren Schmidt .\" All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer, .\" without modification, immediately at the beginning of the file. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" 3. The name of the author may not be used to endorse or promote products .\" derived from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR .\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES .\" OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. .\" IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, .\" INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT .\" NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF .\" THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .\" .\" $FreeBSD$ .\" .Dd June 18, 2012 .Dt ATA 4 .Os .Sh NAME .Nm ata , .Nm ar , .Nm acd , .Nm ad , .Nm afd , .Nm ast .Nd generic ATA/ATAPI disk controller driver .Sh SYNOPSIS For ISA based ATA/ATAPI support: .Cd device isa .Cd device ata .Pp In .Pa /boot/device.hints : .Cd hint.ata.0.at="isa" .Cd hint.ata.0.port="0x1f0" .Cd hint.ata.0.irq="14" .Cd hint.ata.1.at="isa" .Cd hint.ata.1.port="0x170" .Cd hint.ata.1.irq="15" .Pp For PC98 based ATA/ATAPI support: .Cd device isa .Cd device ata .Pp In .Pa /boot/device.hints : .Cd hint.atacbus.0.at="isa" .Cd hint.atacbus.0.port="0x640" .Cd hint.atacbus.0.irq="9" .Pp For PCI based ATA/ATAPI support: .Cd device pci .Cd device ata .Pp To support ATA compliant disk drives: .Cd device atadisk .Pp To support ATA software RAID's: .Cd device ataraid .Pp To support ATAPI CDROM, DVD and CD/DVD burner drives: .Cd device atapicd .Pp To support ATAPI floppy drives: .Cd device atapifd .Pp To support ATAPI tape drives: .Cd device atapist .Pp To turn .Xr ata 4 subsystem controller drivers into cam(4) interface modules (disables all native .Xr ata 4 APIs and periperal drivers): .Cd options ATA_CAM .Pp The following tunables are settable from the loader: .Bl -ohang .It Va hw.ahci.force set to nonzero value for forcing drivers to attach to some known AHCI-capable chips even if they are configured for legacy IDE emulation (the default is 1, force the attach). .It Va hw.ata.ata_dma set to 1 for DMA access, 0 for PIO (default is DMA). .It Va hw.ata.ata_dma_check_80pin set to 0 to disable the 80pin cable check (default is 1, check the cable). .It Va hw.ata.atapi_dma set to 1 for DMA access, 0 for PIO (default is DMA). .It Va hw.ata.wc set to 1 to enable Write Caching, 0 to disable (default is enabled). .Em WARNING : can cause data loss on power failures and crashes. .It Va hint.atapci.X.msi set to 1 to allow Message Signalled Interrupts (MSI) to be used by specified PCI ATA controller, if supported. .It Va hint.ata.X.devX.mode limits initial ATA mode for specified device on specified channel. .It Va hint.ata.X.mode limits initial ATA mode for every device on specified channel. .It Va hint.ata.X.pm_level controls SATA interface Power Management for specified channel, allowing to save some power by the cost of additional command latency. Possible values: .Bl -tag -compact .It 0 interface Power Management is disabled, default value. .It 1 device is allowed to initiate PM state change, host is passive. .It 2 host initiates PARTIAL PM state transition every time port becomes idle. .It 3 host initiates SLUMBER PM state transition every time port becomes idle. .El Modes 2 and 3 are implemented only for AHCI driver now. .El .Sh DESCRIPTION The .Nm driver provides access to ATA (IDE) and SerialATA disk drives, ATAPI CDROM/DVD drives, ZIP/LS120 ATAPI drives and ATAPI tape drives connected to controllers according to the ATA/ATAPI standards. .Pp The currently supported ATA/SATA controller chips are: .Pp .Bl -tag -width "Silicon Image:" -compact .It Acard: ATP850P, ATP860A, ATP860R, ATP865A, ATP865R. .It ALI: M5228, M5229, M5281, M5283, M5287, M5288, M5289. .It AMD: AMD756, AMD766, AMD768, AMD8111, CS5536. .It ATI: IXP200, IXP300, IXP400, IXP600, IXP700, IXP800. .It CMD: CMD646, CMD646U2, CMD648, CMD649. .It Cypress: Cypress 82C693. .It Cyrix: Cyrix 5530. .It HighPoint: HPT302, HPT366, HPT368, HPT370, HPT371, HPT372, HPT372N, HPT374. .It Intel: 6300ESB, 31244, PIIX, PIIX3, PIIX4, ESB2, ICH, ICH0, ICH2, ICH3, ICH4, ICH5, ICH6, ICH7, ICH8, ICH9, ICH10, SCH, PCH. .It ITE: IT8211F, IT8212F, IT8213F. .It JMicron: JMB360, JMB361, JMB363, JMB365, JMB366, JMB368. .It Marvell -88SX5040, 88SX5041, 88SX5080, 88SX5081, 88SX6041, 88SX6042, 88SX6081, 88SX6101, -88SX6102, 88SX6111, 88SX6121, 88SX6141, 88SX6145, 88SX7042. +88SX5040, 88SX5041, 88SX5080, 88SX5081, 88SX6041, 88SX6042, 88SX6081, 88SE6101, +88SE6102, 88SE6111, 88SE6121, 88SE6141, 88SE6145, 88SX7042. .It National: SC1100. .It NetCell: NC3000, NC5000. .It nVidia: nForce, nForce2, nForce2 MCP, nForce3, nForce3 MCP, nForce3 Pro, nForce4, MCP51, MCP55, MCP61, MCP65, MCP67, MCP73, MCP77, MCP79, MCP89. .It Promise: PDC20246, PDC20262, PDC20263, PDC20265, PDC20267, PDC20268, PDC20269, PDC20270, PDC20271, PDC20275, PDC20276, PDC20277, PDC20318, PDC20319, PDC20371, PDC20375, PDC20376, PDC20377, PDC20378, PDC20379, PDC20571, PDC20575, PDC20579, PDC20580, PDC20617, PDC20618, PDC20619, PDC20620, PDC20621, PDC20622, PDC40518, PDC40519, PDC40718, PDC40719. .It ServerWorks: HT1000, ROSB4, CSB5, CSB6, K2, Frodo4, Frodo8. .It Silicon Image: SiI0680, SiI3112, SiI3114, SiI3124, SiI3132, SiI3512. .It SiS: SIS180, SIS181, SIS182, SIS5513, SIS530, SIS540, SIS550, SIS620, SIS630, SIS630S, SIS633, SIS635, SIS730, SIS733, SIS735, SIS745, SIS961, SIS962, SIS963, SIS964, SIS965. .It VIA: VT6410, VT6420, VT6421, VT82C586, VT82C586B, VT82C596, VT82C596B, VT82C686, VT82C686A, VT82C686B, VT8231, VT8233, VT8233A, VT8233C, VT8235, VT8237, VT8237A, VT8237S, VT8251, CX700, VX800, VX855, VX900. .El .Pp Unknown ATA chipsets are supported in PIO modes, and if the standard busmaster DMA registers are present and contain valid setup, DMA is also enabled, although the max mode is limited to UDMA33, as it is not known what the chipset can do and how to program it. .Pp The .Nm driver can change the transfer mode and various other parameters when the system is up and running. See .Xr atacontrol 8 . .Pp The .Nm driver sets the maximum transfer mode supported by the hardware as default. However the .Nm driver sometimes warns: .Dq Sy "DMA limited to UDMA33, non-ATA66 cable or device". This means that the .Nm driver has detected that the required 80 conductor cable is not present or could not be detected properly, or that one of the devices on the channel only accepts up to UDMA2/ATA33. The .Va hw.ata.ata_dma_check_80pin tunable can be set to 0 to disable this check. .Pp Unknown ATAPI devices are initialized to DMA mode if the .Va hw.ata.atapi_dma tunable is set to 1 and they support at least UDMA33 transfers. Otherwise they are set to PIO mode because severe DMA problems are common even if the device capabilities indicate support. You can always try to set DMA mode on an ATAPI device using .Xr atacontrol 8 , but be aware that your hardware might .Em not support it and can potentially .Em hang the entire system causing data loss. .Sh FILES .Bl -tag -width ".Pa /sys/i386/conf/GENERIC" -compact .It Pa /dev/ad* ATA disk device nodes .It Pa /dev/ar* ATA RAID device nodes .It Pa /dev/acd* ATAPI CD-ROM device nodes .It Pa /dev/afd* ATAPI floppy drive device nodes .It Pa /dev/ast* ATAPI tape drive device nodes .It Pa /sys/i386/conf/GENERIC sample generic kernel config file for .Nm based systems .El .Sh NOTES Please remember that in order to use UDMA4/ATA66 and above modes you .Em must use 80 conductor cables. Please assure that ribbon cables are no longer than 45cm. In case of rounded ATA cables, the length depends on the quality of the cables. SATA cables can be up to 1m long according to the specification. .Pp Static device numbering (enabled with the .Dv ATA_STATIC_ID kernel option) reserves a number for each possibly connected disk, even when not present. This is useful in hotswap scenarios where disks should always show up as the same numbered device, and not depend on attach order. .Pp Native Command Queuing (NCQ) on SATA drives is not yet supported. .Sh SEE ALSO .Xr ahci 4 , .Xr cam 4 , .Xr ataraid 4 , .Xr atacontrol 8 , .Xr burncd 8 , .Xr siis 4 .Sh HISTORY The .Nm driver first appeared in .Fx 4.0 . .Sh AUTHORS .An S\(/oren Schmidt .Aq sos@FreeBSD.org . Index: stable/8/share/man/man4 =================================================================== --- stable/8/share/man/man4 (revision 243119) +++ stable/8/share/man/man4 (revision 243120) Property changes on: stable/8/share/man/man4 ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,1 ## Merged /head/share/man/man4:r232380 Index: stable/8/sys/dev/ahci/ahci.c =================================================================== --- stable/8/sys/dev/ahci/ahci.c (revision 243119) +++ stable/8/sys/dev/ahci/ahci.c (revision 243120) @@ -1,2998 +1,2998 @@ /*- * Copyright (c) 2009 Alexander Motin * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer, * without modification, immediately at the beginning of the file. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "ahci.h" #include #include #include #include #include /* local prototypes */ static int ahci_setup_interrupt(device_t dev); static void ahci_intr(void *data); static void ahci_intr_one(void *data); static int ahci_suspend(device_t dev); static int ahci_resume(device_t dev); static int ahci_ch_init(device_t dev); static int ahci_ch_deinit(device_t dev); static int ahci_ch_suspend(device_t dev); static int ahci_ch_resume(device_t dev); static void ahci_ch_pm(void *arg); static void ahci_ch_intr_locked(void *data); static void ahci_ch_intr(void *data); static void ahci_ch_led(void *priv, int onoff); static int ahci_ctlr_reset(device_t dev); static int ahci_ctlr_setup(device_t dev); static void ahci_begin_transaction(device_t dev, union ccb *ccb); static void ahci_dmasetprd(void *arg, bus_dma_segment_t *segs, int nsegs, int error); static void ahci_execute_transaction(struct ahci_slot *slot); static void ahci_timeout(struct ahci_slot *slot); static void ahci_end_transaction(struct ahci_slot *slot, enum ahci_err_type et); static int ahci_setup_fis(device_t dev, struct ahci_cmd_tab *ctp, union ccb *ccb, int tag); static void ahci_dmainit(device_t dev); static void ahci_dmasetupc_cb(void *xsc, bus_dma_segment_t *segs, int nsegs, int error); static void ahci_dmafini(device_t dev); static void ahci_slotsalloc(device_t dev); static void ahci_slotsfree(device_t dev); static void ahci_reset(device_t dev); static void ahci_start(device_t dev, int fbs); static void ahci_stop(device_t dev); static void ahci_clo(device_t dev); static void ahci_start_fr(device_t dev); static void ahci_stop_fr(device_t dev); static int ahci_sata_connect(struct ahci_channel *ch); static int ahci_sata_phy_reset(device_t dev); static int ahci_wait_ready(device_t dev, int t, int t0); static void ahci_issue_recovery(device_t dev); static void ahci_process_read_log(device_t dev, union ccb *ccb); static void ahci_process_request_sense(device_t dev, union ccb *ccb); static void ahciaction(struct cam_sim *sim, union ccb *ccb); static void ahcipoll(struct cam_sim *sim); MALLOC_DEFINE(M_AHCI, "AHCI driver", "AHCI driver data buffers"); static struct { uint32_t id; uint8_t rev; const char *name; int quirks; #define AHCI_Q_NOFORCE 1 #define AHCI_Q_NOPMP 2 #define AHCI_Q_NONCQ 4 #define AHCI_Q_1CH 8 #define AHCI_Q_2CH 16 #define AHCI_Q_4CH 32 #define AHCI_Q_EDGEIS 64 #define AHCI_Q_SATA2 128 #define AHCI_Q_NOBSYRES 256 #define AHCI_Q_NOAA 512 #define AHCI_Q_NOCOUNT 1024 #define AHCI_Q_ALTSIG 2048 } ahci_ids[] = { {0x43801002, 0x00, "ATI IXP600", 0}, {0x43901002, 0x00, "ATI IXP700", 0}, {0x43911002, 0x00, "ATI IXP700", 0}, {0x43921002, 0x00, "ATI IXP700", 0}, {0x43931002, 0x00, "ATI IXP700", 0}, {0x43941002, 0x00, "ATI IXP800", 0}, {0x43951002, 0x00, "ATI IXP800", 0}, {0x06121b21, 0x00, "ASMedia ASM1061", 0}, {0x26528086, 0x00, "Intel ICH6", AHCI_Q_NOFORCE}, {0x26538086, 0x00, "Intel ICH6M", AHCI_Q_NOFORCE}, {0x26818086, 0x00, "Intel ESB2", 0}, {0x26828086, 0x00, "Intel ESB2", 0}, {0x26838086, 0x00, "Intel ESB2", 0}, {0x27c18086, 0x00, "Intel ICH7", 0}, {0x27c38086, 0x00, "Intel ICH7", 0}, {0x27c58086, 0x00, "Intel ICH7M", 0}, {0x27c68086, 0x00, "Intel ICH7M", 0}, {0x28218086, 0x00, "Intel ICH8", 0}, {0x28228086, 0x00, "Intel ICH8", 0}, {0x28248086, 0x00, "Intel ICH8", 0}, {0x28298086, 0x00, "Intel ICH8M", 0}, {0x282a8086, 0x00, "Intel ICH8M", 0}, {0x29228086, 0x00, "Intel ICH9", 0}, {0x29238086, 0x00, "Intel ICH9", 0}, {0x29248086, 0x00, "Intel ICH9", 0}, {0x29258086, 0x00, "Intel ICH9", 0}, {0x29278086, 0x00, "Intel ICH9", 0}, {0x29298086, 0x00, "Intel ICH9M", 0}, {0x292a8086, 0x00, "Intel ICH9M", 0}, {0x292b8086, 0x00, "Intel ICH9M", 0}, {0x292c8086, 0x00, "Intel ICH9M", 0}, {0x292f8086, 0x00, "Intel ICH9M", 0}, {0x294d8086, 0x00, "Intel ICH9", 0}, {0x294e8086, 0x00, "Intel ICH9M", 0}, {0x3a058086, 0x00, "Intel ICH10", 0}, {0x3a228086, 0x00, "Intel ICH10", 0}, {0x3a258086, 0x00, "Intel ICH10", 0}, {0x3b228086, 0x00, "Intel 5 Series/3400 Series", 0}, {0x3b238086, 0x00, "Intel 5 Series/3400 Series", 0}, {0x3b258086, 0x00, "Intel 5 Series/3400 Series", 0}, {0x3b298086, 0x00, "Intel 5 Series/3400 Series", 0}, {0x3b2c8086, 0x00, "Intel 5 Series/3400 Series", 0}, {0x3b2f8086, 0x00, "Intel 5 Series/3400 Series", 0}, {0x1c028086, 0x00, "Intel Cougar Point", 0}, {0x1c038086, 0x00, "Intel Cougar Point", 0}, {0x1c048086, 0x00, "Intel Cougar Point", 0}, {0x1c058086, 0x00, "Intel Cougar Point", 0}, {0x1d028086, 0x00, "Intel Patsburg", 0}, {0x1d048086, 0x00, "Intel Patsburg", 0}, {0x1d068086, 0x00, "Intel Patsburg", 0}, {0x28268086, 0x00, "Intel Patsburg (RAID)", 0}, {0x1e028086, 0x00, "Intel Panther Point", 0}, {0x1e038086, 0x00, "Intel Panther Point", 0}, {0x1e048086, 0x00, "Intel Panther Point", 0}, {0x1e058086, 0x00, "Intel Panther Point", 0}, {0x1e068086, 0x00, "Intel Panther Point", 0}, {0x1e078086, 0x00, "Intel Panther Point", 0}, {0x1e0e8086, 0x00, "Intel Panther Point", 0}, {0x1e0f8086, 0x00, "Intel Panther Point", 0}, {0x23238086, 0x00, "Intel DH89xxCC", 0}, {0x2360197b, 0x00, "JMicron JMB360", 0}, {0x2361197b, 0x00, "JMicron JMB361", AHCI_Q_NOFORCE}, {0x2362197b, 0x00, "JMicron JMB362", 0}, {0x2363197b, 0x00, "JMicron JMB363", AHCI_Q_NOFORCE}, {0x2365197b, 0x00, "JMicron JMB365", AHCI_Q_NOFORCE}, {0x2366197b, 0x00, "JMicron JMB366", AHCI_Q_NOFORCE}, {0x2368197b, 0x00, "JMicron JMB368", AHCI_Q_NOFORCE}, - {0x611111ab, 0x00, "Marvell 88SX6111", AHCI_Q_NOFORCE | AHCI_Q_1CH | + {0x611111ab, 0x00, "Marvell 88SE6111", AHCI_Q_NOFORCE | AHCI_Q_1CH | AHCI_Q_EDGEIS}, - {0x612111ab, 0x00, "Marvell 88SX6121", AHCI_Q_NOFORCE | AHCI_Q_2CH | + {0x612111ab, 0x00, "Marvell 88SE6121", AHCI_Q_NOFORCE | AHCI_Q_2CH | AHCI_Q_EDGEIS | AHCI_Q_NONCQ | AHCI_Q_NOCOUNT}, - {0x614111ab, 0x00, "Marvell 88SX6141", AHCI_Q_NOFORCE | AHCI_Q_4CH | + {0x614111ab, 0x00, "Marvell 88SE6141", AHCI_Q_NOFORCE | AHCI_Q_4CH | AHCI_Q_EDGEIS | AHCI_Q_NONCQ | AHCI_Q_NOCOUNT}, - {0x614511ab, 0x00, "Marvell 88SX6145", AHCI_Q_NOFORCE | AHCI_Q_4CH | + {0x614511ab, 0x00, "Marvell 88SE6145", AHCI_Q_NOFORCE | AHCI_Q_4CH | AHCI_Q_EDGEIS | AHCI_Q_NONCQ | AHCI_Q_NOCOUNT}, {0x91201b4b, 0x00, "Marvell 88SE912x", AHCI_Q_EDGEIS|AHCI_Q_NOBSYRES}, {0x91231b4b, 0x11, "Marvell 88SE912x", AHCI_Q_NOBSYRES|AHCI_Q_ALTSIG}, {0x91231b4b, 0x00, "Marvell 88SE912x", AHCI_Q_EDGEIS|AHCI_Q_SATA2|AHCI_Q_NOBSYRES}, {0x91251b4b, 0x00, "Marvell 88SE9125", AHCI_Q_NOBSYRES}, {0x91281b4b, 0x00, "Marvell 88SE9128", AHCI_Q_NOBSYRES|AHCI_Q_ALTSIG}, {0x91721b4b, 0x00, "Marvell 88SE9172", AHCI_Q_NOBSYRES}, {0x91821b4b, 0x00, "Marvell 88SE9182", AHCI_Q_NOBSYRES}, {0x92201b4b, 0x00, "Marvell 88SE9220", AHCI_Q_NOBSYRES|AHCI_Q_ALTSIG}, {0x92301b4b, 0x00, "Marvell 88SE9230", AHCI_Q_NOBSYRES|AHCI_Q_ALTSIG}, {0x92351b4b, 0x00, "Marvell 88SE9235", AHCI_Q_NOBSYRES}, {0x06201103, 0x00, "HighPoint RocketRAID 620", AHCI_Q_NOBSYRES}, {0x06201b4b, 0x00, "HighPoint RocketRAID 620", AHCI_Q_NOBSYRES}, {0x06221103, 0x00, "HighPoint RocketRAID 622", AHCI_Q_NOBSYRES}, {0x06221b4b, 0x00, "HighPoint RocketRAID 622", AHCI_Q_NOBSYRES}, {0x06401103, 0x00, "HighPoint RocketRAID 640", AHCI_Q_NOBSYRES}, {0x06401b4b, 0x00, "HighPoint RocketRAID 640", AHCI_Q_NOBSYRES}, {0x06441103, 0x00, "HighPoint RocketRAID 644", AHCI_Q_NOBSYRES}, {0x06441b4b, 0x00, "HighPoint RocketRAID 644", AHCI_Q_NOBSYRES}, {0x044c10de, 0x00, "NVIDIA MCP65", AHCI_Q_NOAA}, {0x044d10de, 0x00, "NVIDIA MCP65", AHCI_Q_NOAA}, {0x044e10de, 0x00, "NVIDIA MCP65", AHCI_Q_NOAA}, {0x044f10de, 0x00, "NVIDIA MCP65", AHCI_Q_NOAA}, {0x045c10de, 0x00, "NVIDIA MCP65", AHCI_Q_NOAA}, {0x045d10de, 0x00, "NVIDIA MCP65", AHCI_Q_NOAA}, {0x045e10de, 0x00, "NVIDIA MCP65", AHCI_Q_NOAA}, {0x045f10de, 0x00, "NVIDIA MCP65", AHCI_Q_NOAA}, {0x055010de, 0x00, "NVIDIA MCP67", AHCI_Q_NOAA}, {0x055110de, 0x00, "NVIDIA MCP67", AHCI_Q_NOAA}, {0x055210de, 0x00, "NVIDIA MCP67", AHCI_Q_NOAA}, {0x055310de, 0x00, "NVIDIA MCP67", AHCI_Q_NOAA}, {0x055410de, 0x00, "NVIDIA MCP67", AHCI_Q_NOAA}, {0x055510de, 0x00, "NVIDIA MCP67", AHCI_Q_NOAA}, {0x055610de, 0x00, "NVIDIA MCP67", AHCI_Q_NOAA}, {0x055710de, 0x00, "NVIDIA MCP67", AHCI_Q_NOAA}, {0x055810de, 0x00, "NVIDIA MCP67", AHCI_Q_NOAA}, {0x055910de, 0x00, "NVIDIA MCP67", AHCI_Q_NOAA}, {0x055A10de, 0x00, "NVIDIA MCP67", AHCI_Q_NOAA}, {0x055B10de, 0x00, "NVIDIA MCP67", AHCI_Q_NOAA}, {0x058410de, 0x00, "NVIDIA MCP67", AHCI_Q_NOAA}, {0x07f010de, 0x00, "NVIDIA MCP73", AHCI_Q_NOAA}, {0x07f110de, 0x00, "NVIDIA MCP73", AHCI_Q_NOAA}, {0x07f210de, 0x00, "NVIDIA MCP73", AHCI_Q_NOAA}, {0x07f310de, 0x00, "NVIDIA MCP73", AHCI_Q_NOAA}, {0x07f410de, 0x00, "NVIDIA MCP73", AHCI_Q_NOAA}, {0x07f510de, 0x00, "NVIDIA MCP73", AHCI_Q_NOAA}, {0x07f610de, 0x00, "NVIDIA MCP73", AHCI_Q_NOAA}, {0x07f710de, 0x00, "NVIDIA MCP73", AHCI_Q_NOAA}, {0x07f810de, 0x00, "NVIDIA MCP73", AHCI_Q_NOAA}, {0x07f910de, 0x00, "NVIDIA MCP73", AHCI_Q_NOAA}, {0x07fa10de, 0x00, "NVIDIA MCP73", AHCI_Q_NOAA}, {0x07fb10de, 0x00, "NVIDIA MCP73", AHCI_Q_NOAA}, {0x0ad010de, 0x00, "NVIDIA MCP77", AHCI_Q_NOAA}, {0x0ad110de, 0x00, "NVIDIA MCP77", AHCI_Q_NOAA}, {0x0ad210de, 0x00, "NVIDIA MCP77", AHCI_Q_NOAA}, {0x0ad310de, 0x00, "NVIDIA MCP77", AHCI_Q_NOAA}, {0x0ad410de, 0x00, "NVIDIA MCP77", AHCI_Q_NOAA}, {0x0ad510de, 0x00, "NVIDIA MCP77", AHCI_Q_NOAA}, {0x0ad610de, 0x00, "NVIDIA MCP77", AHCI_Q_NOAA}, {0x0ad710de, 0x00, "NVIDIA MCP77", AHCI_Q_NOAA}, {0x0ad810de, 0x00, "NVIDIA MCP77", AHCI_Q_NOAA}, {0x0ad910de, 0x00, "NVIDIA MCP77", AHCI_Q_NOAA}, {0x0ada10de, 0x00, "NVIDIA MCP77", AHCI_Q_NOAA}, {0x0adb10de, 0x00, "NVIDIA MCP77", AHCI_Q_NOAA}, {0x0ab410de, 0x00, "NVIDIA MCP79", AHCI_Q_NOAA}, {0x0ab510de, 0x00, "NVIDIA MCP79", AHCI_Q_NOAA}, {0x0ab610de, 0x00, "NVIDIA MCP79", AHCI_Q_NOAA}, {0x0ab710de, 0x00, "NVIDIA MCP79", AHCI_Q_NOAA}, {0x0ab810de, 0x00, "NVIDIA MCP79", AHCI_Q_NOAA}, {0x0ab910de, 0x00, "NVIDIA MCP79", AHCI_Q_NOAA}, {0x0aba10de, 0x00, "NVIDIA MCP79", AHCI_Q_NOAA}, {0x0abb10de, 0x00, "NVIDIA MCP79", AHCI_Q_NOAA}, {0x0abc10de, 0x00, "NVIDIA MCP79", AHCI_Q_NOAA}, {0x0abd10de, 0x00, "NVIDIA MCP79", AHCI_Q_NOAA}, {0x0abe10de, 0x00, "NVIDIA MCP79", AHCI_Q_NOAA}, {0x0abf10de, 0x00, "NVIDIA MCP79", AHCI_Q_NOAA}, {0x0d8410de, 0x00, "NVIDIA MCP89", AHCI_Q_NOAA}, {0x0d8510de, 0x00, "NVIDIA MCP89", AHCI_Q_NOFORCE|AHCI_Q_NOAA}, {0x0d8610de, 0x00, "NVIDIA MCP89", AHCI_Q_NOAA}, {0x0d8710de, 0x00, "NVIDIA MCP89", AHCI_Q_NOAA}, {0x0d8810de, 0x00, "NVIDIA MCP89", AHCI_Q_NOAA}, {0x0d8910de, 0x00, "NVIDIA MCP89", AHCI_Q_NOAA}, {0x0d8a10de, 0x00, "NVIDIA MCP89", AHCI_Q_NOAA}, {0x0d8b10de, 0x00, "NVIDIA MCP89", AHCI_Q_NOAA}, {0x0d8c10de, 0x00, "NVIDIA MCP89", AHCI_Q_NOAA}, {0x0d8d10de, 0x00, "NVIDIA MCP89", AHCI_Q_NOAA}, {0x0d8e10de, 0x00, "NVIDIA MCP89", AHCI_Q_NOAA}, {0x0d8f10de, 0x00, "NVIDIA MCP89", AHCI_Q_NOAA}, {0x33491106, 0x00, "VIA VT8251", AHCI_Q_NOPMP|AHCI_Q_NONCQ}, {0x62871106, 0x00, "VIA VT8251", AHCI_Q_NOPMP|AHCI_Q_NONCQ}, {0x11841039, 0x00, "SiS 966", 0}, {0x11851039, 0x00, "SiS 968", 0}, {0x01861039, 0x00, "SiS 968", 0}, {0x00000000, 0x00, NULL, 0} }; #define recovery_type spriv_field0 #define RECOVERY_NONE 0 #define RECOVERY_READ_LOG 1 #define RECOVERY_REQUEST_SENSE 2 #define recovery_slot spriv_field1 static int force_ahci = 1; TUNABLE_INT("hw.ahci.force", &force_ahci); static int ahci_probe(device_t dev) { char buf[64]; int i, valid = 0; uint32_t devid = pci_get_devid(dev); uint8_t revid = pci_get_revid(dev); /* Is this a possible AHCI candidate? */ if (pci_get_class(dev) == PCIC_STORAGE && pci_get_subclass(dev) == PCIS_STORAGE_SATA && pci_get_progif(dev) == PCIP_STORAGE_SATA_AHCI_1_0) valid = 1; /* Is this a known AHCI chip? */ for (i = 0; ahci_ids[i].id != 0; i++) { if (ahci_ids[i].id == devid && ahci_ids[i].rev <= revid && (valid || (force_ahci == 1 && !(ahci_ids[i].quirks & AHCI_Q_NOFORCE)))) { /* Do not attach JMicrons with single PCI function. */ if (pci_get_vendor(dev) == 0x197b && (pci_read_config(dev, 0xdf, 1) & 0x40) == 0) return (ENXIO); snprintf(buf, sizeof(buf), "%s AHCI SATA controller", ahci_ids[i].name); device_set_desc_copy(dev, buf); return (BUS_PROBE_VENDOR); } } if (!valid) return (ENXIO); device_set_desc_copy(dev, "AHCI SATA controller"); return (BUS_PROBE_VENDOR); } static int ahci_ata_probe(device_t dev) { char buf[64]; int i; uint32_t devid = pci_get_devid(dev); uint8_t revid = pci_get_revid(dev); if ((intptr_t)device_get_ivars(dev) >= 0) return (ENXIO); /* Is this a known AHCI chip? */ for (i = 0; ahci_ids[i].id != 0; i++) { if (ahci_ids[i].id == devid && ahci_ids[i].rev <= revid) { snprintf(buf, sizeof(buf), "%s AHCI SATA controller", ahci_ids[i].name); device_set_desc_copy(dev, buf); return (BUS_PROBE_VENDOR); } } device_set_desc_copy(dev, "AHCI SATA controller"); return (BUS_PROBE_VENDOR); } static int ahci_attach(device_t dev) { struct ahci_controller *ctlr = device_get_softc(dev); device_t child; int error, unit, speed, i; uint32_t devid = pci_get_devid(dev); uint8_t revid = pci_get_revid(dev); u_int32_t version; ctlr->dev = dev; i = 0; while (ahci_ids[i].id != 0 && (ahci_ids[i].id != devid || ahci_ids[i].rev > revid)) i++; ctlr->quirks = ahci_ids[i].quirks; resource_int_value(device_get_name(dev), device_get_unit(dev), "ccc", &ctlr->ccc); /* if we have a memory BAR(5) we are likely on an AHCI part */ ctlr->r_rid = PCIR_BAR(5); if (!(ctlr->r_mem = bus_alloc_resource_any(dev, SYS_RES_MEMORY, &ctlr->r_rid, RF_ACTIVE))) return ENXIO; /* Setup our own memory management for channels. */ ctlr->sc_iomem.rm_start = rman_get_start(ctlr->r_mem); ctlr->sc_iomem.rm_end = rman_get_end(ctlr->r_mem); ctlr->sc_iomem.rm_type = RMAN_ARRAY; ctlr->sc_iomem.rm_descr = "I/O memory addresses"; if ((error = rman_init(&ctlr->sc_iomem)) != 0) { bus_release_resource(dev, SYS_RES_MEMORY, ctlr->r_rid, ctlr->r_mem); return (error); } if ((error = rman_manage_region(&ctlr->sc_iomem, rman_get_start(ctlr->r_mem), rman_get_end(ctlr->r_mem))) != 0) { bus_release_resource(dev, SYS_RES_MEMORY, ctlr->r_rid, ctlr->r_mem); rman_fini(&ctlr->sc_iomem); return (error); } pci_enable_busmaster(dev); /* Reset controller */ if ((error = ahci_ctlr_reset(dev)) != 0) { bus_release_resource(dev, SYS_RES_MEMORY, ctlr->r_rid, ctlr->r_mem); rman_fini(&ctlr->sc_iomem); return (error); }; /* Get the HW capabilities */ version = ATA_INL(ctlr->r_mem, AHCI_VS); ctlr->caps = ATA_INL(ctlr->r_mem, AHCI_CAP); if (version >= 0x00010200) ctlr->caps2 = ATA_INL(ctlr->r_mem, AHCI_CAP2); if (ctlr->caps & AHCI_CAP_EMS) ctlr->capsem = ATA_INL(ctlr->r_mem, AHCI_EM_CTL); ctlr->ichannels = ATA_INL(ctlr->r_mem, AHCI_PI); /* Identify and set separate quirks for HBA and RAID f/w Marvells. */ if ((ctlr->quirks & AHCI_Q_NOBSYRES) && (ctlr->quirks & AHCI_Q_ALTSIG) && (ctlr->caps & AHCI_CAP_SPM) == 0) ctlr->quirks &= ~AHCI_Q_NOBSYRES; if (ctlr->quirks & AHCI_Q_1CH) { ctlr->caps &= ~AHCI_CAP_NPMASK; ctlr->ichannels &= 0x01; } if (ctlr->quirks & AHCI_Q_2CH) { ctlr->caps &= ~AHCI_CAP_NPMASK; ctlr->caps |= 1; ctlr->ichannels &= 0x03; } if (ctlr->quirks & AHCI_Q_4CH) { ctlr->caps &= ~AHCI_CAP_NPMASK; ctlr->caps |= 3; ctlr->ichannels &= 0x0f; } ctlr->channels = MAX(flsl(ctlr->ichannels), (ctlr->caps & AHCI_CAP_NPMASK) + 1); if (ctlr->quirks & AHCI_Q_NOPMP) ctlr->caps &= ~AHCI_CAP_SPM; if (ctlr->quirks & AHCI_Q_NONCQ) ctlr->caps &= ~AHCI_CAP_SNCQ; if ((ctlr->caps & AHCI_CAP_CCCS) == 0) ctlr->ccc = 0; mtx_init(&ctlr->em_mtx, "AHCI EM lock", NULL, MTX_DEF); ctlr->emloc = ATA_INL(ctlr->r_mem, AHCI_EM_LOC); ahci_ctlr_setup(dev); /* Setup interrupts. */ if (ahci_setup_interrupt(dev)) { bus_release_resource(dev, SYS_RES_MEMORY, ctlr->r_rid, ctlr->r_mem); rman_fini(&ctlr->sc_iomem); return ENXIO; } /* Announce HW capabilities. */ speed = (ctlr->caps & AHCI_CAP_ISS) >> AHCI_CAP_ISS_SHIFT; device_printf(dev, "AHCI v%x.%02x with %d %sGbps ports, Port Multiplier %s%s\n", ((version >> 20) & 0xf0) + ((version >> 16) & 0x0f), ((version >> 4) & 0xf0) + (version & 0x0f), (ctlr->caps & AHCI_CAP_NPMASK) + 1, ((speed == 1) ? "1.5":((speed == 2) ? "3": ((speed == 3) ? "6":"?"))), (ctlr->caps & AHCI_CAP_SPM) ? "supported" : "not supported", (ctlr->caps & AHCI_CAP_FBSS) ? " with FBS" : ""); if (bootverbose) { device_printf(dev, "Caps:%s%s%s%s%s%s%s%s %sGbps", (ctlr->caps & AHCI_CAP_64BIT) ? " 64bit":"", (ctlr->caps & AHCI_CAP_SNCQ) ? " NCQ":"", (ctlr->caps & AHCI_CAP_SSNTF) ? " SNTF":"", (ctlr->caps & AHCI_CAP_SMPS) ? " MPS":"", (ctlr->caps & AHCI_CAP_SSS) ? " SS":"", (ctlr->caps & AHCI_CAP_SALP) ? " ALP":"", (ctlr->caps & AHCI_CAP_SAL) ? " AL":"", (ctlr->caps & AHCI_CAP_SCLO) ? " CLO":"", ((speed == 1) ? "1.5":((speed == 2) ? "3": ((speed == 3) ? "6":"?")))); printf("%s%s%s%s%s%s %dcmd%s%s%s %dports\n", (ctlr->caps & AHCI_CAP_SAM) ? " AM":"", (ctlr->caps & AHCI_CAP_SPM) ? " PM":"", (ctlr->caps & AHCI_CAP_FBSS) ? " FBS":"", (ctlr->caps & AHCI_CAP_PMD) ? " PMD":"", (ctlr->caps & AHCI_CAP_SSC) ? " SSC":"", (ctlr->caps & AHCI_CAP_PSC) ? " PSC":"", ((ctlr->caps & AHCI_CAP_NCS) >> AHCI_CAP_NCS_SHIFT) + 1, (ctlr->caps & AHCI_CAP_CCCS) ? " CCC":"", (ctlr->caps & AHCI_CAP_EMS) ? " EM":"", (ctlr->caps & AHCI_CAP_SXS) ? " eSATA":"", (ctlr->caps & AHCI_CAP_NPMASK) + 1); } if (bootverbose && version >= 0x00010200) { device_printf(dev, "Caps2:%s%s%s\n", (ctlr->caps2 & AHCI_CAP2_APST) ? " APST":"", (ctlr->caps2 & AHCI_CAP2_NVMP) ? " NVMP":"", (ctlr->caps2 & AHCI_CAP2_BOH) ? " BOH":""); } if (bootverbose && (ctlr->caps & AHCI_CAP_EMS)) { device_printf(dev, "EM Caps:%s%s%s%s%s%s%s%s\n", (ctlr->capsem & AHCI_EM_PM) ? " PM":"", (ctlr->capsem & AHCI_EM_ALHD) ? " ALHD":"", (ctlr->capsem & AHCI_EM_XMT) ? " XMT":"", (ctlr->capsem & AHCI_EM_SMB) ? " SMB":"", (ctlr->capsem & AHCI_EM_SGPIO) ? " SGPIO":"", (ctlr->capsem & AHCI_EM_SES2) ? " SES-2":"", (ctlr->capsem & AHCI_EM_SAFTE) ? " SAF-TE":"", (ctlr->capsem & AHCI_EM_LED) ? " LED":""); } /* Attach all channels on this controller */ for (unit = 0; unit < ctlr->channels; unit++) { child = device_add_child(dev, "ahcich", -1); if (child == NULL) { device_printf(dev, "failed to add channel device\n"); continue; } device_set_ivars(child, (void *)(intptr_t)unit); if ((ctlr->ichannels & (1 << unit)) == 0) device_disable(child); } bus_generic_attach(dev); return 0; } static int ahci_detach(device_t dev) { struct ahci_controller *ctlr = device_get_softc(dev); device_t *children; int nchildren, i; /* Detach & delete all children */ if (!device_get_children(dev, &children, &nchildren)) { for (i = 0; i < nchildren; i++) device_delete_child(dev, children[i]); free(children, M_TEMP); } /* Free interrupts. */ for (i = 0; i < ctlr->numirqs; i++) { if (ctlr->irqs[i].r_irq) { bus_teardown_intr(dev, ctlr->irqs[i].r_irq, ctlr->irqs[i].handle); bus_release_resource(dev, SYS_RES_IRQ, ctlr->irqs[i].r_irq_rid, ctlr->irqs[i].r_irq); } } pci_release_msi(dev); /* Free memory. */ rman_fini(&ctlr->sc_iomem); if (ctlr->r_mem) bus_release_resource(dev, SYS_RES_MEMORY, ctlr->r_rid, ctlr->r_mem); mtx_destroy(&ctlr->em_mtx); return (0); } static int ahci_ctlr_reset(device_t dev) { struct ahci_controller *ctlr = device_get_softc(dev); int timeout; if (pci_read_config(dev, 0x00, 4) == 0x28298086 && (pci_read_config(dev, 0x92, 1) & 0xfe) == 0x04) pci_write_config(dev, 0x92, 0x01, 1); /* Enable AHCI mode */ ATA_OUTL(ctlr->r_mem, AHCI_GHC, AHCI_GHC_AE); /* Reset AHCI controller */ ATA_OUTL(ctlr->r_mem, AHCI_GHC, AHCI_GHC_AE|AHCI_GHC_HR); for (timeout = 1000; timeout > 0; timeout--) { DELAY(1000); if ((ATA_INL(ctlr->r_mem, AHCI_GHC) & AHCI_GHC_HR) == 0) break; } if (timeout == 0) { device_printf(dev, "AHCI controller reset failure\n"); return ENXIO; } /* Reenable AHCI mode */ ATA_OUTL(ctlr->r_mem, AHCI_GHC, AHCI_GHC_AE); return (0); } static int ahci_ctlr_setup(device_t dev) { struct ahci_controller *ctlr = device_get_softc(dev); /* Clear interrupts */ ATA_OUTL(ctlr->r_mem, AHCI_IS, ATA_INL(ctlr->r_mem, AHCI_IS)); /* Configure CCC */ if (ctlr->ccc) { ATA_OUTL(ctlr->r_mem, AHCI_CCCP, ATA_INL(ctlr->r_mem, AHCI_PI)); ATA_OUTL(ctlr->r_mem, AHCI_CCCC, (ctlr->ccc << AHCI_CCCC_TV_SHIFT) | (4 << AHCI_CCCC_CC_SHIFT) | AHCI_CCCC_EN); ctlr->cccv = (ATA_INL(ctlr->r_mem, AHCI_CCCC) & AHCI_CCCC_INT_MASK) >> AHCI_CCCC_INT_SHIFT; if (bootverbose) { device_printf(dev, "CCC with %dms/4cmd enabled on vector %d\n", ctlr->ccc, ctlr->cccv); } } /* Enable AHCI interrupts */ ATA_OUTL(ctlr->r_mem, AHCI_GHC, ATA_INL(ctlr->r_mem, AHCI_GHC) | AHCI_GHC_IE); return (0); } static int ahci_suspend(device_t dev) { struct ahci_controller *ctlr = device_get_softc(dev); bus_generic_suspend(dev); /* Disable interupts, so the state change(s) doesn't trigger */ ATA_OUTL(ctlr->r_mem, AHCI_GHC, ATA_INL(ctlr->r_mem, AHCI_GHC) & (~AHCI_GHC_IE)); return 0; } static int ahci_resume(device_t dev) { int res; if ((res = ahci_ctlr_reset(dev)) != 0) return (res); ahci_ctlr_setup(dev); return (bus_generic_resume(dev)); } static int ahci_setup_interrupt(device_t dev) { struct ahci_controller *ctlr = device_get_softc(dev); int i, msi = 1; /* Process hints. */ resource_int_value(device_get_name(dev), device_get_unit(dev), "msi", &msi); if (msi < 0) msi = 0; else if (msi == 1) msi = min(1, pci_msi_count(dev)); else if (msi > 1) msi = pci_msi_count(dev); /* Allocate MSI if needed/present. */ if (msi && pci_alloc_msi(dev, &msi) == 0) { ctlr->numirqs = msi; } else { msi = 0; ctlr->numirqs = 1; } /* Check for single MSI vector fallback. */ if (ctlr->numirqs > 1 && (ATA_INL(ctlr->r_mem, AHCI_GHC) & AHCI_GHC_MRSM) != 0) { device_printf(dev, "Falling back to one MSI\n"); ctlr->numirqs = 1; } /* Allocate all IRQs. */ for (i = 0; i < ctlr->numirqs; i++) { ctlr->irqs[i].ctlr = ctlr; ctlr->irqs[i].r_irq_rid = i + (msi ? 1 : 0); if (ctlr->numirqs == 1 || i >= ctlr->channels || (ctlr->ccc && i == ctlr->cccv)) ctlr->irqs[i].mode = AHCI_IRQ_MODE_ALL; else if (i == ctlr->numirqs - 1) ctlr->irqs[i].mode = AHCI_IRQ_MODE_AFTER; else ctlr->irqs[i].mode = AHCI_IRQ_MODE_ONE; if (!(ctlr->irqs[i].r_irq = bus_alloc_resource_any(dev, SYS_RES_IRQ, &ctlr->irqs[i].r_irq_rid, RF_SHAREABLE | RF_ACTIVE))) { device_printf(dev, "unable to map interrupt\n"); return ENXIO; } if ((bus_setup_intr(dev, ctlr->irqs[i].r_irq, ATA_INTR_FLAGS, NULL, (ctlr->irqs[i].mode == AHCI_IRQ_MODE_ONE) ? ahci_intr_one : ahci_intr, &ctlr->irqs[i], &ctlr->irqs[i].handle))) { /* SOS XXX release r_irq */ device_printf(dev, "unable to setup interrupt\n"); return ENXIO; } } return (0); } /* * Common case interrupt handler. */ static void ahci_intr(void *data) { struct ahci_controller_irq *irq = data; struct ahci_controller *ctlr = irq->ctlr; u_int32_t is, ise = 0; void *arg; int unit; if (irq->mode == AHCI_IRQ_MODE_ALL) { unit = 0; if (ctlr->ccc) is = ctlr->ichannels; else is = ATA_INL(ctlr->r_mem, AHCI_IS); } else { /* AHCI_IRQ_MODE_AFTER */ unit = irq->r_irq_rid - 1; is = ATA_INL(ctlr->r_mem, AHCI_IS); } /* CCC interrupt is edge triggered. */ if (ctlr->ccc) ise = 1 << ctlr->cccv; /* Some controllers have edge triggered IS. */ if (ctlr->quirks & AHCI_Q_EDGEIS) ise |= is; if (ise != 0) ATA_OUTL(ctlr->r_mem, AHCI_IS, ise); for (; unit < ctlr->channels; unit++) { if ((is & (1 << unit)) != 0 && (arg = ctlr->interrupt[unit].argument)) { ctlr->interrupt[unit].function(arg); } } /* AHCI declares level triggered IS. */ if (!(ctlr->quirks & AHCI_Q_EDGEIS)) ATA_OUTL(ctlr->r_mem, AHCI_IS, is); } /* * Simplified interrupt handler for multivector MSI mode. */ static void ahci_intr_one(void *data) { struct ahci_controller_irq *irq = data; struct ahci_controller *ctlr = irq->ctlr; void *arg; int unit; unit = irq->r_irq_rid - 1; /* Some controllers have edge triggered IS. */ if (ctlr->quirks & AHCI_Q_EDGEIS) ATA_OUTL(ctlr->r_mem, AHCI_IS, 1 << unit); if ((arg = ctlr->interrupt[unit].argument)) ctlr->interrupt[unit].function(arg); /* AHCI declares level triggered IS. */ if (!(ctlr->quirks & AHCI_Q_EDGEIS)) ATA_OUTL(ctlr->r_mem, AHCI_IS, 1 << unit); } static struct resource * ahci_alloc_resource(device_t dev, device_t child, int type, int *rid, u_long start, u_long end, u_long count, u_int flags) { struct ahci_controller *ctlr = device_get_softc(dev); int unit = ((struct ahci_channel *)device_get_softc(child))->unit; struct resource *res = NULL; int offset = AHCI_OFFSET + (unit << 7); long st; switch (type) { case SYS_RES_MEMORY: st = rman_get_start(ctlr->r_mem); res = rman_reserve_resource(&ctlr->sc_iomem, st + offset, st + offset + 127, 128, RF_ACTIVE, child); if (res) { bus_space_handle_t bsh; bus_space_tag_t bst; bsh = rman_get_bushandle(ctlr->r_mem); bst = rman_get_bustag(ctlr->r_mem); bus_space_subregion(bst, bsh, offset, 128, &bsh); rman_set_bushandle(res, bsh); rman_set_bustag(res, bst); } break; case SYS_RES_IRQ: if (*rid == ATA_IRQ_RID) res = ctlr->irqs[0].r_irq; break; } return (res); } static int ahci_release_resource(device_t dev, device_t child, int type, int rid, struct resource *r) { switch (type) { case SYS_RES_MEMORY: rman_release_resource(r); return (0); case SYS_RES_IRQ: if (rid != ATA_IRQ_RID) return ENOENT; return (0); } return (EINVAL); } static int ahci_setup_intr(device_t dev, device_t child, struct resource *irq, int flags, driver_filter_t *filter, driver_intr_t *function, void *argument, void **cookiep) { struct ahci_controller *ctlr = device_get_softc(dev); int unit = (intptr_t)device_get_ivars(child); if (filter != NULL) { printf("ahci.c: we cannot use a filter here\n"); return (EINVAL); } ctlr->interrupt[unit].function = function; ctlr->interrupt[unit].argument = argument; return (0); } static int ahci_teardown_intr(device_t dev, device_t child, struct resource *irq, void *cookie) { struct ahci_controller *ctlr = device_get_softc(dev); int unit = (intptr_t)device_get_ivars(child); ctlr->interrupt[unit].function = NULL; ctlr->interrupt[unit].argument = NULL; return (0); } static int ahci_print_child(device_t dev, device_t child) { int retval; retval = bus_print_child_header(dev, child); retval += printf(" at channel %d", (int)(intptr_t)device_get_ivars(child)); retval += bus_print_child_footer(dev, child); return (retval); } static int ahci_child_location_str(device_t dev, device_t child, char *buf, size_t buflen) { snprintf(buf, buflen, "channel=%d", (int)(intptr_t)device_get_ivars(child)); return (0); } devclass_t ahci_devclass; static device_method_t ahci_methods[] = { DEVMETHOD(device_probe, ahci_probe), DEVMETHOD(device_attach, ahci_attach), DEVMETHOD(device_detach, ahci_detach), DEVMETHOD(device_suspend, ahci_suspend), DEVMETHOD(device_resume, ahci_resume), DEVMETHOD(bus_print_child, ahci_print_child), DEVMETHOD(bus_alloc_resource, ahci_alloc_resource), DEVMETHOD(bus_release_resource, ahci_release_resource), DEVMETHOD(bus_setup_intr, ahci_setup_intr), DEVMETHOD(bus_teardown_intr,ahci_teardown_intr), DEVMETHOD(bus_child_location_str, ahci_child_location_str), { 0, 0 } }; static driver_t ahci_driver = { "ahci", ahci_methods, sizeof(struct ahci_controller) }; DRIVER_MODULE(ahci, pci, ahci_driver, ahci_devclass, 0, 0); static device_method_t ahci_ata_methods[] = { DEVMETHOD(device_probe, ahci_ata_probe), DEVMETHOD(device_attach, ahci_attach), DEVMETHOD(device_detach, ahci_detach), DEVMETHOD(device_suspend, ahci_suspend), DEVMETHOD(device_resume, ahci_resume), DEVMETHOD(bus_print_child, ahci_print_child), DEVMETHOD(bus_alloc_resource, ahci_alloc_resource), DEVMETHOD(bus_release_resource, ahci_release_resource), DEVMETHOD(bus_setup_intr, ahci_setup_intr), DEVMETHOD(bus_teardown_intr,ahci_teardown_intr), DEVMETHOD(bus_child_location_str, ahci_child_location_str), { 0, 0 } }; static driver_t ahci_ata_driver = { "ahci", ahci_ata_methods, sizeof(struct ahci_controller) }; DRIVER_MODULE(ahci, atapci, ahci_ata_driver, ahci_devclass, 0, 0); MODULE_VERSION(ahci, 1); MODULE_DEPEND(ahci, cam, 1, 1, 1); static int ahci_ch_probe(device_t dev) { device_set_desc_copy(dev, "AHCI channel"); return (0); } static int ahci_ch_attach(device_t dev) { struct ahci_controller *ctlr = device_get_softc(device_get_parent(dev)); struct ahci_channel *ch = device_get_softc(dev); struct cam_devq *devq; int rid, error, i, sata_rev = 0; u_int32_t version; char buf[32]; ch->dev = dev; ch->unit = (intptr_t)device_get_ivars(dev); ch->caps = ctlr->caps; ch->caps2 = ctlr->caps2; ch->quirks = ctlr->quirks; ch->numslots = ((ch->caps & AHCI_CAP_NCS) >> AHCI_CAP_NCS_SHIFT) + 1; mtx_init(&ch->mtx, "AHCI channel lock", NULL, MTX_DEF); resource_int_value(device_get_name(dev), device_get_unit(dev), "pm_level", &ch->pm_level); if (ch->pm_level > 3) callout_init_mtx(&ch->pm_timer, &ch->mtx, 0); callout_init_mtx(&ch->reset_timer, &ch->mtx, 0); /* Limit speed for my onboard JMicron external port. * It is not eSATA really. */ if (pci_get_devid(ctlr->dev) == 0x2363197b && pci_get_subvendor(ctlr->dev) == 0x1043 && pci_get_subdevice(ctlr->dev) == 0x81e4 && ch->unit == 0) sata_rev = 1; if (ch->quirks & AHCI_Q_SATA2) sata_rev = 2; resource_int_value(device_get_name(dev), device_get_unit(dev), "sata_rev", &sata_rev); for (i = 0; i < 16; i++) { ch->user[i].revision = sata_rev; ch->user[i].mode = 0; ch->user[i].bytecount = 8192; ch->user[i].tags = ch->numslots; ch->user[i].caps = 0; ch->curr[i] = ch->user[i]; if (ch->pm_level) { ch->user[i].caps = CTS_SATA_CAPS_H_PMREQ | CTS_SATA_CAPS_H_APST | CTS_SATA_CAPS_D_PMREQ | CTS_SATA_CAPS_D_APST; } ch->user[i].caps |= CTS_SATA_CAPS_H_DMAAA | CTS_SATA_CAPS_H_AN; } rid = ch->unit; if (!(ch->r_mem = bus_alloc_resource_any(dev, SYS_RES_MEMORY, &rid, RF_ACTIVE))) return (ENXIO); ahci_dmainit(dev); ahci_slotsalloc(dev); ahci_ch_init(dev); mtx_lock(&ch->mtx); rid = ATA_IRQ_RID; if (!(ch->r_irq = bus_alloc_resource_any(dev, SYS_RES_IRQ, &rid, RF_SHAREABLE | RF_ACTIVE))) { device_printf(dev, "Unable to map interrupt\n"); error = ENXIO; goto err0; } if ((bus_setup_intr(dev, ch->r_irq, ATA_INTR_FLAGS, NULL, ahci_ch_intr_locked, dev, &ch->ih))) { device_printf(dev, "Unable to setup interrupt\n"); error = ENXIO; goto err1; } ch->chcaps = ATA_INL(ch->r_mem, AHCI_P_CMD); version = ATA_INL(ctlr->r_mem, AHCI_VS); if (version < 0x00010200 && (ctlr->caps & AHCI_CAP_FBSS)) ch->chcaps |= AHCI_P_CMD_FBSCP; if (bootverbose) { device_printf(dev, "Caps:%s%s%s%s%s\n", (ch->chcaps & AHCI_P_CMD_HPCP) ? " HPCP":"", (ch->chcaps & AHCI_P_CMD_MPSP) ? " MPSP":"", (ch->chcaps & AHCI_P_CMD_CPD) ? " CPD":"", (ch->chcaps & AHCI_P_CMD_ESP) ? " ESP":"", (ch->chcaps & AHCI_P_CMD_FBSCP) ? " FBSCP":""); } /* Create the device queue for our SIM. */ devq = cam_simq_alloc(ch->numslots); if (devq == NULL) { device_printf(dev, "Unable to allocate simq\n"); error = ENOMEM; goto err1; } /* Construct SIM entry */ ch->sim = cam_sim_alloc(ahciaction, ahcipoll, "ahcich", ch, device_get_unit(dev), &ch->mtx, min(2, ch->numslots), (ch->caps & AHCI_CAP_SNCQ) ? ch->numslots : 0, devq); if (ch->sim == NULL) { cam_simq_free(devq); device_printf(dev, "unable to allocate sim\n"); error = ENOMEM; goto err1; } if (xpt_bus_register(ch->sim, dev, 0) != CAM_SUCCESS) { device_printf(dev, "unable to register xpt bus\n"); error = ENXIO; goto err2; } if (xpt_create_path(&ch->path, /*periph*/NULL, cam_sim_path(ch->sim), CAM_TARGET_WILDCARD, CAM_LUN_WILDCARD) != CAM_REQ_CMP) { device_printf(dev, "unable to create path\n"); error = ENXIO; goto err3; } if (ch->pm_level > 3) { callout_reset(&ch->pm_timer, (ch->pm_level == 4) ? hz / 1000 : hz / 8, ahci_ch_pm, dev); } mtx_unlock(&ch->mtx); if ((ch->caps & AHCI_CAP_EMS) && (ctlr->capsem & AHCI_EM_LED)) { for (i = 0; i < AHCI_NUM_LEDS; i++) { ch->leds[i].dev = dev; ch->leds[i].num = i; } if ((ctlr->capsem & AHCI_EM_ALHD) == 0) { snprintf(buf, sizeof(buf), "%s.act", device_get_nameunit(dev)); ch->leds[0].led = led_create(ahci_ch_led, &ch->leds[0], buf); } snprintf(buf, sizeof(buf), "%s.locate", device_get_nameunit(dev)); ch->leds[1].led = led_create(ahci_ch_led, &ch->leds[1], buf); snprintf(buf, sizeof(buf), "%s.fault", device_get_nameunit(dev)); ch->leds[2].led = led_create(ahci_ch_led, &ch->leds[2], buf); } return (0); err3: xpt_bus_deregister(cam_sim_path(ch->sim)); err2: cam_sim_free(ch->sim, /*free_devq*/TRUE); err1: bus_release_resource(dev, SYS_RES_IRQ, ATA_IRQ_RID, ch->r_irq); err0: bus_release_resource(dev, SYS_RES_MEMORY, ch->unit, ch->r_mem); mtx_unlock(&ch->mtx); mtx_destroy(&ch->mtx); return (error); } static int ahci_ch_detach(device_t dev) { struct ahci_channel *ch = device_get_softc(dev); int i; for (i = 0; i < AHCI_NUM_LEDS; i++) { if (ch->leds[i].led) led_destroy(ch->leds[i].led); } mtx_lock(&ch->mtx); xpt_async(AC_LOST_DEVICE, ch->path, NULL); /* Forget about reset. */ if (ch->resetting) { ch->resetting = 0; xpt_release_simq(ch->sim, TRUE); } xpt_free_path(ch->path); xpt_bus_deregister(cam_sim_path(ch->sim)); cam_sim_free(ch->sim, /*free_devq*/TRUE); mtx_unlock(&ch->mtx); if (ch->pm_level > 3) callout_drain(&ch->pm_timer); callout_drain(&ch->reset_timer); bus_teardown_intr(dev, ch->r_irq, ch->ih); bus_release_resource(dev, SYS_RES_IRQ, ATA_IRQ_RID, ch->r_irq); ahci_ch_deinit(dev); ahci_slotsfree(dev); ahci_dmafini(dev); bus_release_resource(dev, SYS_RES_MEMORY, ch->unit, ch->r_mem); mtx_destroy(&ch->mtx); return (0); } static int ahci_ch_init(device_t dev) { struct ahci_channel *ch = device_get_softc(dev); uint64_t work; /* Disable port interrupts */ ATA_OUTL(ch->r_mem, AHCI_P_IE, 0); /* Setup work areas */ work = ch->dma.work_bus + AHCI_CL_OFFSET; ATA_OUTL(ch->r_mem, AHCI_P_CLB, work & 0xffffffff); ATA_OUTL(ch->r_mem, AHCI_P_CLBU, work >> 32); work = ch->dma.rfis_bus; ATA_OUTL(ch->r_mem, AHCI_P_FB, work & 0xffffffff); ATA_OUTL(ch->r_mem, AHCI_P_FBU, work >> 32); /* Activate the channel and power/spin up device */ ATA_OUTL(ch->r_mem, AHCI_P_CMD, (AHCI_P_CMD_ACTIVE | AHCI_P_CMD_POD | AHCI_P_CMD_SUD | ((ch->pm_level == 2 || ch->pm_level == 3) ? AHCI_P_CMD_ALPE : 0) | ((ch->pm_level > 2) ? AHCI_P_CMD_ASP : 0 ))); ahci_start_fr(dev); ahci_start(dev, 1); return (0); } static int ahci_ch_deinit(device_t dev) { struct ahci_channel *ch = device_get_softc(dev); /* Disable port interrupts. */ ATA_OUTL(ch->r_mem, AHCI_P_IE, 0); /* Reset command register. */ ahci_stop(dev); ahci_stop_fr(dev); ATA_OUTL(ch->r_mem, AHCI_P_CMD, 0); /* Allow everything, including partial and slumber modes. */ ATA_OUTL(ch->r_mem, AHCI_P_SCTL, 0); /* Request slumber mode transition and give some time to get there. */ ATA_OUTL(ch->r_mem, AHCI_P_CMD, AHCI_P_CMD_SLUMBER); DELAY(100); /* Disable PHY. */ ATA_OUTL(ch->r_mem, AHCI_P_SCTL, ATA_SC_DET_DISABLE); return (0); } static int ahci_ch_suspend(device_t dev) { struct ahci_channel *ch = device_get_softc(dev); mtx_lock(&ch->mtx); xpt_freeze_simq(ch->sim, 1); /* Forget about reset. */ if (ch->resetting) { ch->resetting = 0; callout_stop(&ch->reset_timer); xpt_release_simq(ch->sim, TRUE); } while (ch->oslots) msleep(ch, &ch->mtx, PRIBIO, "ahcisusp", hz/100); ahci_ch_deinit(dev); mtx_unlock(&ch->mtx); return (0); } static int ahci_ch_resume(device_t dev) { struct ahci_channel *ch = device_get_softc(dev); mtx_lock(&ch->mtx); ahci_ch_init(dev); ahci_reset(dev); xpt_release_simq(ch->sim, TRUE); mtx_unlock(&ch->mtx); return (0); } devclass_t ahcich_devclass; static device_method_t ahcich_methods[] = { DEVMETHOD(device_probe, ahci_ch_probe), DEVMETHOD(device_attach, ahci_ch_attach), DEVMETHOD(device_detach, ahci_ch_detach), DEVMETHOD(device_suspend, ahci_ch_suspend), DEVMETHOD(device_resume, ahci_ch_resume), { 0, 0 } }; static driver_t ahcich_driver = { "ahcich", ahcich_methods, sizeof(struct ahci_channel) }; DRIVER_MODULE(ahcich, ahci, ahcich_driver, ahcich_devclass, 0, 0); static void ahci_ch_setleds(device_t dev) { struct ahci_channel *ch; struct ahci_controller *ctlr; size_t buf; int i, timeout; int16_t val; ctlr = device_get_softc(device_get_parent(dev)); ch = device_get_softc(dev); val = 0; for (i = 0; i < AHCI_NUM_LEDS; i++) val |= ch->leds[i].state << (i * 3); buf = (ctlr->emloc & 0xffff0000) >> 14; mtx_lock(&ctlr->em_mtx); timeout = 1000; while (ATA_INL(ctlr->r_mem, AHCI_EM_CTL) & (AHCI_EM_TM | AHCI_EM_RST) && --timeout > 0) DELAY(1000); if (timeout == 0) device_printf(dev, "EM timeout\n"); ATA_OUTL(ctlr->r_mem, buf, (1 << 8) | (0 << 16) | (0 << 24)); ATA_OUTL(ctlr->r_mem, buf + 4, ch->unit | (val << 16)); ATA_OUTL(ctlr->r_mem, AHCI_EM_CTL, AHCI_EM_TM); mtx_unlock(&ctlr->em_mtx); } static void ahci_ch_led(void *priv, int onoff) { struct ahci_led *led; led = (struct ahci_led *)priv; led->state = onoff; ahci_ch_setleds(led->dev); } struct ahci_dc_cb_args { bus_addr_t maddr; int error; }; static void ahci_dmainit(device_t dev) { struct ahci_channel *ch = device_get_softc(dev); struct ahci_dc_cb_args dcba; size_t rfsize; if (ch->caps & AHCI_CAP_64BIT) ch->dma.max_address = BUS_SPACE_MAXADDR; else ch->dma.max_address = BUS_SPACE_MAXADDR_32BIT; /* Command area. */ if (bus_dma_tag_create(bus_get_dma_tag(dev), 1024, 0, ch->dma.max_address, BUS_SPACE_MAXADDR, NULL, NULL, AHCI_WORK_SIZE, 1, AHCI_WORK_SIZE, 0, NULL, NULL, &ch->dma.work_tag)) goto error; if (bus_dmamem_alloc(ch->dma.work_tag, (void **)&ch->dma.work, 0, &ch->dma.work_map)) goto error; if (bus_dmamap_load(ch->dma.work_tag, ch->dma.work_map, ch->dma.work, AHCI_WORK_SIZE, ahci_dmasetupc_cb, &dcba, 0) || dcba.error) { bus_dmamem_free(ch->dma.work_tag, ch->dma.work, ch->dma.work_map); goto error; } ch->dma.work_bus = dcba.maddr; /* FIS receive area. */ if (ch->chcaps & AHCI_P_CMD_FBSCP) rfsize = 4096; else rfsize = 256; if (bus_dma_tag_create(bus_get_dma_tag(dev), rfsize, 0, ch->dma.max_address, BUS_SPACE_MAXADDR, NULL, NULL, rfsize, 1, rfsize, 0, NULL, NULL, &ch->dma.rfis_tag)) goto error; if (bus_dmamem_alloc(ch->dma.rfis_tag, (void **)&ch->dma.rfis, 0, &ch->dma.rfis_map)) goto error; if (bus_dmamap_load(ch->dma.rfis_tag, ch->dma.rfis_map, ch->dma.rfis, rfsize, ahci_dmasetupc_cb, &dcba, 0) || dcba.error) { bus_dmamem_free(ch->dma.rfis_tag, ch->dma.rfis, ch->dma.rfis_map); goto error; } ch->dma.rfis_bus = dcba.maddr; /* Data area. */ if (bus_dma_tag_create(bus_get_dma_tag(dev), 2, 0, ch->dma.max_address, BUS_SPACE_MAXADDR, NULL, NULL, AHCI_SG_ENTRIES * PAGE_SIZE * ch->numslots, AHCI_SG_ENTRIES, AHCI_PRD_MAX, 0, busdma_lock_mutex, &ch->mtx, &ch->dma.data_tag)) { goto error; } return; error: device_printf(dev, "WARNING - DMA initialization failed\n"); ahci_dmafini(dev); } static void ahci_dmasetupc_cb(void *xsc, bus_dma_segment_t *segs, int nsegs, int error) { struct ahci_dc_cb_args *dcba = (struct ahci_dc_cb_args *)xsc; if (!(dcba->error = error)) dcba->maddr = segs[0].ds_addr; } static void ahci_dmafini(device_t dev) { struct ahci_channel *ch = device_get_softc(dev); if (ch->dma.data_tag) { bus_dma_tag_destroy(ch->dma.data_tag); ch->dma.data_tag = NULL; } if (ch->dma.rfis_bus) { bus_dmamap_unload(ch->dma.rfis_tag, ch->dma.rfis_map); bus_dmamem_free(ch->dma.rfis_tag, ch->dma.rfis, ch->dma.rfis_map); ch->dma.rfis_bus = 0; ch->dma.rfis_map = NULL; ch->dma.rfis = NULL; } if (ch->dma.work_bus) { bus_dmamap_unload(ch->dma.work_tag, ch->dma.work_map); bus_dmamem_free(ch->dma.work_tag, ch->dma.work, ch->dma.work_map); ch->dma.work_bus = 0; ch->dma.work_map = NULL; ch->dma.work = NULL; } if (ch->dma.work_tag) { bus_dma_tag_destroy(ch->dma.work_tag); ch->dma.work_tag = NULL; } } static void ahci_slotsalloc(device_t dev) { struct ahci_channel *ch = device_get_softc(dev); int i; /* Alloc and setup command/dma slots */ bzero(ch->slot, sizeof(ch->slot)); for (i = 0; i < ch->numslots; i++) { struct ahci_slot *slot = &ch->slot[i]; slot->dev = dev; slot->slot = i; slot->state = AHCI_SLOT_EMPTY; slot->ccb = NULL; callout_init_mtx(&slot->timeout, &ch->mtx, 0); if (bus_dmamap_create(ch->dma.data_tag, 0, &slot->dma.data_map)) device_printf(ch->dev, "FAILURE - create data_map\n"); } } static void ahci_slotsfree(device_t dev) { struct ahci_channel *ch = device_get_softc(dev); int i; /* Free all dma slots */ for (i = 0; i < ch->numslots; i++) { struct ahci_slot *slot = &ch->slot[i]; callout_drain(&slot->timeout); if (slot->dma.data_map) { bus_dmamap_destroy(ch->dma.data_tag, slot->dma.data_map); slot->dma.data_map = NULL; } } } static int ahci_phy_check_events(device_t dev, u_int32_t serr) { struct ahci_channel *ch = device_get_softc(dev); if (((ch->pm_level == 0) && (serr & ATA_SE_PHY_CHANGED)) || ((ch->pm_level != 0 || ch->listening) && (serr & ATA_SE_EXCHANGED))) { u_int32_t status = ATA_INL(ch->r_mem, AHCI_P_SSTS); union ccb *ccb; if (bootverbose) { if ((status & ATA_SS_DET_MASK) != ATA_SS_DET_NO_DEVICE) device_printf(dev, "CONNECT requested\n"); else device_printf(dev, "DISCONNECT requested\n"); } ahci_reset(dev); if ((ccb = xpt_alloc_ccb_nowait()) == NULL) return (0); if (xpt_create_path(&ccb->ccb_h.path, NULL, cam_sim_path(ch->sim), CAM_TARGET_WILDCARD, CAM_LUN_WILDCARD) != CAM_REQ_CMP) { xpt_free_ccb(ccb); return (0); } xpt_rescan(ccb); return (1); } return (0); } static void ahci_cpd_check_events(device_t dev) { struct ahci_channel *ch = device_get_softc(dev); u_int32_t status; union ccb *ccb; if (ch->pm_level == 0) return; status = ATA_INL(ch->r_mem, AHCI_P_CMD); if ((status & AHCI_P_CMD_CPD) == 0) return; if (bootverbose) { if (status & AHCI_P_CMD_CPS) { device_printf(dev, "COLD CONNECT requested\n"); } else device_printf(dev, "COLD DISCONNECT requested\n"); } ahci_reset(dev); if ((ccb = xpt_alloc_ccb_nowait()) == NULL) return; if (xpt_create_path(&ccb->ccb_h.path, NULL, cam_sim_path(ch->sim), CAM_TARGET_WILDCARD, CAM_LUN_WILDCARD) != CAM_REQ_CMP) { xpt_free_ccb(ccb); return; } xpt_rescan(ccb); } static void ahci_notify_events(device_t dev, u_int32_t status) { struct ahci_channel *ch = device_get_softc(dev); struct cam_path *dpath; int i; if (ch->caps & AHCI_CAP_SSNTF) ATA_OUTL(ch->r_mem, AHCI_P_SNTF, status); if (bootverbose) device_printf(dev, "SNTF 0x%04x\n", status); for (i = 0; i < 16; i++) { if ((status & (1 << i)) == 0) continue; if (xpt_create_path(&dpath, NULL, xpt_path_path_id(ch->path), i, 0) == CAM_REQ_CMP) { xpt_async(AC_SCSI_AEN, dpath, NULL); xpt_free_path(dpath); } } } static void ahci_ch_intr_locked(void *data) { device_t dev = (device_t)data; struct ahci_channel *ch = device_get_softc(dev); mtx_lock(&ch->mtx); xpt_batch_start(ch->sim); ahci_ch_intr(data); xpt_batch_done(ch->sim); mtx_unlock(&ch->mtx); } static void ahci_ch_pm(void *arg) { device_t dev = (device_t)arg; struct ahci_channel *ch = device_get_softc(dev); uint32_t work; if (ch->numrslots != 0) return; work = ATA_INL(ch->r_mem, AHCI_P_CMD); if (ch->pm_level == 4) work |= AHCI_P_CMD_PARTIAL; else work |= AHCI_P_CMD_SLUMBER; ATA_OUTL(ch->r_mem, AHCI_P_CMD, work); } static void ahci_ch_intr(void *data) { device_t dev = (device_t)data; struct ahci_channel *ch = device_get_softc(dev); uint32_t istatus, sstatus, cstatus, serr = 0, sntf = 0, ok, err; enum ahci_err_type et; int i, ccs, port, reset = 0; /* Read and clear interrupt statuses. */ istatus = ATA_INL(ch->r_mem, AHCI_P_IS); if (istatus == 0) return; ATA_OUTL(ch->r_mem, AHCI_P_IS, istatus); /* Read command statuses. */ sstatus = ATA_INL(ch->r_mem, AHCI_P_SACT); cstatus = ATA_INL(ch->r_mem, AHCI_P_CI); if (istatus & AHCI_P_IX_SDB) { if (ch->caps & AHCI_CAP_SSNTF) sntf = ATA_INL(ch->r_mem, AHCI_P_SNTF); else if (ch->fbs_enabled) { u_int8_t *fis = ch->dma.rfis + 0x58; for (i = 0; i < 16; i++) { if (fis[1] & 0x80) { fis[1] &= 0x7f; sntf |= 1 << i; } fis += 256; } } else { u_int8_t *fis = ch->dma.rfis + 0x58; if (fis[1] & 0x80) sntf = (1 << (fis[1] & 0x0f)); } } /* Process PHY events */ if (istatus & (AHCI_P_IX_PC | AHCI_P_IX_PRC | AHCI_P_IX_OF | AHCI_P_IX_IF | AHCI_P_IX_HBD | AHCI_P_IX_HBF | AHCI_P_IX_TFE)) { serr = ATA_INL(ch->r_mem, AHCI_P_SERR); if (serr) { ATA_OUTL(ch->r_mem, AHCI_P_SERR, serr); reset = ahci_phy_check_events(dev, serr); } } /* Process cold presence detection events */ if ((istatus & AHCI_P_IX_CPD) && !reset) ahci_cpd_check_events(dev); /* Process command errors */ if (istatus & (AHCI_P_IX_OF | AHCI_P_IX_IF | AHCI_P_IX_HBD | AHCI_P_IX_HBF | AHCI_P_IX_TFE)) { ccs = (ATA_INL(ch->r_mem, AHCI_P_CMD) & AHCI_P_CMD_CCS_MASK) >> AHCI_P_CMD_CCS_SHIFT; //device_printf(dev, "%s ERROR is %08x cs %08x ss %08x rs %08x tfd %02x serr %08x fbs %08x ccs %d\n", // __func__, istatus, cstatus, sstatus, ch->rslots, ATA_INL(ch->r_mem, AHCI_P_TFD), // serr, ATA_INL(ch->r_mem, AHCI_P_FBS), ccs); port = -1; if (ch->fbs_enabled) { uint32_t fbs = ATA_INL(ch->r_mem, AHCI_P_FBS); if (fbs & AHCI_P_FBS_SDE) { port = (fbs & AHCI_P_FBS_DWE) >> AHCI_P_FBS_DWE_SHIFT; } else { for (i = 0; i < 16; i++) { if (ch->numrslotspd[i] == 0) continue; if (port == -1) port = i; else if (port != i) { port = -2; break; } } } } err = ch->rslots & (cstatus | sstatus); } else { ccs = 0; err = 0; port = -1; } /* Complete all successfull commands. */ ok = ch->rslots & ~(cstatus | sstatus); for (i = 0; i < ch->numslots; i++) { if ((ok >> i) & 1) ahci_end_transaction(&ch->slot[i], AHCI_ERR_NONE); } /* On error, complete the rest of commands with error statuses. */ if (err) { if (ch->frozen) { union ccb *fccb = ch->frozen; ch->frozen = NULL; fccb->ccb_h.status = CAM_REQUEUE_REQ | CAM_RELEASE_SIMQ; if (!(fccb->ccb_h.status & CAM_DEV_QFRZN)) { xpt_freeze_devq(fccb->ccb_h.path, 1); fccb->ccb_h.status |= CAM_DEV_QFRZN; } xpt_done(fccb); } for (i = 0; i < ch->numslots; i++) { /* XXX: reqests in loading state. */ if (((err >> i) & 1) == 0) continue; if (port >= 0 && ch->slot[i].ccb->ccb_h.target_id != port) continue; if (istatus & AHCI_P_IX_TFE) { if (port != -2) { /* Task File Error */ if (ch->numtslotspd[ ch->slot[i].ccb->ccb_h.target_id] == 0) { /* Untagged operation. */ if (i == ccs) et = AHCI_ERR_TFE; else et = AHCI_ERR_INNOCENT; } else { /* Tagged operation. */ et = AHCI_ERR_NCQ; } } else { et = AHCI_ERR_TFE; ch->fatalerr = 1; } } else if (istatus & AHCI_P_IX_IF) { if (ch->numtslots == 0 && i != ccs && port != -2) et = AHCI_ERR_INNOCENT; else et = AHCI_ERR_SATA; } else et = AHCI_ERR_INVALID; ahci_end_transaction(&ch->slot[i], et); } /* * We can't reinit port if there are some other * commands active, use resume to complete them. */ if (ch->rslots != 0 && !ch->recoverycmd) ATA_OUTL(ch->r_mem, AHCI_P_FBS, AHCI_P_FBS_EN | AHCI_P_FBS_DEC); } /* Process NOTIFY events */ if (sntf) ahci_notify_events(dev, sntf); } /* Must be called with channel locked. */ static int ahci_check_collision(device_t dev, union ccb *ccb) { struct ahci_channel *ch = device_get_softc(dev); int t = ccb->ccb_h.target_id; if ((ccb->ccb_h.func_code == XPT_ATA_IO) && (ccb->ataio.cmd.flags & CAM_ATAIO_FPDMA)) { /* Tagged command while we have no supported tag free. */ if (((~ch->oslots) & (0xffffffff >> (32 - ch->curr[t].tags))) == 0) return (1); /* If we have FBS */ if (ch->fbs_enabled) { /* Tagged command while untagged are active. */ if (ch->numrslotspd[t] != 0 && ch->numtslotspd[t] == 0) return (1); } else { /* Tagged command while untagged are active. */ if (ch->numrslots != 0 && ch->numtslots == 0) return (1); /* Tagged command while tagged to other target is active. */ if (ch->numtslots != 0 && ch->taggedtarget != ccb->ccb_h.target_id) return (1); } } else { /* If we have FBS */ if (ch->fbs_enabled) { /* Untagged command while tagged are active. */ if (ch->numrslotspd[t] != 0 && ch->numtslotspd[t] != 0) return (1); } else { /* Untagged command while tagged are active. */ if (ch->numrslots != 0 && ch->numtslots != 0) return (1); } } if ((ccb->ccb_h.func_code == XPT_ATA_IO) && (ccb->ataio.cmd.flags & (CAM_ATAIO_CONTROL | CAM_ATAIO_NEEDRESULT))) { /* Atomic command while anything active. */ if (ch->numrslots != 0) return (1); } /* We have some atomic command running. */ if (ch->aslots != 0) return (1); return (0); } /* Must be called with channel locked. */ static void ahci_begin_transaction(device_t dev, union ccb *ccb) { struct ahci_channel *ch = device_get_softc(dev); struct ahci_slot *slot; int tag, tags; /* Choose empty slot. */ tags = ch->numslots; if ((ccb->ccb_h.func_code == XPT_ATA_IO) && (ccb->ataio.cmd.flags & CAM_ATAIO_FPDMA)) tags = ch->curr[ccb->ccb_h.target_id].tags; tag = ch->lastslot; while (1) { if (tag >= tags) tag = 0; if (ch->slot[tag].state == AHCI_SLOT_EMPTY) break; tag++; }; ch->lastslot = tag; /* Occupy chosen slot. */ slot = &ch->slot[tag]; slot->ccb = ccb; /* Stop PM timer. */ if (ch->numrslots == 0 && ch->pm_level > 3) callout_stop(&ch->pm_timer); /* Update channel stats. */ ch->oslots |= (1 << slot->slot); ch->numrslots++; ch->numrslotspd[ccb->ccb_h.target_id]++; if ((ccb->ccb_h.func_code == XPT_ATA_IO) && (ccb->ataio.cmd.flags & CAM_ATAIO_FPDMA)) { ch->numtslots++; ch->numtslotspd[ccb->ccb_h.target_id]++; ch->taggedtarget = ccb->ccb_h.target_id; } if ((ccb->ccb_h.func_code == XPT_ATA_IO) && (ccb->ataio.cmd.flags & (CAM_ATAIO_CONTROL | CAM_ATAIO_NEEDRESULT))) ch->aslots |= (1 << slot->slot); slot->dma.nsegs = 0; /* If request moves data, setup and load SG list */ if ((ccb->ccb_h.flags & CAM_DIR_MASK) != CAM_DIR_NONE) { void *buf; bus_size_t size; slot->state = AHCI_SLOT_LOADING; if (ccb->ccb_h.func_code == XPT_ATA_IO) { buf = ccb->ataio.data_ptr; size = ccb->ataio.dxfer_len; } else { buf = ccb->csio.data_ptr; size = ccb->csio.dxfer_len; } bus_dmamap_load(ch->dma.data_tag, slot->dma.data_map, buf, size, ahci_dmasetprd, slot, 0); } else ahci_execute_transaction(slot); } /* Locked by busdma engine. */ static void ahci_dmasetprd(void *arg, bus_dma_segment_t *segs, int nsegs, int error) { struct ahci_slot *slot = arg; struct ahci_channel *ch = device_get_softc(slot->dev); struct ahci_cmd_tab *ctp; struct ahci_dma_prd *prd; int i; if (error) { device_printf(slot->dev, "DMA load error\n"); ahci_end_transaction(slot, AHCI_ERR_INVALID); return; } KASSERT(nsegs <= AHCI_SG_ENTRIES, ("too many DMA segment entries\n")); /* Get a piece of the workspace for this request */ ctp = (struct ahci_cmd_tab *) (ch->dma.work + AHCI_CT_OFFSET + (AHCI_CT_SIZE * slot->slot)); /* Fill S/G table */ prd = &ctp->prd_tab[0]; for (i = 0; i < nsegs; i++) { prd[i].dba = htole64(segs[i].ds_addr); prd[i].dbc = htole32((segs[i].ds_len - 1) & AHCI_PRD_MASK); } slot->dma.nsegs = nsegs; bus_dmamap_sync(ch->dma.data_tag, slot->dma.data_map, ((slot->ccb->ccb_h.flags & CAM_DIR_IN) ? BUS_DMASYNC_PREREAD : BUS_DMASYNC_PREWRITE)); ahci_execute_transaction(slot); } /* Must be called with channel locked. */ static void ahci_execute_transaction(struct ahci_slot *slot) { device_t dev = slot->dev; struct ahci_channel *ch = device_get_softc(dev); struct ahci_cmd_tab *ctp; struct ahci_cmd_list *clp; union ccb *ccb = slot->ccb; int port = ccb->ccb_h.target_id & 0x0f; int fis_size, i, softreset; uint8_t *fis = ch->dma.rfis + 0x40; uint8_t val; /* Get a piece of the workspace for this request */ ctp = (struct ahci_cmd_tab *) (ch->dma.work + AHCI_CT_OFFSET + (AHCI_CT_SIZE * slot->slot)); /* Setup the FIS for this request */ if (!(fis_size = ahci_setup_fis(dev, ctp, ccb, slot->slot))) { device_printf(ch->dev, "Setting up SATA FIS failed\n"); ahci_end_transaction(slot, AHCI_ERR_INVALID); return; } /* Setup the command list entry */ clp = (struct ahci_cmd_list *) (ch->dma.work + AHCI_CL_OFFSET + (AHCI_CL_SIZE * slot->slot)); clp->cmd_flags = htole16( (ccb->ccb_h.flags & CAM_DIR_OUT ? AHCI_CMD_WRITE : 0) | (ccb->ccb_h.func_code == XPT_SCSI_IO ? (AHCI_CMD_ATAPI | AHCI_CMD_PREFETCH) : 0) | (fis_size / sizeof(u_int32_t)) | (port << 12)); clp->prd_length = htole16(slot->dma.nsegs); /* Special handling for Soft Reset command. */ if ((ccb->ccb_h.func_code == XPT_ATA_IO) && (ccb->ataio.cmd.flags & CAM_ATAIO_CONTROL)) { if (ccb->ataio.cmd.control & ATA_A_RESET) { softreset = 1; /* Kick controller into sane state */ ahci_stop(dev); ahci_clo(dev); ahci_start(dev, 0); clp->cmd_flags |= AHCI_CMD_RESET | AHCI_CMD_CLR_BUSY; } else { softreset = 2; /* Prepare FIS receive area for check. */ for (i = 0; i < 20; i++) fis[i] = 0xff; } } else softreset = 0; clp->bytecount = 0; clp->cmd_table_phys = htole64(ch->dma.work_bus + AHCI_CT_OFFSET + (AHCI_CT_SIZE * slot->slot)); bus_dmamap_sync(ch->dma.work_tag, ch->dma.work_map, BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); bus_dmamap_sync(ch->dma.rfis_tag, ch->dma.rfis_map, BUS_DMASYNC_PREREAD); /* Set ACTIVE bit for NCQ commands. */ if ((ccb->ccb_h.func_code == XPT_ATA_IO) && (ccb->ataio.cmd.flags & CAM_ATAIO_FPDMA)) { ATA_OUTL(ch->r_mem, AHCI_P_SACT, 1 << slot->slot); } /* If FBS is enabled, set PMP port. */ if (ch->fbs_enabled) { ATA_OUTL(ch->r_mem, AHCI_P_FBS, AHCI_P_FBS_EN | (port << AHCI_P_FBS_DEV_SHIFT)); } /* Issue command to the controller. */ slot->state = AHCI_SLOT_RUNNING; ch->rslots |= (1 << slot->slot); ATA_OUTL(ch->r_mem, AHCI_P_CI, (1 << slot->slot)); /* Device reset commands doesn't interrupt. Poll them. */ if (ccb->ccb_h.func_code == XPT_ATA_IO && (ccb->ataio.cmd.command == ATA_DEVICE_RESET || softreset)) { int count, timeout = ccb->ccb_h.timeout * 100; enum ahci_err_type et = AHCI_ERR_NONE; for (count = 0; count < timeout; count++) { DELAY(10); if (!(ATA_INL(ch->r_mem, AHCI_P_CI) & (1 << slot->slot))) break; if ((ATA_INL(ch->r_mem, AHCI_P_TFD) & ATA_S_ERROR) && softreset != 1) { #if 0 device_printf(ch->dev, "Poll error on slot %d, TFD: %04x\n", slot->slot, ATA_INL(ch->r_mem, AHCI_P_TFD)); #endif et = AHCI_ERR_TFE; break; } /* Workaround for ATI SB600/SB700 chipsets. */ if (ccb->ccb_h.target_id == 15 && pci_get_vendor(device_get_parent(dev)) == 0x1002 && (ATA_INL(ch->r_mem, AHCI_P_IS) & AHCI_P_IX_IPM)) { et = AHCI_ERR_TIMEOUT; break; } } /* Marvell controllers do not wait for readyness. */ if ((ch->quirks & AHCI_Q_NOBSYRES) && softreset == 2 && et == AHCI_ERR_NONE) { while ((val = fis[2]) & ATA_S_BUSY) { DELAY(10); if (count++ >= timeout) break; } } if (timeout && (count >= timeout)) { device_printf(dev, "Poll timeout on slot %d port %d\n", slot->slot, port); device_printf(dev, "is %08x cs %08x ss %08x " "rs %08x tfd %02x serr %08x cmd %08x\n", ATA_INL(ch->r_mem, AHCI_P_IS), ATA_INL(ch->r_mem, AHCI_P_CI), ATA_INL(ch->r_mem, AHCI_P_SACT), ch->rslots, ATA_INL(ch->r_mem, AHCI_P_TFD), ATA_INL(ch->r_mem, AHCI_P_SERR), ATA_INL(ch->r_mem, AHCI_P_CMD)); et = AHCI_ERR_TIMEOUT; } /* Kick controller into sane state and enable FBS. */ if (softreset == 2) ch->eslots |= (1 << slot->slot); ahci_end_transaction(slot, et); return; } /* Start command execution timeout */ callout_reset(&slot->timeout, (int)ccb->ccb_h.timeout * hz / 2000, (timeout_t*)ahci_timeout, slot); return; } /* Must be called with channel locked. */ static void ahci_process_timeout(device_t dev) { struct ahci_channel *ch = device_get_softc(dev); int i; mtx_assert(&ch->mtx, MA_OWNED); /* Handle the rest of commands. */ for (i = 0; i < ch->numslots; i++) { /* Do we have a running request on slot? */ if (ch->slot[i].state < AHCI_SLOT_RUNNING) continue; ahci_end_transaction(&ch->slot[i], AHCI_ERR_TIMEOUT); } } /* Must be called with channel locked. */ static void ahci_rearm_timeout(device_t dev) { struct ahci_channel *ch = device_get_softc(dev); int i; mtx_assert(&ch->mtx, MA_OWNED); for (i = 0; i < ch->numslots; i++) { struct ahci_slot *slot = &ch->slot[i]; /* Do we have a running request on slot? */ if (slot->state < AHCI_SLOT_RUNNING) continue; if ((ch->toslots & (1 << i)) == 0) continue; callout_reset(&slot->timeout, (int)slot->ccb->ccb_h.timeout * hz / 2000, (timeout_t*)ahci_timeout, slot); } } /* Locked by callout mechanism. */ static void ahci_timeout(struct ahci_slot *slot) { device_t dev = slot->dev; struct ahci_channel *ch = device_get_softc(dev); uint32_t sstatus; int ccs; int i; /* Check for stale timeout. */ if (slot->state < AHCI_SLOT_RUNNING) return; /* Check if slot was not being executed last time we checked. */ if (slot->state < AHCI_SLOT_EXECUTING) { /* Check if slot started executing. */ sstatus = ATA_INL(ch->r_mem, AHCI_P_SACT); ccs = (ATA_INL(ch->r_mem, AHCI_P_CMD) & AHCI_P_CMD_CCS_MASK) >> AHCI_P_CMD_CCS_SHIFT; if ((sstatus & (1 << slot->slot)) != 0 || ccs == slot->slot || ch->fbs_enabled || ch->wrongccs) slot->state = AHCI_SLOT_EXECUTING; else if ((ch->rslots & (1 << ccs)) == 0) { ch->wrongccs = 1; slot->state = AHCI_SLOT_EXECUTING; } callout_reset(&slot->timeout, (int)slot->ccb->ccb_h.timeout * hz / 2000, (timeout_t*)ahci_timeout, slot); return; } device_printf(dev, "Timeout on slot %d port %d\n", slot->slot, slot->ccb->ccb_h.target_id & 0x0f); device_printf(dev, "is %08x cs %08x ss %08x rs %08x tfd %02x " "serr %08x cmd %08x\n", ATA_INL(ch->r_mem, AHCI_P_IS), ATA_INL(ch->r_mem, AHCI_P_CI), ATA_INL(ch->r_mem, AHCI_P_SACT), ch->rslots, ATA_INL(ch->r_mem, AHCI_P_TFD), ATA_INL(ch->r_mem, AHCI_P_SERR), ATA_INL(ch->r_mem, AHCI_P_CMD)); /* Handle frozen command. */ if (ch->frozen) { union ccb *fccb = ch->frozen; ch->frozen = NULL; fccb->ccb_h.status = CAM_REQUEUE_REQ | CAM_RELEASE_SIMQ; if (!(fccb->ccb_h.status & CAM_DEV_QFRZN)) { xpt_freeze_devq(fccb->ccb_h.path, 1); fccb->ccb_h.status |= CAM_DEV_QFRZN; } xpt_done(fccb); } if (!ch->fbs_enabled && !ch->wrongccs) { /* Without FBS we know real timeout source. */ ch->fatalerr = 1; /* Handle command with timeout. */ ahci_end_transaction(&ch->slot[slot->slot], AHCI_ERR_TIMEOUT); /* Handle the rest of commands. */ for (i = 0; i < ch->numslots; i++) { /* Do we have a running request on slot? */ if (ch->slot[i].state < AHCI_SLOT_RUNNING) continue; ahci_end_transaction(&ch->slot[i], AHCI_ERR_INNOCENT); } } else { /* With FBS we wait for other commands timeout and pray. */ if (ch->toslots == 0) xpt_freeze_simq(ch->sim, 1); ch->toslots |= (1 << slot->slot); if ((ch->rslots & ~ch->toslots) == 0) ahci_process_timeout(dev); else device_printf(dev, " ... waiting for slots %08x\n", ch->rslots & ~ch->toslots); } } /* Must be called with channel locked. */ static void ahci_end_transaction(struct ahci_slot *slot, enum ahci_err_type et) { device_t dev = slot->dev; struct ahci_channel *ch = device_get_softc(dev); union ccb *ccb = slot->ccb; struct ahci_cmd_list *clp; int lastto; uint32_t sig; bus_dmamap_sync(ch->dma.work_tag, ch->dma.work_map, BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); clp = (struct ahci_cmd_list *) (ch->dma.work + AHCI_CL_OFFSET + (AHCI_CL_SIZE * slot->slot)); /* Read result registers to the result struct * May be incorrect if several commands finished same time, * so read only when sure or have to. */ if (ccb->ccb_h.func_code == XPT_ATA_IO) { struct ata_res *res = &ccb->ataio.res; if ((et == AHCI_ERR_TFE) || (ccb->ataio.cmd.flags & CAM_ATAIO_NEEDRESULT)) { u_int8_t *fis = ch->dma.rfis + 0x40; bus_dmamap_sync(ch->dma.rfis_tag, ch->dma.rfis_map, BUS_DMASYNC_POSTREAD); if (ch->fbs_enabled) { fis += ccb->ccb_h.target_id * 256; res->status = fis[2]; res->error = fis[3]; } else { uint16_t tfd = ATA_INL(ch->r_mem, AHCI_P_TFD); res->status = tfd; res->error = tfd >> 8; } res->lba_low = fis[4]; res->lba_mid = fis[5]; res->lba_high = fis[6]; res->device = fis[7]; res->lba_low_exp = fis[8]; res->lba_mid_exp = fis[9]; res->lba_high_exp = fis[10]; res->sector_count = fis[12]; res->sector_count_exp = fis[13]; /* * Some weird controllers do not return signature in * FIS receive area. Read it from PxSIG register. */ if ((ch->quirks & AHCI_Q_ALTSIG) && (ccb->ataio.cmd.flags & CAM_ATAIO_CONTROL) && (ccb->ataio.cmd.control & ATA_A_RESET) == 0) { sig = ATA_INL(ch->r_mem, AHCI_P_SIG); res->lba_high = sig >> 24; res->lba_mid = sig >> 16; res->lba_low = sig >> 8; res->sector_count = sig; } } else bzero(res, sizeof(*res)); if ((ccb->ataio.cmd.flags & CAM_ATAIO_FPDMA) == 0 && (ccb->ccb_h.flags & CAM_DIR_MASK) != CAM_DIR_NONE && (ch->quirks & AHCI_Q_NOCOUNT) == 0) { ccb->ataio.resid = ccb->ataio.dxfer_len - le32toh(clp->bytecount); } } else { if ((ccb->ccb_h.flags & CAM_DIR_MASK) != CAM_DIR_NONE && (ch->quirks & AHCI_Q_NOCOUNT) == 0) { ccb->csio.resid = ccb->csio.dxfer_len - le32toh(clp->bytecount); } } if ((ccb->ccb_h.flags & CAM_DIR_MASK) != CAM_DIR_NONE) { bus_dmamap_sync(ch->dma.data_tag, slot->dma.data_map, (ccb->ccb_h.flags & CAM_DIR_IN) ? BUS_DMASYNC_POSTREAD : BUS_DMASYNC_POSTWRITE); bus_dmamap_unload(ch->dma.data_tag, slot->dma.data_map); } if (et != AHCI_ERR_NONE) ch->eslots |= (1 << slot->slot); /* In case of error, freeze device for proper recovery. */ if ((et != AHCI_ERR_NONE) && (!ch->recoverycmd) && !(ccb->ccb_h.status & CAM_DEV_QFRZN)) { xpt_freeze_devq(ccb->ccb_h.path, 1); ccb->ccb_h.status |= CAM_DEV_QFRZN; } /* Set proper result status. */ ccb->ccb_h.status &= ~CAM_STATUS_MASK; switch (et) { case AHCI_ERR_NONE: ccb->ccb_h.status |= CAM_REQ_CMP; if (ccb->ccb_h.func_code == XPT_SCSI_IO) ccb->csio.scsi_status = SCSI_STATUS_OK; break; case AHCI_ERR_INVALID: ch->fatalerr = 1; ccb->ccb_h.status |= CAM_REQ_INVALID; break; case AHCI_ERR_INNOCENT: ccb->ccb_h.status |= CAM_REQUEUE_REQ; break; case AHCI_ERR_TFE: case AHCI_ERR_NCQ: if (ccb->ccb_h.func_code == XPT_SCSI_IO) { ccb->ccb_h.status |= CAM_SCSI_STATUS_ERROR; ccb->csio.scsi_status = SCSI_STATUS_CHECK_COND; } else { ccb->ccb_h.status |= CAM_ATA_STATUS_ERROR; } break; case AHCI_ERR_SATA: ch->fatalerr = 1; if (!ch->recoverycmd) { xpt_freeze_simq(ch->sim, 1); ccb->ccb_h.status &= ~CAM_STATUS_MASK; ccb->ccb_h.status |= CAM_RELEASE_SIMQ; } ccb->ccb_h.status |= CAM_UNCOR_PARITY; break; case AHCI_ERR_TIMEOUT: if (!ch->recoverycmd) { xpt_freeze_simq(ch->sim, 1); ccb->ccb_h.status &= ~CAM_STATUS_MASK; ccb->ccb_h.status |= CAM_RELEASE_SIMQ; } ccb->ccb_h.status |= CAM_CMD_TIMEOUT; break; default: ch->fatalerr = 1; ccb->ccb_h.status |= CAM_REQ_CMP_ERR; } /* Free slot. */ ch->oslots &= ~(1 << slot->slot); ch->rslots &= ~(1 << slot->slot); ch->aslots &= ~(1 << slot->slot); slot->state = AHCI_SLOT_EMPTY; slot->ccb = NULL; /* Update channel stats. */ ch->numrslots--; ch->numrslotspd[ccb->ccb_h.target_id]--; if ((ccb->ccb_h.func_code == XPT_ATA_IO) && (ccb->ataio.cmd.flags & CAM_ATAIO_FPDMA)) { ch->numtslots--; ch->numtslotspd[ccb->ccb_h.target_id]--; } /* Cancel timeout state if request completed normally. */ if (et != AHCI_ERR_TIMEOUT) { lastto = (ch->toslots == (1 << slot->slot)); ch->toslots &= ~(1 << slot->slot); if (lastto) xpt_release_simq(ch->sim, TRUE); } /* If it was first request of reset sequence and there is no error, * proceed to second request. */ if ((ccb->ccb_h.func_code == XPT_ATA_IO) && (ccb->ataio.cmd.flags & CAM_ATAIO_CONTROL) && (ccb->ataio.cmd.control & ATA_A_RESET) && et == AHCI_ERR_NONE) { ccb->ataio.cmd.control &= ~ATA_A_RESET; ahci_begin_transaction(dev, ccb); return; } /* If it was our READ LOG command - process it. */ if (ccb->ccb_h.recovery_type == RECOVERY_READ_LOG) { ahci_process_read_log(dev, ccb); /* If it was our REQUEST SENSE command - process it. */ } else if (ccb->ccb_h.recovery_type == RECOVERY_REQUEST_SENSE) { ahci_process_request_sense(dev, ccb); /* If it was NCQ or ATAPI command error, put result on hold. */ } else if (et == AHCI_ERR_NCQ || ((ccb->ccb_h.status & CAM_STATUS_MASK) == CAM_SCSI_STATUS_ERROR && (ccb->ccb_h.flags & CAM_DIS_AUTOSENSE) == 0)) { ch->hold[slot->slot] = ccb; ch->numhslots++; } else xpt_done(ccb); /* If we have no other active commands, ... */ if (ch->rslots == 0) { /* if there was fatal error - reset port. */ if (ch->toslots != 0 || ch->fatalerr) { ahci_reset(dev); } else { /* if we have slots in error, we can reinit port. */ if (ch->eslots != 0) { ahci_stop(dev); ahci_clo(dev); ahci_start(dev, 1); } /* if there commands on hold, we can do READ LOG. */ if (!ch->recoverycmd && ch->numhslots) ahci_issue_recovery(dev); } /* If all the rest of commands are in timeout - give them chance. */ } else if ((ch->rslots & ~ch->toslots) == 0 && et != AHCI_ERR_TIMEOUT) ahci_rearm_timeout(dev); /* Unfreeze frozen command. */ if (ch->frozen && !ahci_check_collision(dev, ch->frozen)) { union ccb *fccb = ch->frozen; ch->frozen = NULL; ahci_begin_transaction(dev, fccb); xpt_release_simq(ch->sim, TRUE); } /* Start PM timer. */ if (ch->numrslots == 0 && ch->pm_level > 3 && (ch->curr[ch->pm_present ? 15 : 0].caps & CTS_SATA_CAPS_D_PMREQ)) { callout_schedule(&ch->pm_timer, (ch->pm_level == 4) ? hz / 1000 : hz / 8); } } static void ahci_issue_recovery(device_t dev) { struct ahci_channel *ch = device_get_softc(dev); union ccb *ccb; struct ccb_ataio *ataio; struct ccb_scsiio *csio; int i; /* Find some held command. */ for (i = 0; i < ch->numslots; i++) { if (ch->hold[i]) break; } ccb = xpt_alloc_ccb_nowait(); if (ccb == NULL) { device_printf(dev, "Unable to allocate recovery command\n"); completeall: /* We can't do anything -- complete held commands. */ for (i = 0; i < ch->numslots; i++) { if (ch->hold[i] == NULL) continue; ch->hold[i]->ccb_h.status &= ~CAM_STATUS_MASK; ch->hold[i]->ccb_h.status |= CAM_RESRC_UNAVAIL; xpt_done(ch->hold[i]); ch->hold[i] = NULL; ch->numhslots--; } ahci_reset(dev); return; } ccb->ccb_h = ch->hold[i]->ccb_h; /* Reuse old header. */ if (ccb->ccb_h.func_code == XPT_ATA_IO) { /* READ LOG */ ccb->ccb_h.recovery_type = RECOVERY_READ_LOG; ccb->ccb_h.func_code = XPT_ATA_IO; ccb->ccb_h.flags = CAM_DIR_IN; ccb->ccb_h.timeout = 1000; /* 1s should be enough. */ ataio = &ccb->ataio; ataio->data_ptr = malloc(512, M_AHCI, M_NOWAIT); if (ataio->data_ptr == NULL) { xpt_free_ccb(ccb); device_printf(dev, "Unable to allocate memory for READ LOG command\n"); goto completeall; } ataio->dxfer_len = 512; bzero(&ataio->cmd, sizeof(ataio->cmd)); ataio->cmd.flags = CAM_ATAIO_48BIT; ataio->cmd.command = 0x2F; /* READ LOG EXT */ ataio->cmd.sector_count = 1; ataio->cmd.sector_count_exp = 0; ataio->cmd.lba_low = 0x10; ataio->cmd.lba_mid = 0; ataio->cmd.lba_mid_exp = 0; } else { /* REQUEST SENSE */ ccb->ccb_h.recovery_type = RECOVERY_REQUEST_SENSE; ccb->ccb_h.recovery_slot = i; ccb->ccb_h.func_code = XPT_SCSI_IO; ccb->ccb_h.flags = CAM_DIR_IN; ccb->ccb_h.status = 0; ccb->ccb_h.timeout = 1000; /* 1s should be enough. */ csio = &ccb->csio; csio->data_ptr = (void *)&ch->hold[i]->csio.sense_data; csio->dxfer_len = ch->hold[i]->csio.sense_len; csio->cdb_len = 6; bzero(&csio->cdb_io, sizeof(csio->cdb_io)); csio->cdb_io.cdb_bytes[0] = 0x03; csio->cdb_io.cdb_bytes[4] = csio->dxfer_len; } /* Freeze SIM while doing recovery. */ ch->recoverycmd = 1; xpt_freeze_simq(ch->sim, 1); ahci_begin_transaction(dev, ccb); } static void ahci_process_read_log(device_t dev, union ccb *ccb) { struct ahci_channel *ch = device_get_softc(dev); uint8_t *data; struct ata_res *res; int i; ch->recoverycmd = 0; data = ccb->ataio.data_ptr; if ((ccb->ccb_h.status & CAM_STATUS_MASK) == CAM_REQ_CMP && (data[0] & 0x80) == 0) { for (i = 0; i < ch->numslots; i++) { if (!ch->hold[i]) continue; if (ch->hold[i]->ccb_h.func_code != XPT_ATA_IO) continue; if ((data[0] & 0x1F) == i) { res = &ch->hold[i]->ataio.res; res->status = data[2]; res->error = data[3]; res->lba_low = data[4]; res->lba_mid = data[5]; res->lba_high = data[6]; res->device = data[7]; res->lba_low_exp = data[8]; res->lba_mid_exp = data[9]; res->lba_high_exp = data[10]; res->sector_count = data[12]; res->sector_count_exp = data[13]; } else { ch->hold[i]->ccb_h.status &= ~CAM_STATUS_MASK; ch->hold[i]->ccb_h.status |= CAM_REQUEUE_REQ; } xpt_done(ch->hold[i]); ch->hold[i] = NULL; ch->numhslots--; } } else { if ((ccb->ccb_h.status & CAM_STATUS_MASK) != CAM_REQ_CMP) device_printf(dev, "Error while READ LOG EXT\n"); else if ((data[0] & 0x80) == 0) { device_printf(dev, "Non-queued command error in READ LOG EXT\n"); } for (i = 0; i < ch->numslots; i++) { if (!ch->hold[i]) continue; if (ch->hold[i]->ccb_h.func_code != XPT_ATA_IO) continue; xpt_done(ch->hold[i]); ch->hold[i] = NULL; ch->numhslots--; } } free(ccb->ataio.data_ptr, M_AHCI); xpt_free_ccb(ccb); xpt_release_simq(ch->sim, TRUE); } static void ahci_process_request_sense(device_t dev, union ccb *ccb) { struct ahci_channel *ch = device_get_softc(dev); int i; ch->recoverycmd = 0; i = ccb->ccb_h.recovery_slot; if ((ccb->ccb_h.status & CAM_STATUS_MASK) == CAM_REQ_CMP) { ch->hold[i]->ccb_h.status |= CAM_AUTOSNS_VALID; } else { ch->hold[i]->ccb_h.status &= ~CAM_STATUS_MASK; ch->hold[i]->ccb_h.status |= CAM_AUTOSENSE_FAIL; } xpt_done(ch->hold[i]); ch->hold[i] = NULL; ch->numhslots--; xpt_free_ccb(ccb); xpt_release_simq(ch->sim, TRUE); } static void ahci_start(device_t dev, int fbs) { struct ahci_channel *ch = device_get_softc(dev); u_int32_t cmd; /* Clear SATA error register */ ATA_OUTL(ch->r_mem, AHCI_P_SERR, 0xFFFFFFFF); /* Clear any interrupts pending on this channel */ ATA_OUTL(ch->r_mem, AHCI_P_IS, 0xFFFFFFFF); /* Configure FIS-based switching if supported. */ if (ch->chcaps & AHCI_P_CMD_FBSCP) { ch->fbs_enabled = (fbs && ch->pm_present) ? 1 : 0; ATA_OUTL(ch->r_mem, AHCI_P_FBS, ch->fbs_enabled ? AHCI_P_FBS_EN : 0); } /* Start operations on this channel */ cmd = ATA_INL(ch->r_mem, AHCI_P_CMD); cmd &= ~AHCI_P_CMD_PMA; ATA_OUTL(ch->r_mem, AHCI_P_CMD, cmd | AHCI_P_CMD_ST | (ch->pm_present ? AHCI_P_CMD_PMA : 0)); } static void ahci_stop(device_t dev) { struct ahci_channel *ch = device_get_softc(dev); u_int32_t cmd; int timeout; /* Kill all activity on this channel */ cmd = ATA_INL(ch->r_mem, AHCI_P_CMD); ATA_OUTL(ch->r_mem, AHCI_P_CMD, cmd & ~AHCI_P_CMD_ST); /* Wait for activity stop. */ timeout = 0; do { DELAY(10); if (timeout++ > 50000) { device_printf(dev, "stopping AHCI engine failed\n"); break; } } while (ATA_INL(ch->r_mem, AHCI_P_CMD) & AHCI_P_CMD_CR); ch->eslots = 0; } static void ahci_clo(device_t dev) { struct ahci_channel *ch = device_get_softc(dev); u_int32_t cmd; int timeout; /* Issue Command List Override if supported */ if (ch->caps & AHCI_CAP_SCLO) { cmd = ATA_INL(ch->r_mem, AHCI_P_CMD); cmd |= AHCI_P_CMD_CLO; ATA_OUTL(ch->r_mem, AHCI_P_CMD, cmd); timeout = 0; do { DELAY(10); if (timeout++ > 50000) { device_printf(dev, "executing CLO failed\n"); break; } } while (ATA_INL(ch->r_mem, AHCI_P_CMD) & AHCI_P_CMD_CLO); } } static void ahci_stop_fr(device_t dev) { struct ahci_channel *ch = device_get_softc(dev); u_int32_t cmd; int timeout; /* Kill all FIS reception on this channel */ cmd = ATA_INL(ch->r_mem, AHCI_P_CMD); ATA_OUTL(ch->r_mem, AHCI_P_CMD, cmd & ~AHCI_P_CMD_FRE); /* Wait for FIS reception stop. */ timeout = 0; do { DELAY(10); if (timeout++ > 50000) { device_printf(dev, "stopping AHCI FR engine failed\n"); break; } } while (ATA_INL(ch->r_mem, AHCI_P_CMD) & AHCI_P_CMD_FR); } static void ahci_start_fr(device_t dev) { struct ahci_channel *ch = device_get_softc(dev); u_int32_t cmd; /* Start FIS reception on this channel */ cmd = ATA_INL(ch->r_mem, AHCI_P_CMD); ATA_OUTL(ch->r_mem, AHCI_P_CMD, cmd | AHCI_P_CMD_FRE); } static int ahci_wait_ready(device_t dev, int t, int t0) { struct ahci_channel *ch = device_get_softc(dev); int timeout = 0; uint32_t val; while ((val = ATA_INL(ch->r_mem, AHCI_P_TFD)) & (ATA_S_BUSY | ATA_S_DRQ)) { if (timeout > t) { if (t != 0) { device_printf(dev, "AHCI reset: device not ready after %dms " "(tfd = %08x)\n", MAX(t, 0) + t0, val); } return (EBUSY); } DELAY(1000); timeout++; } if (bootverbose) device_printf(dev, "AHCI reset: device ready after %dms\n", timeout + t0); return (0); } static void ahci_reset_to(void *arg) { device_t dev = arg; struct ahci_channel *ch = device_get_softc(dev); if (ch->resetting == 0) return; ch->resetting--; if (ahci_wait_ready(dev, ch->resetting == 0 ? -1 : 0, (310 - ch->resetting) * 100) == 0) { ch->resetting = 0; ahci_start(dev, 1); xpt_release_simq(ch->sim, TRUE); return; } if (ch->resetting == 0) { ahci_clo(dev); ahci_start(dev, 1); xpt_release_simq(ch->sim, TRUE); return; } callout_schedule(&ch->reset_timer, hz / 10); } static void ahci_reset(device_t dev) { struct ahci_channel *ch = device_get_softc(dev); struct ahci_controller *ctlr = device_get_softc(device_get_parent(dev)); int i; xpt_freeze_simq(ch->sim, 1); if (bootverbose) device_printf(dev, "AHCI reset...\n"); /* Forget about previous reset. */ if (ch->resetting) { ch->resetting = 0; callout_stop(&ch->reset_timer); xpt_release_simq(ch->sim, TRUE); } /* Requeue freezed command. */ if (ch->frozen) { union ccb *fccb = ch->frozen; ch->frozen = NULL; fccb->ccb_h.status = CAM_REQUEUE_REQ | CAM_RELEASE_SIMQ; if (!(fccb->ccb_h.status & CAM_DEV_QFRZN)) { xpt_freeze_devq(fccb->ccb_h.path, 1); fccb->ccb_h.status |= CAM_DEV_QFRZN; } xpt_done(fccb); } /* Kill the engine and requeue all running commands. */ ahci_stop(dev); for (i = 0; i < ch->numslots; i++) { /* Do we have a running request on slot? */ if (ch->slot[i].state < AHCI_SLOT_RUNNING) continue; /* XXX; Commands in loading state. */ ahci_end_transaction(&ch->slot[i], AHCI_ERR_INNOCENT); } for (i = 0; i < ch->numslots; i++) { if (!ch->hold[i]) continue; xpt_done(ch->hold[i]); ch->hold[i] = NULL; ch->numhslots--; } if (ch->toslots != 0) xpt_release_simq(ch->sim, TRUE); ch->eslots = 0; ch->toslots = 0; ch->wrongccs = 0; ch->fatalerr = 0; /* Tell the XPT about the event */ xpt_async(AC_BUS_RESET, ch->path, NULL); /* Disable port interrupts */ ATA_OUTL(ch->r_mem, AHCI_P_IE, 0); /* Reset and reconnect PHY, */ if (!ahci_sata_phy_reset(dev)) { if (bootverbose) device_printf(dev, "AHCI reset: device not found\n"); ch->devices = 0; /* Enable wanted port interrupts */ ATA_OUTL(ch->r_mem, AHCI_P_IE, (((ch->pm_level != 0) ? AHCI_P_IX_CPD | AHCI_P_IX_MP : 0) | AHCI_P_IX_PRC | AHCI_P_IX_PC)); xpt_release_simq(ch->sim, TRUE); return; } if (bootverbose) device_printf(dev, "AHCI reset: device found\n"); /* Wait for clearing busy status. */ if (ahci_wait_ready(dev, dumping ? 31000 : 0, 0)) { if (dumping) ahci_clo(dev); else ch->resetting = 310; } ch->devices = 1; /* Enable wanted port interrupts */ ATA_OUTL(ch->r_mem, AHCI_P_IE, (((ch->pm_level != 0) ? AHCI_P_IX_CPD | AHCI_P_IX_MP : 0) | AHCI_P_IX_TFE | AHCI_P_IX_HBF | AHCI_P_IX_HBD | AHCI_P_IX_IF | AHCI_P_IX_OF | ((ch->pm_level == 0) ? AHCI_P_IX_PRC : 0) | AHCI_P_IX_PC | AHCI_P_IX_DP | AHCI_P_IX_UF | (ctlr->ccc ? 0 : AHCI_P_IX_SDB) | AHCI_P_IX_DS | AHCI_P_IX_PS | (ctlr->ccc ? 0 : AHCI_P_IX_DHR))); if (ch->resetting) callout_reset(&ch->reset_timer, hz / 10, ahci_reset_to, dev); else { ahci_start(dev, 1); xpt_release_simq(ch->sim, TRUE); } } static int ahci_setup_fis(device_t dev, struct ahci_cmd_tab *ctp, union ccb *ccb, int tag) { struct ahci_channel *ch = device_get_softc(dev); u_int8_t *fis = &ctp->cfis[0]; bzero(ctp->cfis, 64); fis[0] = 0x27; /* host to device */ fis[1] = (ccb->ccb_h.target_id & 0x0f); if (ccb->ccb_h.func_code == XPT_SCSI_IO) { fis[1] |= 0x80; fis[2] = ATA_PACKET_CMD; if ((ccb->ccb_h.flags & CAM_DIR_MASK) != CAM_DIR_NONE && ch->curr[ccb->ccb_h.target_id].mode >= ATA_DMA) fis[3] = ATA_F_DMA; else { fis[5] = ccb->csio.dxfer_len; fis[6] = ccb->csio.dxfer_len >> 8; } fis[7] = ATA_D_LBA; fis[15] = ATA_A_4BIT; bzero(ctp->acmd, 32); bcopy((ccb->ccb_h.flags & CAM_CDB_POINTER) ? ccb->csio.cdb_io.cdb_ptr : ccb->csio.cdb_io.cdb_bytes, ctp->acmd, ccb->csio.cdb_len); } else if ((ccb->ataio.cmd.flags & CAM_ATAIO_CONTROL) == 0) { fis[1] |= 0x80; fis[2] = ccb->ataio.cmd.command; fis[3] = ccb->ataio.cmd.features; fis[4] = ccb->ataio.cmd.lba_low; fis[5] = ccb->ataio.cmd.lba_mid; fis[6] = ccb->ataio.cmd.lba_high; fis[7] = ccb->ataio.cmd.device; fis[8] = ccb->ataio.cmd.lba_low_exp; fis[9] = ccb->ataio.cmd.lba_mid_exp; fis[10] = ccb->ataio.cmd.lba_high_exp; fis[11] = ccb->ataio.cmd.features_exp; if (ccb->ataio.cmd.flags & CAM_ATAIO_FPDMA) { fis[12] = tag << 3; fis[13] = 0; } else { fis[12] = ccb->ataio.cmd.sector_count; fis[13] = ccb->ataio.cmd.sector_count_exp; } fis[15] = ATA_A_4BIT; } else { fis[15] = ccb->ataio.cmd.control; } return (20); } static int ahci_sata_connect(struct ahci_channel *ch) { u_int32_t status; int timeout, found = 0; /* Wait up to 100ms for "connect well" */ for (timeout = 0; timeout < 1000 ; timeout++) { status = ATA_INL(ch->r_mem, AHCI_P_SSTS); if ((status & ATA_SS_DET_MASK) != ATA_SS_DET_NO_DEVICE) found = 1; if (((status & ATA_SS_DET_MASK) == ATA_SS_DET_PHY_ONLINE) && ((status & ATA_SS_SPD_MASK) != ATA_SS_SPD_NO_SPEED) && ((status & ATA_SS_IPM_MASK) == ATA_SS_IPM_ACTIVE)) break; if ((status & ATA_SS_DET_MASK) == ATA_SS_DET_PHY_OFFLINE) { if (bootverbose) { device_printf(ch->dev, "SATA offline status=%08x\n", status); } return (0); } if (found == 0 && timeout >= 100) break; DELAY(100); } if (timeout >= 1000 || !found) { if (bootverbose) { device_printf(ch->dev, "SATA connect timeout time=%dus status=%08x\n", timeout * 100, status); } return (0); } if (bootverbose) { device_printf(ch->dev, "SATA connect time=%dus status=%08x\n", timeout * 100, status); } /* Clear SATA error register */ ATA_OUTL(ch->r_mem, AHCI_P_SERR, 0xffffffff); return (1); } static int ahci_sata_phy_reset(device_t dev) { struct ahci_channel *ch = device_get_softc(dev); int sata_rev; uint32_t val; if (ch->listening) { val = ATA_INL(ch->r_mem, AHCI_P_CMD); val |= AHCI_P_CMD_SUD; ATA_OUTL(ch->r_mem, AHCI_P_CMD, val); ch->listening = 0; } sata_rev = ch->user[ch->pm_present ? 15 : 0].revision; if (sata_rev == 1) val = ATA_SC_SPD_SPEED_GEN1; else if (sata_rev == 2) val = ATA_SC_SPD_SPEED_GEN2; else if (sata_rev == 3) val = ATA_SC_SPD_SPEED_GEN3; else val = 0; ATA_OUTL(ch->r_mem, AHCI_P_SCTL, ATA_SC_DET_RESET | val | ATA_SC_IPM_DIS_PARTIAL | ATA_SC_IPM_DIS_SLUMBER); DELAY(1000); ATA_OUTL(ch->r_mem, AHCI_P_SCTL, ATA_SC_DET_IDLE | val | ((ch->pm_level > 0) ? 0 : (ATA_SC_IPM_DIS_PARTIAL | ATA_SC_IPM_DIS_SLUMBER))); if (!ahci_sata_connect(ch)) { if (ch->caps & AHCI_CAP_SSS) { val = ATA_INL(ch->r_mem, AHCI_P_CMD); val &= ~AHCI_P_CMD_SUD; ATA_OUTL(ch->r_mem, AHCI_P_CMD, val); ch->listening = 1; } else if (ch->pm_level > 0) ATA_OUTL(ch->r_mem, AHCI_P_SCTL, ATA_SC_DET_DISABLE); return (0); } return (1); } static int ahci_check_ids(device_t dev, union ccb *ccb) { struct ahci_channel *ch = device_get_softc(dev); if (ccb->ccb_h.target_id > ((ch->caps & AHCI_CAP_SPM) ? 15 : 0)) { ccb->ccb_h.status = CAM_TID_INVALID; xpt_done(ccb); return (-1); } if (ccb->ccb_h.target_lun != 0) { ccb->ccb_h.status = CAM_LUN_INVALID; xpt_done(ccb); return (-1); } return (0); } static void ahciaction(struct cam_sim *sim, union ccb *ccb) { device_t dev; struct ahci_channel *ch; CAM_DEBUG(ccb->ccb_h.path, CAM_DEBUG_TRACE, ("ahciaction func_code=%x\n", ccb->ccb_h.func_code)); ch = (struct ahci_channel *)cam_sim_softc(sim); dev = ch->dev; switch (ccb->ccb_h.func_code) { /* Common cases first */ case XPT_ATA_IO: /* Execute the requested I/O operation */ case XPT_SCSI_IO: if (ahci_check_ids(dev, ccb)) return; if (ch->devices == 0 || (ch->pm_present == 0 && ccb->ccb_h.target_id > 0 && ccb->ccb_h.target_id < 15)) { ccb->ccb_h.status = CAM_SEL_TIMEOUT; break; } ccb->ccb_h.recovery_type = RECOVERY_NONE; /* Check for command collision. */ if (ahci_check_collision(dev, ccb)) { /* Freeze command. */ ch->frozen = ccb; /* We have only one frozen slot, so freeze simq also. */ xpt_freeze_simq(ch->sim, 1); return; } ahci_begin_transaction(dev, ccb); return; case XPT_EN_LUN: /* Enable LUN as a target */ case XPT_TARGET_IO: /* Execute target I/O request */ case XPT_ACCEPT_TARGET_IO: /* Accept Host Target Mode CDB */ case XPT_CONT_TARGET_IO: /* Continue Host Target I/O Connection*/ case XPT_ABORT: /* Abort the specified CCB */ /* XXX Implement */ ccb->ccb_h.status = CAM_REQ_INVALID; break; case XPT_SET_TRAN_SETTINGS: { struct ccb_trans_settings *cts = &ccb->cts; struct ahci_device *d; if (ahci_check_ids(dev, ccb)) return; if (cts->type == CTS_TYPE_CURRENT_SETTINGS) d = &ch->curr[ccb->ccb_h.target_id]; else d = &ch->user[ccb->ccb_h.target_id]; if (cts->xport_specific.sata.valid & CTS_SATA_VALID_REVISION) d->revision = cts->xport_specific.sata.revision; if (cts->xport_specific.sata.valid & CTS_SATA_VALID_MODE) d->mode = cts->xport_specific.sata.mode; if (cts->xport_specific.sata.valid & CTS_SATA_VALID_BYTECOUNT) d->bytecount = min(8192, cts->xport_specific.sata.bytecount); if (cts->xport_specific.sata.valid & CTS_SATA_VALID_TAGS) d->tags = min(ch->numslots, cts->xport_specific.sata.tags); if (cts->xport_specific.sata.valid & CTS_SATA_VALID_PM) ch->pm_present = cts->xport_specific.sata.pm_present; if (cts->xport_specific.sata.valid & CTS_SATA_VALID_ATAPI) d->atapi = cts->xport_specific.sata.atapi; if (cts->xport_specific.sata.valid & CTS_SATA_VALID_CAPS) d->caps = cts->xport_specific.sata.caps; ccb->ccb_h.status = CAM_REQ_CMP; break; } case XPT_GET_TRAN_SETTINGS: /* Get default/user set transfer settings for the target */ { struct ccb_trans_settings *cts = &ccb->cts; struct ahci_device *d; uint32_t status; if (ahci_check_ids(dev, ccb)) return; if (cts->type == CTS_TYPE_CURRENT_SETTINGS) d = &ch->curr[ccb->ccb_h.target_id]; else d = &ch->user[ccb->ccb_h.target_id]; cts->protocol = PROTO_UNSPECIFIED; cts->protocol_version = PROTO_VERSION_UNSPECIFIED; cts->transport = XPORT_SATA; cts->transport_version = XPORT_VERSION_UNSPECIFIED; cts->proto_specific.valid = 0; cts->xport_specific.sata.valid = 0; if (cts->type == CTS_TYPE_CURRENT_SETTINGS && (ccb->ccb_h.target_id == 15 || (ccb->ccb_h.target_id == 0 && !ch->pm_present))) { status = ATA_INL(ch->r_mem, AHCI_P_SSTS) & ATA_SS_SPD_MASK; if (status & 0x0f0) { cts->xport_specific.sata.revision = (status & 0x0f0) >> 4; cts->xport_specific.sata.valid |= CTS_SATA_VALID_REVISION; } cts->xport_specific.sata.caps = d->caps & CTS_SATA_CAPS_D; if (ch->pm_level) { if (ch->caps & (AHCI_CAP_PSC | AHCI_CAP_SSC)) cts->xport_specific.sata.caps |= CTS_SATA_CAPS_H_PMREQ; if (ch->caps2 & AHCI_CAP2_APST) cts->xport_specific.sata.caps |= CTS_SATA_CAPS_H_APST; } if ((ch->caps & AHCI_CAP_SNCQ) && (ch->quirks & AHCI_Q_NOAA) == 0) cts->xport_specific.sata.caps |= CTS_SATA_CAPS_H_DMAAA; cts->xport_specific.sata.caps |= CTS_SATA_CAPS_H_AN; cts->xport_specific.sata.caps &= ch->user[ccb->ccb_h.target_id].caps; cts->xport_specific.sata.valid |= CTS_SATA_VALID_CAPS; } else { cts->xport_specific.sata.revision = d->revision; cts->xport_specific.sata.valid |= CTS_SATA_VALID_REVISION; cts->xport_specific.sata.caps = d->caps; cts->xport_specific.sata.valid |= CTS_SATA_VALID_CAPS; } cts->xport_specific.sata.mode = d->mode; cts->xport_specific.sata.valid |= CTS_SATA_VALID_MODE; cts->xport_specific.sata.bytecount = d->bytecount; cts->xport_specific.sata.valid |= CTS_SATA_VALID_BYTECOUNT; cts->xport_specific.sata.pm_present = ch->pm_present; cts->xport_specific.sata.valid |= CTS_SATA_VALID_PM; cts->xport_specific.sata.tags = d->tags; cts->xport_specific.sata.valid |= CTS_SATA_VALID_TAGS; cts->xport_specific.sata.atapi = d->atapi; cts->xport_specific.sata.valid |= CTS_SATA_VALID_ATAPI; ccb->ccb_h.status = CAM_REQ_CMP; break; } case XPT_RESET_BUS: /* Reset the specified SCSI bus */ case XPT_RESET_DEV: /* Bus Device Reset the specified SCSI device */ ahci_reset(dev); ccb->ccb_h.status = CAM_REQ_CMP; break; case XPT_TERM_IO: /* Terminate the I/O process */ /* XXX Implement */ ccb->ccb_h.status = CAM_REQ_INVALID; break; case XPT_PATH_INQ: /* Path routing inquiry */ { struct ccb_pathinq *cpi = &ccb->cpi; cpi->version_num = 1; /* XXX??? */ cpi->hba_inquiry = PI_SDTR_ABLE; if (ch->caps & AHCI_CAP_SNCQ) cpi->hba_inquiry |= PI_TAG_ABLE; if (ch->caps & AHCI_CAP_SPM) cpi->hba_inquiry |= PI_SATAPM; cpi->target_sprt = 0; cpi->hba_misc = PIM_SEQSCAN; cpi->hba_eng_cnt = 0; if (ch->caps & AHCI_CAP_SPM) cpi->max_target = 15; else cpi->max_target = 0; cpi->max_lun = 0; cpi->initiator_id = 0; cpi->bus_id = cam_sim_bus(sim); cpi->base_transfer_speed = 150000; strncpy(cpi->sim_vid, "FreeBSD", SIM_IDLEN); strncpy(cpi->hba_vid, "AHCI", HBA_IDLEN); strncpy(cpi->dev_name, cam_sim_name(sim), DEV_IDLEN); cpi->unit_number = cam_sim_unit(sim); cpi->transport = XPORT_SATA; cpi->transport_version = XPORT_VERSION_UNSPECIFIED; cpi->protocol = PROTO_ATA; cpi->protocol_version = PROTO_VERSION_UNSPECIFIED; cpi->maxio = MAXPHYS; /* ATI SB600 can't handle 256 sectors with FPDMA (NCQ). */ if (pci_get_devid(device_get_parent(dev)) == 0x43801002) cpi->maxio = min(cpi->maxio, 128 * 512); cpi->ccb_h.status = CAM_REQ_CMP; break; } default: ccb->ccb_h.status = CAM_REQ_INVALID; break; } xpt_done(ccb); } static void ahcipoll(struct cam_sim *sim) { struct ahci_channel *ch = (struct ahci_channel *)cam_sim_softc(sim); ahci_ch_intr(ch->dev); if (ch->resetting != 0 && (--ch->resetpolldiv <= 0 || !callout_pending(&ch->reset_timer))) { ch->resetpolldiv = 1000; ahci_reset_to(ch->dev); } } Index: stable/8/sys/dev/ahci =================================================================== --- stable/8/sys/dev/ahci (revision 243119) +++ stable/8/sys/dev/ahci (revision 243120) Property changes on: stable/8/sys/dev/ahci ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,1 ## Merged /head/sys/dev/ahci:r232380 Index: stable/8/sys/dev/ata/ata-pci.h =================================================================== --- stable/8/sys/dev/ata/ata-pci.h (revision 243119) +++ stable/8/sys/dev/ata/ata-pci.h (revision 243120) @@ -1,625 +1,625 @@ /*- * Copyright (c) 2003 - 2008 Søren Schmidt * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer, * without modification, immediately at the beginning of the file. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * $FreeBSD$ */ /* structure holding chipset config info */ struct ata_chip_id { u_int32_t chipid; u_int8_t chiprev; int cfg1; int cfg2; u_int8_t max_dma; const char *text; }; #define ATA_PCI_MAX_CH 8 /* structure describing a PCI ATA controller */ struct ata_pci_controller { device_t dev; int r_type1; int r_rid1; struct resource *r_res1; int r_type2; int r_rid2; struct resource *r_res2; int r_irq_rid; struct resource *r_irq; void *handle; const struct ata_chip_id *chip; int legacy; int channels; int ichannels; int (*chipinit)(device_t); int (*chipdeinit)(device_t); int (*suspend)(device_t); int (*resume)(device_t); int (*ch_attach)(device_t); int (*ch_detach)(device_t); int (*ch_suspend)(device_t); int (*ch_resume)(device_t); #ifndef ATA_CAM int (*locking)(device_t, int); #endif void (*reset)(device_t); int (*setmode)(device_t, int, int); int (*getrev)(device_t, int); struct { void (*function)(void *); void *argument; } interrupt[ATA_PCI_MAX_CH]; void *chipset_data; }; /* defines for known chipset PCI id's */ #define ATA_ACARD_ID 0x1191 #define ATA_ATP850 0x00021191 #define ATA_ATP850A 0x00041191 #define ATA_ATP850R 0x00051191 #define ATA_ATP860A 0x00061191 #define ATA_ATP860R 0x00071191 #define ATA_ATP865A 0x00081191 #define ATA_ATP865R 0x00091191 #define ATA_ACER_LABS_ID 0x10b9 #define ATA_ALI_1533 0x153310b9 #define ATA_ALI_5228 0x522810b9 #define ATA_ALI_5229 0x522910b9 #define ATA_ALI_5281 0x528110b9 #define ATA_ALI_5287 0x528710b9 #define ATA_ALI_5288 0x528810b9 #define ATA_ALI_5289 0x528910b9 #define ATA_AMD_ID 0x1022 #define ATA_AMD755 0x74011022 #define ATA_AMD756 0x74091022 #define ATA_AMD766 0x74111022 #define ATA_AMD768 0x74411022 #define ATA_AMD8111 0x74691022 #define ATA_AMD5536 0x209a1022 #define ATA_ADAPTEC_ID 0x9005 #define ATA_ADAPTEC_1420 0x02419005 #define ATA_ADAPTEC_1430 0x02439005 #define ATA_ATI_ID 0x1002 #define ATA_ATI_IXP200 0x43491002 #define ATA_ATI_IXP300 0x43691002 #define ATA_ATI_IXP300_S1 0x436e1002 #define ATA_ATI_IXP400 0x43761002 #define ATA_ATI_IXP400_S1 0x43791002 #define ATA_ATI_IXP400_S2 0x437a1002 #define ATA_ATI_IXP600 0x438c1002 #define ATA_ATI_IXP600_S1 0x43801002 #define ATA_ATI_IXP700 0x439c1002 #define ATA_ATI_IXP700_S1 0x43901002 #define ATA_ATI_IXP700_S2 0x43911002 #define ATA_ATI_IXP700_S3 0x43921002 #define ATA_ATI_IXP700_S4 0x43931002 #define ATA_ATI_IXP800_S1 0x43941002 #define ATA_ATI_IXP800_S2 0x43951002 #define ATA_CENATEK_ID 0x16ca #define ATA_CENATEK_ROCKET 0x000116ca #define ATA_CYRIX_ID 0x1078 #define ATA_CYRIX_5530 0x01021078 #define ATA_CYPRESS_ID 0x1080 #define ATA_CYPRESS_82C693 0xc6931080 #define ATA_DEC_21150 0x00221011 #define ATA_DEC_21150_1 0x00231011 #define ATA_HIGHPOINT_ID 0x1103 #define ATA_HPT366 0x00041103 #define ATA_HPT372 0x00051103 #define ATA_HPT302 0x00061103 #define ATA_HPT371 0x00071103 #define ATA_HPT374 0x00081103 #define ATA_INTEL_ID 0x8086 #define ATA_I960RM 0x09628086 #define ATA_I82371FB 0x12308086 #define ATA_I82371SB 0x70108086 #define ATA_I82371AB 0x71118086 #define ATA_I82443MX 0x71998086 #define ATA_I82451NX 0x84ca8086 #define ATA_I82372FB 0x76018086 #define ATA_I82801AB 0x24218086 #define ATA_I82801AA 0x24118086 #define ATA_I82801BA 0x244a8086 #define ATA_I82801BA_1 0x244b8086 #define ATA_I82801CA 0x248a8086 #define ATA_I82801CA_1 0x248b8086 #define ATA_I82801DB 0x24cb8086 #define ATA_I82801DB_1 0x24ca8086 #define ATA_I82801EB 0x24db8086 #define ATA_I82801EB_S1 0x24d18086 #define ATA_I82801EB_R1 0x24df8086 #define ATA_I6300ESB 0x25a28086 #define ATA_I6300ESB_S1 0x25a38086 #define ATA_I6300ESB_R1 0x25b08086 #define ATA_I63XXESB2 0x269e8086 #define ATA_I63XXESB2_S1 0x26808086 #define ATA_I63XXESB2_S2 0x26818086 #define ATA_I63XXESB2_R1 0x26828086 #define ATA_I63XXESB2_R2 0x26838086 #define ATA_I82801FB 0x266f8086 #define ATA_I82801FB_S1 0x26518086 #define ATA_I82801FB_R1 0x26528086 #define ATA_I82801FBM 0x26538086 #define ATA_I82801GB 0x27df8086 #define ATA_I82801GB_S1 0x27c08086 #define ATA_I82801GB_AH 0x27c18086 #define ATA_I82801GB_R1 0x27c38086 #define ATA_I82801GBM_S1 0x27c48086 #define ATA_I82801GBM_AH 0x27c58086 #define ATA_I82801GBM_R1 0x27c68086 #define ATA_I82801HB_S1 0x28208086 #define ATA_I82801HB_AH6 0x28218086 #define ATA_I82801HB_R1 0x28228086 #define ATA_I82801HB_AH4 0x28248086 #define ATA_I82801HB_S2 0x28258086 #define ATA_I82801HBM 0x28508086 #define ATA_I82801HBM_S1 0x28288086 #define ATA_I82801HBM_S2 0x28298086 #define ATA_I82801HBM_S3 0x282a8086 #define ATA_I82801IB_S1 0x29208086 #define ATA_I82801IB_S3 0x29218086 #define ATA_I82801IB_AH6 0x29228086 #define ATA_I82801IB_AH4 0x29238086 #define ATA_I82801IB_R1 0x29258086 #define ATA_I82801IB_S2 0x29268086 #define ATA_I82801IBM_S1 0x29288086 #define ATA_I82801IBM_AH 0x29298086 #define ATA_I82801IBM_R1 0x292a8086 #define ATA_I82801IBM_S2 0x292d8086 #define ATA_I82801JIB_S1 0x3a208086 #define ATA_I82801JIB_AH 0x3a228086 #define ATA_I82801JIB_R1 0x3a258086 #define ATA_I82801JIB_S2 0x3a268086 #define ATA_I82801JD_S1 0x3a008086 #define ATA_I82801JD_AH 0x3a028086 #define ATA_I82801JD_R1 0x3a058086 #define ATA_I82801JD_S2 0x3a068086 #define ATA_I82801JI_S1 0x3a208086 #define ATA_I82801JI_AH 0x3a228086 #define ATA_I82801JI_R1 0x3a258086 #define ATA_I82801JI_S2 0x3a268086 #define ATA_5Series_S1 0x3b208086 #define ATA_5Series_S2 0x3b218086 #define ATA_5Series_AH1 0x3b228086 #define ATA_5Series_AH2 0x3b238086 #define ATA_5Series_R1 0x3b258086 #define ATA_5Series_S3 0x3b268086 #define ATA_5Series_S4 0x3b288086 #define ATA_5Series_AH3 0x3b298086 #define ATA_5Series_R2 0x3b2c8086 #define ATA_5Series_S5 0x3b2d8086 #define ATA_5Series_S6 0x3b2e8086 #define ATA_5Series_AH4 0x3b2f8086 #define ATA_CPT_S1 0x1c008086 #define ATA_CPT_S2 0x1c018086 #define ATA_CPT_AH1 0x1c028086 #define ATA_CPT_AH2 0x1c038086 #define ATA_CPT_R1 0x1c048086 #define ATA_CPT_R2 0x1c058086 #define ATA_CPT_S3 0x1c088086 #define ATA_CPT_S4 0x1c098086 #define ATA_PBG_S1 0x1d008086 #define ATA_PBG_AH1 0x1d028086 #define ATA_PBG_R1 0x1d048086 #define ATA_PBG_R2 0x1d068086 #define ATA_PBG_R3 0x28268086 #define ATA_PBG_S2 0x1d088086 #define ATA_PPT_S1 0x1e008086 #define ATA_PPT_S2 0x1e018086 #define ATA_PPT_AH1 0x1e028086 #define ATA_PPT_AH2 0x1e038086 #define ATA_PPT_R1 0x1e048086 #define ATA_PPT_R2 0x1e058086 #define ATA_PPT_R3 0x1e068086 #define ATA_PPT_R4 0x1e078086 #define ATA_PPT_S3 0x1e088086 #define ATA_PPT_S4 0x1e098086 #define ATA_PPT_R5 0x1e0e8086 #define ATA_PPT_R6 0x1e0f8086 #define ATA_I31244 0x32008086 #define ATA_ISCH 0x811a8086 #define ATA_DH89XXCC 0x23238086 #define ATA_ITE_ID 0x1283 #define ATA_IT8211F 0x82111283 #define ATA_IT8212F 0x82121283 #define ATA_IT8213F 0x82131283 #define ATA_JMICRON_ID 0x197b #define ATA_JMB360 0x2360197b #define ATA_JMB361 0x2361197b #define ATA_JMB362 0x2362197b #define ATA_JMB363 0x2363197b #define ATA_JMB365 0x2365197b #define ATA_JMB366 0x2366197b #define ATA_JMB368 0x2368197b #define ATA_MARVELL_ID 0x11ab #define ATA_M88SX5040 0x504011ab #define ATA_M88SX5041 0x504111ab #define ATA_M88SX5080 0x508011ab #define ATA_M88SX5081 0x508111ab #define ATA_M88SX6041 0x604111ab #define ATA_M88SX6042 0x604211ab #define ATA_M88SX6081 0x608111ab #define ATA_M88SX7042 0x704211ab -#define ATA_M88SX6101 0x610111ab -#define ATA_M88SX6102 0x610211ab -#define ATA_M88SX6111 0x611111ab -#define ATA_M88SX6121 0x612111ab -#define ATA_M88SX6141 0x614111ab -#define ATA_M88SX6145 0x614511ab +#define ATA_M88SE6101 0x610111ab +#define ATA_M88SE6102 0x610211ab +#define ATA_M88SE6111 0x611111ab +#define ATA_M88SE6121 0x612111ab +#define ATA_M88SE6141 0x614111ab +#define ATA_M88SE6145 0x614511ab #define ATA_MARVELL2_ID 0x1b4b #define ATA_MICRON_ID 0x1042 #define ATA_MICRON_RZ1000 0x10001042 #define ATA_MICRON_RZ1001 0x10011042 #define ATA_NATIONAL_ID 0x100b #define ATA_SC1100 0x0502100b #define ATA_NETCELL_ID 0x169c #define ATA_NETCELL_SR 0x0044169c #define ATA_NVIDIA_ID 0x10de #define ATA_NFORCE1 0x01bc10de #define ATA_NFORCE2 0x006510de #define ATA_NFORCE2_PRO 0x008510de #define ATA_NFORCE2_PRO_S1 0x008e10de #define ATA_NFORCE3 0x00d510de #define ATA_NFORCE3_PRO 0x00e510de #define ATA_NFORCE3_PRO_S1 0x00e310de #define ATA_NFORCE3_PRO_S2 0x00ee10de #define ATA_NFORCE_MCP04 0x003510de #define ATA_NFORCE_MCP04_S1 0x003610de #define ATA_NFORCE_MCP04_S2 0x003e10de #define ATA_NFORCE_CK804 0x005310de #define ATA_NFORCE_CK804_S1 0x005410de #define ATA_NFORCE_CK804_S2 0x005510de #define ATA_NFORCE_MCP51 0x026510de #define ATA_NFORCE_MCP51_S1 0x026610de #define ATA_NFORCE_MCP51_S2 0x026710de #define ATA_NFORCE_MCP55 0x036e10de #define ATA_NFORCE_MCP55_S1 0x037e10de #define ATA_NFORCE_MCP55_S2 0x037f10de #define ATA_NFORCE_MCP61 0x03ec10de #define ATA_NFORCE_MCP61_S1 0x03e710de #define ATA_NFORCE_MCP61_S2 0x03f610de #define ATA_NFORCE_MCP61_S3 0x03f710de #define ATA_NFORCE_MCP65 0x044810de #define ATA_NFORCE_MCP65_A0 0x044c10de #define ATA_NFORCE_MCP65_A1 0x044d10de #define ATA_NFORCE_MCP65_A2 0x044e10de #define ATA_NFORCE_MCP65_A3 0x044f10de #define ATA_NFORCE_MCP65_A4 0x045c10de #define ATA_NFORCE_MCP65_A5 0x045d10de #define ATA_NFORCE_MCP65_A6 0x045e10de #define ATA_NFORCE_MCP65_A7 0x045f10de #define ATA_NFORCE_MCP67 0x056010de #define ATA_NFORCE_MCP67_A0 0x055010de #define ATA_NFORCE_MCP67_A1 0x055110de #define ATA_NFORCE_MCP67_A2 0x055210de #define ATA_NFORCE_MCP67_A3 0x055310de #define ATA_NFORCE_MCP67_A4 0x055410de #define ATA_NFORCE_MCP67_A5 0x055510de #define ATA_NFORCE_MCP67_A6 0x055610de #define ATA_NFORCE_MCP67_A7 0x055710de #define ATA_NFORCE_MCP67_A8 0x055810de #define ATA_NFORCE_MCP67_A9 0x055910de #define ATA_NFORCE_MCP67_AA 0x055A10de #define ATA_NFORCE_MCP67_AB 0x055B10de #define ATA_NFORCE_MCP67_AC 0x058410de #define ATA_NFORCE_MCP73 0x056c10de #define ATA_NFORCE_MCP73_A0 0x07f010de #define ATA_NFORCE_MCP73_A1 0x07f110de #define ATA_NFORCE_MCP73_A2 0x07f210de #define ATA_NFORCE_MCP73_A3 0x07f310de #define ATA_NFORCE_MCP73_A4 0x07f410de #define ATA_NFORCE_MCP73_A5 0x07f510de #define ATA_NFORCE_MCP73_A6 0x07f610de #define ATA_NFORCE_MCP73_A7 0x07f710de #define ATA_NFORCE_MCP73_A8 0x07f810de #define ATA_NFORCE_MCP73_A9 0x07f910de #define ATA_NFORCE_MCP73_AA 0x07fa10de #define ATA_NFORCE_MCP73_AB 0x07fb10de #define ATA_NFORCE_MCP77 0x075910de #define ATA_NFORCE_MCP77_A0 0x0ad010de #define ATA_NFORCE_MCP77_A1 0x0ad110de #define ATA_NFORCE_MCP77_A2 0x0ad210de #define ATA_NFORCE_MCP77_A3 0x0ad310de #define ATA_NFORCE_MCP77_A4 0x0ad410de #define ATA_NFORCE_MCP77_A5 0x0ad510de #define ATA_NFORCE_MCP77_A6 0x0ad610de #define ATA_NFORCE_MCP77_A7 0x0ad710de #define ATA_NFORCE_MCP77_A8 0x0ad810de #define ATA_NFORCE_MCP77_A9 0x0ad910de #define ATA_NFORCE_MCP77_AA 0x0ada10de #define ATA_NFORCE_MCP77_AB 0x0adb10de #define ATA_NFORCE_MCP79_A0 0x0ab410de #define ATA_NFORCE_MCP79_A1 0x0ab510de #define ATA_NFORCE_MCP79_A2 0x0ab610de #define ATA_NFORCE_MCP79_A3 0x0ab710de #define ATA_NFORCE_MCP79_A4 0x0ab810de #define ATA_NFORCE_MCP79_A5 0x0ab910de #define ATA_NFORCE_MCP79_A6 0x0aba10de #define ATA_NFORCE_MCP79_A7 0x0abb10de #define ATA_NFORCE_MCP79_A8 0x0abc10de #define ATA_NFORCE_MCP79_A9 0x0abd10de #define ATA_NFORCE_MCP79_AA 0x0abe10de #define ATA_NFORCE_MCP79_AB 0x0abf10de #define ATA_NFORCE_MCP89_A0 0x0d8410de #define ATA_NFORCE_MCP89_A1 0x0d8510de #define ATA_NFORCE_MCP89_A2 0x0d8610de #define ATA_NFORCE_MCP89_A3 0x0d8710de #define ATA_NFORCE_MCP89_A4 0x0d8810de #define ATA_NFORCE_MCP89_A5 0x0d8910de #define ATA_NFORCE_MCP89_A6 0x0d8a10de #define ATA_NFORCE_MCP89_A7 0x0d8b10de #define ATA_NFORCE_MCP89_A8 0x0d8c10de #define ATA_NFORCE_MCP89_A9 0x0d8d10de #define ATA_NFORCE_MCP89_AA 0x0d8e10de #define ATA_NFORCE_MCP89_AB 0x0d8f10de #define ATA_PROMISE_ID 0x105a #define ATA_PDC20246 0x4d33105a #define ATA_PDC20262 0x4d38105a #define ATA_PDC20263 0x0d38105a #define ATA_PDC20265 0x0d30105a #define ATA_PDC20267 0x4d30105a #define ATA_PDC20268 0x4d68105a #define ATA_PDC20269 0x4d69105a #define ATA_PDC20270 0x6268105a #define ATA_PDC20271 0x6269105a #define ATA_PDC20275 0x1275105a #define ATA_PDC20276 0x5275105a #define ATA_PDC20277 0x7275105a #define ATA_PDC20318 0x3318105a #define ATA_PDC20319 0x3319105a #define ATA_PDC20371 0x3371105a #define ATA_PDC20375 0x3375105a #define ATA_PDC20376 0x3376105a #define ATA_PDC20377 0x3377105a #define ATA_PDC20378 0x3373105a #define ATA_PDC20379 0x3372105a #define ATA_PDC20571 0x3571105a #define ATA_PDC20575 0x3d75105a #define ATA_PDC20579 0x3574105a #define ATA_PDC20771 0x3570105a #define ATA_PDC40518 0x3d18105a #define ATA_PDC40519 0x3519105a #define ATA_PDC40718 0x3d17105a #define ATA_PDC40719 0x3515105a #define ATA_PDC40775 0x3d73105a #define ATA_PDC40779 0x3577105a #define ATA_PDC20617 0x6617105a #define ATA_PDC20618 0x6626105a #define ATA_PDC20619 0x6629105a #define ATA_PDC20620 0x6620105a #define ATA_PDC20621 0x6621105a #define ATA_PDC20622 0x6622105a #define ATA_PDC20624 0x6624105a #define ATA_PDC81518 0x8002105a #define ATA_SERVERWORKS_ID 0x1166 #define ATA_ROSB4_ISA 0x02001166 #define ATA_ROSB4 0x02111166 #define ATA_CSB5 0x02121166 #define ATA_CSB6 0x02131166 #define ATA_CSB6_1 0x02171166 #define ATA_HT1000 0x02141166 #define ATA_HT1000_S1 0x024b1166 #define ATA_HT1000_S2 0x024a1166 #define ATA_K2 0x02401166 #define ATA_FRODO4 0x02411166 #define ATA_FRODO8 0x02421166 #define ATA_SILICON_IMAGE_ID 0x1095 #define ATA_SII3114 0x31141095 #define ATA_SII3512 0x35121095 #define ATA_SII3112 0x31121095 #define ATA_SII3112_1 0x02401095 #define ATA_SII3124 0x31241095 #define ATA_SII3132 0x31321095 #define ATA_SII3132_1 0x02421095 #define ATA_SII3132_2 0x02441095 #define ATA_SII0680 0x06801095 #define ATA_CMD646 0x06461095 #define ATA_CMD648 0x06481095 #define ATA_CMD649 0x06491095 #define ATA_SIS_ID 0x1039 #define ATA_SISSOUTH 0x00081039 #define ATA_SIS5511 0x55111039 #define ATA_SIS5513 0x55131039 #define ATA_SIS5517 0x55171039 #define ATA_SIS5518 0x55181039 #define ATA_SIS5571 0x55711039 #define ATA_SIS5591 0x55911039 #define ATA_SIS5596 0x55961039 #define ATA_SIS5597 0x55971039 #define ATA_SIS5598 0x55981039 #define ATA_SIS5600 0x56001039 #define ATA_SIS530 0x05301039 #define ATA_SIS540 0x05401039 #define ATA_SIS550 0x05501039 #define ATA_SIS620 0x06201039 #define ATA_SIS630 0x06301039 #define ATA_SIS635 0x06351039 #define ATA_SIS633 0x06331039 #define ATA_SIS640 0x06401039 #define ATA_SIS645 0x06451039 #define ATA_SIS646 0x06461039 #define ATA_SIS648 0x06481039 #define ATA_SIS650 0x06501039 #define ATA_SIS651 0x06511039 #define ATA_SIS652 0x06521039 #define ATA_SIS655 0x06551039 #define ATA_SIS658 0x06581039 #define ATA_SIS661 0x06611039 #define ATA_SIS730 0x07301039 #define ATA_SIS733 0x07331039 #define ATA_SIS735 0x07351039 #define ATA_SIS740 0x07401039 #define ATA_SIS745 0x07451039 #define ATA_SIS746 0x07461039 #define ATA_SIS748 0x07481039 #define ATA_SIS750 0x07501039 #define ATA_SIS751 0x07511039 #define ATA_SIS752 0x07521039 #define ATA_SIS755 0x07551039 #define ATA_SIS961 0x09611039 #define ATA_SIS962 0x09621039 #define ATA_SIS963 0x09631039 #define ATA_SIS964 0x09641039 #define ATA_SIS965 0x09651039 #define ATA_SIS180 0x01801039 #define ATA_SIS181 0x01811039 #define ATA_SIS182 0x01821039 #define ATA_VIA_ID 0x1106 #define ATA_VIA82C571 0x05711106 #define ATA_VIA82C586 0x05861106 #define ATA_VIA82C596 0x05961106 #define ATA_VIA82C686 0x06861106 #define ATA_VIA8231 0x82311106 #define ATA_VIA8233 0x30741106 #define ATA_VIA8233A 0x31471106 #define ATA_VIA8233C 0x31091106 #define ATA_VIA8235 0x31771106 #define ATA_VIA8237 0x32271106 #define ATA_VIA8237A 0x05911106 #define ATA_VIA8237S 0x53371106 #define ATA_VIA8237_5372 0x53721106 #define ATA_VIA8237_7372 0x73721106 #define ATA_VIA8251 0x33491106 #define ATA_VIA8361 0x31121106 #define ATA_VIA8363 0x03051106 #define ATA_VIA8371 0x03911106 #define ATA_VIA8662 0x31021106 #define ATA_VIA6410 0x31641106 #define ATA_VIA6420 0x31491106 #define ATA_VIA6421 0x32491106 #define ATA_VIACX700IDE 0x05811106 #define ATA_VIACX700 0x83241106 #define ATA_VIASATAIDE 0x53241106 #define ATA_VIAVX800 0x83531106 #define ATA_VIASATAIDE2 0xc4091106 #define ATA_VIAVX855 0x84091106 #define ATA_VIASATAIDE3 0x90011106 #define ATA_VIAVX900 0x84101106 /* global prototypes ata-pci.c */ int ata_pci_probe(device_t dev); int ata_pci_attach(device_t dev); int ata_pci_detach(device_t dev); int ata_pci_suspend(device_t dev); int ata_pci_resume(device_t dev); int ata_pci_read_ivar(device_t dev, device_t child, int which, uintptr_t *result); int ata_pci_write_ivar(device_t dev, device_t child, int which, uintptr_t value); uint32_t ata_pci_read_config(device_t dev, device_t child, int reg, int width); void ata_pci_write_config(device_t dev, device_t child, int reg, uint32_t val, int width); int ata_pci_print_child(device_t dev, device_t child); int ata_pci_child_location_str(device_t dev, device_t child, char *buf, size_t buflen); struct resource * ata_pci_alloc_resource(device_t dev, device_t child, int type, int *rid, u_long start, u_long end, u_long count, u_int flags); int ata_pci_release_resource(device_t dev, device_t child, int type, int rid, struct resource *r); int ata_pci_setup_intr(device_t dev, device_t child, struct resource *irq, int flags, driver_filter_t *filter, driver_intr_t *function, void *argument, void **cookiep); int ata_pci_teardown_intr(device_t dev, device_t child, struct resource *irq, void *cookie); int ata_pci_ch_attach(device_t dev); int ata_pci_ch_detach(device_t dev); int ata_pci_status(device_t dev); void ata_pci_hw(device_t dev); void ata_pci_dmainit(device_t dev); void ata_pci_dmafini(device_t dev); const char *ata_pcivendor2str(device_t dev); int ata_legacy(device_t); void ata_generic_intr(void *data); int ata_generic_chipinit(device_t dev); int ata_generic_setmode(device_t dev, int target, int mode); int ata_setup_interrupt(device_t dev, void *intr_func); void ata_set_desc(device_t dev); const struct ata_chip_id *ata_match_chip(device_t dev, const struct ata_chip_id *index); const struct ata_chip_id *ata_find_chip(device_t dev, const struct ata_chip_id *index, int slot); int ata_mode2idx(int mode); /* global prototypes from chipsets/ata-*.c */ int ata_ahci_chipinit(device_t); int ata_marvell_edma_chipinit(device_t); int ata_sii_chipinit(device_t); /* externs */ extern devclass_t ata_pci_devclass; MALLOC_DECLARE(M_ATAPCI); /* macro for easy definition of all driver module stuff */ #define ATA_DECLARE_DRIVER(dname) \ static device_method_t __CONCAT(dname,_methods)[] = { \ DEVMETHOD(device_probe, __CONCAT(dname,_probe)), \ DEVMETHOD(device_attach, ata_pci_attach), \ DEVMETHOD(device_detach, ata_pci_detach), \ DEVMETHOD(device_suspend, ata_pci_suspend), \ DEVMETHOD(device_resume, ata_pci_resume), \ DEVMETHOD(device_shutdown, bus_generic_shutdown), \ DEVMETHOD(bus_read_ivar, ata_pci_read_ivar), \ DEVMETHOD(bus_write_ivar, ata_pci_write_ivar), \ DEVMETHOD(bus_alloc_resource, ata_pci_alloc_resource), \ DEVMETHOD(bus_release_resource, ata_pci_release_resource), \ DEVMETHOD(bus_activate_resource, bus_generic_activate_resource), \ DEVMETHOD(bus_deactivate_resource, bus_generic_deactivate_resource), \ DEVMETHOD(bus_setup_intr, ata_pci_setup_intr), \ DEVMETHOD(bus_teardown_intr, ata_pci_teardown_intr), \ DEVMETHOD(pci_read_config, ata_pci_read_config), \ DEVMETHOD(pci_write_config, ata_pci_write_config), \ DEVMETHOD(bus_print_child, ata_pci_print_child), \ DEVMETHOD(bus_child_location_str, ata_pci_child_location_str), \ DEVMETHOD_END \ }; \ static driver_t __CONCAT(dname,_driver) = { \ "atapci", \ __CONCAT(dname,_methods), \ sizeof(struct ata_pci_controller) \ }; \ DRIVER_MODULE(dname, pci, __CONCAT(dname,_driver), ata_pci_devclass, NULL, NULL); \ MODULE_VERSION(dname, 1); \ MODULE_DEPEND(dname, ata, 1, 1, 1); \ MODULE_DEPEND(dname, atapci, 1, 1, 1); Index: stable/8/sys/dev/ata/chipsets/ata-ahci.c =================================================================== --- stable/8/sys/dev/ata/chipsets/ata-ahci.c (revision 243119) +++ stable/8/sys/dev/ata/chipsets/ata-ahci.c (revision 243120) @@ -1,1067 +1,1067 @@ /*- * Copyright (c) 1998 - 2008 Søren Schmidt * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer, * without modification, immediately at the beginning of the file. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include "opt_ata.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include /* local prototypes */ static int ata_ahci_ch_attach(device_t dev); static int ata_ahci_ch_detach(device_t dev); static int ata_ahci_ch_suspend(device_t dev); static int ata_ahci_ch_resume(device_t dev); static int ata_ahci_ctlr_reset(device_t dev); static void ata_ahci_reset(device_t dev); static int ata_ahci_suspend(device_t dev); static int ata_ahci_status(device_t dev); static int ata_ahci_begin_transaction(struct ata_request *request); static int ata_ahci_end_transaction(struct ata_request *request); static int ata_ahci_pm_read(device_t dev, int port, int reg, u_int32_t *result); static int ata_ahci_pm_write(device_t dev, int port, int reg, u_int32_t result); static int ata_ahci_hardreset(device_t dev, int port, uint32_t *signature); static u_int32_t ata_ahci_softreset(device_t dev, int port); static void ata_ahci_dmasetprd(void *xsc, bus_dma_segment_t *segs, int nsegs, int error); static int ata_ahci_setup_fis(struct ata_ahci_cmd_tab *ctp, struct ata_request *equest); static void ata_ahci_dmainit(device_t dev); static void ata_ahci_start(device_t dev); static void ata_ahci_stop(device_t dev); static void ata_ahci_clo(device_t dev); static void ata_ahci_start_fr(device_t dev); static void ata_ahci_stop_fr(device_t dev); /* * AHCI v1.x compliant SATA chipset support functions */ static int ata_ahci_probe(device_t dev) { struct ata_pci_controller *ctlr = device_get_softc(dev); char buffer[64]; /* is this a possible AHCI candidate ? */ if (pci_get_class(dev) != PCIC_STORAGE || pci_get_subclass(dev) != PCIS_STORAGE_SATA) return (ENXIO); /* is this PCI device flagged as an AHCI compliant chip ? */ if (pci_get_progif(dev) != PCIP_STORAGE_SATA_AHCI_1_0) return (ENXIO); if (bootverbose) sprintf(buffer, "%s (ID=%08x) AHCI controller", ata_pcivendor2str(dev), pci_get_devid(dev)); else sprintf(buffer, "%s AHCI controller", ata_pcivendor2str(dev)); device_set_desc_copy(dev, buffer); ctlr->chipinit = ata_ahci_chipinit; return (BUS_PROBE_GENERIC); } static int ata_ahci_ata_probe(device_t dev) { struct ata_pci_controller *ctlr = device_get_softc(dev); if ((intptr_t)device_get_ivars(dev) >= 0) return (ENXIO); device_set_desc_copy(dev, "AHCI SATA controller"); ctlr->chipinit = ata_ahci_chipinit; return (BUS_PROBE_GENERIC); } static int ata_ahci_ata_attach(device_t dev) { struct ata_pci_controller *ctlr = device_get_softc(dev); device_t child; int unit; /* do chipset specific setups only needed once */ ctlr->legacy = 0; ctlr->ichannels = -1; ctlr->ch_attach = ata_pci_ch_attach; ctlr->ch_detach = ata_pci_ch_detach; ctlr->dev = dev; if (ctlr->chipinit(dev)) return ENXIO; /* attach all channels on this controller */ for (unit = 0; unit < ctlr->channels; unit++) { if ((ctlr->ichannels & (1 << unit)) == 0) continue; child = device_add_child(dev, "ata", ((unit == 0 || unit == 1) && ctlr->legacy) ? unit : devclass_find_free_unit(ata_devclass, 2)); if (child == NULL) device_printf(dev, "failed to add ata child device\n"); else device_set_ivars(child, (void *)(intptr_t)unit); } bus_generic_attach(dev); return 0; } int ata_ahci_chipinit(device_t dev) { struct ata_pci_controller *ctlr = device_get_softc(dev); int error, speed; u_int32_t caps, version; /* if we have a memory BAR(5) we are likely on an AHCI part */ ctlr->r_type2 = SYS_RES_MEMORY; ctlr->r_rid2 = PCIR_BAR(5); if (!(ctlr->r_res2 = bus_alloc_resource_any(dev, ctlr->r_type2, &ctlr->r_rid2, RF_ACTIVE))) return ENXIO; /* setup interrupt delivery if not done allready by a vendor driver */ if (!ctlr->r_irq) { if (ata_setup_interrupt(dev, ata_generic_intr)) { bus_release_resource(dev, ctlr->r_type2, ctlr->r_rid2, ctlr->r_res2); return ENXIO; } } else device_printf(dev, "AHCI called from vendor specific driver\n"); /* reset controller */ if ((error = ata_ahci_ctlr_reset(dev)) != 0) { bus_release_resource(dev, ctlr->r_type2, ctlr->r_rid2, ctlr->r_res2); return (error); }; /* get the number of HW channels */ ctlr->ichannels = ATA_INL(ctlr->r_res2, ATA_AHCI_PI); ctlr->channels = MAX(flsl(ctlr->ichannels), (ATA_INL(ctlr->r_res2, ATA_AHCI_CAP) & ATA_AHCI_CAP_NPMASK) + 1); - if (pci_get_devid(dev) == ATA_M88SX6111) + if (pci_get_devid(dev) == ATA_M88SE6111) ctlr->channels = 1; - else if (pci_get_devid(dev) == ATA_M88SX6121) + else if (pci_get_devid(dev) == ATA_M88SE6121) ctlr->channels = 2; - else if (pci_get_devid(dev) == ATA_M88SX6141 || - pci_get_devid(dev) == ATA_M88SX6145) + else if (pci_get_devid(dev) == ATA_M88SE6141 || + pci_get_devid(dev) == ATA_M88SE6145) ctlr->channels = 4; ctlr->reset = ata_ahci_reset; ctlr->ch_attach = ata_ahci_ch_attach; ctlr->ch_detach = ata_ahci_ch_detach; ctlr->ch_suspend = ata_ahci_ch_suspend; ctlr->ch_resume = ata_ahci_ch_resume; ctlr->setmode = ata_sata_setmode; ctlr->getrev = ata_sata_getrev; ctlr->suspend = ata_ahci_suspend; ctlr->resume = ata_ahci_ctlr_reset; /* announce we support the HW */ version = ATA_INL(ctlr->r_res2, ATA_AHCI_VS); caps = ATA_INL(ctlr->r_res2, ATA_AHCI_CAP); speed = (caps & ATA_AHCI_CAP_ISS) >> ATA_AHCI_CAP_ISS_SHIFT; device_printf(dev, "AHCI v%x.%02x controller with %d %sGbps ports, PM %s\n", ((version >> 20) & 0xf0) + ((version >> 16) & 0x0f), ((version >> 4) & 0xf0) + (version & 0x0f), (caps & ATA_AHCI_CAP_NPMASK) + 1, ((speed == 1) ? "1.5":((speed == 2) ? "3": ((speed == 3) ? "6":"?"))), (caps & ATA_AHCI_CAP_SPM) ? "supported" : "not supported"); if (bootverbose) { device_printf(dev, "Caps:%s%s%s%s%s%s%s%s %sGbps", (caps & ATA_AHCI_CAP_64BIT) ? " 64bit":"", (caps & ATA_AHCI_CAP_SNCQ) ? " NCQ":"", (caps & ATA_AHCI_CAP_SSNTF) ? " SNTF":"", (caps & ATA_AHCI_CAP_SMPS) ? " MPS":"", (caps & ATA_AHCI_CAP_SSS) ? " SS":"", (caps & ATA_AHCI_CAP_SALP) ? " ALP":"", (caps & ATA_AHCI_CAP_SAL) ? " AL":"", (caps & ATA_AHCI_CAP_SCLO) ? " CLO":"", ((speed == 1) ? "1.5":((speed == 2) ? "3": ((speed == 3) ? "6":"?")))); printf("%s%s%s%s%s%s %dcmd%s%s%s %dports\n", (caps & ATA_AHCI_CAP_SAM) ? " AM":"", (caps & ATA_AHCI_CAP_SPM) ? " PM":"", (caps & ATA_AHCI_CAP_FBSS) ? " FBS":"", (caps & ATA_AHCI_CAP_PMD) ? " PMD":"", (caps & ATA_AHCI_CAP_SSC) ? " SSC":"", (caps & ATA_AHCI_CAP_PSC) ? " PSC":"", ((caps & ATA_AHCI_CAP_NCS) >> ATA_AHCI_CAP_NCS_SHIFT) + 1, (caps & ATA_AHCI_CAP_CCCS) ? " CCC":"", (caps & ATA_AHCI_CAP_EMS) ? " EM":"", (caps & ATA_AHCI_CAP_SXS) ? " eSATA":"", (caps & ATA_AHCI_CAP_NPMASK) + 1); } return 0; } static int ata_ahci_ctlr_reset(device_t dev) { struct ata_pci_controller *ctlr = device_get_softc(dev); int timeout; /* enable AHCI mode */ ATA_OUTL(ctlr->r_res2, ATA_AHCI_GHC, ATA_AHCI_GHC_AE); /* reset AHCI controller */ ATA_OUTL(ctlr->r_res2, ATA_AHCI_GHC, ATA_AHCI_GHC_AE|ATA_AHCI_GHC_HR); for (timeout = 1000; timeout > 0; timeout--) { DELAY(1000); if ((ATA_INL(ctlr->r_res2, ATA_AHCI_GHC) & ATA_AHCI_GHC_HR) == 0) break; } if (timeout == 0) { device_printf(dev, "AHCI controller reset failure\n"); return ENXIO; } /* reenable AHCI mode */ ATA_OUTL(ctlr->r_res2, ATA_AHCI_GHC, ATA_AHCI_GHC_AE); /* clear interrupts */ ATA_OUTL(ctlr->r_res2, ATA_AHCI_IS, ATA_INL(ctlr->r_res2, ATA_AHCI_IS)); /* enable AHCI interrupts */ ATA_OUTL(ctlr->r_res2, ATA_AHCI_GHC, ATA_INL(ctlr->r_res2, ATA_AHCI_GHC) | ATA_AHCI_GHC_IE); return 0; } static int ata_ahci_suspend(device_t dev) { struct ata_pci_controller *ctlr = device_get_softc(dev); /* disable interupts so the state change(s) doesn't trigger */ ATA_OUTL(ctlr->r_res2, ATA_AHCI_GHC, ATA_INL(ctlr->r_res2, ATA_AHCI_GHC) & (~ATA_AHCI_GHC_IE)); return 0; } static int ata_ahci_ch_attach(device_t dev) { struct ata_pci_controller *ctlr = device_get_softc(device_get_parent(dev)); struct ata_channel *ch = device_get_softc(dev); int offset = ch->unit << 7; ata_ahci_dmainit(dev); /* set the SATA resources */ ch->r_io[ATA_SSTATUS].res = ctlr->r_res2; ch->r_io[ATA_SSTATUS].offset = ATA_AHCI_P_SSTS + offset; ch->r_io[ATA_SERROR].res = ctlr->r_res2; ch->r_io[ATA_SERROR].offset = ATA_AHCI_P_SERR + offset; ch->r_io[ATA_SCONTROL].res = ctlr->r_res2; ch->r_io[ATA_SCONTROL].offset = ATA_AHCI_P_SCTL + offset; ch->r_io[ATA_SACTIVE].res = ctlr->r_res2; ch->r_io[ATA_SACTIVE].offset = ATA_AHCI_P_SACT + offset; ch->hw.status = ata_ahci_status; ch->hw.begin_transaction = ata_ahci_begin_transaction; ch->hw.end_transaction = ata_ahci_end_transaction; ch->hw.command = NULL; /* not used here */ ch->hw.softreset = ata_ahci_softreset; ch->hw.pm_read = ata_ahci_pm_read; ch->hw.pm_write = ata_ahci_pm_write; ch->flags |= ATA_NO_SLAVE; ch->flags |= ATA_SATA; ata_ahci_ch_resume(dev); return 0; } static int ata_ahci_ch_detach(device_t dev) { struct ata_channel *ch = device_get_softc(dev); if (ch->dma.work_tag && ch->dma.work_map) bus_dmamap_sync(ch->dma.work_tag, ch->dma.work_map, BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); ata_ahci_ch_suspend(dev); ata_dmafini(dev); return (0); } static int ata_ahci_ch_suspend(device_t dev) { struct ata_pci_controller *ctlr = device_get_softc(device_get_parent(dev)); struct ata_channel *ch = device_get_softc(dev); int offset = ch->unit << 7; /* Disable port interrupts. */ ATA_OUTL(ctlr->r_res2, ATA_AHCI_P_IE + offset, 0); /* Reset command register. */ ata_ahci_stop(dev); ata_ahci_stop_fr(dev); ATA_OUTL(ctlr->r_res2, ATA_AHCI_P_CMD + offset, 0); /* Allow everything including partial and slumber modes. */ ATA_IDX_OUTL(ch, ATA_SCONTROL, 0); /* Request slumber mode transition and give some time to get there. */ ATA_OUTL(ctlr->r_res2, ATA_AHCI_P_CMD + offset, ATA_AHCI_P_CMD_SLUMBER); DELAY(100); /* Disable PHY. */ ATA_IDX_OUTL(ch, ATA_SCONTROL, ATA_SC_DET_DISABLE); return (0); } static int ata_ahci_ch_resume(device_t dev) { struct ata_pci_controller *ctlr = device_get_softc(device_get_parent(dev)); struct ata_channel *ch = device_get_softc(dev); uint64_t work; int offset = ch->unit << 7; /* Disable port interrupts */ ATA_OUTL(ctlr->r_res2, ATA_AHCI_P_IE + offset, 0); /* setup work areas */ work = ch->dma.work_bus + ATA_AHCI_CL_OFFSET; ATA_OUTL(ctlr->r_res2, ATA_AHCI_P_CLB + offset, work & 0xffffffff); ATA_OUTL(ctlr->r_res2, ATA_AHCI_P_CLBU + offset, work >> 32); work = ch->dma.work_bus + ATA_AHCI_FB_OFFSET; ATA_OUTL(ctlr->r_res2, ATA_AHCI_P_FB + offset, work & 0xffffffff); ATA_OUTL(ctlr->r_res2, ATA_AHCI_P_FBU + offset, work >> 32); /* activate the channel and power/spin up device */ ATA_OUTL(ctlr->r_res2, ATA_AHCI_P_CMD + offset, (ATA_AHCI_P_CMD_ACTIVE | ATA_AHCI_P_CMD_POD | ATA_AHCI_P_CMD_SUD | ((ch->pm_level > 1) ? ATA_AHCI_P_CMD_ALPE : 0) | ((ch->pm_level > 2) ? ATA_AHCI_P_CMD_ASP : 0 ))); ata_ahci_start_fr(dev); ata_ahci_start(dev); return (0); } static int ata_ahci_status(device_t dev) { struct ata_pci_controller *ctlr = device_get_softc(device_get_parent(dev)); struct ata_channel *ch = device_get_softc(dev); u_int32_t action = ATA_INL(ctlr->r_res2, ATA_AHCI_IS); int offset = ch->unit << 7; #define ATA_AHCI_STATBITS \ (ATA_AHCI_P_IX_IF|ATA_AHCI_P_IX_HBD|ATA_AHCI_P_IX_HBF|ATA_AHCI_P_IX_TFE) if (action & (1 << ch->unit)) { u_int32_t istatus = ATA_INL(ctlr->r_res2, ATA_AHCI_P_IS + offset); u_int32_t cstatus = ATA_INL(ctlr->r_res2, ATA_AHCI_P_CI + offset); /* clear interrupt(s) */ ATA_OUTL(ctlr->r_res2, ATA_AHCI_P_IS + offset, istatus); ATA_OUTL(ctlr->r_res2, ATA_AHCI_IS, 1 << ch->unit); /* do we have any PHY events ? */ if (istatus & (ATA_AHCI_P_IX_PRC | ATA_AHCI_P_IX_PC)) ata_sata_phy_check_events(dev, -1); /* do we have a potentially hanging engine to take care of? */ /* XXX SOS what todo on NCQ */ if ((istatus & ATA_AHCI_STATBITS) && (cstatus & 1)) { u_int32_t cmd = ATA_INL(ctlr->r_res2, ATA_AHCI_P_CMD + offset); int timeout = 0; /* kill off all activity on this channel */ ATA_OUTL(ctlr->r_res2, ATA_AHCI_P_CMD + offset, cmd & ~(ATA_AHCI_P_CMD_FRE | ATA_AHCI_P_CMD_ST)); /* XXX SOS this is not entirely wrong */ do { DELAY(1000); if (timeout++ > 1000) { device_printf(dev, "stopping AHCI engine failed\n"); break; } } while (ATA_INL(ctlr->r_res2, ATA_AHCI_P_CMD + offset) & ATA_AHCI_P_CMD_CR); /* start operations on this channel */ ATA_OUTL(ctlr->r_res2, ATA_AHCI_P_CMD + offset, cmd | (ATA_AHCI_P_CMD_FRE | ATA_AHCI_P_CMD_ST)); return 1; } else /* XXX SOS what todo on NCQ */ return (!(cstatus & 1)); } return 0; } /* must be called with ATA channel locked and state_mtx held */ static int ata_ahci_begin_transaction(struct ata_request *request) { struct ata_pci_controller *ctlr=device_get_softc(device_get_parent(request->parent)); struct ata_channel *ch = device_get_softc(request->parent); struct ata_ahci_cmd_tab *ctp; struct ata_ahci_cmd_list *clp; int offset = ch->unit << 7; int port = request->unit & 0x0f; int entries = 0; int fis_size; /* get a piece of the workspace for this request */ ctp = (struct ata_ahci_cmd_tab *) (ch->dma.work + ATA_AHCI_CT_OFFSET); /* setup the FIS for this request */ if (!(fis_size = ata_ahci_setup_fis(ctp, request))) { device_printf(request->parent, "setting up SATA FIS failed\n"); request->result = EIO; return ATA_OP_FINISHED; } /* if request moves data setup and load SG list */ if (request->flags & (ATA_R_READ | ATA_R_WRITE)) { if (ch->dma.load(request, ctp->prd_tab, &entries)) { device_printf(request->parent, "setting up DMA failed\n"); request->result = EIO; return ATA_OP_FINISHED; } } /* setup the command list entry */ clp = (struct ata_ahci_cmd_list *) (ch->dma.work + ATA_AHCI_CL_OFFSET); clp->prd_length = entries; clp->cmd_flags = (request->flags & ATA_R_WRITE ? ATA_AHCI_CMD_WRITE : 0) | (request->flags & ATA_R_ATAPI ? (ATA_AHCI_CMD_ATAPI | ATA_AHCI_CMD_PREFETCH) : 0) | (fis_size / sizeof(u_int32_t)) | (port << 12); clp->bytecount = 0; clp->cmd_table_phys = htole64(ch->dma.work_bus + ATA_AHCI_CT_OFFSET); /* set command type bit */ if (request->flags & ATA_R_ATAPI) ATA_OUTL(ctlr->r_res2, ATA_AHCI_P_CMD + offset, ATA_INL(ctlr->r_res2, ATA_AHCI_P_CMD + offset) | ATA_AHCI_P_CMD_ATAPI); else ATA_OUTL(ctlr->r_res2, ATA_AHCI_P_CMD + offset, ATA_INL(ctlr->r_res2, ATA_AHCI_P_CMD + offset) & ~ATA_AHCI_P_CMD_ATAPI); bus_dmamap_sync(ch->dma.work_tag, ch->dma.work_map, BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); /* issue command to controller */ ATA_OUTL(ctlr->r_res2, ATA_AHCI_P_CI + offset, 1); if (!(request->flags & ATA_R_ATAPI)) { /* device reset doesn't interrupt */ if (request->u.ata.command == ATA_DEVICE_RESET) { u_int32_t tf_data; int timeout = 1000000; do { DELAY(10); tf_data = ATA_INL(ctlr->r_res2, ATA_AHCI_P_TFD + (ch->unit<<7)); } while ((tf_data & ATA_S_BUSY) && timeout--); if (bootverbose) device_printf(ch->dev, "device_reset timeout=%dus\n", (1000000-timeout)*10); request->status = tf_data; if (request->status & ATA_S_ERROR) request->error = tf_data >> 8; return ATA_OP_FINISHED; } } /* start the timeout */ callout_reset(&request->callout, request->timeout * hz, (timeout_t*)ata_timeout, request); return ATA_OP_CONTINUES; } /* must be called with ATA channel locked and state_mtx held */ static int ata_ahci_end_transaction(struct ata_request *request) { struct ata_pci_controller *ctlr=device_get_softc(device_get_parent(request->parent)); struct ata_channel *ch = device_get_softc(request->parent); struct ata_ahci_cmd_list *clp; u_int32_t tf_data; int offset = ch->unit << 7; /* kill the timeout */ callout_stop(&request->callout); bus_dmamap_sync(ch->dma.work_tag, ch->dma.work_map, BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); /* get status */ tf_data = ATA_INL(ctlr->r_res2, ATA_AHCI_P_TFD + offset); request->status = tf_data; /* if error status get details */ if (request->status & ATA_S_ERROR) request->error = tf_data >> 8; /* on control commands read back registers to the request struct */ if (request->flags & ATA_R_CONTROL) { u_int8_t *fis = ch->dma.work + ATA_AHCI_FB_OFFSET + 0x40; request->u.ata.count = fis[12] | ((u_int16_t)fis[13] << 8); request->u.ata.lba = fis[4] | ((u_int64_t)fis[5] << 8) | ((u_int64_t)fis[6] << 16); if (request->flags & ATA_R_48BIT) request->u.ata.lba |= ((u_int64_t)fis[8] << 24) | ((u_int64_t)fis[9] << 32) | ((u_int64_t)fis[10] << 40); else request->u.ata.lba |= ((u_int64_t)(fis[7] & 0x0f) << 24); } /* record how much data we actually moved */ clp = (struct ata_ahci_cmd_list *) (ch->dma.work + ATA_AHCI_CL_OFFSET); request->donecount = le32toh(clp->bytecount); /* release SG list etc */ ch->dma.unload(request); return ATA_OP_FINISHED; } static int ata_ahci_issue_cmd(device_t dev, u_int16_t flags, int timeout) { struct ata_pci_controller *ctlr = device_get_softc(device_get_parent(dev)); struct ata_channel *ch = device_get_softc(dev); struct ata_ahci_cmd_list *clp = (struct ata_ahci_cmd_list *)(ch->dma.work + ATA_AHCI_CL_OFFSET); struct ata_ahci_cmd_tab *ctp = (struct ata_ahci_cmd_tab *)(ch->dma.work + ATA_AHCI_CT_OFFSET); u_int32_t status = 0; int offset = ch->unit << 7; int port = (ctp->cfis[1] & 0x0f); int count; clp->prd_length = 0; clp->cmd_flags = (20 / sizeof(u_int32_t)) | flags | (port << 12); clp->bytecount = 0; clp->cmd_table_phys = htole64(ch->dma.work_bus + ATA_AHCI_CT_OFFSET); bus_dmamap_sync(ch->dma.work_tag, ch->dma.work_map, BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); /* issue command to controller */ ATA_OUTL(ctlr->r_res2, ATA_AHCI_P_CI + offset, 1); /* poll for command finished */ for (count = 0; count < timeout; count++) { DELAY(1000); if (!((status = ATA_INL(ctlr->r_res2, ATA_AHCI_P_CI + offset)) & 1)) break; } bus_dmamap_sync(ch->dma.work_tag, ch->dma.work_map, BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); /* clear interrupts */ ATA_OUTL(ctlr->r_res2, ATA_AHCI_P_IS + offset, ATA_INL(ctlr->r_res2, ATA_AHCI_P_IS + offset)); if (timeout && (count >= timeout)) { if (bootverbose) { device_printf(dev, "ahci_issue_cmd timeout: %d of %dms, status=%08x\n", count, timeout, status); } return EIO; } return 0; } static int ata_ahci_pm_read(device_t dev, int port, int reg, u_int32_t *result) { struct ata_channel *ch = device_get_softc(dev); struct ata_ahci_cmd_tab *ctp = (struct ata_ahci_cmd_tab *)(ch->dma.work + ATA_AHCI_CT_OFFSET); u_int8_t *fis = ch->dma.work + ATA_AHCI_FB_OFFSET + 0x40; if (port < 0) { *result = ATA_IDX_INL(ch, reg); return (0); } if (port < ATA_PM) { switch (reg) { case ATA_SSTATUS: reg = 0; break; case ATA_SERROR: reg = 1; break; case ATA_SCONTROL: reg = 2; break; default: return (EINVAL); } } bzero(ctp->cfis, 64); ctp->cfis[0] = 0x27; /* host to device */ ctp->cfis[1] = 0x8f; /* command FIS to PM port */ ctp->cfis[2] = ATA_READ_PM; ctp->cfis[3] = reg; ctp->cfis[7] = port | ATA_D_LBA; ctp->cfis[15] = ATA_A_4BIT; if (ata_ahci_issue_cmd(dev, 0, 10)) { device_printf(dev, "error reading PM port\n"); return EIO; } *result = fis[12] | (fis[4] << 8) | (fis[5] << 16) | (fis[6] << 24); return 0; } static int ata_ahci_pm_write(device_t dev, int port, int reg, u_int32_t value) { struct ata_pci_controller *ctlr = device_get_softc(device_get_parent(dev)); struct ata_channel *ch = device_get_softc(dev); struct ata_ahci_cmd_tab *ctp = (struct ata_ahci_cmd_tab *)(ch->dma.work + ATA_AHCI_CT_OFFSET); int offset = ch->unit << 7; if (port < 0) { ATA_IDX_OUTL(ch, reg, value); return (0); } if (port < ATA_PM) { switch (reg) { case ATA_SSTATUS: reg = 0; break; case ATA_SERROR: reg = 1; break; case ATA_SCONTROL: reg = 2; break; default: return (EINVAL); } } bzero(ctp->cfis, 64); ctp->cfis[0] = 0x27; /* host to device */ ctp->cfis[1] = 0x8f; /* command FIS to PM port */ ctp->cfis[2] = ATA_WRITE_PM; ctp->cfis[3] = reg; ctp->cfis[7] = port | ATA_D_LBA; ctp->cfis[12] = value & 0xff; ctp->cfis[4] = (value >> 8) & 0xff; ctp->cfis[5] = (value >> 16) & 0xff; ctp->cfis[6] = (value >> 24) & 0xff; ctp->cfis[15] = ATA_A_4BIT; if (ata_ahci_issue_cmd(dev, 0, 100)) { device_printf(dev, "error writing PM port\n"); return ATA_E_ABORT; } return (ATA_INL(ctlr->r_res2, ATA_AHCI_P_TFD + offset) >> 8) & 0xff; } static void ata_ahci_stop(device_t dev) { struct ata_pci_controller *ctlr = device_get_softc(device_get_parent(dev)); struct ata_channel *ch = device_get_softc(dev); u_int32_t cmd; int offset = ch->unit << 7; int timeout; /* kill off all activity on this channel */ cmd = ATA_INL(ctlr->r_res2, ATA_AHCI_P_CMD + offset); ATA_OUTL(ctlr->r_res2, ATA_AHCI_P_CMD + offset, cmd & ~ATA_AHCI_P_CMD_ST); /* XXX SOS this is not entirely wrong */ timeout = 0; do { DELAY(1000); if (timeout++ > 1000) { device_printf(dev, "stopping AHCI engine failed\n"); break; } } while (ATA_INL(ctlr->r_res2, ATA_AHCI_P_CMD + offset) & ATA_AHCI_P_CMD_CR); } static void ata_ahci_clo(device_t dev) { struct ata_pci_controller *ctlr = device_get_softc(device_get_parent(dev)); struct ata_channel *ch = device_get_softc(dev); u_int32_t cmd; int offset = ch->unit << 7; int timeout; /* issue Command List Override if supported */ if (ATA_INL(ctlr->r_res2, ATA_AHCI_CAP) & ATA_AHCI_CAP_SCLO) { cmd = ATA_INL(ctlr->r_res2, ATA_AHCI_P_CMD + offset); cmd |= ATA_AHCI_P_CMD_CLO; ATA_OUTL(ctlr->r_res2, ATA_AHCI_P_CMD + offset, cmd); timeout = 0; do { DELAY(1000); if (timeout++ > 1000) { device_printf(dev, "executing CLO failed\n"); break; } } while (ATA_INL(ctlr->r_res2, ATA_AHCI_P_CMD+offset)&ATA_AHCI_P_CMD_CLO); } } static void ata_ahci_start(device_t dev) { struct ata_pci_controller *ctlr = device_get_softc(device_get_parent(dev)); struct ata_channel *ch = device_get_softc(dev); u_int32_t cmd; int offset = ch->unit << 7; /* clear SATA error register */ ATA_IDX_OUTL(ch, ATA_SERROR, ATA_IDX_INL(ch, ATA_SERROR)); /* clear any interrupts pending on this channel */ ATA_OUTL(ctlr->r_res2, ATA_AHCI_P_IS + offset, ATA_INL(ctlr->r_res2, ATA_AHCI_P_IS + offset)); /* start operations on this channel */ cmd = ATA_INL(ctlr->r_res2, ATA_AHCI_P_CMD + offset); ATA_OUTL(ctlr->r_res2, ATA_AHCI_P_CMD + offset, cmd | ATA_AHCI_P_CMD_ST | (ch->devices & ATA_PORTMULTIPLIER ? ATA_AHCI_P_CMD_PMA : 0)); } static void ata_ahci_stop_fr(device_t dev) { struct ata_pci_controller *ctlr = device_get_softc(device_get_parent(dev)); struct ata_channel *ch = device_get_softc(dev); u_int32_t cmd; int offset = ch->unit << 7; int timeout; /* kill off all activity on this channel */ cmd = ATA_INL(ctlr->r_res2, ATA_AHCI_P_CMD + offset); ATA_OUTL(ctlr->r_res2, ATA_AHCI_P_CMD + offset, cmd & ~ATA_AHCI_P_CMD_FRE); timeout = 0; do { DELAY(1000); if (timeout++ > 1000) { device_printf(dev, "stopping AHCI FR engine failed\n"); break; } } while (ATA_INL(ctlr->r_res2, ATA_AHCI_P_CMD + offset) & ATA_AHCI_P_CMD_FR); } static void ata_ahci_start_fr(device_t dev) { struct ata_pci_controller *ctlr = device_get_softc(device_get_parent(dev)); struct ata_channel *ch = device_get_softc(dev); u_int32_t cmd; int offset = ch->unit << 7; /* start FIS reception on this channel */ cmd = ATA_INL(ctlr->r_res2, ATA_AHCI_P_CMD + offset); ATA_OUTL(ctlr->r_res2, ATA_AHCI_P_CMD + offset, cmd | ATA_AHCI_P_CMD_FRE); } static int ata_ahci_wait_ready(device_t dev, int t) { struct ata_pci_controller *ctlr = device_get_softc(device_get_parent(dev)); struct ata_channel *ch = device_get_softc(dev); int offset = ch->unit << 7; int timeout = 0; uint32_t val; while ((val = ATA_INL(ctlr->r_res2, ATA_AHCI_P_TFD + offset)) & (ATA_S_BUSY | ATA_S_DRQ)) { DELAY(1000); if (timeout++ > t) { device_printf(dev, "port is not ready (timeout %dms) tfd = %08x\n", t, val); return (EBUSY); } } if (bootverbose) device_printf(dev, "ready wait time=%dms\n", timeout); return (0); } static int ata_ahci_hardreset(device_t dev, int port, uint32_t *signature) { struct ata_pci_controller *ctlr = device_get_softc(device_get_parent(dev)); struct ata_channel *ch = device_get_softc(dev); int offset = ch->unit << 7; *signature = 0xffffffff; ata_ahci_stop(dev); /* Reset port */ if (!ata_sata_phy_reset(dev, port, 0)) return (ENOENT); /* Wait for clearing busy status. */ if (ata_ahci_wait_ready(dev, 15000)) { device_printf(dev, "hardware reset timeout\n"); return (EBUSY); } *signature = ATA_INL(ctlr->r_res2, ATA_AHCI_P_SIG + offset); ata_ahci_start(dev); return (0); } static u_int32_t ata_ahci_softreset(device_t dev, int port) { struct ata_channel *ch = device_get_softc(dev); struct ata_ahci_cmd_tab *ctp = (struct ata_ahci_cmd_tab *)(ch->dma.work + ATA_AHCI_CT_OFFSET); u_int8_t *fis = ch->dma.work + ATA_AHCI_FB_OFFSET + 0x40; if (bootverbose) device_printf(dev, "software reset port %d...\n", port); /* kick controller into sane state */ ata_ahci_stop(dev); ata_ahci_clo(dev); ata_ahci_start(dev); /* pull reset active */ bzero(ctp->cfis, 64); ctp->cfis[0] = 0x27; ctp->cfis[1] = port & 0x0f; //ctp->cfis[7] = ATA_D_LBA | ATA_D_IBM; ctp->cfis[15] = (ATA_A_4BIT | ATA_A_RESET); if (ata_ahci_issue_cmd(dev, ATA_AHCI_CMD_RESET | ATA_AHCI_CMD_CLR_BUSY,100)) { device_printf(dev, "software reset set timeout\n"); return (-1); } ata_udelay(50); /* pull reset inactive -> device softreset */ bzero(ctp->cfis, 64); ctp->cfis[0] = 0x27; ctp->cfis[1] = port & 0x0f; //ctp->cfis[7] = ATA_D_LBA | ATA_D_IBM; ctp->cfis[15] = ATA_A_4BIT; ata_ahci_issue_cmd(dev, 0, 3000); if (ata_ahci_wait_ready(dev, 0)) { device_printf(dev, "software reset clear timeout\n"); return (-1); } return (((u_int32_t)fis[6] << 24) | ((u_int32_t)fis[5] << 16) | ((u_int32_t)fis[4] << 8) | (u_int32_t)fis[12]); } static void ata_ahci_reset(device_t dev) { struct ata_pci_controller *ctlr = device_get_softc(device_get_parent(dev)); struct ata_channel *ch = device_get_softc(dev); u_int32_t signature; int offset = ch->unit << 7; if (bootverbose) device_printf(dev, "AHCI reset...\n"); /* Disable port interrupts */ ATA_OUTL(ctlr->r_res2, ATA_AHCI_P_IE + offset, 0); if (ata_ahci_hardreset(dev, -1, &signature)) { if (bootverbose) device_printf(dev, "AHCI reset done: phy reset found no device\n"); ch->devices = 0; /* enable wanted port interrupts */ ATA_OUTL(ctlr->r_res2, ATA_AHCI_P_IE + offset, (ATA_AHCI_P_IX_CPD | ATA_AHCI_P_IX_PRC | ATA_AHCI_P_IX_PC)); return; } /* enable wanted port interrupts */ ATA_OUTL(ctlr->r_res2, ATA_AHCI_P_IE + offset, (ATA_AHCI_P_IX_CPD | ATA_AHCI_P_IX_TFE | ATA_AHCI_P_IX_HBF | ATA_AHCI_P_IX_HBD | ATA_AHCI_P_IX_IF | ATA_AHCI_P_IX_OF | ((ch->pm_level == 0) ? ATA_AHCI_P_IX_PRC | ATA_AHCI_P_IX_PC : 0) | ATA_AHCI_P_IX_DP | ATA_AHCI_P_IX_UF | ATA_AHCI_P_IX_SDB | ATA_AHCI_P_IX_DS | ATA_AHCI_P_IX_PS | ATA_AHCI_P_IX_DHR)); /* * Only probe for PortMultiplier if HW has support. * Ignore Marvell, which is not working, */ if ((ATA_INL(ctlr->r_res2, ATA_AHCI_CAP) & ATA_AHCI_CAP_SPM) && pci_get_vendor(ctlr->dev) != 0x11ab) { signature = ata_ahci_softreset(dev, ATA_PM); /* Workaround for some ATI chips, failing to soft-reset * when port multiplicator supported, but absent. * XXX: We can also check PxIS.IPMS==1 here to be sure. */ if (signature == 0xffffffff) signature = ata_ahci_softreset(dev, 0); } else { signature = ata_ahci_softreset(dev, 0); } if (bootverbose) device_printf(dev, "SIGNATURE: %08x\n", signature); switch (signature >> 16) { case 0x0000: ch->devices = ATA_ATA_MASTER; break; case 0x9669: ch->devices = ATA_PORTMULTIPLIER; ata_pm_identify(dev); break; case 0xeb14: ch->devices = ATA_ATAPI_MASTER; break; default: /* SOS XXX */ if (bootverbose) device_printf(dev, "Unknown signature, assuming disk device\n"); ch->devices = ATA_ATA_MASTER; } if (bootverbose) device_printf(dev, "AHCI reset done: devices=%08x\n", ch->devices); } static void ata_ahci_dmasetprd(void *xsc, bus_dma_segment_t *segs, int nsegs, int error) { struct ata_dmasetprd_args *args = xsc; struct ata_ahci_dma_prd *prd = args->dmatab; int i; if (!(args->error = error)) { for (i = 0; i < nsegs; i++) { prd[i].dba = htole64(segs[i].ds_addr); prd[i].dbc = htole32((segs[i].ds_len - 1) & ATA_AHCI_PRD_MASK); } } KASSERT(nsegs <= ATA_AHCI_DMA_ENTRIES, ("too many DMA segment entries\n")); args->nsegs = nsegs; } static void ata_ahci_dmainit(device_t dev) { struct ata_pci_controller *ctlr = device_get_softc(device_get_parent(dev)); struct ata_channel *ch = device_get_softc(dev); /* note start and stop are not used here */ ch->dma.setprd = ata_ahci_dmasetprd; ch->dma.max_iosize = (ATA_AHCI_DMA_ENTRIES - 1) * PAGE_SIZE; if (ATA_INL(ctlr->r_res2, ATA_AHCI_CAP) & ATA_AHCI_CAP_64BIT) ch->dma.max_address = BUS_SPACE_MAXADDR; ata_dmainit(dev); } static int ata_ahci_setup_fis(struct ata_ahci_cmd_tab *ctp, struct ata_request *request) { bzero(ctp->cfis, 64); if (request->flags & ATA_R_ATAPI) { bzero(ctp->acmd, 32); bcopy(request->u.atapi.ccb, ctp->acmd, 16); } return ata_request2fis_h2d(request, &ctp->cfis[0]); } ATA_DECLARE_DRIVER(ata_ahci); static device_method_t ata_ahci_ata_methods[] = { DEVMETHOD(device_probe, ata_ahci_ata_probe), DEVMETHOD(device_attach, ata_ahci_ata_attach), DEVMETHOD(device_detach, ata_pci_detach), DEVMETHOD(device_suspend, ata_pci_suspend), DEVMETHOD(device_resume, ata_pci_resume), DEVMETHOD(device_shutdown, bus_generic_shutdown), DEVMETHOD(bus_read_ivar, ata_pci_read_ivar), DEVMETHOD(bus_write_ivar, ata_pci_write_ivar), DEVMETHOD(bus_alloc_resource, ata_pci_alloc_resource), DEVMETHOD(bus_release_resource, ata_pci_release_resource), DEVMETHOD(bus_activate_resource, bus_generic_activate_resource), DEVMETHOD(bus_deactivate_resource, bus_generic_deactivate_resource), DEVMETHOD(bus_setup_intr, ata_pci_setup_intr), DEVMETHOD(bus_teardown_intr, ata_pci_teardown_intr), DEVMETHOD_END }; static driver_t ata_ahci_ata_driver = { "atapci", ata_ahci_ata_methods, sizeof(struct ata_pci_controller) }; DRIVER_MODULE(ata_ahci_ata, atapci, ata_ahci_ata_driver, ata_pci_devclass, NULL, NULL); Index: stable/8/sys/dev/ata/chipsets/ata-marvell.c =================================================================== --- stable/8/sys/dev/ata/chipsets/ata-marvell.c (revision 243119) +++ stable/8/sys/dev/ata/chipsets/ata-marvell.c (revision 243120) @@ -1,630 +1,630 @@ /*- * Copyright (c) 1998 - 2008 Søren Schmidt * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer, * without modification, immediately at the beginning of the file. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include "opt_ata.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include /* local prototypes */ static int ata_marvell_chipinit(device_t dev); static int ata_marvell_ch_attach(device_t dev); static int ata_marvell_setmode(device_t dev, int target, int mode); static int ata_marvell_dummy_chipinit(device_t dev); static int ata_marvell_edma_ch_attach(device_t dev); static int ata_marvell_edma_ch_detach(device_t dev); static int ata_marvell_edma_status(device_t dev); static int ata_marvell_edma_begin_transaction(struct ata_request *request); static int ata_marvell_edma_end_transaction(struct ata_request *request); static void ata_marvell_edma_reset(device_t dev); static void ata_marvell_edma_dmasetprd(void *xsc, bus_dma_segment_t *segs, int nsegs, int error); static void ata_marvell_edma_dmainit(device_t dev); /* misc defines */ #define MV_50XX 50 #define MV_60XX 60 #define MV_6042 62 #define MV_7042 72 #define MV_61XX 61 #define MV_91XX 91 /* * Marvell chipset support functions */ #define ATA_MV_HOST_BASE(ch) \ ((ch->unit & 3) * 0x0100) + (ch->unit > 3 ? 0x30000 : 0x20000) #define ATA_MV_EDMA_BASE(ch) \ ((ch->unit & 3) * 0x2000) + (ch->unit > 3 ? 0x30000 : 0x20000) struct ata_marvell_response { u_int16_t tag; u_int8_t edma_status; u_int8_t dev_status; u_int32_t timestamp; }; struct ata_marvell_dma_prdentry { u_int32_t addrlo; u_int32_t count; u_int32_t addrhi; u_int32_t reserved; }; static int ata_marvell_probe(device_t dev) { struct ata_pci_controller *ctlr = device_get_softc(dev); static const struct ata_chip_id ids[] = {{ ATA_M88SX5040, 0, 4, MV_50XX, ATA_SA150, "88SX5040" }, { ATA_M88SX5041, 0, 4, MV_50XX, ATA_SA150, "88SX5041" }, { ATA_M88SX5080, 0, 8, MV_50XX, ATA_SA150, "88SX5080" }, { ATA_M88SX5081, 0, 8, MV_50XX, ATA_SA150, "88SX5081" }, { ATA_M88SX6041, 0, 4, MV_60XX, ATA_SA300, "88SX6041" }, { ATA_M88SX6042, 0, 4, MV_6042, ATA_SA300, "88SX6042" }, { ATA_M88SX6081, 0, 8, MV_60XX, ATA_SA300, "88SX6081" }, { ATA_M88SX7042, 0, 4, MV_7042, ATA_SA300, "88SX7042" }, - { ATA_M88SX6101, 0, 0, MV_61XX, ATA_UDMA6, "88SX6101" }, - { ATA_M88SX6102, 0, 0, MV_61XX, ATA_UDMA6, "88SX6102" }, - { ATA_M88SX6111, 0, 1, MV_61XX, ATA_UDMA6, "88SX6111" }, - { ATA_M88SX6121, 0, 2, MV_61XX, ATA_UDMA6, "88SX6121" }, - { ATA_M88SX6141, 0, 4, MV_61XX, ATA_UDMA6, "88SX6141" }, - { ATA_M88SX6145, 0, 4, MV_61XX, ATA_UDMA6, "88SX6145" }, + { ATA_M88SE6101, 0, 0, MV_61XX, ATA_UDMA6, "88SE6101" }, + { ATA_M88SE6102, 0, 0, MV_61XX, ATA_UDMA6, "88SE6102" }, + { ATA_M88SE6111, 0, 1, MV_61XX, ATA_UDMA6, "88SE6111" }, + { ATA_M88SE6121, 0, 2, MV_61XX, ATA_UDMA6, "88SE6121" }, + { ATA_M88SE6141, 0, 4, MV_61XX, ATA_UDMA6, "88SE6141" }, + { ATA_M88SE6145, 0, 4, MV_61XX, ATA_UDMA6, "88SE6145" }, { 0x91a41b4b, 0, 0, MV_91XX, ATA_UDMA6, "88SE912x" }, { 0, 0, 0, 0, 0, 0}}; if (pci_get_vendor(dev) != ATA_MARVELL_ID && pci_get_vendor(dev) != ATA_MARVELL2_ID) return ENXIO; if (!(ctlr->chip = ata_match_chip(dev, ids))) return ENXIO; ata_set_desc(dev); switch (ctlr->chip->cfg2) { case MV_50XX: case MV_60XX: case MV_6042: case MV_7042: ctlr->chipinit = ata_marvell_edma_chipinit; break; case MV_61XX: ctlr->chipinit = ata_marvell_chipinit; break; case MV_91XX: ctlr->chipinit = ata_marvell_dummy_chipinit; break; } return (BUS_PROBE_DEFAULT); } static int ata_marvell_chipinit(device_t dev) { struct ata_pci_controller *ctlr = device_get_softc(dev); device_t child; if (ata_setup_interrupt(dev, ata_generic_intr)) return ENXIO; /* Create AHCI subdevice if AHCI part present. */ if (ctlr->chip->cfg1) { child = device_add_child(dev, NULL, -1); if (child != NULL) { device_set_ivars(child, (void *)(intptr_t)-1); bus_generic_attach(dev); } } ctlr->ch_attach = ata_marvell_ch_attach; ctlr->ch_detach = ata_pci_ch_detach; ctlr->reset = ata_generic_reset; ctlr->setmode = ata_marvell_setmode; ctlr->channels = 1; return (0); } static int ata_marvell_ch_attach(device_t dev) { struct ata_channel *ch = device_get_softc(dev); int error; error = ata_pci_ch_attach(dev); /* dont use 32 bit PIO transfers */ ch->flags |= ATA_USE_16BIT; ch->flags |= ATA_CHECKS_CABLE; return (error); } static int ata_marvell_setmode(device_t dev, int target, int mode) { struct ata_pci_controller *ctlr = device_get_softc(device_get_parent(dev)); struct ata_channel *ch = device_get_softc(dev); mode = min(mode, ctlr->chip->max_dma); /* Check for 80pin cable present. */ if (ata_dma_check_80pin && mode > ATA_UDMA2 && ATA_IDX_INB(ch, ATA_BMDEVSPEC_0) & 0x01) { ata_print_cable(dev, "controller"); mode = ATA_UDMA2; } /* Nothing to do to setup mode, the controller snoop SET_FEATURE cmd. */ return (mode); } static int ata_marvell_dummy_chipinit(device_t dev) { struct ata_pci_controller *ctlr = device_get_softc(dev); ctlr->channels = 0; return (0); } int ata_marvell_edma_chipinit(device_t dev) { struct ata_pci_controller *ctlr = device_get_softc(dev); if (ata_setup_interrupt(dev, ata_generic_intr)) return ENXIO; ctlr->r_type1 = SYS_RES_MEMORY; ctlr->r_rid1 = PCIR_BAR(0); if (!(ctlr->r_res1 = bus_alloc_resource_any(dev, ctlr->r_type1, &ctlr->r_rid1, RF_ACTIVE))) return ENXIO; /* mask all host controller interrupts */ ATA_OUTL(ctlr->r_res1, 0x01d64, 0x00000000); /* mask all PCI interrupts */ ATA_OUTL(ctlr->r_res1, 0x01d5c, 0x00000000); ctlr->ch_attach = ata_marvell_edma_ch_attach; ctlr->ch_detach = ata_marvell_edma_ch_detach; ctlr->reset = ata_marvell_edma_reset; ctlr->setmode = ata_sata_setmode; ctlr->getrev = ata_sata_getrev; ctlr->channels = ctlr->chip->cfg1; /* clear host controller interrupts */ ATA_OUTL(ctlr->r_res1, 0x20014, 0x00000000); if (ctlr->chip->cfg1 > 4) ATA_OUTL(ctlr->r_res1, 0x30014, 0x00000000); /* clear PCI interrupts */ ATA_OUTL(ctlr->r_res1, 0x01d58, 0x00000000); /* unmask PCI interrupts we want */ ATA_OUTL(ctlr->r_res1, 0x01d5c, 0x007fffff); /* unmask host controller interrupts we want */ ATA_OUTL(ctlr->r_res1, 0x01d64, 0x000000ff/*HC0*/ | 0x0001fe00/*HC1*/ | /*(1<<19) | (1<<20) | (1<<21) |*/(1<<22) | (1<<24) | (0x7f << 25)); return 0; } static int ata_marvell_edma_ch_attach(device_t dev) { struct ata_pci_controller *ctlr = device_get_softc(device_get_parent(dev)); struct ata_channel *ch = device_get_softc(dev); u_int64_t work; int i; ata_marvell_edma_dmainit(dev); work = ch->dma.work_bus; /* clear work area */ bzero(ch->dma.work, 1024+256); bus_dmamap_sync(ch->dma.work_tag, ch->dma.work_map, BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); /* set legacy ATA resources */ for (i = ATA_DATA; i <= ATA_COMMAND; i++) { ch->r_io[i].res = ctlr->r_res1; ch->r_io[i].offset = 0x02100 + (i << 2) + ATA_MV_EDMA_BASE(ch); } ch->r_io[ATA_CONTROL].res = ctlr->r_res1; ch->r_io[ATA_CONTROL].offset = 0x02120 + ATA_MV_EDMA_BASE(ch); ch->r_io[ATA_IDX_ADDR].res = ctlr->r_res1; ata_default_registers(dev); /* set SATA resources */ switch (ctlr->chip->cfg2) { case MV_50XX: ch->r_io[ATA_SSTATUS].res = ctlr->r_res1; ch->r_io[ATA_SSTATUS].offset = 0x00100 + ATA_MV_HOST_BASE(ch); ch->r_io[ATA_SERROR].res = ctlr->r_res1; ch->r_io[ATA_SERROR].offset = 0x00104 + ATA_MV_HOST_BASE(ch); ch->r_io[ATA_SCONTROL].res = ctlr->r_res1; ch->r_io[ATA_SCONTROL].offset = 0x00108 + ATA_MV_HOST_BASE(ch); break; case MV_60XX: case MV_6042: case MV_7042: ch->r_io[ATA_SSTATUS].res = ctlr->r_res1; ch->r_io[ATA_SSTATUS].offset = 0x02300 + ATA_MV_EDMA_BASE(ch); ch->r_io[ATA_SERROR].res = ctlr->r_res1; ch->r_io[ATA_SERROR].offset = 0x02304 + ATA_MV_EDMA_BASE(ch); ch->r_io[ATA_SCONTROL].res = ctlr->r_res1; ch->r_io[ATA_SCONTROL].offset = 0x02308 + ATA_MV_EDMA_BASE(ch); ch->r_io[ATA_SACTIVE].res = ctlr->r_res1; ch->r_io[ATA_SACTIVE].offset = 0x02350 + ATA_MV_EDMA_BASE(ch); break; } ch->flags |= ATA_NO_SLAVE; ch->flags |= ATA_USE_16BIT; /* XXX SOS needed ? */ ch->flags |= ATA_SATA; ata_generic_hw(dev); ch->hw.begin_transaction = ata_marvell_edma_begin_transaction; ch->hw.end_transaction = ata_marvell_edma_end_transaction; ch->hw.status = ata_marvell_edma_status; /* disable the EDMA machinery */ ATA_OUTL(ctlr->r_res1, 0x02028 + ATA_MV_EDMA_BASE(ch), 0x00000002); DELAY(100000); /* SOS should poll for disabled */ /* set configuration to non-queued 128b read transfers stop on error */ ATA_OUTL(ctlr->r_res1, 0x02000 + ATA_MV_EDMA_BASE(ch), (1<<11) | (1<<13)); /* request queue base high */ ATA_OUTL(ctlr->r_res1, 0x02010 + ATA_MV_EDMA_BASE(ch), work >> 32); /* request queue in ptr */ ATA_OUTL(ctlr->r_res1, 0x02014 + ATA_MV_EDMA_BASE(ch), work & 0xffffffff); /* request queue out ptr */ ATA_OUTL(ctlr->r_res1, 0x02018 + ATA_MV_EDMA_BASE(ch), 0x0); /* response queue base high */ work += 1024; ATA_OUTL(ctlr->r_res1, 0x0201c + ATA_MV_EDMA_BASE(ch), work >> 32); /* response queue in ptr */ ATA_OUTL(ctlr->r_res1, 0x02020 + ATA_MV_EDMA_BASE(ch), 0x0); /* response queue out ptr */ ATA_OUTL(ctlr->r_res1, 0x02024 + ATA_MV_EDMA_BASE(ch), work & 0xffffffff); /* clear SATA error register */ ATA_IDX_OUTL(ch, ATA_SERROR, ATA_IDX_INL(ch, ATA_SERROR)); /* clear any outstanding error interrupts */ ATA_OUTL(ctlr->r_res1, 0x02008 + ATA_MV_EDMA_BASE(ch), 0x0); /* unmask all error interrupts */ ATA_OUTL(ctlr->r_res1, 0x0200c + ATA_MV_EDMA_BASE(ch), ~0x0); /* enable EDMA machinery */ ATA_OUTL(ctlr->r_res1, 0x02028 + ATA_MV_EDMA_BASE(ch), 0x00000001); return 0; } static int ata_marvell_edma_ch_detach(device_t dev) { struct ata_channel *ch = device_get_softc(dev); if (ch->dma.work_tag && ch->dma.work_map) bus_dmamap_sync(ch->dma.work_tag, ch->dma.work_map, BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); ata_dmafini(dev); return (0); } static int ata_marvell_edma_status(device_t dev) { struct ata_pci_controller *ctlr = device_get_softc(device_get_parent(dev)); struct ata_channel *ch = device_get_softc(dev); u_int32_t cause = ATA_INL(ctlr->r_res1, 0x01d60); int shift = (ch->unit << 1) + (ch->unit > 3); if (cause & (1 << shift)) { /* clear interrupt(s) */ ATA_OUTL(ctlr->r_res1, 0x02008 + ATA_MV_EDMA_BASE(ch), 0x0); /* do we have any PHY events ? */ ata_sata_phy_check_events(dev, -1); } /* do we have any device action ? */ return (cause & (2 << shift)); } /* must be called with ATA channel locked and state_mtx held */ static int ata_marvell_edma_begin_transaction(struct ata_request *request) { struct ata_pci_controller *ctlr=device_get_softc(device_get_parent(request->parent)); struct ata_channel *ch = device_get_softc(request->parent); u_int32_t req_in; u_int8_t *bytep; int i; int error, slot; /* only DMA R/W goes through the EMDA machine */ if (request->u.ata.command != ATA_READ_DMA && request->u.ata.command != ATA_WRITE_DMA && request->u.ata.command != ATA_READ_DMA48 && request->u.ata.command != ATA_WRITE_DMA48) { /* disable the EDMA machinery */ if (ATA_INL(ctlr->r_res1, 0x02028 + ATA_MV_EDMA_BASE(ch)) & 0x00000001) ATA_OUTL(ctlr->r_res1, 0x02028 + ATA_MV_EDMA_BASE(ch), 0x00000002); return ata_begin_transaction(request); } /* check sanity, setup SG list and DMA engine */ if ((error = ch->dma.load(request, NULL, NULL))) { device_printf(request->parent, "setting up DMA failed\n"); request->result = error; return ATA_OP_FINISHED; } /* get next free request queue slot */ req_in = ATA_INL(ctlr->r_res1, 0x02014 + ATA_MV_EDMA_BASE(ch)); slot = (((req_in & ~0xfffffc00) >> 5) + 0) & 0x1f; bytep = (u_int8_t *)(ch->dma.work); bytep += (slot << 5); /* fill in this request */ le32enc(bytep + 0 * sizeof(u_int32_t), request->dma->sg_bus & 0xffffffff); le32enc(bytep + 1 * sizeof(u_int32_t), (u_int64_t)request->dma->sg_bus >> 32); if (ctlr->chip->cfg2 != MV_6042 && ctlr->chip->cfg2 != MV_7042) { le16enc(bytep + 4 * sizeof(u_int16_t), (request->flags & ATA_R_READ ? 0x01 : 0x00) | (request->tag << 1)); i = 10; bytep[i++] = (request->u.ata.count >> 8) & 0xff; bytep[i++] = 0x10 | ATA_COUNT; bytep[i++] = request->u.ata.count & 0xff; bytep[i++] = 0x10 | ATA_COUNT; bytep[i++] = (request->u.ata.lba >> 24) & 0xff; bytep[i++] = 0x10 | ATA_SECTOR; bytep[i++] = request->u.ata.lba & 0xff; bytep[i++] = 0x10 | ATA_SECTOR; bytep[i++] = (request->u.ata.lba >> 32) & 0xff; bytep[i++] = 0x10 | ATA_CYL_LSB; bytep[i++] = (request->u.ata.lba >> 8) & 0xff; bytep[i++] = 0x10 | ATA_CYL_LSB; bytep[i++] = (request->u.ata.lba >> 40) & 0xff; bytep[i++] = 0x10 | ATA_CYL_MSB; bytep[i++] = (request->u.ata.lba >> 16) & 0xff; bytep[i++] = 0x10 | ATA_CYL_MSB; bytep[i++] = ATA_D_LBA | ATA_D_IBM | ((request->u.ata.lba >> 24) & 0xf); bytep[i++] = 0x10 | ATA_DRIVE; bytep[i++] = request->u.ata.command; bytep[i++] = 0x90 | ATA_COMMAND; } else { le32enc(bytep + 2 * sizeof(u_int32_t), (request->flags & ATA_R_READ ? 0x01 : 0x00) | (request->tag << 1)); i = 16; bytep[i++] = 0; bytep[i++] = 0; bytep[i++] = request->u.ata.command; bytep[i++] = request->u.ata.feature & 0xff; bytep[i++] = request->u.ata.lba & 0xff; bytep[i++] = (request->u.ata.lba >> 8) & 0xff; bytep[i++] = (request->u.ata.lba >> 16) & 0xff; bytep[i++] = ATA_D_LBA | ATA_D_IBM | ((request->u.ata.lba >> 24) & 0x0f); bytep[i++] = (request->u.ata.lba >> 24) & 0xff; bytep[i++] = (request->u.ata.lba >> 32) & 0xff; bytep[i++] = (request->u.ata.lba >> 40) & 0xff; bytep[i++] = (request->u.ata.feature >> 8) & 0xff; bytep[i++] = request->u.ata.count & 0xff; bytep[i++] = (request->u.ata.count >> 8) & 0xff; bytep[i++] = 0; bytep[i++] = 0; } bus_dmamap_sync(ch->dma.work_tag, ch->dma.work_map, BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); /* enable EDMA machinery if needed */ if (!(ATA_INL(ctlr->r_res1, 0x02028 + ATA_MV_EDMA_BASE(ch)) & 0x00000001)) { ATA_OUTL(ctlr->r_res1, 0x02028 + ATA_MV_EDMA_BASE(ch), 0x00000001); while (!(ATA_INL(ctlr->r_res1, 0x02028 + ATA_MV_EDMA_BASE(ch)) & 0x00000001)) DELAY(10); } /* tell EDMA it has a new request */ slot = (((req_in & ~0xfffffc00) >> 5) + 1) & 0x1f; req_in &= 0xfffffc00; req_in += (slot << 5); ATA_OUTL(ctlr->r_res1, 0x02014 + ATA_MV_EDMA_BASE(ch), req_in); return ATA_OP_CONTINUES; } /* must be called with ATA channel locked and state_mtx held */ static int ata_marvell_edma_end_transaction(struct ata_request *request) { struct ata_pci_controller *ctlr=device_get_softc(device_get_parent(request->parent)); struct ata_channel *ch = device_get_softc(request->parent); int offset = (ch->unit > 3 ? 0x30014 : 0x20014); u_int32_t icr = ATA_INL(ctlr->r_res1, offset); int res; /* EDMA interrupt */ if ((icr & (0x0001 << (ch->unit & 3)))) { struct ata_marvell_response *response; u_int32_t rsp_in, rsp_out; int slot; /* stop timeout */ callout_stop(&request->callout); /* get response ptr's */ rsp_in = ATA_INL(ctlr->r_res1, 0x02020 + ATA_MV_EDMA_BASE(ch)); rsp_out = ATA_INL(ctlr->r_res1, 0x02024 + ATA_MV_EDMA_BASE(ch)); slot = (((rsp_in & ~0xffffff00) >> 3)) & 0x1f; rsp_out &= 0xffffff00; rsp_out += (slot << 3); bus_dmamap_sync(ch->dma.work_tag, ch->dma.work_map, BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); response = (struct ata_marvell_response *) (ch->dma.work + 1024 + (slot << 3)); /* record status for this request */ request->status = response->dev_status; request->error = 0; /* ack response */ ATA_OUTL(ctlr->r_res1, 0x02024 + ATA_MV_EDMA_BASE(ch), rsp_out); /* update progress */ if (!(request->status & ATA_S_ERROR) && !(request->flags & ATA_R_TIMEOUT)) request->donecount = request->bytecount; /* unload SG list */ ch->dma.unload(request); res = ATA_OP_FINISHED; } /* legacy ATA interrupt */ else { res = ata_end_transaction(request); } /* ack interrupt */ ATA_OUTL(ctlr->r_res1, offset, ~(icr & (0x0101 << (ch->unit & 3)))); return res; } static void ata_marvell_edma_reset(device_t dev) { struct ata_pci_controller *ctlr = device_get_softc(device_get_parent(dev)); struct ata_channel *ch = device_get_softc(dev); /* disable the EDMA machinery */ ATA_OUTL(ctlr->r_res1, 0x02028 + ATA_MV_EDMA_BASE(ch), 0x00000002); while ((ATA_INL(ctlr->r_res1, 0x02028 + ATA_MV_EDMA_BASE(ch)) & 0x00000001)) DELAY(10); /* clear SATA error register */ ATA_IDX_OUTL(ch, ATA_SERROR, ATA_IDX_INL(ch, ATA_SERROR)); /* clear any outstanding error interrupts */ ATA_OUTL(ctlr->r_res1, 0x02008 + ATA_MV_EDMA_BASE(ch), 0x0); /* unmask all error interrupts */ ATA_OUTL(ctlr->r_res1, 0x0200c + ATA_MV_EDMA_BASE(ch), ~0x0); /* enable channel and test for devices */ if (ata_sata_phy_reset(dev, -1, 1)) ata_generic_reset(dev); else ch->devices = 0; /* enable EDMA machinery */ ATA_OUTL(ctlr->r_res1, 0x02028 + ATA_MV_EDMA_BASE(ch), 0x00000001); } static void ata_marvell_edma_dmasetprd(void *xsc, bus_dma_segment_t *segs, int nsegs, int error) { struct ata_dmasetprd_args *args = xsc; struct ata_marvell_dma_prdentry *prd = args->dmatab; int i; if ((args->error = error)) return; for (i = 0; i < nsegs; i++) { prd[i].addrlo = htole32(segs[i].ds_addr); prd[i].count = htole32(segs[i].ds_len); prd[i].addrhi = htole32((u_int64_t)segs[i].ds_addr >> 32); prd[i].reserved = 0; } prd[i - 1].count |= htole32(ATA_DMA_EOT); KASSERT(nsegs <= ATA_DMA_ENTRIES, ("too many DMA segment entries\n")); args->nsegs = nsegs; } static void ata_marvell_edma_dmainit(device_t dev) { struct ata_pci_controller *ctlr = device_get_softc(device_get_parent(dev)); struct ata_channel *ch = device_get_softc(dev); /* note start and stop are not used here */ ch->dma.setprd = ata_marvell_edma_dmasetprd; /* if 64bit support present adjust max address used */ if (ATA_INL(ctlr->r_res1, 0x00d00) & 0x00000004) ch->dma.max_address = BUS_SPACE_MAXADDR; /* chip does not reliably do 64K DMA transfers */ if (ctlr->chip->cfg2 == MV_50XX || ctlr->chip->cfg2 == MV_60XX) ch->dma.max_iosize = 64 * DEV_BSIZE; ata_dmainit(dev); } ATA_DECLARE_DRIVER(ata_marvell); Index: stable/8/sys/dev/ata =================================================================== --- stable/8/sys/dev/ata (revision 243119) +++ stable/8/sys/dev/ata (revision 243120) Property changes on: stable/8/sys/dev/ata ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,1 ## Merged /head/sys/dev/ata:r232380 Index: stable/8/sys/dev =================================================================== --- stable/8/sys/dev (revision 243119) +++ stable/8/sys/dev (revision 243120) Property changes on: stable/8/sys/dev ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,1 ## Merged /head/sys/dev:r232380 Index: stable/8/sys =================================================================== --- stable/8/sys (revision 243119) +++ stable/8/sys (revision 243120) Property changes on: stable/8/sys ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,1 ## Merged /head/sys:r232380