Index: user/ngie/more-tests2/MAINTAINERS =================================================================== --- user/ngie/more-tests2/MAINTAINERS (revision 292053) +++ user/ngie/more-tests2/MAINTAINERS (revision 292054) @@ -1,156 +1,159 @@ $FreeBSD$ Please note that the content of this file is strictly advisory. No locks listed here are valid. The only strict review requirements are granted by core. These are documented in head/LOCKS and enforced by svnadmin/conf/approvers. The source tree is a community effort. However, some folks go to the trouble of looking after particular areas of the tree. In return for their active caretaking of the code it is polite to coordinate changes with them. This is a list of people who have expressed an interest in part of the code or listed their active caretaking role so that other committers can easily find somebody who is familiar with it. The notes should specify if there is a 3rd party source tree involved or other things that should be kept in mind. However, this is not a 'big stick', it is an offer to help and a source of guidance. It does not override the communal nature of the tree. It is not a registry of 'turf' or private property. *** This list is prone to becoming stale quickly. The best way to find the recent maintainer of a sub-system is to check recent logs for that directory or sub-system. *** subsystem login notes ----------------------------- opencrypto jmg Pre-commit review requested. Documentation Required. kqueue jmg Pre-commit review requested. Documentation Required. share/mk imp, bapt, bdrewery, emaste, sjg Make is hard. ath(4) adrian Pre-commit review requested, send to freebsd-wireless@freebsd.org net80211 adrian Pre-commit review requested, send to freebsd-wireless@freebsd.org iwn(4) adrian Pre-commit review requested, send to freebsd-wireless@freebsd.org iwm(4) adrian Pre-commit review requested, send to freebsd-wireless@freebsd.org otus(4) adrian Pre-commit review requested, send to freebsd-wireless@freebsd.org dev/usb/wlan adrian Pre-commit review requested, send to freebsd-wireless@freebsd.org openssl benl,jkim Pre-commit review requested. release/release.sh gjb,re Pre-commit review and regression tests requested. sh(1) jilles Pre-commit review requested. This also applies to kill(1), printf(1) and test(1) which are compiled in as builtins. isci(4) jimharris Pre-commit review requested. nvme(4) jimharris Pre-commit review requested. nvd(4) jimharris Pre-commit review requested. nvmecontrol(8) jimharris Pre-commit review requested. libfetch des Pre-commit review requested. fetch des Pre-commit review requested. libpam des Pre-commit review requested. openssh des Pre-commit review requested. pseudofs des Pre-commit review requested. procfs des Pre-commit review requested. linprocfs des Pre-commit review requested. contrib/compiler-rt dim Pre-commit review preferred. contrib/libc++ dim Pre-commit review preferred. contrib/libcxxrt dim Pre-commit review preferred. contrib/llvm dim Pre-commit review preferred. contrib/llvm/tools/lldb emaste Pre-commit review preferred. atf freebsd-testing,jmmv,ngie Pre-commit review requested. contrib/netbsd-tests freebsd-testing,ngie Pre-commit review requested. contrib/pjdfstest freebsd-testing,ngie,pjd Pre-commit review requested. share/mk/*.test.mk freebsd-testing,ngie (same list as share/mk too) Pre-commit review requested. tests freebsd-testing,ngie Pre-commit review requested. sys/dev/usb hselasky If in doubt, ask. sys/dev/sound/usb hselasky If in doubt, ask. sys/compat/linuxkpi hselasky If in doubt, ask. sys/dev/e1000 erj Pre-commit phabricator review requested. sys/dev/ixgbe erj Pre-commit phabricator review requested. sys/dev/ixl erj Pre-commit phabricator review requested. sys/netinet/ip_carp.c glebius Pre-commit review recommended. sys/netpfil/pf kp,glebius Pre-commit review recommended. +sctp rrs,tuexen Pre-commit review requested (changes need to be backported to github). +pmcstudy(8) rrs Pre-commit review requested. +callout_*(9) rrs Pre-commit review requested -- becareful its tricksy code :o. usr.sbin/pkg pkg@ Please coordinate behavior or flag changes with pkg team. lpr gad Pre-commit review requested, particularly for lpd/recvjob.c and lpd/printjob.c. nis(8), yp(8) araujo Pre-commit review requested. ---- OLD ---- libc/posix1e rwatson Pre-commit review requested. POSIX.1e ACLs rwatson Pre-commit review requested. UFS EAs rwatson Pre-commit review requested. MAC Framework rwatson Pre-commit review requested. MAC Modules rwatson Pre-commit review requested. contrib/openbsm rwatson Pre-commit review requested. sys/security/audit rwatson Pre-commit review requested. ahc(4) gibbs Pre-commit review requested. ahd(4) gibbs Pre-commit review requested. pci bus imp,jhb Pre-commit review requested. cdboot jhb Pre-commit review requested. pxeboot jhb Pre-commit review requested. witness jhb Pre-commit review requested. CAM gibbs, ken Pre-commit review requested. send to scsi@freebsd.org devstat(9) ken Pre-commit review requested. camcontrol(8) ken Pre-commit review requested. libcam ken Pre-commit review requested. libdevstat ken Pre-commit review requested. iostat(8) ken Pre-commit review requested. cd(4) ken Pre-commit review requested. pass(4) ken Pre-commit review requested. ch(4) ken Pre-commit review requested. em(4) jfv Pre-commit review requested. sendmail gshapiro Pre-commit review requested. etc/mail gshapiro Pre-commit review requested. Keep in sync with -STABLE. etc/sendmail gshapiro Pre-commit review requested. Keep in sync with -STABLE. nvi peter Try not to break it. libz peter Try not to break it. groff ru Recommends pre-commit review. ipfw ipfw Pre-commit review preferred. send to ipfw@freebsd.org drm rnoland Just keep me informed of changes, try not to break it. unifdef(1) fanf Pre-commit review requested. ntp roberto Pre-commit review requested. inetd dwmalone Recommends pre-commit review. contrib/smbfs bp Open for in-tree committs. In case of functional changes pre-commit review requested. file obrien Insists to keep file blocked from other's unapproved commits contrib/bzip2 obrien Pre-commit review required. geom freebsd-geom@FreeBSD.org geom_concat pjd Pre-commit review preferred. geom_eli pjd Pre-commit review preferred. geom_gate pjd Pre-commit review preferred. geom_label pjd Pre-commit review preferred. geom_mirror pjd Pre-commit review preferred. geom_nop pjd Pre-commit review preferred. geom_raid3 pjd Pre-commit review preferred. geom_shsec pjd Pre-commit review preferred. geom_stripe pjd Pre-commit review preferred. geom_zero pjd Pre-commit review preferred. sbin/geom pjd Pre-commit review preferred. zfs freebsd-fs@FreeBSD.org nfs freebsd-fs@FreeBSD.org, rmacklem is best for reviews. linux emul emulation Please discuss changes here. bs{diff,patch} cperciva Pre-commit review requested. portsnap cperciva Pre-commit review requested. freebsd-update cperciva Pre-commit review requested. sys/netgraph/bluetooth emax Pre-commit review preferred. lib/libbluetooth emax Pre-commit review preferred. lib/libsdp emax Pre-commit review preferred. usr.bin/bluetooth emax Pre-commit review preferred. usr.sbin/bluetooth emax Pre-commit review preferred. *env(3) secteam Due to the problematic security history of this code, please have patches reviewed by secteam. share/zoneinfo edwin Heads-up appreciated, since our data is coming from a third party source. usr.sbin/zic edwin Heads-up appreciated, since this code is maintained by a third party source. lib/libc/stdtime edwin Heads-up appreciated, since parts of this code is maintained by a third party source. sbin/routed bms Pre-commit review; notify vendor at rhyolite.com cmx daniel@roe.ch Pre-commit review preferred. filemon obrien Pre-commit review preferred. sysdoc trhodes Pre-commit review preferred. nanobsd imp Pre-commit review requested for coordination. vmm(4) neel,grehan Pre-commit review requested. Property changes on: user/ngie/more-tests2/MAINTAINERS ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,1 ## Merged /head/MAINTAINERS:r291039-292053 Index: user/ngie/more-tests2/contrib/ofed/usr.bin/ibaddr/Makefile =================================================================== --- user/ngie/more-tests2/contrib/ofed/usr.bin/ibaddr/Makefile (revision 292053) +++ user/ngie/more-tests2/contrib/ofed/usr.bin/ibaddr/Makefile (revision 292054) @@ -1,14 +1,14 @@ # $FreeBSD$ .include "../Makefile.inc" .PATH: ${DIAGPATH}/src ${DIAGPATH}/man PROG= ibaddr SRCS= ibaddr.c ibdiag_common.c -LDADD= -libumad -libcommon -libmad +LIBADD= ibumad ibcommon ibmad CFLAGS+= -I${DIAGPATH}/include MAN= ibaddr.8 WARNS?= 1 .include Index: user/ngie/more-tests2/contrib/ofed/usr.bin/ibnetdiscover/Makefile =================================================================== --- user/ngie/more-tests2/contrib/ofed/usr.bin/ibnetdiscover/Makefile (revision 292053) +++ user/ngie/more-tests2/contrib/ofed/usr.bin/ibnetdiscover/Makefile (revision 292054) @@ -1,14 +1,14 @@ # $FreeBSD$ .include "../Makefile.inc" .PATH: ${DIAGPATH}/src ${DIAGPATH}/man PROG= ibnetdiscover SRCS= ibnetdiscover.c grouping.c ibdiag_common.c -LDADD= -libumad -libcommon -libmad -losmcomp -CFLAGS+= -pthread -I${DIAGPATH}/include +LIBADD= ibumad ibcommon ibmad osmcomp pthread +CFLAGS+= -I${DIAGPATH}/include MAN= ibnetdiscover.8 WARNS?= 1 .include Index: user/ngie/more-tests2/contrib/ofed/usr.bin/ibping/Makefile =================================================================== --- user/ngie/more-tests2/contrib/ofed/usr.bin/ibping/Makefile (revision 292053) +++ user/ngie/more-tests2/contrib/ofed/usr.bin/ibping/Makefile (revision 292054) @@ -1,14 +1,14 @@ # $FreeBSD$ .include "../Makefile.inc" .PATH: ${DIAGPATH}/src ${DIAGPATH}/man PROG= ibping SRCS= ibping.c ibdiag_common.c -LDADD= -libumad -libcommon -libmad +LIBADD= ibumad ibcommon ibmad CFLAGS+= -I${DIAGPATH}/include MAN= ibping.8 WARNS?= 1 .include Index: user/ngie/more-tests2/contrib/ofed/usr.bin/ibportstate/Makefile =================================================================== --- user/ngie/more-tests2/contrib/ofed/usr.bin/ibportstate/Makefile (revision 292053) +++ user/ngie/more-tests2/contrib/ofed/usr.bin/ibportstate/Makefile (revision 292054) @@ -1,14 +1,14 @@ # $FreeBSD$ .include "../Makefile.inc" .PATH: ${DIAGPATH}/src ${DIAGPATH}/man PROG= ibportstate SRCS= ibportstate.c ibdiag_common.c -LDADD= -libumad -libcommon -libmad +LIBADD= ibumad ibcommon ibmad CFLAGS+= -I${DIAGPATH}/include MAN= ibportstate.8 WARNS?= 1 .include Index: user/ngie/more-tests2/contrib/ofed/usr.bin/ibroute/Makefile =================================================================== --- user/ngie/more-tests2/contrib/ofed/usr.bin/ibroute/Makefile (revision 292053) +++ user/ngie/more-tests2/contrib/ofed/usr.bin/ibroute/Makefile (revision 292054) @@ -1,14 +1,14 @@ # $FreeBSD$ .include "../Makefile.inc" .PATH: ${DIAGPATH}/src ${DIAGPATH}/man PROG= ibroute SRCS= ibroute.c ibdiag_common.c -LDADD= -libumad -libcommon -libmad -losmcomp -CFLAGS+= -pthread -I${DIAGPATH}/include +LIBADD= ibumad ibcommon ibmad osmcomp pthread +CFLAGS+= -I${DIAGPATH}/include MAN= ibroute.8 WARNS?= 1 .include Index: user/ngie/more-tests2/contrib/ofed/usr.bin/ibsendtrap/Makefile =================================================================== --- user/ngie/more-tests2/contrib/ofed/usr.bin/ibsendtrap/Makefile (revision 292053) +++ user/ngie/more-tests2/contrib/ofed/usr.bin/ibsendtrap/Makefile (revision 292054) @@ -1,14 +1,14 @@ # $FreeBSD$ .include "../Makefile.inc" .PATH: ${DIAGPATH}/src ${DIAGPATH}/man PROG= ibsendtrap SRCS= ibsendtrap.c ibdiag_common.c -LDADD= -libumad -libcommon -libmad +LIBADD= ibumad ibcommon ibmad CFLAGS+= -I${DIAGPATH}/include MAN= WARNS?= 1 .include Index: user/ngie/more-tests2/contrib/ofed/usr.bin/ibstat/Makefile =================================================================== --- user/ngie/more-tests2/contrib/ofed/usr.bin/ibstat/Makefile (revision 292053) +++ user/ngie/more-tests2/contrib/ofed/usr.bin/ibstat/Makefile (revision 292054) @@ -1,14 +1,14 @@ # $FreeBSD$ .include "../Makefile.inc" .PATH: ${DIAGPATH}/src ${DIAGPATH}/man PROG= ibstat SRCS= ibstat.c -LDADD= -libumad -libcommon +LIBADD= ibumad ibcommon CFLAGS+= -I${DIAGPATH}/include MAN= ibstat.8 WARNS?= 1 .include Index: user/ngie/more-tests2/contrib/ofed/usr.bin/ibsysstat/Makefile =================================================================== --- user/ngie/more-tests2/contrib/ofed/usr.bin/ibsysstat/Makefile (revision 292053) +++ user/ngie/more-tests2/contrib/ofed/usr.bin/ibsysstat/Makefile (revision 292054) @@ -1,14 +1,14 @@ # $FreeBSD$ .include "../Makefile.inc" .PATH: ${DIAGPATH}/src ${DIAGPATH}/man PROG= ibsysstat SRCS= ibsysstat.c ibdiag_common.c -LDADD= -libumad -libcommon -libmad +LIBADD= ibumad ibcommon ibmad CFLAGS+= -I${DIAGPATH}/include MAN= ibsysstat.8 WARNS?= 1 .include Index: user/ngie/more-tests2/contrib/ofed/usr.bin/ibtracert/Makefile =================================================================== --- user/ngie/more-tests2/contrib/ofed/usr.bin/ibtracert/Makefile (revision 292053) +++ user/ngie/more-tests2/contrib/ofed/usr.bin/ibtracert/Makefile (revision 292054) @@ -1,14 +1,14 @@ # $FreeBSD$ .include "../Makefile.inc" .PATH: ${DIAGPATH}/src ${DIAGPATH}/man PROG= ibtracert SRCS= ibtracert.c ibdiag_common.c -LDADD= -libumad -libcommon -libmad -losmcomp -CFLAGS+= -pthread -I${DIAGPATH}/include +LIBADD= ibumad ibcommon ibmad osmcomp pthread +CFLAGS+= -I${DIAGPATH}/include MAN= ibtracert.8 WARNS?= 1 .include Index: user/ngie/more-tests2/contrib/ofed/usr.bin/opensm/Makefile =================================================================== --- user/ngie/more-tests2/contrib/ofed/usr.bin/opensm/Makefile (revision 292053) +++ user/ngie/more-tests2/contrib/ofed/usr.bin/opensm/Makefile (revision 292054) @@ -1,40 +1,39 @@ # $FreeBSD$ .include "../Makefile.inc" OPENSM = ${.CURDIR}/../../management/opensm .PATH: ${OPENSM}/opensm ${OPENSM}/man PROG= opensm SRCS= main.c osm_console_io.c osm_console.c osm_db_files.c SRCS+= osm_db_pack.c osm_drop_mgr.c osm_inform.c osm_lid_mgr.c SRCS+= osm_lin_fwd_rcv.c osm_link_mgr.c osm_mcast_fwd_rcv.c osm_mcast_mgr.c SRCS+= osm_mcast_tbl.c osm_mcm_info.c osm_mcm_port.c osm_mtree.c SRCS+= osm_multicast.c osm_node.c osm_node_desc_rcv.c osm_node_info_rcv.c SRCS+= osm_opensm.c osm_pkey.c osm_pkey_mgr.c osm_pkey_rcv.c osm_port.c SRCS+= osm_port_info_rcv.c osm_remote_sm.c osm_req.c osm_resp.c osm_sa.c SRCS+= osm_sa_class_port_info.c osm_sa_informinfo.c osm_sa_lft_record.c SRCS+= osm_sa_mft_record.c osm_sa_link_record.c osm_sa_mad_ctrl.c SRCS+= osm_sa_mcmember_record.c osm_sa_node_record.c osm_sa_path_record.c SRCS+= osm_sa_pkey_record.c osm_sa_portinfo_record.c osm_sa_guidinfo_record.c SRCS+= osm_sa_multipath_record.c osm_sa_service_record.c osm_sa_slvl_record.c SRCS+= osm_sa_sminfo_record.c osm_sa_vlarb_record.c osm_sa_sw_info_record.c SRCS+= osm_service.c osm_slvl_map_rcv.c osm_sm.c osm_sminfo_rcv.c SRCS+= osm_sm_mad_ctrl.c osm_sm_state_mgr.c osm_state_mgr.c osm_subnet.c SRCS+= osm_sw_info_rcv.c osm_switch.c osm_prtn.c osm_prtn_config.c osm_qos.c SRCS+= osm_router.c osm_trap_rcv.c osm_ucast_mgr.c osm_ucast_updn.c SRCS+= osm_ucast_lash.c osm_ucast_file.c osm_ucast_ftree.c osm_vl15intf.c SRCS+= osm_vl_arb_rcv.c st.c osm_perfmgr.c osm_perfmgr_db.c osm_event_plugin.c SRCS+= osm_dump.c osm_ucast_cache.c osm_qos_parser_y.y osm_qos_parser_l.l SRCS+= osm_qos_policy.c -LDADD= -lopensm -losmvendor -losmcomp -libmad -libumad -libcommon -CFLAGS+= -pthread +LIBADD= opensm osmvendor osmcomp ibmad ibumad ibcommon pthread CFLAGS+= -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP MAN= opensm.8 WARNS?= 1 .include Index: user/ngie/more-tests2/contrib/ofed/usr.bin/osmtest/Makefile =================================================================== --- user/ngie/more-tests2/contrib/ofed/usr.bin/osmtest/Makefile (revision 292053) +++ user/ngie/more-tests2/contrib/ofed/usr.bin/osmtest/Makefile (revision 292054) @@ -1,35 +1,28 @@ # $FreeBSD$ .include "../Makefile.inc" OPENSM= ${.CURDIR}/../../management/opensm .PATH: ${OPENSM}/osmtest ${OPENSM}/man BINDIR= ${TESTSBASE}/usr.bin/opensm PROG= osmtest SRCS= main.c \ osmt_inform.c \ osmt_mtl_regular_qp.c \ osmt_multicast.c \ osmt_service.c \ osmt_slvl_vl_arb.c \ osmtest.c -LDADD= -libcommon \ - -libmad \ - -libumad \ - -losmvendor \ - -losmcomp \ - -lopensm \ - -LIBADD+= pthread +LIBADD= ibcommon ibmad ibumad osmvendor osmcomp opensm pthread CFLAGS= -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP \ -I${OPENSM}/osmtest/include MAN= osmtest.8 WARNS?= 1 .include Index: user/ngie/more-tests2/contrib/ofed/usr.bin/perfquery/Makefile =================================================================== --- user/ngie/more-tests2/contrib/ofed/usr.bin/perfquery/Makefile (revision 292053) +++ user/ngie/more-tests2/contrib/ofed/usr.bin/perfquery/Makefile (revision 292054) @@ -1,14 +1,14 @@ # $FreeBSD$ .include "../Makefile.inc" .PATH: ${DIAGPATH}/src ${DIAGPATH}/man PROG= perfquery SRCS= perfquery.c ibdiag_common.c -LDADD= -libumad -libcommon -libmad +LIBADD= ibumad ibcommon ibmad CFLAGS+= -I${DIAGPATH}/include MAN= perfquery.8 WARNS?= 1 .include Index: user/ngie/more-tests2/contrib/ofed/usr.bin/saquery/Makefile =================================================================== --- user/ngie/more-tests2/contrib/ofed/usr.bin/saquery/Makefile (revision 292053) +++ user/ngie/more-tests2/contrib/ofed/usr.bin/saquery/Makefile (revision 292054) @@ -1,16 +1,15 @@ # $FreeBSD$ .include "../Makefile.inc" .PATH: ${DIAGPATH}/src ${DIAGPATH}/man PROG= saquery SRCS= saquery.c ibdiag_common.c -LDADD= -libumad -libcommon -libmad -losmcomp -losmvendor -lopensm +LIBADD= ibumad ibcommon ibmad osmcomp osmvendor opensm pthread CFLAGS+= -I${DIAGPATH}/include CFLAGS+= -DOSM_VENDOR_INTF_OPENIB -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP -CFLAGS+= -pthread MAN= saquery.8 WARNS?= 1 .include Index: user/ngie/more-tests2/contrib/ofed/usr.bin/sminfo/Makefile =================================================================== --- user/ngie/more-tests2/contrib/ofed/usr.bin/sminfo/Makefile (revision 292053) +++ user/ngie/more-tests2/contrib/ofed/usr.bin/sminfo/Makefile (revision 292054) @@ -1,14 +1,14 @@ # $FreeBSD$ .include "../Makefile.inc" .PATH: ${DIAGPATH}/src ${DIAGPATH}/man PROG= sminfo SRCS= sminfo.c ibdiag_common.c -LDADD= -libumad -libcommon -libmad +LIBADD= ibumad ibcommon ibmad CFLAGS+= -I${DIAGPATH}/include MAN= sminfo.8 WARNS?= 1 .include Index: user/ngie/more-tests2/contrib/ofed/usr.bin/smpdump/Makefile =================================================================== --- user/ngie/more-tests2/contrib/ofed/usr.bin/smpdump/Makefile (revision 292053) +++ user/ngie/more-tests2/contrib/ofed/usr.bin/smpdump/Makefile (revision 292054) @@ -1,14 +1,14 @@ # $FreeBSD$ .include "../Makefile.inc" .PATH: ${DIAGPATH}/src ${DIAGPATH}/man PROG= smpdump SRCS= smpdump.c -LDADD= -libumad -libcommon -libmad +LIBADD= ibumad ibcommon ibmad CFLAGS+= -I${DIAGPATH}/include MAN= smpdump.8 WARNS?= 1 .include Index: user/ngie/more-tests2/contrib/ofed/usr.bin/smpquery/Makefile =================================================================== --- user/ngie/more-tests2/contrib/ofed/usr.bin/smpquery/Makefile (revision 292053) +++ user/ngie/more-tests2/contrib/ofed/usr.bin/smpquery/Makefile (revision 292054) @@ -1,14 +1,14 @@ # $FreeBSD$ .include "../Makefile.inc" .PATH: ${DIAGPATH}/src ${DIAGPATH}/man PROG= smpquery SRCS= smpquery.c ibdiag_common.c -LDADD= -libumad -libcommon -libmad -losmcomp -CFLAGS+= -pthread -I${DIAGPATH}/include +LIBADD= ibumad ibcommon ibmad osmcomp pthread +CFLAGS+= -I${DIAGPATH}/include MAN= smpquery.8 WARNS?= 1 .include Index: user/ngie/more-tests2/contrib/ofed/usr.bin/vendstat/Makefile =================================================================== --- user/ngie/more-tests2/contrib/ofed/usr.bin/vendstat/Makefile (revision 292053) +++ user/ngie/more-tests2/contrib/ofed/usr.bin/vendstat/Makefile (revision 292054) @@ -1,14 +1,14 @@ # $FreeBSD$ .include "../Makefile.inc" .PATH: ${DIAGPATH}/src ${DIAGPATH}/man PROG= vendstat SRCS= vendstat.c ibdiag_common.c -LDADD= -libumad -libcommon -libmad +LIBADD= ibumad ibcommon ibmad CFLAGS+= -I${DIAGPATH}/include MAN= vendstat.8 WARNS?= 1 .include Index: user/ngie/more-tests2/lib/libc/rpc/svc_vc.c =================================================================== --- user/ngie/more-tests2/lib/libc/rpc/svc_vc.c (revision 292053) +++ user/ngie/more-tests2/lib/libc/rpc/svc_vc.c (revision 292054) @@ -1,769 +1,783 @@ /* $NetBSD: svc_vc.c,v 1.7 2000/08/03 00:01:53 fvdl Exp $ */ /*- * Copyright (c) 2009, Sun Microsystems, Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are met: * - Redistributions of source code must retain the above copyright notice, * this list of conditions and the following disclaimer. * - Redistributions in binary form must reproduce the above copyright notice, * this list of conditions and the following disclaimer in the documentation * and/or other materials provided with the distribution. * - Neither the name of Sun Microsystems, Inc. nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE * POSSIBILITY OF SUCH DAMAGE. */ #if defined(LIBC_SCCS) && !defined(lint) static char *sccsid2 = "@(#)svc_tcp.c 1.21 87/08/11 Copyr 1984 Sun Micro"; static char *sccsid = "@(#)svc_tcp.c 2.2 88/08/01 4.0 RPCSRC"; #endif #include __FBSDID("$FreeBSD$"); /* * svc_vc.c, Server side for Connection Oriented based RPC. * * Actually implements two flavors of transporter - * a tcp rendezvouser (a listner and connection establisher) * and a record/tcp stream. */ #include "namespace.h" #include "reentrant.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "rpc_com.h" #include "mt_misc.h" #include "un-namespace.h" static SVCXPRT *makefd_xprt(int, u_int, u_int); static bool_t rendezvous_request(SVCXPRT *, struct rpc_msg *); static enum xprt_stat rendezvous_stat(SVCXPRT *); static void svc_vc_destroy(SVCXPRT *); static void __svc_vc_dodestroy (SVCXPRT *); static int read_vc(void *, void *, int); static int write_vc(void *, void *, int); static enum xprt_stat svc_vc_stat(SVCXPRT *); static bool_t svc_vc_recv(SVCXPRT *, struct rpc_msg *); static bool_t svc_vc_getargs(SVCXPRT *, xdrproc_t, void *); static bool_t svc_vc_freeargs(SVCXPRT *, xdrproc_t, void *); static bool_t svc_vc_reply(SVCXPRT *, struct rpc_msg *); static void svc_vc_rendezvous_ops(SVCXPRT *); static void svc_vc_ops(SVCXPRT *); static bool_t svc_vc_control(SVCXPRT *xprt, const u_int rq, void *in); static bool_t svc_vc_rendezvous_control (SVCXPRT *xprt, const u_int rq, void *in); struct cf_rendezvous { /* kept in xprt->xp_p1 for rendezvouser */ u_int sendsize; u_int recvsize; int maxrec; }; struct cf_conn { /* kept in xprt->xp_p1 for actual connection */ enum xprt_stat strm_stat; u_int32_t x_id; XDR xdrs; char verf_body[MAX_AUTH_BYTES]; u_int sendsize; u_int recvsize; int maxrec; bool_t nonblock; struct timeval last_recv_time; }; /* * Usage: * xprt = svc_vc_create(sock, send_buf_size, recv_buf_size); * * Creates, registers, and returns a (rpc) tcp based transporter. * Once *xprt is initialized, it is registered as a transporter * see (svc.h, xprt_register). This routine returns * a NULL if a problem occurred. * * The filedescriptor passed in is expected to refer to a bound, but * not yet connected socket. * * Since streams do buffered io similar to stdio, the caller can specify * how big the send and receive buffers are via the second and third parms; * 0 => use the system default. */ SVCXPRT * svc_vc_create(int fd, u_int sendsize, u_int recvsize) { SVCXPRT *xprt = NULL; struct cf_rendezvous *r = NULL; struct __rpc_sockinfo si; struct sockaddr_storage sslocal; socklen_t slen; if (!__rpc_fd2sockinfo(fd, &si)) return NULL; r = mem_alloc(sizeof(*r)); if (r == NULL) { warnx("svc_vc_create: out of memory"); goto cleanup_svc_vc_create; } r->sendsize = __rpc_get_t_size(si.si_af, si.si_proto, (int)sendsize); r->recvsize = __rpc_get_t_size(si.si_af, si.si_proto, (int)recvsize); r->maxrec = __svc_maxrec; xprt = svc_xprt_alloc(); if (xprt == NULL) { warnx("svc_vc_create: out of memory"); goto cleanup_svc_vc_create; } xprt->xp_p1 = r; xprt->xp_verf = _null_auth; svc_vc_rendezvous_ops(xprt); xprt->xp_port = (u_short)-1; /* It is the rendezvouser */ xprt->xp_fd = fd; slen = sizeof (struct sockaddr_storage); if (_getsockname(fd, (struct sockaddr *)(void *)&sslocal, &slen) < 0) { warnx("svc_vc_create: could not retrieve local addr"); goto cleanup_svc_vc_create; } xprt->xp_ltaddr.maxlen = xprt->xp_ltaddr.len = sslocal.ss_len; xprt->xp_ltaddr.buf = mem_alloc((size_t)sslocal.ss_len); if (xprt->xp_ltaddr.buf == NULL) { warnx("svc_vc_create: no mem for local addr"); goto cleanup_svc_vc_create; } memcpy(xprt->xp_ltaddr.buf, &sslocal, (size_t)sslocal.ss_len); xprt->xp_rtaddr.maxlen = sizeof (struct sockaddr_storage); xprt_register(xprt); return (xprt); cleanup_svc_vc_create: if (xprt) mem_free(xprt, sizeof(*xprt)); if (r != NULL) mem_free(r, sizeof(*r)); return (NULL); } /* * Like svtcp_create(), except the routine takes any *open* UNIX file * descriptor as its first input. */ SVCXPRT * svc_fd_create(int fd, u_int sendsize, u_int recvsize) { struct sockaddr_storage ss; socklen_t slen; SVCXPRT *ret; assert(fd != -1); ret = makefd_xprt(fd, sendsize, recvsize); if (ret == NULL) return NULL; slen = sizeof (struct sockaddr_storage); if (_getsockname(fd, (struct sockaddr *)(void *)&ss, &slen) < 0) { warnx("svc_fd_create: could not retrieve local addr"); goto freedata; } ret->xp_ltaddr.maxlen = ret->xp_ltaddr.len = ss.ss_len; ret->xp_ltaddr.buf = mem_alloc((size_t)ss.ss_len); if (ret->xp_ltaddr.buf == NULL) { warnx("svc_fd_create: no mem for local addr"); goto freedata; } memcpy(ret->xp_ltaddr.buf, &ss, (size_t)ss.ss_len); slen = sizeof (struct sockaddr_storage); if (_getpeername(fd, (struct sockaddr *)(void *)&ss, &slen) < 0) { warnx("svc_fd_create: could not retrieve remote addr"); goto freedata; } ret->xp_rtaddr.maxlen = ret->xp_rtaddr.len = ss.ss_len; ret->xp_rtaddr.buf = mem_alloc((size_t)ss.ss_len); if (ret->xp_rtaddr.buf == NULL) { warnx("svc_fd_create: no mem for local addr"); goto freedata; } memcpy(ret->xp_rtaddr.buf, &ss, (size_t)ss.ss_len); #ifdef PORTMAP if (ss.ss_family == AF_INET || ss.ss_family == AF_LOCAL) { ret->xp_raddr = *(struct sockaddr_in *)ret->xp_rtaddr.buf; ret->xp_addrlen = sizeof (struct sockaddr_in); } #endif /* PORTMAP */ return ret; freedata: if (ret->xp_ltaddr.buf != NULL) mem_free(ret->xp_ltaddr.buf, rep->xp_ltaddr.maxlen); return NULL; } static SVCXPRT * makefd_xprt(int fd, u_int sendsize, u_int recvsize) { SVCXPRT *xprt; struct cf_conn *cd; const char *netid; struct __rpc_sockinfo si; assert(fd != -1); xprt = svc_xprt_alloc(); if (xprt == NULL) { warnx("svc_vc: makefd_xprt: out of memory"); goto done; } cd = mem_alloc(sizeof(struct cf_conn)); if (cd == NULL) { warnx("svc_tcp: makefd_xprt: out of memory"); svc_xprt_free(xprt); xprt = NULL; goto done; } cd->strm_stat = XPRT_IDLE; xdrrec_create(&(cd->xdrs), sendsize, recvsize, xprt, read_vc, write_vc); xprt->xp_p1 = cd; xprt->xp_verf.oa_base = cd->verf_body; svc_vc_ops(xprt); /* truely deals with calls */ xprt->xp_port = 0; /* this is a connection, not a rendezvouser */ xprt->xp_fd = fd; if (__rpc_fd2sockinfo(fd, &si) && __rpc_sockinfo2netid(&si, &netid)) xprt->xp_netid = strdup(netid); xprt_register(xprt); done: return (xprt); } /*ARGSUSED*/ static bool_t rendezvous_request(SVCXPRT *xprt, struct rpc_msg *msg) { int sock, flags; struct cf_rendezvous *r; struct cf_conn *cd; - struct sockaddr_storage addr; - socklen_t len; + struct sockaddr_storage addr, sslocal; + socklen_t len, slen; struct __rpc_sockinfo si; SVCXPRT *newxprt; fd_set cleanfds; assert(xprt != NULL); assert(msg != NULL); r = (struct cf_rendezvous *)xprt->xp_p1; again: len = sizeof addr; if ((sock = _accept(xprt->xp_fd, (struct sockaddr *)(void *)&addr, &len)) < 0) { if (errno == EINTR) goto again; /* * Clean out the most idle file descriptor when we're * running out. */ if (errno == EMFILE || errno == ENFILE) { cleanfds = svc_fdset; __svc_clean_idle(&cleanfds, 0, FALSE); goto again; } return (FALSE); } /* * make a new transporter (re-uses xprt) */ newxprt = makefd_xprt(sock, r->sendsize, r->recvsize); newxprt->xp_rtaddr.buf = mem_alloc(len); if (newxprt->xp_rtaddr.buf == NULL) return (FALSE); memcpy(newxprt->xp_rtaddr.buf, &addr, len); newxprt->xp_rtaddr.len = len; #ifdef PORTMAP if (addr.ss_family == AF_INET || addr.ss_family == AF_LOCAL) { newxprt->xp_raddr = *(struct sockaddr_in *)newxprt->xp_rtaddr.buf; newxprt->xp_addrlen = sizeof (struct sockaddr_in); } #endif /* PORTMAP */ if (__rpc_fd2sockinfo(sock, &si) && si.si_proto == IPPROTO_TCP) { len = 1; /* XXX fvdl - is this useful? */ _setsockopt(sock, IPPROTO_TCP, TCP_NODELAY, &len, sizeof (len)); } cd = (struct cf_conn *)newxprt->xp_p1; cd->recvsize = r->recvsize; cd->sendsize = r->sendsize; cd->maxrec = r->maxrec; if (cd->maxrec != 0) { flags = _fcntl(sock, F_GETFL, 0); if (flags == -1) return (FALSE); if (_fcntl(sock, F_SETFL, flags | O_NONBLOCK) == -1) return (FALSE); if (cd->recvsize > cd->maxrec) cd->recvsize = cd->maxrec; cd->nonblock = TRUE; __xdrrec_setnonblock(&cd->xdrs, cd->maxrec); } else cd->nonblock = FALSE; + slen = sizeof(struct sockaddr_storage); + if(_getsockname(sock, (struct sockaddr *)(void *)&sslocal, &slen) < 0) { + warnx("svc_vc_create: could not retrieve local addr"); + newxprt->xp_ltaddr.maxlen = newxprt->xp_ltaddr.len = 0; + } else { + newxprt->xp_ltaddr.maxlen = newxprt->xp_ltaddr.len = sslocal.ss_len; + newxprt->xp_ltaddr.buf = mem_alloc((size_t)sslocal.ss_len); + if (newxprt->xp_ltaddr.buf == NULL) { + warnx("svc_vc_create: no mem for local addr"); + newxprt->xp_ltaddr.maxlen = newxprt->xp_ltaddr.len = 0; + } else { + memcpy(newxprt->xp_ltaddr.buf, &sslocal, (size_t)sslocal.ss_len); + } + } gettimeofday(&cd->last_recv_time, NULL); return (FALSE); /* there is never an rpc msg to be processed */ } /*ARGSUSED*/ static enum xprt_stat rendezvous_stat(SVCXPRT *xprt) { return (XPRT_IDLE); } static void svc_vc_destroy(SVCXPRT *xprt) { assert(xprt != NULL); xprt_unregister(xprt); __svc_vc_dodestroy(xprt); } static void __svc_vc_dodestroy(SVCXPRT *xprt) { struct cf_conn *cd; struct cf_rendezvous *r; cd = (struct cf_conn *)xprt->xp_p1; if (xprt->xp_fd != RPC_ANYFD) (void)_close(xprt->xp_fd); if (xprt->xp_port != 0) { /* a rendezvouser socket */ r = (struct cf_rendezvous *)xprt->xp_p1; mem_free(r, sizeof (struct cf_rendezvous)); xprt->xp_port = 0; } else { /* an actual connection socket */ XDR_DESTROY(&(cd->xdrs)); mem_free(cd, sizeof(struct cf_conn)); } if (xprt->xp_rtaddr.buf) mem_free(xprt->xp_rtaddr.buf, xprt->xp_rtaddr.maxlen); if (xprt->xp_ltaddr.buf) mem_free(xprt->xp_ltaddr.buf, xprt->xp_ltaddr.maxlen); free(xprt->xp_tp); free(xprt->xp_netid); svc_xprt_free(xprt); } /*ARGSUSED*/ static bool_t svc_vc_control(SVCXPRT *xprt, const u_int rq, void *in) { return (FALSE); } static bool_t svc_vc_rendezvous_control(SVCXPRT *xprt, const u_int rq, void *in) { struct cf_rendezvous *cfp; cfp = (struct cf_rendezvous *)xprt->xp_p1; if (cfp == NULL) return (FALSE); switch (rq) { case SVCGET_CONNMAXREC: *(int *)in = cfp->maxrec; break; case SVCSET_CONNMAXREC: cfp->maxrec = *(int *)in; break; default: return (FALSE); } return (TRUE); } /* * reads data from the tcp or uip connection. * any error is fatal and the connection is closed. * (And a read of zero bytes is a half closed stream => error.) * All read operations timeout after 35 seconds. A timeout is * fatal for the connection. */ static int read_vc(void *xprtp, void *buf, int len) { SVCXPRT *xprt; int sock; int milliseconds = 35 * 1000; struct pollfd pollfd; struct cf_conn *cfp; xprt = (SVCXPRT *)xprtp; assert(xprt != NULL); sock = xprt->xp_fd; cfp = (struct cf_conn *)xprt->xp_p1; if (cfp->nonblock) { len = _read(sock, buf, (size_t)len); if (len < 0) { if (errno == EAGAIN) len = 0; else goto fatal_err; } if (len != 0) gettimeofday(&cfp->last_recv_time, NULL); return len; } do { pollfd.fd = sock; pollfd.events = POLLIN; pollfd.revents = 0; switch (_poll(&pollfd, 1, milliseconds)) { case -1: if (errno == EINTR) continue; /*FALLTHROUGH*/ case 0: goto fatal_err; default: break; } } while ((pollfd.revents & POLLIN) == 0); if ((len = _read(sock, buf, (size_t)len)) > 0) { gettimeofday(&cfp->last_recv_time, NULL); return (len); } fatal_err: ((struct cf_conn *)(xprt->xp_p1))->strm_stat = XPRT_DIED; return (-1); } /* * writes data to the tcp connection. * Any error is fatal and the connection is closed. */ static int write_vc(void *xprtp, void *buf, int len) { SVCXPRT *xprt; int i, cnt; struct cf_conn *cd; struct timeval tv0, tv1; xprt = (SVCXPRT *)xprtp; assert(xprt != NULL); cd = (struct cf_conn *)xprt->xp_p1; if (cd->nonblock) gettimeofday(&tv0, NULL); for (cnt = len; cnt > 0; cnt -= i, buf = (char *)buf + i) { i = _write(xprt->xp_fd, buf, (size_t)cnt); if (i < 0) { if (errno != EAGAIN || !cd->nonblock) { cd->strm_stat = XPRT_DIED; return (-1); } if (cd->nonblock) { /* * For non-blocking connections, do not * take more than 2 seconds writing the * data out. * * XXX 2 is an arbitrary amount. */ gettimeofday(&tv1, NULL); if (tv1.tv_sec - tv0.tv_sec >= 2) { cd->strm_stat = XPRT_DIED; return (-1); } } i = 0; } } return (len); } static enum xprt_stat svc_vc_stat(SVCXPRT *xprt) { struct cf_conn *cd; assert(xprt != NULL); cd = (struct cf_conn *)(xprt->xp_p1); if (cd->strm_stat == XPRT_DIED) return (XPRT_DIED); if (! xdrrec_eof(&(cd->xdrs))) return (XPRT_MOREREQS); return (XPRT_IDLE); } static bool_t svc_vc_recv(SVCXPRT *xprt, struct rpc_msg *msg) { struct cf_conn *cd; XDR *xdrs; assert(xprt != NULL); assert(msg != NULL); cd = (struct cf_conn *)(xprt->xp_p1); xdrs = &(cd->xdrs); if (cd->nonblock) { if (!__xdrrec_getrec(xdrs, &cd->strm_stat, TRUE)) return FALSE; } else { (void)xdrrec_skiprecord(xdrs); } xdrs->x_op = XDR_DECODE; if (xdr_callmsg(xdrs, msg)) { cd->x_id = msg->rm_xid; return (TRUE); } cd->strm_stat = XPRT_DIED; return (FALSE); } static bool_t svc_vc_getargs(SVCXPRT *xprt, xdrproc_t xdr_args, void *args_ptr) { struct cf_conn *cd; assert(xprt != NULL); cd = (struct cf_conn *)(xprt->xp_p1); return (SVCAUTH_UNWRAP(&SVC_AUTH(xprt), &cd->xdrs, xdr_args, args_ptr)); } static bool_t svc_vc_freeargs(SVCXPRT *xprt, xdrproc_t xdr_args, void *args_ptr) { XDR *xdrs; assert(xprt != NULL); /* args_ptr may be NULL */ xdrs = &(((struct cf_conn *)(xprt->xp_p1))->xdrs); xdrs->x_op = XDR_FREE; return ((*xdr_args)(xdrs, args_ptr)); } static bool_t svc_vc_reply(SVCXPRT *xprt, struct rpc_msg *msg) { struct cf_conn *cd; XDR *xdrs; bool_t rstat; xdrproc_t xdr_proc; caddr_t xdr_where; u_int pos; assert(xprt != NULL); assert(msg != NULL); cd = (struct cf_conn *)(xprt->xp_p1); xdrs = &(cd->xdrs); xdrs->x_op = XDR_ENCODE; msg->rm_xid = cd->x_id; rstat = TRUE; if (msg->rm_reply.rp_stat == MSG_ACCEPTED && msg->rm_reply.rp_acpt.ar_stat == SUCCESS) { xdr_proc = msg->acpted_rply.ar_results.proc; xdr_where = msg->acpted_rply.ar_results.where; msg->acpted_rply.ar_results.proc = (xdrproc_t) xdr_void; msg->acpted_rply.ar_results.where = NULL; pos = XDR_GETPOS(xdrs); if (!xdr_replymsg(xdrs, msg) || !SVCAUTH_WRAP(&SVC_AUTH(xprt), xdrs, xdr_proc, xdr_where)) { XDR_SETPOS(xdrs, pos); rstat = FALSE; } } else { rstat = xdr_replymsg(xdrs, msg); } if (rstat) (void)xdrrec_endofrecord(xdrs, TRUE); return (rstat); } static void svc_vc_ops(SVCXPRT *xprt) { static struct xp_ops ops; static struct xp_ops2 ops2; /* VARIABLES PROTECTED BY ops_lock: ops, ops2 */ mutex_lock(&ops_lock); if (ops.xp_recv == NULL) { ops.xp_recv = svc_vc_recv; ops.xp_stat = svc_vc_stat; ops.xp_getargs = svc_vc_getargs; ops.xp_reply = svc_vc_reply; ops.xp_freeargs = svc_vc_freeargs; ops.xp_destroy = svc_vc_destroy; ops2.xp_control = svc_vc_control; } xprt->xp_ops = &ops; xprt->xp_ops2 = &ops2; mutex_unlock(&ops_lock); } static void svc_vc_rendezvous_ops(SVCXPRT *xprt) { static struct xp_ops ops; static struct xp_ops2 ops2; mutex_lock(&ops_lock); if (ops.xp_recv == NULL) { ops.xp_recv = rendezvous_request; ops.xp_stat = rendezvous_stat; ops.xp_getargs = (bool_t (*)(SVCXPRT *, xdrproc_t, void *))abort; ops.xp_reply = (bool_t (*)(SVCXPRT *, struct rpc_msg *))abort; ops.xp_freeargs = (bool_t (*)(SVCXPRT *, xdrproc_t, void *))abort, ops.xp_destroy = svc_vc_destroy; ops2.xp_control = svc_vc_rendezvous_control; } xprt->xp_ops = &ops; xprt->xp_ops2 = &ops2; mutex_unlock(&ops_lock); } /* * Get the effective UID of the sending process. Used by rpcbind, keyserv * and rpc.yppasswdd on AF_LOCAL. */ int __rpc_get_local_uid(SVCXPRT *transp, uid_t *uid) { int sock, ret; gid_t egid; uid_t euid; struct sockaddr *sa; sock = transp->xp_fd; sa = (struct sockaddr *)transp->xp_rtaddr.buf; if (sa->sa_family == AF_LOCAL) { ret = getpeereid(sock, &euid, &egid); if (ret == 0) *uid = euid; return (ret); } else return (-1); } /* * Destroy xprts that have not have had any activity in 'timeout' seconds. * If 'cleanblock' is true, blocking connections (the default) are also * cleaned. If timeout is 0, the least active connection is picked. */ bool_t __svc_clean_idle(fd_set *fds, int timeout, bool_t cleanblock) { int i, ncleaned; SVCXPRT *xprt, *least_active; struct timeval tv, tdiff, tmax; struct cf_conn *cd; gettimeofday(&tv, NULL); tmax.tv_sec = tmax.tv_usec = 0; least_active = NULL; rwlock_wrlock(&svc_fd_lock); for (i = ncleaned = 0; i <= svc_maxfd; i++) { if (FD_ISSET(i, fds)) { xprt = __svc_xports[i]; if (xprt == NULL || xprt->xp_ops == NULL || xprt->xp_ops->xp_recv != svc_vc_recv) continue; cd = (struct cf_conn *)xprt->xp_p1; if (!cleanblock && !cd->nonblock) continue; if (timeout == 0) { timersub(&tv, &cd->last_recv_time, &tdiff); if (timercmp(&tdiff, &tmax, >)) { tmax = tdiff; least_active = xprt; } continue; } if (tv.tv_sec - cd->last_recv_time.tv_sec > timeout) { __xprt_unregister_unlocked(xprt); __svc_vc_dodestroy(xprt); ncleaned++; } } } if (timeout == 0 && least_active != NULL) { __xprt_unregister_unlocked(least_active); __svc_vc_dodestroy(least_active); ncleaned++; } rwlock_unlock(&svc_fd_lock); return ncleaned > 0 ? TRUE : FALSE; } Index: user/ngie/more-tests2/lib/libc/stdio/open_memstream.c =================================================================== --- user/ngie/more-tests2/lib/libc/stdio/open_memstream.c (revision 292053) +++ user/ngie/more-tests2/lib/libc/stdio/open_memstream.c (revision 292054) @@ -1,212 +1,212 @@ /*- * Copyright (c) 2013 Hudson River Trading LLC * Written by: John H. Baldwin * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include "namespace.h" #include #include +#include #ifdef DEBUG -#include +#include #endif -#include #include #include #include #include #include "un-namespace.h" /* XXX: There is no FPOS_MAX. This assumes fpos_t is an off_t. */ #define FPOS_MAX OFF_MAX struct memstream { char **bufp; size_t *sizep; ssize_t len; fpos_t offset; }; static int memstream_grow(struct memstream *ms, fpos_t newoff) { char *buf; ssize_t newsize; if (newoff < 0 || newoff >= SSIZE_MAX) newsize = SSIZE_MAX - 1; else newsize = newoff; if (newsize > ms->len) { buf = realloc(*ms->bufp, newsize + 1); if (buf != NULL) { #ifdef DEBUG fprintf(stderr, "MS: %p growing from %zd to %zd\n", ms, ms->len, newsize); #endif memset(buf + ms->len + 1, 0, newsize - ms->len); *ms->bufp = buf; ms->len = newsize; return (1); } return (0); } return (1); } static void memstream_update(struct memstream *ms) { assert(ms->len >= 0 && ms->offset >= 0); *ms->sizep = ms->len < ms->offset ? ms->len : ms->offset; } static int memstream_write(void *cookie, const char *buf, int len) { struct memstream *ms; ssize_t tocopy; ms = cookie; if (!memstream_grow(ms, ms->offset + len)) return (-1); tocopy = ms->len - ms->offset; if (len < tocopy) tocopy = len; memcpy(*ms->bufp + ms->offset, buf, tocopy); ms->offset += tocopy; memstream_update(ms); #ifdef DEBUG fprintf(stderr, "MS: write(%p, %d) = %zd\n", ms, len, tocopy); #endif return (tocopy); } static fpos_t memstream_seek(void *cookie, fpos_t pos, int whence) { struct memstream *ms; #ifdef DEBUG fpos_t old; #endif ms = cookie; #ifdef DEBUG old = ms->offset; #endif switch (whence) { case SEEK_SET: /* _fseeko() checks for negative offsets. */ assert(pos >= 0); ms->offset = pos; break; case SEEK_CUR: /* This is only called by _ftello(). */ assert(pos == 0); break; case SEEK_END: if (pos < 0) { if (pos + ms->len < 0) { #ifdef DEBUG fprintf(stderr, "MS: bad SEEK_END: pos %jd, len %zd\n", (intmax_t)pos, ms->len); #endif errno = EINVAL; return (-1); } } else { if (FPOS_MAX - ms->len < pos) { #ifdef DEBUG fprintf(stderr, "MS: bad SEEK_END: pos %jd, len %zd\n", (intmax_t)pos, ms->len); #endif errno = EOVERFLOW; return (-1); } } ms->offset = ms->len + pos; break; } memstream_update(ms); #ifdef DEBUG fprintf(stderr, "MS: seek(%p, %jd, %d) %jd -> %jd\n", ms, (intmax_t)pos, whence, (intmax_t)old, (intmax_t)ms->offset); #endif return (ms->offset); } static int memstream_close(void *cookie) { free(cookie); return (0); } FILE * open_memstream(char **bufp, size_t *sizep) { struct memstream *ms; int save_errno; FILE *fp; if (bufp == NULL || sizep == NULL) { errno = EINVAL; return (NULL); } *bufp = calloc(1, 1); if (*bufp == NULL) return (NULL); ms = malloc(sizeof(*ms)); if (ms == NULL) { save_errno = errno; free(*bufp); *bufp = NULL; errno = save_errno; return (NULL); } ms->bufp = bufp; ms->sizep = sizep; ms->len = 0; ms->offset = 0; memstream_update(ms); fp = funopen(ms, NULL, memstream_write, memstream_seek, memstream_close); if (fp == NULL) { save_errno = errno; free(ms); free(*bufp); *bufp = NULL; errno = save_errno; return (NULL); } fwide(fp, -1); return (fp); } Index: user/ngie/more-tests2/lib/libc/stdio/open_wmemstream.c =================================================================== --- user/ngie/more-tests2/lib/libc/stdio/open_wmemstream.c (revision 292053) +++ user/ngie/more-tests2/lib/libc/stdio/open_wmemstream.c (revision 292054) @@ -1,274 +1,274 @@ /*- * Copyright (c) 2013 Hudson River Trading LLC * Written by: John H. Baldwin * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include "namespace.h" #include #include +#include #ifdef DEBUG -#include +#include #endif -#include #include #include #include #include #include "un-namespace.h" /* XXX: There is no FPOS_MAX. This assumes fpos_t is an off_t. */ #define FPOS_MAX OFF_MAX struct wmemstream { wchar_t **bufp; size_t *sizep; ssize_t len; fpos_t offset; mbstate_t mbstate; }; static int wmemstream_grow(struct wmemstream *ms, fpos_t newoff) { wchar_t *buf; ssize_t newsize; if (newoff < 0 || newoff >= SSIZE_MAX / sizeof(wchar_t)) newsize = SSIZE_MAX / sizeof(wchar_t) - 1; else newsize = newoff; if (newsize > ms->len) { buf = realloc(*ms->bufp, (newsize + 1) * sizeof(wchar_t)); if (buf != NULL) { #ifdef DEBUG fprintf(stderr, "WMS: %p growing from %zd to %zd\n", ms, ms->len, newsize); #endif wmemset(buf + ms->len + 1, 0, newsize - ms->len); *ms->bufp = buf; ms->len = newsize; return (1); } return (0); } return (1); } static void wmemstream_update(struct wmemstream *ms) { assert(ms->len >= 0 && ms->offset >= 0); *ms->sizep = ms->len < ms->offset ? ms->len : ms->offset; } /* * Based on a starting multibyte state and an input buffer, determine * how many wchar_t's would be output. This doesn't use mbsnrtowcs() * so that it can handle embedded null characters. */ static size_t wbuflen(const mbstate_t *state, const char *buf, int len) { mbstate_t lenstate; size_t charlen, count; count = 0; lenstate = *state; while (len > 0) { charlen = mbrlen(buf, len, &lenstate); if (charlen == (size_t)-1) return (-1); if (charlen == (size_t)-2) break; if (charlen == 0) /* XXX: Not sure how else to handle this. */ charlen = 1; len -= charlen; buf += charlen; count++; } return (count); } static int wmemstream_write(void *cookie, const char *buf, int len) { struct wmemstream *ms; ssize_t consumed, wlen; size_t charlen; ms = cookie; wlen = wbuflen(&ms->mbstate, buf, len); if (wlen < 0) { errno = EILSEQ; return (-1); } if (!wmemstream_grow(ms, ms->offset + wlen)) return (-1); /* * This copies characters one at a time rather than using * mbsnrtowcs() so it can properly handle embedded null * characters. */ consumed = 0; while (len > 0 && ms->offset < ms->len) { charlen = mbrtowc(*ms->bufp + ms->offset, buf, len, &ms->mbstate); if (charlen == (size_t)-1) { if (consumed == 0) { errno = EILSEQ; return (-1); } /* Treat it as a successful short write. */ break; } if (charlen == 0) /* XXX: Not sure how else to handle this. */ charlen = 1; if (charlen == (size_t)-2) { consumed += len; len = 0; } else { consumed += charlen; buf += charlen; len -= charlen; ms->offset++; } } wmemstream_update(ms); #ifdef DEBUG fprintf(stderr, "WMS: write(%p, %d) = %zd\n", ms, len, consumed); #endif return (consumed); } static fpos_t wmemstream_seek(void *cookie, fpos_t pos, int whence) { struct wmemstream *ms; fpos_t old; ms = cookie; old = ms->offset; switch (whence) { case SEEK_SET: /* _fseeko() checks for negative offsets. */ assert(pos >= 0); ms->offset = pos; break; case SEEK_CUR: /* This is only called by _ftello(). */ assert(pos == 0); break; case SEEK_END: if (pos < 0) { if (pos + ms->len < 0) { #ifdef DEBUG fprintf(stderr, "WMS: bad SEEK_END: pos %jd, len %zd\n", (intmax_t)pos, ms->len); #endif errno = EINVAL; return (-1); } } else { if (FPOS_MAX - ms->len < pos) { #ifdef DEBUG fprintf(stderr, "WMS: bad SEEK_END: pos %jd, len %zd\n", (intmax_t)pos, ms->len); #endif errno = EOVERFLOW; return (-1); } } ms->offset = ms->len + pos; break; } /* Reset the multibyte state if a seek changes the position. */ if (ms->offset != old) memset(&ms->mbstate, 0, sizeof(ms->mbstate)); wmemstream_update(ms); #ifdef DEBUG fprintf(stderr, "WMS: seek(%p, %jd, %d) %jd -> %jd\n", ms, (intmax_t)pos, whence, (intmax_t)old, (intmax_t)ms->offset); #endif return (ms->offset); } static int wmemstream_close(void *cookie) { free(cookie); return (0); } FILE * open_wmemstream(wchar_t **bufp, size_t *sizep) { struct wmemstream *ms; int save_errno; FILE *fp; if (bufp == NULL || sizep == NULL) { errno = EINVAL; return (NULL); } *bufp = calloc(1, sizeof(wchar_t)); if (*bufp == NULL) return (NULL); ms = malloc(sizeof(*ms)); if (ms == NULL) { save_errno = errno; free(*bufp); *bufp = NULL; errno = save_errno; return (NULL); } ms->bufp = bufp; ms->sizep = sizep; ms->len = 0; ms->offset = 0; memset(&ms->mbstate, 0, sizeof(mbstate_t)); wmemstream_update(ms); fp = funopen(ms, NULL, wmemstream_write, wmemstream_seek, wmemstream_close); if (fp == NULL) { save_errno = errno; free(ms); free(*bufp); *bufp = NULL; errno = save_errno; return (NULL); } fwide(fp, 1); return (fp); } Index: user/ngie/more-tests2/lib/libc =================================================================== --- user/ngie/more-tests2/lib/libc (revision 292053) +++ user/ngie/more-tests2/lib/libc (revision 292054) Property changes on: user/ngie/more-tests2/lib/libc ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,1 ## Merged /head/lib/libc:r292006-292053 Index: user/ngie/more-tests2/lib/libopenbsd/Makefile =================================================================== --- user/ngie/more-tests2/lib/libopenbsd/Makefile (revision 292053) +++ user/ngie/more-tests2/lib/libopenbsd/Makefile (revision 292054) @@ -1,15 +1,14 @@ # $FreeBSD$ LIB= openbsd SRCS= getdtablecount.c \ imsg-buffer.c \ imsg.c \ ohash.c INTERNALLIB= CFLAGS+= -I${.CURDIR} WARNS= 3 -NO_WERROR= .include Index: user/ngie/more-tests2/lib/libopenbsd/imsg.c =================================================================== --- user/ngie/more-tests2/lib/libopenbsd/imsg.c (revision 292053) +++ user/ngie/more-tests2/lib/libopenbsd/imsg.c (revision 292054) @@ -1,307 +1,304 @@ -/* $OpenBSD: imsg.c,v 1.10 2015/07/19 07:18:59 nicm Exp $ */ +/* $OpenBSD: imsg.c,v 1.13 2015/12/09 11:54:12 tb Exp $ */ /* * Copyright (c) 2003, 2004 Henning Brauer * * Permission to use, copy, modify, and distribute this software for any * purpose with or without fee is hereby granted, provided that the above * copyright notice and this permission notice appear in all copies. * * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. * * $FreeBSD$ */ #include #include #include #include #include #include #include #include #include "imsg.h" int imsg_fd_overhead = 0; int imsg_get_fd(struct imsgbuf *); void imsg_init(struct imsgbuf *ibuf, int fd) { msgbuf_init(&ibuf->w); memset(&ibuf->r, 0, sizeof(ibuf->r)); ibuf->fd = fd; ibuf->w.fd = fd; ibuf->pid = getpid(); TAILQ_INIT(&ibuf->fds); } ssize_t imsg_read(struct imsgbuf *ibuf) { struct msghdr msg; struct cmsghdr *cmsg; union { struct cmsghdr hdr; char buf[CMSG_SPACE(sizeof(int) * 1)]; } cmsgbuf; struct iovec iov; ssize_t n = -1; int fd; struct imsg_fd *ifd; memset(&msg, 0, sizeof(msg)); memset(&cmsgbuf, 0, sizeof(cmsgbuf)); iov.iov_base = ibuf->r.buf + ibuf->r.wpos; iov.iov_len = sizeof(ibuf->r.buf) - ibuf->r.wpos; msg.msg_iov = &iov; msg.msg_iovlen = 1; msg.msg_control = &cmsgbuf.buf; msg.msg_controllen = sizeof(cmsgbuf.buf); if ((ifd = calloc(1, sizeof(struct imsg_fd))) == NULL) return (-1); again: if (getdtablecount() + imsg_fd_overhead + - (CMSG_SPACE(sizeof(int))-CMSG_SPACE(0))/sizeof(int) + (int)((CMSG_SPACE(sizeof(int))-CMSG_SPACE(0))/sizeof(int)) >= getdtablesize()) { errno = EAGAIN; free(ifd); return (-1); } if ((n = recvmsg(ibuf->fd, &msg, 0)) == -1) { - if (errno == EMSGSIZE) - goto fail; - if (errno != EINTR && errno != EAGAIN) - goto fail; - goto again; + if (errno == EINTR) + goto again; + goto fail; } ibuf->r.wpos += n; for (cmsg = CMSG_FIRSTHDR(&msg); cmsg != NULL; cmsg = CMSG_NXTHDR(&msg, cmsg)) { if (cmsg->cmsg_level == SOL_SOCKET && cmsg->cmsg_type == SCM_RIGHTS) { int i; int j; /* * We only accept one file descriptor. Due to C * padding rules, our control buffer might contain * more than one fd, and we must close them. */ j = ((char *)cmsg + cmsg->cmsg_len - (char *)CMSG_DATA(cmsg)) / sizeof(int); for (i = 0; i < j; i++) { fd = ((int *)CMSG_DATA(cmsg))[i]; if (ifd != NULL) { ifd->fd = fd; TAILQ_INSERT_TAIL(&ibuf->fds, ifd, entry); ifd = NULL; } else close(fd); } } /* we do not handle other ctl data level */ } fail: - if (ifd) - free(ifd); + free(ifd); return (n); } ssize_t imsg_get(struct imsgbuf *ibuf, struct imsg *imsg) { size_t av, left, datalen; av = ibuf->r.wpos; if (IMSG_HEADER_SIZE > av) return (0); memcpy(&imsg->hdr, ibuf->r.buf, sizeof(imsg->hdr)); if (imsg->hdr.len < IMSG_HEADER_SIZE || imsg->hdr.len > MAX_IMSGSIZE) { errno = ERANGE; return (-1); } if (imsg->hdr.len > av) return (0); datalen = imsg->hdr.len - IMSG_HEADER_SIZE; ibuf->r.rptr = ibuf->r.buf + IMSG_HEADER_SIZE; if (datalen == 0) imsg->data = NULL; else if ((imsg->data = malloc(datalen)) == NULL) return (-1); if (imsg->hdr.flags & IMSGF_HASFD) imsg->fd = imsg_get_fd(ibuf); else imsg->fd = -1; memcpy(imsg->data, ibuf->r.rptr, datalen); if (imsg->hdr.len < av) { left = av - imsg->hdr.len; memmove(&ibuf->r.buf, ibuf->r.buf + imsg->hdr.len, left); ibuf->r.wpos = left; } else ibuf->r.wpos = 0; return (datalen + IMSG_HEADER_SIZE); } int imsg_compose(struct imsgbuf *ibuf, u_int32_t type, u_int32_t peerid, pid_t pid, int fd, const void *data, u_int16_t datalen) { struct ibuf *wbuf; if ((wbuf = imsg_create(ibuf, type, peerid, pid, datalen)) == NULL) return (-1); if (imsg_add(wbuf, data, datalen) == -1) return (-1); wbuf->fd = fd; imsg_close(ibuf, wbuf); return (1); } int imsg_composev(struct imsgbuf *ibuf, u_int32_t type, u_int32_t peerid, pid_t pid, int fd, const struct iovec *iov, int iovcnt) { struct ibuf *wbuf; int i, datalen = 0; for (i = 0; i < iovcnt; i++) datalen += iov[i].iov_len; if ((wbuf = imsg_create(ibuf, type, peerid, pid, datalen)) == NULL) return (-1); for (i = 0; i < iovcnt; i++) if (imsg_add(wbuf, iov[i].iov_base, iov[i].iov_len) == -1) return (-1); wbuf->fd = fd; imsg_close(ibuf, wbuf); return (1); } /* ARGSUSED */ struct ibuf * imsg_create(struct imsgbuf *ibuf, u_int32_t type, u_int32_t peerid, pid_t pid, u_int16_t datalen) { struct ibuf *wbuf; struct imsg_hdr hdr; datalen += IMSG_HEADER_SIZE; if (datalen > MAX_IMSGSIZE) { errno = ERANGE; return (NULL); } hdr.type = type; hdr.flags = 0; hdr.peerid = peerid; if ((hdr.pid = pid) == 0) hdr.pid = ibuf->pid; if ((wbuf = ibuf_dynamic(datalen, MAX_IMSGSIZE)) == NULL) { return (NULL); } if (imsg_add(wbuf, &hdr, sizeof(hdr)) == -1) return (NULL); return (wbuf); } int imsg_add(struct ibuf *msg, const void *data, u_int16_t datalen) { if (datalen) if (ibuf_add(msg, data, datalen) == -1) { ibuf_free(msg); return (-1); } return (datalen); } void imsg_close(struct imsgbuf *ibuf, struct ibuf *msg) { struct imsg_hdr *hdr; hdr = (struct imsg_hdr *)msg->buf; hdr->flags &= ~IMSGF_HASFD; if (msg->fd != -1) hdr->flags |= IMSGF_HASFD; hdr->len = (u_int16_t)msg->wpos; ibuf_close(&ibuf->w, msg); } void imsg_free(struct imsg *imsg) { free(imsg->data); } int imsg_get_fd(struct imsgbuf *ibuf) { int fd; struct imsg_fd *ifd; if ((ifd = TAILQ_FIRST(&ibuf->fds)) == NULL) return (-1); fd = ifd->fd; TAILQ_REMOVE(&ibuf->fds, ifd, entry); free(ifd); return (fd); } int imsg_flush(struct imsgbuf *ibuf) { while (ibuf->w.queued) if (msgbuf_write(&ibuf->w) <= 0) return (-1); return (0); } void imsg_clear(struct imsgbuf *ibuf) { int fd; msgbuf_clear(&ibuf->w); while ((fd = imsg_get_fd(ibuf)) != -1) close(fd); } Index: user/ngie/more-tests2/sbin/devd/devd.cc =================================================================== --- user/ngie/more-tests2/sbin/devd/devd.cc (revision 292053) +++ user/ngie/more-tests2/sbin/devd/devd.cc (revision 292054) @@ -1,1243 +1,1254 @@ /*- * Copyright (c) 2002-2010 M. Warner Losh. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * my_system is a variation on lib/libc/stdlib/system.c: * * Copyright (c) 1988, 1993 * The Regents of the University of California. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * DEVD control daemon. */ // TODO list: // o devd.conf and devd man pages need a lot of help: // - devd needs to document the unix domain socket // - devd.conf needs more details on the supported statements. #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "devd.h" /* C compatible definitions */ #include "devd.hh" /* C++ class definitions */ #define STREAMPIPE "/var/run/devd.pipe" #define SEQPACKETPIPE "/var/run/devd.seqpacket.pipe" #define CF "/etc/devd.conf" #define SYSCTL "hw.bus.devctl_queue" /* * Since the client socket is nonblocking, we must increase its send buffer to * handle brief event storms. On FreeBSD, AF_UNIX sockets don't have a receive - * buffer, so the client can't increate the buffersize by itself. + * buffer, so the client can't increase the buffersize by itself. * * For example, when creating a ZFS pool, devd emits one 165 character - * resource.fs.zfs.statechange message for each vdev in the pool. A 64k - * buffer has enough space for almost 400 drives, which would be very large but - * not impossibly large pool. A 128k buffer has enough space for 794 drives, - * which is more than can fit in a rack with modern technology. + * resource.fs.zfs.statechange message for each vdev in the pool. The kernel + * allocates a 4608B mbuf for each message. Modern technology places a limit of + * roughly 450 drives/rack, and it's unlikely that a zpool will ever be larger + * than that. + * + * 450 drives * 165 bytes / drive = 74250B of data in the sockbuf + * 450 drives * 4608B / drive = 2073600B of mbufs in the sockbuf + * + * We can't directly set the sockbuf's mbuf limit, but we can do it indirectly. + * The kernel sets it to the minimum of a hard-coded maximum value and sbcc * + * kern.ipc.sockbuf_waste_factor, where sbcc is the socket buffer size set by + * the user. The default value of kern.ipc.sockbuf_waste_factor is 8. If we + * set the bufsize to 256k and use the kern.ipc.sockbuf_waste_factor, then the + * kernel will set the mbuf limit to 2MB, which is just large enough for 450 + * drives. It also happens to be the same as the hardcoded maximum value. */ -#define CLIENT_BUFSIZE 131072 +#define CLIENT_BUFSIZE 262144 using namespace std; typedef struct client { int fd; int socktype; } client_t; extern FILE *yyin; extern int lineno; static const char notify = '!'; static const char nomatch = '?'; static const char attach = '+'; static const char detach = '-'; static struct pidfh *pfh; static int no_daemon = 0; static int daemonize_quick = 0; static int quiet_mode = 0; static unsigned total_events = 0; static volatile sig_atomic_t got_siginfo = 0; static volatile sig_atomic_t romeo_must_die = 0; static const char *configfile = CF; static void devdlog(int priority, const char* message, ...) __printflike(2, 3); static void event_loop(void); static void usage(void); template void delete_and_clear(vector &v) { typename vector::const_iterator i; for (i = v.begin(); i != v.end(); ++i) delete *i; v.clear(); } config cfg; event_proc::event_proc() : _prio(-1) { _epsvec.reserve(4); } event_proc::~event_proc() { delete_and_clear(_epsvec); } void event_proc::add(eps *eps) { _epsvec.push_back(eps); } bool event_proc::matches(config &c) const { vector::const_iterator i; for (i = _epsvec.begin(); i != _epsvec.end(); ++i) if (!(*i)->do_match(c)) return (false); return (true); } bool event_proc::run(config &c) const { vector::const_iterator i; for (i = _epsvec.begin(); i != _epsvec.end(); ++i) if (!(*i)->do_action(c)) return (false); return (true); } action::action(const char *cmd) : _cmd(cmd) { // nothing } action::~action() { // nothing } static int my_system(const char *command) { pid_t pid, savedpid; int pstat; struct sigaction ign, intact, quitact; sigset_t newsigblock, oldsigblock; if (!command) /* just checking... */ return (1); /* * Ignore SIGINT and SIGQUIT, block SIGCHLD. Remember to save * existing signal dispositions. */ ign.sa_handler = SIG_IGN; ::sigemptyset(&ign.sa_mask); ign.sa_flags = 0; ::sigaction(SIGINT, &ign, &intact); ::sigaction(SIGQUIT, &ign, &quitact); ::sigemptyset(&newsigblock); ::sigaddset(&newsigblock, SIGCHLD); ::sigprocmask(SIG_BLOCK, &newsigblock, &oldsigblock); switch (pid = ::fork()) { case -1: /* error */ break; case 0: /* child */ /* * Restore original signal dispositions and exec the command. */ ::sigaction(SIGINT, &intact, NULL); ::sigaction(SIGQUIT, &quitact, NULL); ::sigprocmask(SIG_SETMASK, &oldsigblock, NULL); /* * Close the PID file, and all other open descriptors. * Inherit std{in,out,err} only. */ cfg.close_pidfile(); ::closefrom(3); ::execl(_PATH_BSHELL, "sh", "-c", command, (char *)NULL); ::_exit(127); default: /* parent */ savedpid = pid; do { pid = ::wait4(savedpid, &pstat, 0, (struct rusage *)0); } while (pid == -1 && errno == EINTR); break; } ::sigaction(SIGINT, &intact, NULL); ::sigaction(SIGQUIT, &quitact, NULL); ::sigprocmask(SIG_SETMASK, &oldsigblock, NULL); return (pid == -1 ? -1 : pstat); } bool action::do_action(config &c) { string s = c.expand_string(_cmd.c_str()); devdlog(LOG_INFO, "Executing '%s'\n", s.c_str()); my_system(s.c_str()); return (true); } match::match(config &c, const char *var, const char *re) : _inv(re[0] == '!'), _var(var), _re(c.expand_string(_inv ? re + 1 : re, "^", "$")) { regcomp(&_regex, _re.c_str(), REG_EXTENDED | REG_NOSUB | REG_ICASE); } match::~match() { regfree(&_regex); } bool match::do_match(config &c) { const string &value = c.get_variable(_var); bool retval; /* * This function gets called WAY too often to justify calling syslog() * each time, even at LOG_DEBUG. Because if syslogd isn't running, it * can consume excessive amounts of systime inside of connect(). Only * log when we're in -d mode. */ if (no_daemon) { devdlog(LOG_DEBUG, "Testing %s=%s against %s, invert=%d\n", _var.c_str(), value.c_str(), _re.c_str(), _inv); } retval = (regexec(&_regex, value.c_str(), 0, NULL, 0) == 0); if (_inv == 1) retval = (retval == 0) ? 1 : 0; return (retval); } #include #include #include media::media(config &, const char *var, const char *type) : _var(var), _type(-1) { static struct ifmedia_description media_types[] = { { IFM_ETHER, "Ethernet" }, { IFM_TOKEN, "Tokenring" }, { IFM_FDDI, "FDDI" }, { IFM_IEEE80211, "802.11" }, { IFM_ATM, "ATM" }, { -1, "unknown" }, { 0, NULL }, }; for (int i = 0; media_types[i].ifmt_string != NULL; ++i) if (strcasecmp(type, media_types[i].ifmt_string) == 0) { _type = media_types[i].ifmt_word; break; } } media::~media() { } bool media::do_match(config &c) { string value; struct ifmediareq ifmr; bool retval; int s; // Since we can be called from both a device attach/detach // context where device-name is defined and what we want, // as well as from a link status context, where subsystem is // the name of interest, first try device-name and fall back // to subsystem if none exists. value = c.get_variable("device-name"); if (value.empty()) value = c.get_variable("subsystem"); devdlog(LOG_DEBUG, "Testing media type of %s against 0x%x\n", value.c_str(), _type); retval = false; s = socket(PF_INET, SOCK_DGRAM, 0); if (s >= 0) { memset(&ifmr, 0, sizeof(ifmr)); strncpy(ifmr.ifm_name, value.c_str(), sizeof(ifmr.ifm_name)); if (ioctl(s, SIOCGIFMEDIA, (caddr_t)&ifmr) >= 0 && ifmr.ifm_status & IFM_AVALID) { devdlog(LOG_DEBUG, "%s has media type 0x%x\n", value.c_str(), IFM_TYPE(ifmr.ifm_active)); retval = (IFM_TYPE(ifmr.ifm_active) == _type); } else if (_type == -1) { devdlog(LOG_DEBUG, "%s has unknown media type\n", value.c_str()); retval = true; } close(s); } return (retval); } const string var_list::bogus = "_$_$_$_$_B_O_G_U_S_$_$_$_$_"; const string var_list::nothing = ""; const string & var_list::get_variable(const string &var) const { map::const_iterator i; i = _vars.find(var); if (i == _vars.end()) return (var_list::bogus); return (i->second); } bool var_list::is_set(const string &var) const { return (_vars.find(var) != _vars.end()); } void var_list::set_variable(const string &var, const string &val) { /* * This function gets called WAY too often to justify calling syslog() * each time, even at LOG_DEBUG. Because if syslogd isn't running, it * can consume excessive amounts of systime inside of connect(). Only * log when we're in -d mode. */ if (no_daemon) devdlog(LOG_DEBUG, "setting %s=%s\n", var.c_str(), val.c_str()); _vars[var] = val; } void config::reset(void) { _dir_list.clear(); delete_and_clear(_var_list_table); delete_and_clear(_attach_list); delete_and_clear(_detach_list); delete_and_clear(_nomatch_list); delete_and_clear(_notify_list); } void config::parse_one_file(const char *fn) { devdlog(LOG_DEBUG, "Parsing %s\n", fn); yyin = fopen(fn, "r"); if (yyin == NULL) err(1, "Cannot open config file %s", fn); lineno = 1; if (yyparse() != 0) errx(1, "Cannot parse %s at line %d", fn, lineno); fclose(yyin); } void config::parse_files_in_dir(const char *dirname) { DIR *dirp; struct dirent *dp; char path[PATH_MAX]; devdlog(LOG_DEBUG, "Parsing files in %s\n", dirname); dirp = opendir(dirname); if (dirp == NULL) return; readdir(dirp); /* Skip . */ readdir(dirp); /* Skip .. */ while ((dp = readdir(dirp)) != NULL) { if (strcmp(dp->d_name + dp->d_namlen - 5, ".conf") == 0) { snprintf(path, sizeof(path), "%s/%s", dirname, dp->d_name); parse_one_file(path); } } closedir(dirp); } class epv_greater { public: int operator()(event_proc *const&l1, event_proc *const&l2) const { return (l1->get_priority() > l2->get_priority()); } }; void config::sort_vector(vector &v) { stable_sort(v.begin(), v.end(), epv_greater()); } void config::parse(void) { vector::const_iterator i; parse_one_file(configfile); for (i = _dir_list.begin(); i != _dir_list.end(); ++i) parse_files_in_dir((*i).c_str()); sort_vector(_attach_list); sort_vector(_detach_list); sort_vector(_nomatch_list); sort_vector(_notify_list); } void config::open_pidfile() { pid_t otherpid; if (_pidfile.empty()) return; pfh = pidfile_open(_pidfile.c_str(), 0600, &otherpid); if (pfh == NULL) { if (errno == EEXIST) errx(1, "devd already running, pid: %d", (int)otherpid); warn("cannot open pid file"); } } void config::write_pidfile() { pidfile_write(pfh); } void config::close_pidfile() { pidfile_close(pfh); } void config::remove_pidfile() { pidfile_remove(pfh); } void config::add_attach(int prio, event_proc *p) { p->set_priority(prio); _attach_list.push_back(p); } void config::add_detach(int prio, event_proc *p) { p->set_priority(prio); _detach_list.push_back(p); } void config::add_directory(const char *dir) { _dir_list.push_back(string(dir)); } void config::add_nomatch(int prio, event_proc *p) { p->set_priority(prio); _nomatch_list.push_back(p); } void config::add_notify(int prio, event_proc *p) { p->set_priority(prio); _notify_list.push_back(p); } void config::set_pidfile(const char *fn) { _pidfile = fn; } void config::push_var_table() { var_list *vl; vl = new var_list(); _var_list_table.push_back(vl); devdlog(LOG_DEBUG, "Pushing table\n"); } void config::pop_var_table() { delete _var_list_table.back(); _var_list_table.pop_back(); devdlog(LOG_DEBUG, "Popping table\n"); } void config::set_variable(const char *var, const char *val) { _var_list_table.back()->set_variable(var, val); } const string & config::get_variable(const string &var) { vector::reverse_iterator i; for (i = _var_list_table.rbegin(); i != _var_list_table.rend(); ++i) { if ((*i)->is_set(var)) return ((*i)->get_variable(var)); } return (var_list::nothing); } bool config::is_id_char(char ch) const { return (ch != '\0' && (isalpha(ch) || isdigit(ch) || ch == '_' || ch == '-')); } void config::expand_one(const char *&src, string &dst) { int count; string buffer; src++; // $$ -> $ if (*src == '$') { dst += *src++; return; } // $(foo) -> $(foo) // Not sure if I want to support this or not, so for now we just pass // it through. if (*src == '(') { dst += '$'; count = 1; /* If the string ends before ) is matched , return. */ while (count > 0 && *src) { if (*src == ')') count--; else if (*src == '(') count++; dst += *src++; } return; } // $[^A-Za-z] -> $\1 if (!isalpha(*src)) { dst += '$'; dst += *src++; return; } // $var -> replace with value do { buffer += *src++; } while (is_id_char(*src)); dst.append(get_variable(buffer)); } const string config::expand_string(const char *src, const char *prepend, const char *append) { const char *var_at; string dst; /* * 128 bytes is enough for 2427 of 2438 expansions that happen * while parsing config files, as tested on 2013-01-30. */ dst.reserve(128); if (prepend != NULL) dst = prepend; for (;;) { var_at = strchr(src, '$'); if (var_at == NULL) { dst.append(src); break; } dst.append(src, var_at - src); src = var_at; expand_one(src, dst); } if (append != NULL) dst.append(append); return (dst); } bool config::chop_var(char *&buffer, char *&lhs, char *&rhs) const { char *walker; if (*buffer == '\0') return (false); walker = lhs = buffer; while (is_id_char(*walker)) walker++; if (*walker != '=') return (false); walker++; // skip = if (*walker == '"') { walker++; // skip " rhs = walker; while (*walker && *walker != '"') walker++; if (*walker != '"') return (false); rhs[-2] = '\0'; *walker++ = '\0'; } else { rhs = walker; while (*walker && !isspace(*walker)) walker++; if (*walker != '\0') *walker++ = '\0'; rhs[-1] = '\0'; } while (isspace(*walker)) walker++; buffer = walker; return (true); } char * config::set_vars(char *buffer) { char *lhs; char *rhs; while (1) { if (!chop_var(buffer, lhs, rhs)) break; set_variable(lhs, rhs); } return (buffer); } void config::find_and_execute(char type) { vector *l; vector::const_iterator i; const char *s; switch (type) { default: return; case notify: l = &_notify_list; s = "notify"; break; case nomatch: l = &_nomatch_list; s = "nomatch"; break; case attach: l = &_attach_list; s = "attach"; break; case detach: l = &_detach_list; s = "detach"; break; } devdlog(LOG_DEBUG, "Processing %s event\n", s); for (i = l->begin(); i != l->end(); ++i) { if ((*i)->matches(*this)) { (*i)->run(*this); break; } } } static void process_event(char *buffer) { char type; char *sp; sp = buffer + 1; devdlog(LOG_INFO, "Processing event '%s'\n", buffer); type = *buffer++; cfg.push_var_table(); // No match doesn't have a device, and the format is a little // different, so handle it separately. switch (type) { case notify: sp = cfg.set_vars(sp); break; case nomatch: //? at location pnp-info on bus sp = strchr(sp, ' '); if (sp == NULL) return; /* Can't happen? */ *sp++ = '\0'; while (isspace(*sp)) sp++; if (strncmp(sp, "at ", 3) == 0) sp += 3; sp = cfg.set_vars(sp); while (isspace(*sp)) sp++; if (strncmp(sp, "on ", 3) == 0) cfg.set_variable("bus", sp + 3); break; case attach: /*FALLTHROUGH*/ case detach: sp = strchr(sp, ' '); if (sp == NULL) return; /* Can't happen? */ *sp++ = '\0'; cfg.set_variable("device-name", buffer); while (isspace(*sp)) sp++; if (strncmp(sp, "at ", 3) == 0) sp += 3; sp = cfg.set_vars(sp); while (isspace(*sp)) sp++; if (strncmp(sp, "on ", 3) == 0) cfg.set_variable("bus", sp + 3); break; } cfg.find_and_execute(type); cfg.pop_var_table(); } int create_socket(const char *name, int socktype) { int fd, slen; struct sockaddr_un sun; if ((fd = socket(PF_LOCAL, socktype, 0)) < 0) err(1, "socket"); bzero(&sun, sizeof(sun)); sun.sun_family = AF_UNIX; strlcpy(sun.sun_path, name, sizeof(sun.sun_path)); slen = SUN_LEN(&sun); unlink(name); if (fcntl(fd, F_SETFL, O_NONBLOCK) < 0) err(1, "fcntl"); if (::bind(fd, (struct sockaddr *) & sun, slen) < 0) err(1, "bind"); listen(fd, 4); chown(name, 0, 0); /* XXX - root.wheel */ chmod(name, 0666); return (fd); } unsigned int max_clients = 10; /* Default, can be overridden on cmdline. */ unsigned int num_clients; list clients; void notify_clients(const char *data, int len) { list::iterator i; /* * Deliver the data to all clients. Throw clients overboard at the * first sign of trouble. This reaps clients who've died or closed * their sockets, and also clients who are alive but failing to keep up * (or who are maliciously not reading, to consume buffer space in * kernel memory or tie up the limited number of available connections). */ for (i = clients.begin(); i != clients.end(); ) { int flags; if (i->socktype == SOCK_SEQPACKET) flags = MSG_EOR; else flags = 0; if (send(i->fd, data, len, flags) != len) { --num_clients; close(i->fd); i = clients.erase(i); devdlog(LOG_WARNING, "notify_clients: send() failed; " "dropping unresponsive client\n"); } else ++i; } } void check_clients(void) { int s; struct pollfd pfd; list::iterator i; /* * Check all existing clients to see if any of them have disappeared. * Normally we reap clients when we get an error trying to send them an * event. This check eliminates the problem of an ever-growing list of * zombie clients because we're never writing to them on a system * without frequent device-change activity. */ pfd.events = 0; for (i = clients.begin(); i != clients.end(); ) { pfd.fd = i->fd; s = poll(&pfd, 1, 0); if ((s < 0 && s != EINTR ) || (s > 0 && (pfd.revents & POLLHUP))) { --num_clients; close(i->fd); i = clients.erase(i); devdlog(LOG_NOTICE, "check_clients: " "dropping disconnected client\n"); } else ++i; } } void new_client(int fd, int socktype) { client_t s; int sndbuf_size; /* * First go reap any zombie clients, then accept the connection, and * shut down the read side to stop clients from consuming kernel memory * by sending large buffers full of data we'll never read. */ check_clients(); s.socktype = socktype; s.fd = accept(fd, NULL, NULL); if (s.fd != -1) { sndbuf_size = CLIENT_BUFSIZE; if (setsockopt(s.fd, SOL_SOCKET, SO_SNDBUF, &sndbuf_size, sizeof(sndbuf_size))) err(1, "setsockopt"); shutdown(s.fd, SHUT_RD); clients.push_back(s); ++num_clients; } else err(1, "accept"); } static void event_loop(void) { int rv; int fd; char buffer[DEVCTL_MAXBUF]; int once = 0; int stream_fd, seqpacket_fd, max_fd; int accepting; timeval tv; fd_set fds; fd = open(PATH_DEVCTL, O_RDONLY | O_CLOEXEC); if (fd == -1) err(1, "Can't open devctl device %s", PATH_DEVCTL); stream_fd = create_socket(STREAMPIPE, SOCK_STREAM); seqpacket_fd = create_socket(SEQPACKETPIPE, SOCK_SEQPACKET); accepting = 1; max_fd = max(fd, max(stream_fd, seqpacket_fd)) + 1; while (!romeo_must_die) { if (!once && !no_daemon && !daemonize_quick) { // Check to see if we have any events pending. tv.tv_sec = 0; tv.tv_usec = 0; FD_ZERO(&fds); FD_SET(fd, &fds); rv = select(fd + 1, &fds, &fds, &fds, &tv); // No events -> we've processed all pending events if (rv == 0) { devdlog(LOG_DEBUG, "Calling daemon\n"); cfg.remove_pidfile(); cfg.open_pidfile(); daemon(0, 0); cfg.write_pidfile(); once++; } } /* * When we've already got the max number of clients, stop * accepting new connections (don't put the listening sockets in * the set), shrink the accept() queue to reject connections * quickly, and poll the existing clients more often, so that we * notice more quickly when any of them disappear to free up * client slots. */ FD_ZERO(&fds); FD_SET(fd, &fds); if (num_clients < max_clients) { if (!accepting) { listen(stream_fd, max_clients); listen(seqpacket_fd, max_clients); accepting = 1; } FD_SET(stream_fd, &fds); FD_SET(seqpacket_fd, &fds); tv.tv_sec = 60; tv.tv_usec = 0; } else { if (accepting) { listen(stream_fd, 0); listen(seqpacket_fd, 0); accepting = 0; } tv.tv_sec = 2; tv.tv_usec = 0; } rv = select(max_fd, &fds, NULL, NULL, &tv); if (got_siginfo) { devdlog(LOG_NOTICE, "Events received so far=%u\n", total_events); got_siginfo = 0; } if (rv == -1) { if (errno == EINTR) continue; err(1, "select"); } else if (rv == 0) check_clients(); if (FD_ISSET(fd, &fds)) { rv = read(fd, buffer, sizeof(buffer) - 1); if (rv > 0) { total_events++; if (rv == sizeof(buffer) - 1) { devdlog(LOG_WARNING, "Warning: " "available event data exceeded " "buffer space\n"); } notify_clients(buffer, rv); buffer[rv] = '\0'; while (buffer[--rv] == '\n') buffer[rv] = '\0'; process_event(buffer); } else if (rv < 0) { if (errno != EINTR) break; } else { /* EOF */ break; } } if (FD_ISSET(stream_fd, &fds)) new_client(stream_fd, SOCK_STREAM); /* * Aside from the socket type, both sockets use the same * protocol, so we can process clients the same way. */ if (FD_ISSET(seqpacket_fd, &fds)) new_client(seqpacket_fd, SOCK_SEQPACKET); } close(fd); } /* * functions that the parser uses. */ void add_attach(int prio, event_proc *p) { cfg.add_attach(prio, p); } void add_detach(int prio, event_proc *p) { cfg.add_detach(prio, p); } void add_directory(const char *dir) { cfg.add_directory(dir); free(const_cast(dir)); } void add_nomatch(int prio, event_proc *p) { cfg.add_nomatch(prio, p); } void add_notify(int prio, event_proc *p) { cfg.add_notify(prio, p); } event_proc * add_to_event_proc(event_proc *ep, eps *eps) { if (ep == NULL) ep = new event_proc(); ep->add(eps); return (ep); } eps * new_action(const char *cmd) { eps *e = new action(cmd); free(const_cast(cmd)); return (e); } eps * new_match(const char *var, const char *re) { eps *e = new match(cfg, var, re); free(const_cast(var)); free(const_cast(re)); return (e); } eps * new_media(const char *var, const char *re) { eps *e = new media(cfg, var, re); free(const_cast(var)); free(const_cast(re)); return (e); } void set_pidfile(const char *name) { cfg.set_pidfile(name); free(const_cast(name)); } void set_variable(const char *var, const char *val) { cfg.set_variable(var, val); free(const_cast(var)); free(const_cast(val)); } static void gensighand(int) { romeo_must_die = 1; } /* * SIGINFO handler. Will print useful statistics to the syslog or stderr * as appropriate */ static void siginfohand(int) { got_siginfo = 1; } /* * Local logging function. Prints to syslog if we're daemonized; stderr * otherwise. */ static void devdlog(int priority, const char* fmt, ...) { va_list argp; va_start(argp, fmt); if (no_daemon) vfprintf(stderr, fmt, argp); else if ((! quiet_mode) || (priority <= LOG_WARNING)) vsyslog(priority, fmt, argp); va_end(argp); } static void usage() { fprintf(stderr, "usage: %s [-dnq] [-l connlimit] [-f file]\n", getprogname()); exit(1); } static void check_devd_enabled() { int val = 0; size_t len; len = sizeof(val); if (sysctlbyname(SYSCTL, &val, &len, NULL, 0) != 0) errx(1, "devctl sysctl missing from kernel!"); if (val == 0) { warnx("Setting " SYSCTL " to 1000"); val = 1000; sysctlbyname(SYSCTL, NULL, NULL, &val, sizeof(val)); } } /* * main */ int main(int argc, char **argv) { int ch; check_devd_enabled(); while ((ch = getopt(argc, argv, "df:l:nq")) != -1) { switch (ch) { case 'd': no_daemon = 1; break; case 'f': configfile = optarg; break; case 'l': max_clients = MAX(1, strtoul(optarg, NULL, 0)); break; case 'n': daemonize_quick = 1; break; case 'q': quiet_mode = 1; break; default: usage(); } } cfg.parse(); if (!no_daemon && daemonize_quick) { cfg.open_pidfile(); daemon(0, 0); cfg.write_pidfile(); } signal(SIGPIPE, SIG_IGN); signal(SIGHUP, gensighand); signal(SIGINT, gensighand); signal(SIGTERM, gensighand); signal(SIGINFO, siginfohand); event_loop(); return (0); } Index: user/ngie/more-tests2/sbin/geom/class/multipath/geom_multipath.c =================================================================== --- user/ngie/more-tests2/sbin/geom/class/multipath/geom_multipath.c (revision 292053) +++ user/ngie/more-tests2/sbin/geom/class/multipath/geom_multipath.c (revision 292054) @@ -1,322 +1,323 @@ /*- * Copyright (c) 2006 Mathew Jacob * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include "core/geom.h" #include "misc/subr.h" uint32_t lib_version = G_LIB_VERSION; uint32_t version = G_MULTIPATH_VERSION; static void mp_main(struct gctl_req *, unsigned int); static void mp_label(struct gctl_req *); static void mp_clear(struct gctl_req *); static void mp_prefer(struct gctl_req *); struct g_command class_commands[] = { { "create", G_FLAG_VERBOSE | G_FLAG_LOADKLD, NULL, { { 'A', "active_active", NULL, G_TYPE_BOOL }, { 'R', "active_read", NULL, G_TYPE_BOOL }, G_OPT_SENTINEL }, "[-vAR] name prov ..." }, { "label", G_FLAG_VERBOSE | G_FLAG_LOADKLD, mp_main, { { 'A', "active_active", NULL, G_TYPE_BOOL }, { 'R', "active_read", NULL, G_TYPE_BOOL }, G_OPT_SENTINEL }, "[-vAR] name prov ..." }, { "configure", G_FLAG_VERBOSE, NULL, { { 'A', "active_active", NULL, G_TYPE_BOOL }, { 'P', "active_passive", NULL, G_TYPE_BOOL }, { 'R', "active_read", NULL, G_TYPE_BOOL }, G_OPT_SENTINEL }, "[-vAPR] name" }, { "add", G_FLAG_VERBOSE, NULL, G_NULL_OPTS, "[-v] name prov" }, { "remove", G_FLAG_VERBOSE, NULL, G_NULL_OPTS, "[-v] name prov" }, { "prefer", G_FLAG_VERBOSE, mp_main, G_NULL_OPTS, "[-v] prov ..." }, { "fail", G_FLAG_VERBOSE, NULL, G_NULL_OPTS, "[-v] name prov" }, { "restore", G_FLAG_VERBOSE, NULL, G_NULL_OPTS, "[-v] name prov" }, { "rotate", G_FLAG_VERBOSE, NULL, G_NULL_OPTS, "[-v] name" }, { "getactive", G_FLAG_VERBOSE, NULL, G_NULL_OPTS, "[-v] name" }, { "destroy", G_FLAG_VERBOSE, NULL, G_NULL_OPTS, "[-v] name" }, { "stop", G_FLAG_VERBOSE, NULL, G_NULL_OPTS, "[-v] name" }, { "clear", G_FLAG_VERBOSE, mp_main, G_NULL_OPTS, "[-v] prov ..." }, G_CMD_SENTINEL }; static void mp_main(struct gctl_req *req, unsigned int flags __unused) { const char *name; name = gctl_get_ascii(req, "verb"); if (name == NULL) { gctl_error(req, "No '%s' argument.", "verb"); return; } if (strcmp(name, "label") == 0) { mp_label(req); } else if (strcmp(name, "clear") == 0) { mp_clear(req); } else if (strcmp(name, "prefer") == 0) { mp_prefer(req); } else { gctl_error(req, "Unknown command: %s.", name); } } static void mp_label(struct gctl_req *req) { struct g_multipath_metadata md; off_t disksize = 0, msize; uint8_t *sector, *rsector; char *ptr; uuid_t uuid; ssize_t secsize = 0, ssize; uint32_t status; const char *name, *name2, *mpname; int error, i, nargs, fd; nargs = gctl_get_int(req, "nargs"); if (nargs < 2) { gctl_error(req, "wrong number of arguments."); return; } /* * First, check each provider to make sure it's the same size. * This also gets us our size and sectorsize for the metadata. */ for (i = 1; i < nargs; i++) { name = gctl_get_ascii(req, "arg%d", i); msize = g_get_mediasize(name); ssize = g_get_sectorsize(name); if (msize == 0 || ssize == 0) { gctl_error(req, "cannot get information about %s: %s.", name, strerror(errno)); return; } if (i == 1) { secsize = ssize; disksize = msize; } else { if (secsize != ssize) { gctl_error(req, "%s sector size %ju different.", name, (intmax_t)ssize); return; } if (disksize != msize) { gctl_error(req, "%s media size %ju different.", name, (intmax_t)msize); return; } } } /* * Generate metadata. */ strlcpy(md.md_magic, G_MULTIPATH_MAGIC, sizeof(md.md_magic)); md.md_version = G_MULTIPATH_VERSION; mpname = gctl_get_ascii(req, "arg0"); strlcpy(md.md_name, mpname, sizeof(md.md_name)); md.md_size = disksize; md.md_sectorsize = secsize; uuid_create(&uuid, &status); if (status != uuid_s_ok) { gctl_error(req, "cannot create a UUID."); return; } uuid_to_string(&uuid, &ptr, &status); if (status != uuid_s_ok) { gctl_error(req, "cannot stringify a UUID."); return; } strlcpy(md.md_uuid, ptr, sizeof (md.md_uuid)); md.md_active_active = gctl_get_int(req, "active_active"); if (gctl_get_int(req, "active_read")) md.md_active_active = 2; free(ptr); /* * Allocate a sector to write as metadata. */ - sector = malloc(secsize); + sector = calloc(1, secsize); if (sector == NULL) { gctl_error(req, "unable to allocate metadata buffer"); return; } - memset(sector, 0, secsize); rsector = malloc(secsize); if (rsector == NULL) { - free(sector); gctl_error(req, "unable to allocate metadata buffer"); - return; + goto done; } /* * encode the metadata */ multipath_metadata_encode(&md, sector); /* * Store metadata on the initial provider. */ name = gctl_get_ascii(req, "arg1"); error = g_metadata_store(name, sector, secsize); if (error != 0) { gctl_error(req, "cannot store metadata on %s: %s.", name, strerror(error)); - return; + goto done; } /* * Now touch the rest of the providers to hint retaste. */ for (i = 2; i < nargs; i++) { name2 = gctl_get_ascii(req, "arg%d", i); fd = g_open(name2, 1); if (fd < 0) { fprintf(stderr, "Unable to open %s: %s.\n", name2, strerror(errno)); continue; } if (pread(fd, rsector, secsize, disksize - secsize) != (ssize_t)secsize) { fprintf(stderr, "Unable to read metadata from %s: %s.\n", name2, strerror(errno)); g_close(fd); continue; } g_close(fd); if (memcmp(sector, rsector, secsize)) { fprintf(stderr, "No metadata found on %s." " It is not a path of %s.\n", name2, name); } } +done: + free(rsector); + free(sector); } static void mp_clear(struct gctl_req *req) { const char *name; int error, i, nargs; nargs = gctl_get_int(req, "nargs"); if (nargs < 1) { gctl_error(req, "Too few arguments."); return; } for (i = 0; i < nargs; i++) { name = gctl_get_ascii(req, "arg%d", i); error = g_metadata_clear(name, G_MULTIPATH_MAGIC); if (error != 0) { fprintf(stderr, "Can't clear metadata on %s: %s.\n", name, strerror(error)); gctl_error(req, "Not fully done."); continue; } } } static void mp_prefer(struct gctl_req *req) { const char *name, *comp, *errstr; int nargs; nargs = gctl_get_int(req, "nargs"); if (nargs != 2) { gctl_error(req, "Usage: prefer GEOM PROVIDER"); return; } name = gctl_get_ascii(req, "arg0"); comp = gctl_get_ascii(req, "arg1"); errstr = gctl_issue (req); if (errstr != NULL) { fprintf(stderr, "Can't set %s preferred provider to %s: %s.\n", name, comp, errstr); } } Index: user/ngie/more-tests2/sbin/sysctl/sysctl.8 =================================================================== --- user/ngie/more-tests2/sbin/sysctl/sysctl.8 (revision 292053) +++ user/ngie/more-tests2/sbin/sysctl/sysctl.8 (revision 292054) @@ -1,324 +1,326 @@ .\" Copyright (c) 1993 .\" The Regents of the University of California. All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" 4. Neither the name of the University nor the names of its contributors .\" may be used to endorse or promote products derived from this software .\" without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" From: @(#)sysctl.8 8.1 (Berkeley) 6/6/93 .\" $FreeBSD$ .\" -.Dd February 12, 2015 +.Dd December 10, 2015 .Dt SYSCTL 8 .Os .Sh NAME .Nm sysctl .Nd get or set kernel state .Sh SYNOPSIS .Nm -.Op Fl bdehiNnoRTqx +.Op Fl bdehiNnoRTtqx .Op Fl B Ar bufsize .Op Fl f Ar filename .Ar name Ns Op = Ns Ar value .Ar ... .Nm -.Op Fl bdehNnoRTqx +.Op Fl bdehNnoRTtqx .Op Fl B Ar bufsize .Fl a .Sh DESCRIPTION The .Nm utility retrieves kernel state and allows processes with appropriate privilege to set kernel state. The state to be retrieved or set is described using a .Dq Management Information Base .Pq Dq MIB style name, described as a dotted set of components. .Pp The following options are available: .Bl -tag -width indent .It Fl A Equivalent to .Fl o a (for compatibility). .It Fl a List all the currently available non-opaque values. This option is ignored if one or more variable names are specified on the command line. .It Fl b Force the value of the variable(s) to be output in raw, binary format. No names are printed and no terminating newlines are output. This is mostly useful with a single variable. .It Fl B Ar bufsize Set the buffer size to read from the .Nm to .Ar bufsize . This is necessary for a .Nm that has variable length, and the probe value of 0 is a valid length, such as .Va kern.arandom . .It Fl d Print the description of the variable instead of its value. .It Fl e Separate the name and the value of the variable(s) with .Ql = . This is useful for producing output which can be fed back to the .Nm utility. This option is ignored if either .Fl N or .Fl n is specified, or a variable is being set. .It Fl f Ar filename Specify a file which contains a pair of name and value in each line. .Nm reads and processes the specified file first and then processes the name and value pairs in the command line argument. .It Fl h Format output for human, rather than machine, readability. .It Fl i Ignore unknown OIDs. The purpose is to make use of .Nm for collecting data from a variety of machines (not all of which are necessarily running exactly the same software) easier. .It Fl N Show only variable names, not their values. This is particularly useful with shells that offer programmable completion. To enable completion of variable names in .Xr zsh 1 Pq Pa ports/shells/zsh , use the following code: .Bd -literal -offset indent listsysctls () { set -A reply $(sysctl -AN ${1%.*}) } compctl -K listsysctls sysctl .Ed .Pp To enable completion of variable names in .Xr tcsh 1 , use: .Pp .Dl "complete sysctl 'n/*/`sysctl -Na`/'" .It Fl n Show only variable values, not their names. This option is useful for setting shell variables. For instance, to save the pagesize in variable .Va psize , use: .Pp .Dl "set psize=`sysctl -n hw.pagesize`" .It Fl o Show opaque variables (which are normally suppressed). The format and length are printed, as well as a hex dump of the first sixteen bytes of the value. .It Fl q Suppress some warnings generated by .Nm to standard error. .It Fl T Display only variables that are settable via loader (CTLFLAG_TUN). +.It Fl t +Print the type of the variable. .It Fl W Display only writable variables that are not statistical. Useful for determining the set of runtime tunable sysctls. .It Fl X Equivalent to .Fl x a (for compatibility). .It Fl x As .Fl o , but prints a hex dump of the entire value instead of just the first few bytes. .El .Pp The information available from .Nm consists of integers, strings, and opaque types. The .Nm utility only knows about a couple of opaque types, and will resort to hexdumps for the rest. The opaque information is much more useful if retrieved by special purpose programs such as .Xr ps 1 , .Xr systat 1 , and .Xr netstat 1 . .Pp Some of the variables which cannot be modified during normal system operation can be initialized via .Xr loader 8 tunables. This can for example be done by setting them in .Xr loader.conf 5 . Please refer to .Xr loader.conf 5 for more information on which tunables are available and how to set them. .Pp The string and integer information is summarized below. For a detailed description of these variable see .Xr sysctl 3 . .Pp The changeable column indicates whether a process with appropriate privilege can change the value. String and integer values can be set using .Nm . .Bl -column security.bsd.unprivileged_read_msgbuf integerxxx .It Sy "Name Type Changeable" .It "kern.ostype string no" .It "kern.osrelease string no" .It "kern.osrevision integer no" .It "kern.version string no" .It "kern.maxvnodes integer yes" .It "kern.maxproc integer no" .It "kern.maxprocperuid integer yes" .It "kern.maxfiles integer yes" .It "kern.maxfilesperproc integer yes" .It "kern.argmax integer no" .It "kern.securelevel integer raise only" .It "kern.hostname string yes" .It "kern.hostid integer yes" .It "kern.clockrate struct no" .It "kern.posix1version integer no" .It "kern.ngroups integer no" .It "kern.job_control integer no" .It "kern.saved_ids integer no" .It "kern.boottime struct no" .It "kern.domainname string yes" .It "kern.filedelay integer yes" .It "kern.dirdelay integer yes" .It "kern.metadelay integer yes" .It "kern.osreldate string no" .It "kern.bootfile string yes" .It "kern.corefile string yes" .It "kern.logsigexit integer yes" .It "security.bsd.suser_enabled integer yes" .It "security.bsd.see_other_uids integer yes" .It "security.bsd.unprivileged_proc_debug integer yes" .It "security.bsd.unprivileged_read_msgbuf integer yes" .It "vm.loadavg struct no" .It "hw.machine string no" .It "hw.model string no" .It "hw.ncpu integer no" .It "hw.byteorder integer no" .It "hw.physmem integer no" .It "hw.usermem integer no" .It "hw.pagesize integer no" .It "hw.floatingpoint integer no" .It "hw.machine_arch string no" .It "hw.realmem integer no" .It "machdep.adjkerntz integer yes" .It "machdep.disable_rtc_set integer yes" .It "machdep.guessed_bootdev string no" .It "user.cs_path string no" .It "user.bc_base_max integer no" .It "user.bc_dim_max integer no" .It "user.bc_scale_max integer no" .It "user.bc_string_max integer no" .It "user.coll_weights_max integer no" .It "user.expr_nest_max integer no" .It "user.line_max integer no" .It "user.re_dup_max integer no" .It "user.posix2_version integer no" .It "user.posix2_c_bind integer no" .It "user.posix2_c_dev integer no" .It "user.posix2_char_term integer no" .It "user.posix2_fort_dev integer no" .It "user.posix2_fort_run integer no" .It "user.posix2_localedef integer no" .It "user.posix2_sw_dev integer no" .It "user.posix2_upe integer no" .It "user.stream_max integer no" .It "user.tzname_max integer no" .El .Sh FILES .Bl -tag -width ".In netinet/icmp_var.h" -compact .It In sys/sysctl.h definitions for top level identifiers, second level kernel and hardware identifiers, and user level identifiers .It In sys/socket.h definitions for second level network identifiers .It In sys/gmon.h definitions for third level profiling identifiers .It In vm/vm_param.h definitions for second level virtual memory identifiers .It In netinet/in.h definitions for third level Internet identifiers and fourth level IP identifiers .It In netinet/icmp_var.h definitions for fourth level ICMP identifiers .It In netinet/udp_var.h definitions for fourth level UDP identifiers .El .Sh EXAMPLES For example, to retrieve the maximum number of processes allowed in the system, one would use the following request: .Pp .Dl "sysctl kern.maxproc" .Pp To set the maximum number of processes allowed per uid to 1000, one would use the following request: .Pp .Dl "sysctl kern.maxprocperuid=1000" .Pp Information about the system clock rate may be obtained with: .Pp .Dl "sysctl kern.clockrate" .Pp Information about the load average history may be obtained with: .Pp .Dl "sysctl vm.loadavg" .Pp More variables than these exist, and the best and likely only place to search for their deeper meaning is undoubtedly the source where they are defined. .Sh COMPATIBILITY The .Fl w option has been deprecated and is silently ignored. .Sh SEE ALSO .Xr sysctl 3 , .Xr loader.conf 5 , .Xr sysctl.conf 5 , .Xr loader 8 .Sh HISTORY A .Nm utility first appeared in .Bx 4.4 . .Pp In .Fx 2.2 , .Nm was significantly remodeled. .Sh BUGS The .Nm utility presently exploits an undocumented interface to the kernel sysctl facility to traverse the sysctl tree and to retrieve format and name information. This correct interface is being thought about for the time being. Index: user/ngie/more-tests2/sbin/sysctl/sysctl.c =================================================================== --- user/ngie/more-tests2/sbin/sysctl/sysctl.c (revision 292053) +++ user/ngie/more-tests2/sbin/sysctl/sysctl.c (revision 292054) @@ -1,1123 +1,1139 @@ /* * Copyright (c) 1993 * The Regents of the University of California. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #ifndef lint static const char copyright[] = "@(#) Copyright (c) 1993\n\ The Regents of the University of California. All rights reserved.\n"; #endif /* not lint */ #ifndef lint #if 0 static char sccsid[] = "@(#)from: sysctl.c 8.1 (Berkeley) 6/6/93"; #endif static const char rcsid[] = "$FreeBSD$"; #endif /* not lint */ #include #include #include #include #include #include #ifdef __amd64__ #include #include #endif #if defined(__amd64__) || defined(__i386__) #include #endif #include #include #include #include #include #include #include #include #include #include #include static const char *conffile; static int aflag, bflag, Bflag, dflag, eflag, hflag, iflag; -static int Nflag, nflag, oflag, qflag, Tflag, Wflag, xflag; +static int Nflag, nflag, oflag, qflag, tflag, Tflag, Wflag, xflag; static int oidfmt(int *, int, char *, u_int *); static int parsefile(const char *); static int parse(const char *, int); static int show_var(int *, int); static int sysctl_all(int *oid, int len); static int name2oid(const char *, int *); static int strIKtoi(const char *, char **, const char *); static int ctl_sign[CTLTYPE+1] = { [CTLTYPE_INT] = 1, [CTLTYPE_LONG] = 1, [CTLTYPE_S8] = 1, [CTLTYPE_S16] = 1, [CTLTYPE_S32] = 1, [CTLTYPE_S64] = 1, }; static int ctl_size[CTLTYPE+1] = { [CTLTYPE_INT] = sizeof(int), [CTLTYPE_UINT] = sizeof(u_int), [CTLTYPE_LONG] = sizeof(long), [CTLTYPE_ULONG] = sizeof(u_long), [CTLTYPE_S8] = sizeof(int8_t), [CTLTYPE_S16] = sizeof(int16_t), [CTLTYPE_S32] = sizeof(int32_t), [CTLTYPE_S64] = sizeof(int64_t), [CTLTYPE_U8] = sizeof(uint8_t), [CTLTYPE_U16] = sizeof(uint16_t), [CTLTYPE_U32] = sizeof(uint32_t), [CTLTYPE_U64] = sizeof(uint64_t), }; static const char *ctl_typename[CTLTYPE+1] = { [CTLTYPE_INT] = "integer", [CTLTYPE_UINT] = "unsigned integer", [CTLTYPE_LONG] = "long integer", [CTLTYPE_ULONG] = "unsigned long", [CTLTYPE_U8] = "uint8_t", [CTLTYPE_U16] = "uint16_t", [CTLTYPE_U32] = "uint16_t", [CTLTYPE_U64] = "uint64_t", [CTLTYPE_S8] = "int8_t", [CTLTYPE_S16] = "int16_t", [CTLTYPE_S32] = "int32_t", [CTLTYPE_S64] = "int64_t", + [CTLTYPE_NODE] = "node", + [CTLTYPE_STRING] = "string", + [CTLTYPE_OPAQUE] = "opaque", }; static void usage(void) { (void)fprintf(stderr, "%s\n%s\n", - "usage: sysctl [-bdehiNnoqTWx] [ -B ] [-f filename] name[=value] ...", - " sysctl [-bdehNnoqTWx] [ -B ] -a"); + "usage: sysctl [-bdehiNnoqTtWx] [ -B ] [-f filename] name[=value] ...", + " sysctl [-bdehNnoqTtWx] [ -B ] -a"); exit(1); } int main(int argc, char **argv) { int ch; int warncount = 0; setlocale(LC_NUMERIC, ""); setbuf(stdout,0); setbuf(stderr,0); - while ((ch = getopt(argc, argv, "AabB:def:hiNnoqTwWxX")) != -1) { + while ((ch = getopt(argc, argv, "AabB:def:hiNnoqtTwWxX")) != -1) { switch (ch) { case 'A': /* compatibility */ aflag = oflag = 1; break; case 'a': aflag = 1; break; case 'b': bflag = 1; break; case 'B': Bflag = strtol(optarg, NULL, 0); break; case 'd': dflag = 1; break; case 'e': eflag = 1; break; case 'f': conffile = optarg; break; case 'h': hflag = 1; break; case 'i': iflag = 1; break; case 'N': Nflag = 1; break; case 'n': nflag = 1; break; case 'o': oflag = 1; break; case 'q': qflag = 1; break; + case 't': + tflag = 1; + break; case 'T': Tflag = 1; break; case 'w': /* compatibility */ /* ignored */ break; case 'W': Wflag = 1; break; case 'X': /* compatibility */ aflag = xflag = 1; break; case 'x': xflag = 1; break; default: usage(); } } argc -= optind; argv += optind; if (Nflag && nflag) usage(); if (aflag && argc == 0) exit(sysctl_all(0, 0)); if (argc == 0 && conffile == NULL) usage(); warncount = 0; if (conffile != NULL) warncount += parsefile(conffile); while (argc-- > 0) warncount += parse(*argv++, 0); return (warncount); } /* * Parse a name into a MIB entry. * Lookup and print out the MIB entry if it exists. * Set a new value if requested. */ static int parse(const char *string, int lineno) { int len, i, j; const void *newval; const char *newvalstr = NULL; int8_t i8val; uint8_t u8val; int16_t i16val; uint16_t u16val; int32_t i32val; uint32_t u32val; int intval; unsigned int uintval; long longval; unsigned long ulongval; size_t newsize = Bflag; int64_t i64val; uint64_t u64val; int mib[CTL_MAXNAME]; char *cp, *bufp, buf[BUFSIZ], *endptr = NULL, fmt[BUFSIZ], line[BUFSIZ]; u_int kind; if (lineno) snprintf(line, sizeof(line), " at line %d", lineno); else line[0] = '\0'; cp = buf; if (snprintf(buf, BUFSIZ, "%s", string) >= BUFSIZ) { warnx("oid too long: '%s'%s", string, line); return (1); } bufp = strsep(&cp, "=:"); if (cp != NULL) { /* Tflag just lists tunables, do not allow assignment */ if (Tflag || Wflag) { warnx("Can't set variables when using -T or -W"); usage(); } while (isspace(*cp)) cp++; /* Strip a pair of " or ' if any. */ switch (*cp) { case '\"': case '\'': if (cp[strlen(cp) - 1] == *cp) cp[strlen(cp) - 1] = '\0'; cp++; } newvalstr = cp; newsize = strlen(cp); } /* Trim spaces */ cp = bufp + strlen(bufp) - 1; while (cp >= bufp && isspace((int)*cp)) { *cp = '\0'; cp--; } len = name2oid(bufp, mib); if (len < 0) { if (iflag) return (0); if (qflag) return (1); else { if (errno == ENOENT) { warnx("unknown oid '%s'%s", bufp, line); } else { warn("unknown oid '%s'%s", bufp, line); } return (1); } } if (oidfmt(mib, len, fmt, &kind)) { warn("couldn't find format of oid '%s'%s", bufp, line); if (iflag) return (1); else exit(1); } if (newvalstr == NULL || dflag) { if ((kind & CTLTYPE) == CTLTYPE_NODE) { if (dflag) { i = show_var(mib, len); if (!i && !bflag) putchar('\n'); } sysctl_all(mib, len); } else { i = show_var(mib, len); if (!i && !bflag) putchar('\n'); } } else { if ((kind & CTLTYPE) == CTLTYPE_NODE) { warnx("oid '%s' isn't a leaf node%s", bufp, line); return (1); } if (!(kind & CTLFLAG_WR)) { if (kind & CTLFLAG_TUN) { warnx("oid '%s' is a read only tunable%s", bufp, line); warnx("Tunable values are set in /boot/loader.conf"); } else warnx("oid '%s' is read only%s", bufp, line); return (1); } switch (kind & CTLTYPE) { case CTLTYPE_INT: case CTLTYPE_UINT: case CTLTYPE_LONG: case CTLTYPE_ULONG: case CTLTYPE_S8: case CTLTYPE_S16: case CTLTYPE_S32: case CTLTYPE_S64: case CTLTYPE_U8: case CTLTYPE_U16: case CTLTYPE_U32: case CTLTYPE_U64: if (strlen(newvalstr) == 0) { warnx("empty numeric value"); return (1); } /* FALLTHROUGH */ case CTLTYPE_STRING: break; default: warnx("oid '%s' is type %d," " cannot set that%s", bufp, kind & CTLTYPE, line); return (1); } errno = 0; switch (kind & CTLTYPE) { case CTLTYPE_INT: if (strncmp(fmt, "IK", 2) == 0) intval = strIKtoi(newvalstr, &endptr, fmt); else intval = (int)strtol(newvalstr, &endptr, 0); newval = &intval; newsize = sizeof(intval); break; case CTLTYPE_UINT: uintval = (int) strtoul(newvalstr, &endptr, 0); newval = &uintval; newsize = sizeof(uintval); break; case CTLTYPE_LONG: longval = strtol(newvalstr, &endptr, 0); newval = &longval; newsize = sizeof(longval); break; case CTLTYPE_ULONG: ulongval = strtoul(newvalstr, &endptr, 0); newval = &ulongval; newsize = sizeof(ulongval); break; case CTLTYPE_STRING: newval = newvalstr; break; case CTLTYPE_S8: i8val = (int8_t)strtol(newvalstr, &endptr, 0); newval = &i8val; newsize = sizeof(i8val); break; case CTLTYPE_S16: i16val = (int16_t)strtol(newvalstr, &endptr, 0); newval = &i16val; newsize = sizeof(i16val); break; case CTLTYPE_S32: i32val = (int32_t)strtol(newvalstr, &endptr, 0); newval = &i32val; newsize = sizeof(i32val); break; case CTLTYPE_S64: i64val = strtoimax(newvalstr, &endptr, 0); newval = &i64val; newsize = sizeof(i64val); break; case CTLTYPE_U8: u8val = (uint8_t)strtoul(newvalstr, &endptr, 0); newval = &u8val; newsize = sizeof(u8val); break; case CTLTYPE_U16: u16val = (uint16_t)strtoul(newvalstr, &endptr, 0); newval = &u16val; newsize = sizeof(u16val); break; case CTLTYPE_U32: u32val = (uint32_t)strtoul(newvalstr, &endptr, 0); newval = &u32val; newsize = sizeof(u32val); break; case CTLTYPE_U64: u64val = strtoumax(newvalstr, &endptr, 0); newval = &u64val; newsize = sizeof(u64val); break; default: /* NOTREACHED */ abort(); } if (errno != 0 || endptr == newvalstr || (endptr != NULL && *endptr != '\0')) { warnx("invalid %s '%s'%s", ctl_typename[kind & CTLTYPE], newvalstr, line); return (1); } i = show_var(mib, len); if (sysctl(mib, len, 0, 0, newval, newsize) == -1) { if (!i && !bflag) putchar('\n'); switch (errno) { case EOPNOTSUPP: warnx("%s: value is not available%s", string, line); return (1); case ENOTDIR: warnx("%s: specification is incomplete%s", string, line); return (1); case ENOMEM: warnx("%s: type is unknown to this program%s", string, line); return (1); default: warn("%s%s", string, line); return (1); } } if (!bflag) printf(" -> "); i = nflag; nflag = 1; j = show_var(mib, len); if (!j && !bflag) putchar('\n'); nflag = i; } return (0); } static int parsefile(const char *filename) { FILE *file; char line[BUFSIZ], *p, *pq, *pdq; int warncount = 0, lineno = 0; file = fopen(filename, "r"); if (file == NULL) err(EX_NOINPUT, "%s", filename); while (fgets(line, sizeof(line), file) != NULL) { lineno++; p = line; pq = strchr(line, '\''); pdq = strchr(line, '\"'); /* Replace the first # with \0. */ while((p = strchr(p, '#')) != NULL) { if (pq != NULL && p > pq) { if ((p = strchr(pq+1, '\'')) != NULL) *(++p) = '\0'; break; } else if (pdq != NULL && p > pdq) { if ((p = strchr(pdq+1, '\"')) != NULL) *(++p) = '\0'; break; } else if (p == line || *(p-1) != '\\') { *p = '\0'; break; } p++; } /* Trim spaces */ p = line + strlen(line) - 1; while (p >= line && isspace((int)*p)) { *p = '\0'; p--; } p = line; while (isspace((int)*p)) p++; if (*p == '\0') continue; else warncount += parse(p, lineno); } fclose(file); return (warncount); } /* These functions will dump out various interesting structures. */ static int S_clockinfo(size_t l2, void *p) { struct clockinfo *ci = (struct clockinfo*)p; if (l2 != sizeof(*ci)) { warnx("S_clockinfo %zu != %zu", l2, sizeof(*ci)); return (1); } printf(hflag ? "{ hz = %'d, tick = %'d, profhz = %'d, stathz = %'d }" : "{ hz = %d, tick = %d, profhz = %d, stathz = %d }", ci->hz, ci->tick, ci->profhz, ci->stathz); return (0); } static int S_loadavg(size_t l2, void *p) { struct loadavg *tv = (struct loadavg*)p; if (l2 != sizeof(*tv)) { warnx("S_loadavg %zu != %zu", l2, sizeof(*tv)); return (1); } printf(hflag ? "{ %'.2f %'.2f %'.2f }" : "{ %.2f %.2f %.2f }", (double)tv->ldavg[0]/(double)tv->fscale, (double)tv->ldavg[1]/(double)tv->fscale, (double)tv->ldavg[2]/(double)tv->fscale); return (0); } static int S_timeval(size_t l2, void *p) { struct timeval *tv = (struct timeval*)p; time_t tv_sec; char *p1, *p2; if (l2 != sizeof(*tv)) { warnx("S_timeval %zu != %zu", l2, sizeof(*tv)); return (1); } printf(hflag ? "{ sec = %'jd, usec = %'ld } " : "{ sec = %jd, usec = %ld } ", (intmax_t)tv->tv_sec, tv->tv_usec); tv_sec = tv->tv_sec; p1 = strdup(ctime(&tv_sec)); for (p2=p1; *p2 ; p2++) if (*p2 == '\n') *p2 = '\0'; fputs(p1, stdout); free(p1); return (0); } static int S_vmtotal(size_t l2, void *p) { struct vmtotal *v = (struct vmtotal *)p; int pageKilo = getpagesize() / 1024; if (l2 != sizeof(*v)) { warnx("S_vmtotal %zu != %zu", l2, sizeof(*v)); return (1); } printf( "\nSystem wide totals computed every five seconds:" " (values in kilobytes)\n"); printf("===============================================\n"); printf( "Processes:\t\t(RUNQ: %hd Disk Wait: %hd Page Wait: " "%hd Sleep: %hd)\n", v->t_rq, v->t_dw, v->t_pw, v->t_sl); printf( "Virtual Memory:\t\t(Total: %dK Active: %dK)\n", v->t_vm * pageKilo, v->t_avm * pageKilo); printf("Real Memory:\t\t(Total: %dK Active: %dK)\n", v->t_rm * pageKilo, v->t_arm * pageKilo); printf("Shared Virtual Memory:\t(Total: %dK Active: %dK)\n", v->t_vmshr * pageKilo, v->t_avmshr * pageKilo); printf("Shared Real Memory:\t(Total: %dK Active: %dK)\n", v->t_rmshr * pageKilo, v->t_armshr * pageKilo); printf("Free Memory:\t%dK", v->t_free * pageKilo); return (0); } #ifdef __amd64__ #define efi_next_descriptor(ptr, size) \ ((struct efi_md *)(((uint8_t *) ptr) + size)) static int S_efi_map(size_t l2, void *p) { struct efi_map_header *efihdr; struct efi_md *map; const char *type; size_t efisz; int ndesc, i; static const char *types[] = { "Reserved", "LoaderCode", "LoaderData", "BootServicesCode", "BootServicesData", "RuntimeServicesCode", "RuntimeServicesData", "ConventionalMemory", "UnusableMemory", "ACPIReclaimMemory", "ACPIMemoryNVS", "MemoryMappedIO", "MemoryMappedIOPortSpace", "PalCode" }; /* * Memory map data provided by UEFI via the GetMemoryMap * Boot Services API. */ if (l2 < sizeof(*efihdr)) { warnx("S_efi_map length less than header"); return (1); } efihdr = p; efisz = (sizeof(struct efi_map_header) + 0xf) & ~0xf; map = (struct efi_md *)((uint8_t *)efihdr + efisz); if (efihdr->descriptor_size == 0) return (0); if (l2 != efisz + efihdr->memory_size) { warnx("S_efi_map length mismatch %zu vs %zu", l2, efisz + efihdr->memory_size); return (1); } ndesc = efihdr->memory_size / efihdr->descriptor_size; printf("\n%23s %12s %12s %8s %4s", "Type", "Physical", "Virtual", "#Pages", "Attr"); for (i = 0; i < ndesc; i++, map = efi_next_descriptor(map, efihdr->descriptor_size)) { if (map->md_type <= EFI_MD_TYPE_PALCODE) type = types[map->md_type]; else type = ""; printf("\n%23s %012lx %12p %08lx ", type, map->md_phys, map->md_virt, map->md_pages); if (map->md_attr & EFI_MD_ATTR_UC) printf("UC "); if (map->md_attr & EFI_MD_ATTR_WC) printf("WC "); if (map->md_attr & EFI_MD_ATTR_WT) printf("WT "); if (map->md_attr & EFI_MD_ATTR_WB) printf("WB "); if (map->md_attr & EFI_MD_ATTR_UCE) printf("UCE "); if (map->md_attr & EFI_MD_ATTR_WP) printf("WP "); if (map->md_attr & EFI_MD_ATTR_RP) printf("RP "); if (map->md_attr & EFI_MD_ATTR_XP) printf("XP "); if (map->md_attr & EFI_MD_ATTR_RT) printf("RUNTIME"); } return (0); } #endif #if defined(__amd64__) || defined(__i386__) static int S_bios_smap_xattr(size_t l2, void *p) { struct bios_smap_xattr *smap, *end; if (l2 % sizeof(*smap) != 0) { warnx("S_bios_smap_xattr %zu is not a multiple of %zu", l2, sizeof(*smap)); return (1); } end = (struct bios_smap_xattr *)((char *)p + l2); for (smap = p; smap < end; smap++) printf("\nSMAP type=%02x, xattr=%02x, base=%016jx, len=%016jx", smap->type, smap->xattr, (uintmax_t)smap->base, (uintmax_t)smap->length); return (0); } #endif static int strIKtoi(const char *str, char **endptrp, const char *fmt) { int kelv; float temp; size_t len; const char *p; int prec, i; assert(errno == 0); len = strlen(str); /* caller already checked this */ assert(len > 0); /* * A format of "IK" is in deciKelvin. A format of "IK3" is in * milliKelvin. The single digit following IK is log10 of the * multiplying factor to convert Kelvin into the untis of this sysctl, * or the dividing factor to convert the sysctl value to Kelvin. Numbers * larger than 6 will run into precision issues with 32-bit integers. * Characters that aren't ASCII digits after the 'K' are ignored. No * localization is present because this is an interface from the kernel * to this program (eg not an end-user interface), so isdigit() isn't * used here. */ if (fmt[2] != '\0' && fmt[2] >= '0' && fmt[2] <= '9') prec = fmt[2] - '0'; else prec = 1; p = &str[len - 1]; if (*p == 'C' || *p == 'F' || *p == 'K') { temp = strtof(str, endptrp); if (*endptrp != str && *endptrp == p && errno == 0) { if (*p == 'F') temp = (temp - 32) * 5 / 9; *endptrp = NULL; if (*p != 'K') temp += 273.15; for (i = 0; i < prec; i++) temp *= 10.0; return ((int)(temp + 0.5)); } } else { /* No unit specified -> treat it as a raw number */ kelv = (int)strtol(str, endptrp, 10); if (*endptrp != str && *endptrp == p && errno == 0) { *endptrp = NULL; return (kelv); } } errno = ERANGE; return (0); } /* * These functions uses a presently undocumented interface to the kernel * to walk the tree and get the type so it can print the value. * This interface is under work and consideration, and should probably * be killed with a big axe by the first person who can find the time. * (be aware though, that the proper interface isn't as obvious as it * may seem, there are various conflicting requirements. */ static int name2oid(const char *name, int *oidp) { int oid[2]; int i; size_t j; oid[0] = 0; oid[1] = 3; j = CTL_MAXNAME * sizeof(int); i = sysctl(oid, 2, oidp, &j, name, strlen(name)); if (i < 0) return (i); j /= sizeof(int); return (j); } static int oidfmt(int *oid, int len, char *fmt, u_int *kind) { int qoid[CTL_MAXNAME+2]; u_char buf[BUFSIZ]; int i; size_t j; qoid[0] = 0; qoid[1] = 4; memcpy(qoid + 2, oid, len * sizeof(int)); j = sizeof(buf); i = sysctl(qoid, len + 2, buf, &j, 0, 0); if (i) err(1, "sysctl fmt %d %zu %d", i, j, errno); if (kind) *kind = *(u_int *)buf; if (fmt) strcpy(fmt, (char *)(buf + sizeof(u_int))); return (0); } /* * This formats and outputs the value of one variable * * Returns zero if anything was actually output. * Returns one if didn't know what to do with this. * Return minus one if we had errors. */ static int show_var(int *oid, int nlen) { u_char buf[BUFSIZ], *val, *oval, *p; char name[BUFSIZ], fmt[BUFSIZ]; - const char *sep, *sep1; + const char *sep, *sep1, *prntype; int qoid[CTL_MAXNAME+2]; uintmax_t umv; intmax_t mv; int i, hexlen, sign, ctltype; size_t intlen; size_t j, len; u_int kind; float base; int (*func)(size_t, void *); int prec; /* Silence GCC. */ umv = mv = intlen = 0; bzero(buf, BUFSIZ); bzero(fmt, BUFSIZ); bzero(name, BUFSIZ); qoid[0] = 0; memcpy(qoid + 2, oid, nlen * sizeof(int)); qoid[1] = 1; j = sizeof(name); i = sysctl(qoid, nlen + 2, name, &j, 0, 0); if (i || !j) err(1, "sysctl name %d %zu %d", i, j, errno); oidfmt(oid, nlen, fmt, &kind); /* if Wflag then only list sysctls that are writeable and not stats. */ if (Wflag && ((kind & CTLFLAG_WR) == 0 || (kind & CTLFLAG_STATS) != 0)) return 1; /* if Tflag then only list sysctls that are tuneables. */ if (Tflag && (kind & CTLFLAG_TUN) == 0) return 1; if (Nflag) { printf("%s", name); return (0); } if (eflag) sep = "="; else sep = ": "; - if (dflag) { /* just print description */ + ctltype = (kind & CTLTYPE); + if (tflag || dflag) { + if (!nflag) + printf("%s%s", name, sep); + if (ctl_typename[ctltype] != NULL) + prntype = ctl_typename[ctltype]; + else + prntype = "unknown"; + if (tflag && dflag) + printf("%s%s", prntype, sep); + else if (tflag) { + printf("%s", prntype); + return (0); + } qoid[1] = 5; j = sizeof(buf); i = sysctl(qoid, nlen + 2, buf, &j, 0, 0); - if (!nflag) - printf("%s%s", name, sep); printf("%s", buf); return (0); } /* find an estimate of how much we need for this var */ if (Bflag) j = Bflag; else { j = 0; i = sysctl(oid, nlen, 0, &j, 0, 0); j += j; /* we want to be sure :-) */ } val = oval = malloc(j + 1); if (val == NULL) { warnx("malloc failed"); return (1); } - ctltype = (kind & CTLTYPE); len = j; i = sysctl(oid, nlen, val, &len, 0, 0); if (i != 0 || (len == 0 && ctltype != CTLTYPE_STRING)) { free(oval); return (1); } if (bflag) { fwrite(val, 1, len, stdout); free(oval); return (0); } val[len] = '\0'; p = val; sign = ctl_sign[ctltype]; intlen = ctl_size[ctltype]; switch (ctltype) { case CTLTYPE_STRING: if (!nflag) printf("%s%s", name, sep); printf("%.*s", (int)len, p); free(oval); return (0); case CTLTYPE_INT: case CTLTYPE_UINT: case CTLTYPE_LONG: case CTLTYPE_ULONG: case CTLTYPE_S8: case CTLTYPE_S16: case CTLTYPE_S32: case CTLTYPE_S64: case CTLTYPE_U8: case CTLTYPE_U16: case CTLTYPE_U32: case CTLTYPE_U64: if (!nflag) printf("%s%s", name, sep); hexlen = 2 + (intlen * CHAR_BIT + 3) / 4; sep1 = ""; while (len >= intlen) { switch (kind & CTLTYPE) { case CTLTYPE_INT: case CTLTYPE_UINT: umv = *(u_int *)p; mv = *(int *)p; break; case CTLTYPE_LONG: case CTLTYPE_ULONG: umv = *(u_long *)p; mv = *(long *)p; break; case CTLTYPE_S8: case CTLTYPE_U8: umv = *(uint8_t *)p; mv = *(int8_t *)p; break; case CTLTYPE_S16: case CTLTYPE_U16: umv = *(uint16_t *)p; mv = *(int16_t *)p; break; case CTLTYPE_S32: case CTLTYPE_U32: umv = *(uint32_t *)p; mv = *(int32_t *)p; break; case CTLTYPE_S64: case CTLTYPE_U64: umv = *(uint64_t *)p; mv = *(int64_t *)p; break; } fputs(sep1, stdout); if (xflag) printf("%#0*jx", hexlen, umv); else if (!sign) printf(hflag ? "%'ju" : "%ju", umv); else if (fmt[1] == 'K') { if (mv < 0) printf("%jd", mv); else { /* * See strIKtoi for details on fmt. */ prec = 1; if (fmt[2] != '\0') prec = fmt[2] - '0'; base = 1.0; for (int i = 0; i < prec; i++) base *= 10.0; printf("%.*fC", prec, (float)mv / base - 273.15); } } else printf(hflag ? "%'jd" : "%jd", mv); sep1 = " "; len -= intlen; p += intlen; } free(oval); return (0); case CTLTYPE_OPAQUE: i = 0; if (strcmp(fmt, "S,clockinfo") == 0) func = S_clockinfo; else if (strcmp(fmt, "S,timeval") == 0) func = S_timeval; else if (strcmp(fmt, "S,loadavg") == 0) func = S_loadavg; else if (strcmp(fmt, "S,vmtotal") == 0) func = S_vmtotal; #ifdef __amd64__ else if (strcmp(fmt, "S,efi_map_header") == 0) func = S_efi_map; #endif #if defined(__amd64__) || defined(__i386__) else if (strcmp(fmt, "S,bios_smap_xattr") == 0) func = S_bios_smap_xattr; #endif else func = NULL; if (func) { if (!nflag) printf("%s%s", name, sep); i = (*func)(len, p); free(oval); return (i); } /* FALLTHROUGH */ default: if (!oflag && !xflag) { free(oval); return (1); } if (!nflag) printf("%s%s", name, sep); printf("Format:%s Length:%zu Dump:0x", fmt, len); while (len-- && (xflag || p < val + 16)) printf("%02x", *p++); if (!xflag && len > 16) printf("..."); free(oval); return (0); } free(oval); return (1); } static int sysctl_all(int *oid, int len) { int name1[22], name2[22]; int i, j; size_t l1, l2; name1[0] = 0; name1[1] = 2; l1 = 2; if (len) { memcpy(name1+2, oid, len * sizeof(int)); l1 += len; } else { name1[2] = 1; l1++; } for (;;) { l2 = sizeof(name2); j = sysctl(name1, l1, name2, &l2, 0, 0); if (j < 0) { if (errno == ENOENT) return (0); else err(1, "sysctl(getnext) %d %zu", j, l2); } l2 /= sizeof(int); if (len < 0 || l2 < (unsigned int)len) return (0); for (i = 0; i < len; i++) if (name2[i] != oid[i]) return (0); i = show_var(name2, l2); if (!i && !bflag) putchar('\n'); memcpy(name1+2, name2, l2 * sizeof(int)); l1 = 2 + l2; } } Index: user/ngie/more-tests2/sbin =================================================================== --- user/ngie/more-tests2/sbin (revision 292053) +++ user/ngie/more-tests2/sbin (revision 292054) Property changes on: user/ngie/more-tests2/sbin ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,1 ## Merged /head/sbin:r292006-292053 Index: user/ngie/more-tests2/share/man/man4/ioat.4 =================================================================== --- user/ngie/more-tests2/share/man/man4/ioat.4 (revision 292053) +++ user/ngie/more-tests2/share/man/man4/ioat.4 (revision 292054) @@ -1,216 +1,227 @@ .\" Copyright (c) 2015 EMC / Isilon Storage Division .\" All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" $FreeBSD$ .\" -.Dd October 31, 2015 +.Dd December 9, 2015 .Dt IOAT 4 .Os .Sh NAME .Nm I/OAT .Nd Intel I/O Acceleration Technology .Sh SYNOPSIS To compile this driver into your kernel, place the following line in your kernel configuration file: .Bd -ragged -offset indent .Cd "device ioat" .Ed .Pp Or, to load the driver as a module at boot, place the following line in .Xr loader.conf 5 : .Bd -literal -offset indent ioat_load="YES" .Ed .Pp In .Xr loader.conf 5 : .Pp .Cd hw.ioat.force_legacy_interrupts=0 .Pp In .Xr loader.conf 5 or .Xr sysctl.conf 5 : .Pp .Cd hw.ioat.enable_ioat_test=0 .Cd hw.ioat.debug_level=0 (only critical errors; maximum of 3) .Pp .Ft typedef void .Fn (*bus_dmaengine_callback_t) "void *arg" "int error" .Pp .Ft bus_dmaengine_t .Fn ioat_get_dmaengine "uint32_t channel_index" .Ft void .Fn ioat_put_dmaengine "bus_dmaengine_t dmaengine" .Ft void .Fn ioat_acquire "bus_dmaengine_t dmaengine" .Ft void .Fn ioat_release "bus_dmaengine_t dmaengine" .Ft struct bus_dmadesc * .Fo ioat_copy .Fa "bus_dmaengine_t dmaengine" .Fa "bus_addr_t dst" .Fa "bus_addr_t src" .Fa "bus_size_t len" .Fa "bus_dmaengine_callback_t callback_fn" .Fa "void *callback_arg" .Fa "uint32_t flags" .Fc .Ft struct bus_dmadesc * +.Fo ioat_copy_8k_aligned +.Fa "bus_dmaengine_t dmaengine" +.Fa "bus_addr_t dst1" +.Fa "bus_addr_t dst2" +.Fa "bus_addr_t src1" +.Fa "bus_addr_t src2" +.Fa "bus_dmaengine_callback_t callback_fn" +.Fa "void *callback_arg" +.Fa "uint32_t flags" +.Fc +.Ft struct bus_dmadesc * .Fo ioat_blockfill .Fa "bus_dmaengine_t dmaengine" .Fa "bus_addr_t dst" .Fa "uint64_t fillpattern" .Fa "bus_size_t len" .Fa "bus_dmaengine_callback_t callback_fn" .Fa "void *callback_arg" .Fa "uint32_t flags" .Fc .Ft struct bus_dmadesc * .Fo ioat_null .Fa "bus_dmaengine_t dmaengine" .Fa "bus_dmaengine_callback_t callback_fn" .Fa "void *callback_arg" .Fa "uint32_t flags" .Fc .Sh DESCRIPTION The .Nm driver provides a kernel API to a variety of DMA engines on some Intel server platforms. .Pp There is a number of DMA channels per CPU package. (Typically 4 or 8.) Each may be used independently. Operations on a single channel proceed sequentially. .Pp Blockfill operations can be used to write a 64-bit pattern to memory. .Pp Copy operations can be used to offload memory copies to the DMA engines. .Pp Null operations do nothing, but may be used to test the interrupt and callback mechanism. .Pp All operations can optionally trigger an interrupt at completion with the .Ar DMA_EN_INT flag. For example, a user might submit multiple operations to the same channel and only enable an interrupt and callback for the last operation. .Pp All operations are safe to use in a non-blocking context with the .Ar DMA_NO_WAIT flag. (Of course, allocations may fail and operations requested with .Ar DMA_NO_WAIT may return NULL.) .Pp All operations, as well as .Fn ioat_get_dmaengine , can return NULL in special circumstances. For example, if the .Nm driver is being unloaded, or the administrator has induced a hardware reset, or a usage error has resulted in a hardware error state that needs to be recovered from. .Pp It is invalid to attempt to submit new DMA operations in a .Fa bus_dmaengine_callback_t context. .Sh USAGE A typical user will lookup the DMA engine object for a given channel with .Fn ioat_get_dmaengine . When the user wants to offload a copy, they will first .Fn ioat_acquire the .Ar bus_dmaengine_t object for exclusive access to enqueue operations on that channel. Then, they will submit one or more operations using .Fn ioat_blockfill , .Fn ioat_copy , or .Fn ioat_null . -After queueing one or more individual DMA operations, they will +After queuing one or more individual DMA operations, they will .Fn ioat_release the .Ar bus_dmaengine_t to drop their exclusive access to the channel. The routine they provided for the .Fa callback_fn argument will be invoked with the provided .Fa callback_arg when the operation is complete. When they are finished with the .Ar bus_dmaengine_t , the user should .Fn ioat_put_dmaengine . .Pp Users MUST NOT block between .Fn ioat_acquire and .Fn ioat_release . Users SHOULD NOT hold .Ar bus_dmaengine_t references for a very long time to enable fault recovery and kernel module unload. .Pp For an example of usage, see .Pa src/sys/dev/ioat/ioat_test.c . .Sh FILES .Bl -tag -compat .It Pa /dev/ioat_test test device for .Xr ioatcontrol 8 .El .Sh SEE ALSO .Xr ioatcontrol 8 .Sh HISTORY The .Nm driver first appeared in .Fx 11.0 . .Sh AUTHORS The .Nm driver was developed by .An \&Jim Harris Aq Mt jimharris@FreeBSD.org , .An \&Carl Delsey Aq Mt carl.r.delsey@intel.com , and .An \&Conrad Meyer Aq Mt cem@FreeBSD.org . This manual page was written by .An \&Conrad Meyer Aq Mt cem@FreeBSD.org . .Sh CAVEATS Copy operation takes bus addresses as parameters, not virtual addresses. .Pp Buffers for individual copy operations must be physically contiguous. .Pp Copies larger than max transfer size (1MB, but may vary by hardware) are not supported. Future versions will likely support this by breaking up the transfer into smaller sizes. .Sh BUGS The .Nm driver only supports blockfill, copy, and null operations at this time. The driver does not yet support advanced DMA modes, such as XOR, that some I/OAT devices support. Index: user/ngie/more-tests2/share/man/man4/isp.4 =================================================================== --- user/ngie/more-tests2/share/man/man4/isp.4 (revision 292053) +++ user/ngie/more-tests2/share/man/man4/isp.4 (revision 292054) @@ -1,214 +1,210 @@ .\" Copyright (c) 2009-2015 Alexander Motin .\" Copyright (c) 2006 Marcus Alves Grando .\" Copyright (c) 1998-2001 Matthew Jacob, for NASA/Ames Research Center .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" 3. The name of the author may not be used to endorse or promote products .\" derived from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR .\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES .\" OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. .\" IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, .\" INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT .\" NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF .\" THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .\" .\" $FreeBSD$ .\" -.Dd November 22, 2015 +.Dd December 9, 2015 .Dt ISP 4 .Os .Sh NAME .Nm isp .Nd Qlogic based SPI and FibreChannel SCSI Host Adapters .Sh SYNOPSIS To compile this driver into the kernel, place the following lines in your kernel configuration file: .Bd -ragged -offset indent .Cd "device scbus" .Cd "device isp" .Ed .Pp Alternatively, to load the driver as a module at boot time, place the following line in .Xr loader.conf 5 : .Bd -literal -offset indent isp_load="YES" .Ed .Sh DESCRIPTION This driver provides access to .Tn SPI or .Tn FibreChannel SCSI devices. .Pp SPI supports initiator mode for Ultra SCSI and wide mode transactions for .Tn SCSI , Ultra2 LVD (1080, 1280), and Ultra3 LVD (10160, 12160). .Pp Fibre Channel supports initiator and target modes of FCP SCSI profile, utilizing Class 3 and Class 2 (2200 and later) connections. Support is available for Public and Private loops, Point-to-Point and Fabric connections. .Sh FIRMWARE Firmware loading is supported if the .Xr ispfw 4 module is loaded. It is strongly recommended that you use the firmware available from .Xr ispfw 4 as it is the most likely to have been tested with this driver. .Sh HARDWARE Cards supported by the .Nm driver include: .Bl -tag -width xxxxxx -offset indent .It Qlogic 1000 Fast Wide, Ultra Fast Wide cards, Single Ended or Differential SBus cards. .It Qlogic 1020 Fast Wide and Differential Fast Wide SCSI PCI cards. .It Qlogic 1040 Ultra Wide and Differential Ultra Wide SCSI PCI cards. Also known as the DEC KZPBA-CA (single ended) and KZPBA-CB (HVD differential). .It Qlogic 1080 LVD Ultra2 Wide SCSI PCI cards. .It Qlogic 10160 LVD Ultra3 Wide SCSI PCI cards. .It Qlogic 1240 Dual Bus Ultra Wide and Differential Ultra Wide SCSI PCI cards. .It Qlogic 1280 Dual Bus LVD Ultra2 Wide SCSI PCI cards. .It Qlogic 12160 Dual Bus LVD Ultra3 Wide SCSI PCI cards. .It Qlogic 210X Copper and Optical Fibre Channel Arbitrated Loop PCI cards (single, dual). .It Qlogic 220X Copper and Optical Fibre Channel Arbitrated Loop PCI cards (single, dual, quad). .It Qlogic 2300 Optical 2Gb Fibre Channel PCI cards. .It Qlogic 2312 Optical 2Gb Fibre Channel PCI cards. .It Qlogic 234X Optical 2Gb Fibre Channel PCI cards (2312 chipset, single and dual attach). .It Qlogic 2322 Optical 2Gb Fibre Channel PCIe cards. .It Qlogic 200 Dell branded version of the QLogic 2312. .It Qlogic 2422 Optical 4Gb Fibre Channel PCI cards. -.It Qlogic 2432 +.It Qlogic 246x (aka 2432) Optical 4Gb Fibre Channel PCIe cards. -.It Qlogic 2532 +.It Qlogic 256x (aka 2532) Optical 8Gb Fibre Channel PCIe cards. +.It Qlogic 267x/836x (aka 2031/8031) +Optical 16Gb FC/FCoE PCIe cards. .El .Sh CONFIGURATION OPTIONS Target mode support for Fibre Channel adapters may be enabled with the .Pp .Cd options ISP_TARGET_MODE .Pp option. .Sh BOOT OPTIONS The following options are switchable by setting values in .Pa /boot/device.hints . .Pp They are: .Bl -tag -width indent .It Va hint.isp.0.fwload_disable A hint value to disable loading of firmware .Xr ispfw 4 . -.It Va hint.isp.0.prefer_memmap -A hint value to use PCI memory space instead of I/O space -access for. -.It Va hint.isp.0.prefer_iomap -A hint value to use PCI I/O space instead of Memory space -access for. .It Va hint.isp.0.ignore_nvram A hint value to ignore board NVRAM settings for. Otherwise use NVRAM settings. .It Va hint.isp.0.fullduplex A hint value to set full duplex mode. .It Va hint.isp.0.topology A hint value to select topology of connection. Supported values are: .Pp .Bl -tag -width ".Li lport-only" -compact .It Li lport Prefer loopback and fallback to point to point. .It Li nport Prefer point to point and fallback to loopback. .It Li lport-only Loopback only. .It Li nport-only Point to point only. .El .It Va hint.isp.0.portwwn This should be the full 64 bit World Wide Port Name you would like to use, overriding the value in NVRAM for the card. .It Va hint.isp.0.nodewwn This should be the full 64 bit World Wide Node Name you would like to use, overriding the value in NVRAM for the card. .It Va hint.isp.0.iid A hint to override or set the Initiator ID or Loop ID. For Fibre Channel cards in Local Loop topologies it is .Ar strongly recommended that you set this value to non-zero. .It Va hint.isp.0.role A hint to define default role for isp instance (0 -- none, 1 -- target, 2 -- initiator, 3 -- both). .It Va hint.isp.0.debug A hint value for a driver debug level (see the file .Pa /usr/src/sys/dev/isp/ispvar.h for the values. .It Va hint.isp.0.vports A hint to create specified number of additional virtual ports. .El .Sh SYSCTL OPTIONS .Bl -tag -width indent .It Va dev.isp.N.loop_down_limit This value says how long to wait in seconds after loop has gone down before giving up and expiring all of the devices that were visible. The default is 300 seconds (5 minutes). A separate (nonadjustable) timeout is used when booting to not stop booting on lack of FC connectivity. .It Va dev.isp.N.gone_device_time This value says how long to wait for devices to reappear if they (temporarily) disappear due to loop or fabric events. While this timeout is running, I/O to those devices will simply be held. .It Va dev.isp.N.wwnn This is the readonly World Wide Node Name value for this port. .It Va dev.isp.N.wwpn This is the readonly World Wide Port Name value for this port. .El .Sh SEE ALSO .Xr da 4 , .Xr intro 4 , .Xr ispfw 4 , .Xr sa 4 , .Xr scsi 4 , .Xr gmultipath 8 .Sh AUTHORS The .Nm driver was written by .An Matthew Jacob originally for NetBSD at NASA/Ames Research Center. Some later improvement was done by .An Alexander Motin Aq Mt mav@FreeBSD.org . .Sh BUGS The driver currently ignores some NVRAM settings. .Pp Fabric support for 2100 cards has been so problematic, and these cards are so old now that it is just not worth your time to try it. Index: user/ngie/more-tests2/share/man/man4 =================================================================== --- user/ngie/more-tests2/share/man/man4 (revision 292053) +++ user/ngie/more-tests2/share/man/man4 (revision 292054) Property changes on: user/ngie/more-tests2/share/man/man4 ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,1 ## Merged /head/share/man/man4:r291880-292053 Index: user/ngie/more-tests2/share/man/man9/BUS_DESCRIBE_INTR.9 =================================================================== --- user/ngie/more-tests2/share/man/man9/BUS_DESCRIBE_INTR.9 (revision 292053) +++ user/ngie/more-tests2/share/man/man9/BUS_DESCRIBE_INTR.9 (revision 292054) @@ -1,104 +1,104 @@ .\" -*- nroff -*- .\" .\" Copyright (c) 2009 Hudson River Trading LLC .\" Written by: John H. Baldwin .\" All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" $FreeBSD$ .\" -.Dd October 14, 2009 +.Dd December 9, 2015 .Dt BUS_DESCRIBE_INTR 9 .Os .Sh NAME .Nm BUS_DESCRIBE_INTR , .Nm bus_describe_intr .Nd "associate a description with an active interrupt handler" .Sh SYNOPSIS .In sys/param.h .In sys/bus.h .Ft int -.Fo BUS_BIND_INTR +.Fo BUS_DESCRIBE_INTR .Fa "device_t dev" "device_t child" "struct resource *irq" "void *cookie" .Fa "const char *descr" .Fc .Ft int .Fo bus_describe_intr .Fa "device_t dev" "struct resource *irq" "void *cookie" "const char *fmt" .Fa ... .Fc .Sh DESCRIPTION The .Fn BUS_DESCRIBE_INTR method associates a description with an active interrupt handler. The .Fa cookie parameter must be the value returned by a successful call to .Xr BUS_SETUP_INTR 9 for the interrupt .Fa irq . .Pp The .Fn bus_describe_intr function is a simple wrapper around .Fn BUS_DESCRIBE_INTR . As a convenience, .Fn bus_describe_intr allows the caller to use .Xr printf 9 style formatting to build the description string using .Fa fmt . .Pp When an interrupt handler is established by .Xr BUS_SETUP_INTR 9 , the handler is named after the device the handler is established for. This name is then used in various places such as interrupt statistics displayed by .Xr systat 1 and .Xr vmstat 8 . For devices that use a single interrupt, the device name is sufficiently unique to identify the interrupt handler. However, for devices that use multiple interrupts it can be useful to distinguish the interrupt handlers. When a description is set for an active interrupt handler, a colon followed by the description is appended to the device name to form the interrupt handler name. .Sh RETURN VALUES Zero is returned on success, otherwise an appropriate error is returned. .Sh SEE ALSO .Xr systat 1 , .Xr vmstat 8 , .Xr BUS_SETUP_INTR 9 , .Xr device 9 , .Xr printf 9 .Sh HISTORY The .Fn BUS_DESCRIBE_INTR method and .Fn bus_describe_intr functions first appeared in .Fx 8.1 . .Sh BUGS It is not currently possible to remove a description from an active interrupt handler. Index: user/ngie/more-tests2/share/mk/bsd.libnames.mk =================================================================== --- user/ngie/more-tests2/share/mk/bsd.libnames.mk (revision 292053) +++ user/ngie/more-tests2/share/mk/bsd.libnames.mk (revision 292054) @@ -1,173 +1,186 @@ # $FreeBSD$ # The include file define library names. # Other include files (e.g. bsd.prog.mk, bsd.lib.mk) include this # file where necessary. .if !target(____) .error bsd.libnames.mk cannot be included directly. .endif .sinclude # Src directory locations are also defined in src.libnames.mk. LIBCRT0?= ${DESTDIR}${LIBDIR}/crt0.o LIB80211?= ${DESTDIR}${LIBDIR}/lib80211.a LIBALIAS?= ${DESTDIR}${LIBDIR}/libalias.a LIBARCHIVE?= ${DESTDIR}${LIBDIR}/libarchive.a LIBASN1?= ${DESTDIR}${LIBDIR}/libasn1.a LIBATM?= ${DESTDIR}${LIBDIR}/libatm.a LIBAUDITD?= ${DESTDIR}${LIBDIR}/libauditd.a LIBAVL?= ${DESTDIR}${LIBDIR}/libavl.a LIBBEGEMOT?= ${DESTDIR}${LIBDIR}/libbegemot.a LIBBLUETOOTH?= ${DESTDIR}${LIBDIR}/libbluetooth.a LIBBSDXML?= ${DESTDIR}${LIBDIR}/libbsdxml.a LIBBSM?= ${DESTDIR}${LIBDIR}/libbsm.a LIBBSNMP?= ${DESTDIR}${LIBDIR}/libbsnmp.a LIBBZ2?= ${DESTDIR}${LIBDIR}/libbz2.a -LIBCXXRT?= ${DESTDIR}${LIBDIR}/libcxxrt.a -LIBCPLUSPLUS?= ${DESTDIR}${LIBDIR}/libc++.a LIBC?= ${DESTDIR}${LIBDIR}/libc.a -LIBC_PIC?= ${DESTDIR}${LIBDIR}/libc_pic.a LIBCALENDAR?= ${DESTDIR}${LIBDIR}/libcalendar.a LIBCAM?= ${DESTDIR}${LIBDIR}/libcam.a LIBCAPSICUM?= ${DESTDIR}${LIBDIR}/libcapsicum.a LIBCASPER?= ${DESTDIR}${LIBDIR}/libcasper.a -LIBCOM_ERR?= ${DESTDIR}${LIBDIR}/libcom_err.a LIBCOMPAT?= ${DESTDIR}${LIBDIR}/libcompat.a LIBCOMPILER_RT?=${DESTDIR}${LIBDIR}/libcompiler_rt.a +LIBCOM_ERR?= ${DESTDIR}${LIBDIR}/libcom_err.a +LIBCPLUSPLUS?= ${DESTDIR}${LIBDIR}/libc++.a LIBCRYPT?= ${DESTDIR}${LIBDIR}/libcrypt.a LIBCRYPTO?= ${DESTDIR}${LIBDIR}/libcrypto.a LIBCTF?= ${DESTDIR}${LIBDIR}/libctf.a LIBCURSES?= ${DESTDIR}${LIBDIR}/libcurses.a LIBCUSE?= ${DESTDIR}${LIBDIR}/libcuse.a +LIBCXGB4?= ${DESTDIR}${LIBDIR}/libcxgb4.a +LIBCXXRT?= ${DESTDIR}${LIBDIR}/libcxxrt.a +LIBC_PIC?= ${DESTDIR}${LIBDIR}/libc_pic.a LIBDEVCTL?= ${DESTDIR}${LIBDIR}/libdevctl.a LIBDEVINFO?= ${DESTDIR}${LIBDIR}/libdevinfo.a LIBDEVSTAT?= ${DESTDIR}${LIBDIR}/libdevstat.a LIBDIALOG?= ${DESTDIR}${LIBDIR}/libdialog.a LIBDNS?= ${DESTDIR}${LIBDIR}/libdns.a LIBDPV?= ${DESTDIR}${LIBDIR}/libdpv.a LIBDTRACE?= ${DESTDIR}${LIBDIR}/libdtrace.a LIBDWARF?= ${DESTDIR}${LIBDIR}/libdwarf.a LIBEDIT?= ${DESTDIR}${LIBDIR}/libedit.a LIBELF?= ${DESTDIR}${LIBDIR}/libelf.a LIBEXECINFO?= ${DESTDIR}${LIBDIR}/libexecinfo.a LIBFETCH?= ${DESTDIR}${LIBDIR}/libfetch.a LIBFIGPAR?= ${DESTDIR}${LIBDIR}/libfigpar.a LIBFL?= "don't use LIBFL, use LIBL" LIBFORM?= ${DESTDIR}${LIBDIR}/libform.a LIBG2C?= ${DESTDIR}${LIBDIR}/libg2c.a -LIBGPIO?= ${DESTDIR}${LIBDIR}/libgpio.a LIBGEOM?= ${DESTDIR}${LIBDIR}/libgeom.a LIBGNUREGEX?= ${DESTDIR}${LIBDIR}/libgnuregex.a +LIBGPIO?= ${DESTDIR}${LIBDIR}/libgpio.a LIBGSSAPI?= ${DESTDIR}${LIBDIR}/libgssapi.a LIBGSSAPI_KRB5?= ${DESTDIR}${LIBDIR}/libgssapi_krb5.a LIBHDB?= ${DESTDIR}${LIBDIR}/libhdb.a LIBHEIMBASE?= ${DESTDIR}${LIBDIR}/libheimbase.a LIBHEIMNTLM?= ${DESTDIR}${LIBDIR}/libheimntlm.a LIBHEIMSQLITE?= ${DESTDIR}${LIBDIR}/libheimsqlite.a LIBHX509?= ${DESTDIR}${LIBDIR}/libhx509.a +LIBIBCM?= ${DESTDIR}${LIBDIR}/libibcm.a +LIBIBCOMMON?= ${DESTDIR}${LIBDIR}/libibcommon.a +LIBIBMAD?= ${DESTDIR}${LIBDIR}/libibmad.a +LIBIBSDP?= ${DESTDIR}${LIBDIR}/libibsdp.a +LIBIBUMAD?= ${DESTDIR}${LIBDIR}/libibumad.a +LIBIBVERBS?= ${DESTDIR}${LIBDIR}/libibverbs.a LIBIPSEC?= ${DESTDIR}${LIBDIR}/libipsec.a LIBJAIL?= ${DESTDIR}${LIBDIR}/libjail.a LIBKADM5CLNT?= ${DESTDIR}${LIBDIR}/libkadm5clnt.a LIBKADM5SRV?= ${DESTDIR}${LIBDIR}/libkadm5srv.a LIBKAFS5?= ${DESTDIR}${LIBDIR}/libkafs5.a LIBKDC?= ${DESTDIR}${LIBDIR}/libkdc.a LIBKEYCAP?= ${DESTDIR}${LIBDIR}/libkeycap.a LIBKICONV?= ${DESTDIR}${LIBDIR}/libkiconv.a LIBKRB5?= ${DESTDIR}${LIBDIR}/libkrb5.a LIBKVM?= ${DESTDIR}${LIBDIR}/libkvm.a LIBL?= ${DESTDIR}${LIBDIR}/libl.a LIBLN?= "don't use LIBLN, use LIBL" LIBLZMA?= ${DESTDIR}${LIBDIR}/liblzma.a LIBM?= ${DESTDIR}${LIBDIR}/libm.a LIBMAGIC?= ${DESTDIR}${LIBDIR}/libmagic.a LIBMD?= ${DESTDIR}${LIBDIR}/libmd.a LIBMEMSTAT?= ${DESTDIR}${LIBDIR}/libmemstat.a LIBMENU?= ${DESTDIR}${LIBDIR}/libmenu.a LIBMILTER?= ${DESTDIR}${LIBDIR}/libmilter.a +LIBMLX4?= ${DESTDIR}${LIBDIR}/libmlx4.a LIBMP?= ${DESTDIR}${LIBDIR}/libmp.a LIBMT?= ${DESTDIR}${LIBDIR}/libmt.a +LIBMTHCA?= ${DESTDIR}${LIBDIR}/libmthca.a LIBNANDFS?= ${DESTDIR}${LIBDIR}/libnandfs.a LIBNCURSES?= ${DESTDIR}${LIBDIR}/libncurses.a LIBNCURSESW?= ${DESTDIR}${LIBDIR}/libncursesw.a LIBNETGRAPH?= ${DESTDIR}${LIBDIR}/libnetgraph.a LIBNGATM?= ${DESTDIR}${LIBDIR}/libngatm.a LIBNV?= ${DESTDIR}${LIBDIR}/libnv.a LIBNVPAIR?= ${DESTDIR}${LIBDIR}/libnvpair.a +LIBOPENSM?= ${DESTDIR}${LIBDIR}/libopensm.a LIBOPIE?= ${DESTDIR}${LIBDIR}/libopie.a +LIBOSMCOMP?= ${DESTDIR}${LIBDIR}/libosmcomp.a +LIBOSMVENDOR?= ${DESTDIR}${LIBDIR}/libosmvendor.a LIBPAM?= ${DESTDIR}${LIBDIR}/libpam.a LIBPANEL?= ${DESTDIR}${LIBDIR}/libpanel.a LIBPANELW?= ${DESTDIR}${LIBDIR}/libpanelw.a LIBPCAP?= ${DESTDIR}${LIBDIR}/libpcap.a LIBPJDLOG?= ${DESTDIR}${LIBDIR}/libpjdlog.a LIBPMC?= ${DESTDIR}${LIBDIR}/libpmc.a LIBPROC?= ${DESTDIR}${LIBDIR}/libproc.a LIBPROCSTAT?= ${DESTDIR}${LIBDIR}/libprocstat.a LIBPTHREAD?= ${DESTDIR}${LIBDIR}/libpthread.a LIBRADIUS?= ${DESTDIR}${LIBDIR}/libradius.a +LIBRDMACM?= ${DESTDIR}${LIBDIR}/librdmacm.a LIBROKEN?= ${DESTDIR}${LIBDIR}/libroken.a -LIBRPCSVC?= ${DESTDIR}${LIBDIR}/librpcsvc.a LIBRPCSEC_GSS?= ${DESTDIR}${LIBDIR}/librpcsec_gss.a +LIBRPCSVC?= ${DESTDIR}${LIBDIR}/librpcsvc.a LIBRT?= ${DESTDIR}${LIBDIR}/librt.a LIBRTLD_DB?= ${DESTDIR}${LIBDIR}/librtld_db.a LIBSBUF?= ${DESTDIR}${LIBDIR}/libsbuf.a LIBSDP?= ${DESTDIR}${LIBDIR}/libsdp.a LIBSMB?= ${DESTDIR}${LIBDIR}/libsmb.a LIBSSL?= ${DESTDIR}${LIBDIR}/libssl.a LIBSSP_NONSHARED?= ${DESTDIR}${LIBDIR}/libssp_nonshared.a LIBSTAND?= ${DESTDIR}${LIBDIR}/libstand.a LIBSTDCPLUSPLUS?= ${DESTDIR}${LIBDIR}/libstdc++.a LIBSTDTHREADS?= ${DESTDIR}${LIBDIR}/libstdthreads.a LIBTACPLUS?= ${DESTDIR}${LIBDIR}/libtacplus.a LIBTERMCAP?= ${DESTDIR}${LIBDIR}/libtermcap.a LIBTERMCAPW?= ${DESTDIR}${LIBDIR}/libtermcapw.a LIBTERMLIB?= "don't use LIBTERMLIB, use LIBTERMCAP" LIBTINFO?= "don't use LIBTINFO, use LIBNCURSES" LIBUFS?= ${DESTDIR}${LIBDIR}/libufs.a LIBUGIDFW?= ${DESTDIR}${LIBDIR}/libugidfw.a +LIBULOG?= ${DESTDIR}${LIBDIR}/libulog.a LIBUMEM?= ${DESTDIR}${LIBDIR}/libumem.a -LIBUSBHID?= ${DESTDIR}${LIBDIR}/libusbhid.a LIBUSB?= ${DESTDIR}${LIBDIR}/libusb.a -LIBULOG?= ${DESTDIR}${LIBDIR}/libulog.a +LIBUSBHID?= ${DESTDIR}${LIBDIR}/libusbhid.a LIBUTIL?= ${DESTDIR}${LIBDIR}/libutil.a LIBUUTIL?= ${DESTDIR}${LIBDIR}/libuutil.a LIBVGL?= ${DESTDIR}${LIBDIR}/libvgl.a LIBVMMAPI?= ${DESTDIR}${LIBDIR}/libvmmapi.a LIBWIND?= ${DESTDIR}${LIBDIR}/libwind.a LIBWRAP?= ${DESTDIR}${LIBDIR}/libwrap.a -LIBXPG4?= ${DESTDIR}${LIBDIR}/libxpg4.a LIBXO?= ${DESTDIR}${LIBDIR}/libxo.a +LIBXPG4?= ${DESTDIR}${LIBDIR}/libxpg4.a LIBY?= ${DESTDIR}${LIBDIR}/liby.a LIBYPCLNT?= ${DESTDIR}${LIBDIR}/libypclnt.a LIBZ?= ${DESTDIR}${LIBDIR}/libz.a LIBZFS?= ${DESTDIR}${LIBDIR}/libzfs.a LIBZFS_CORE?= ${DESTDIR}${LIBDIR}/libzfs_core.a LIBZPOOL?= ${DESTDIR}${LIBDIR}/libzpool.a # enforce the 2 -lpthread and -lc to always be the last in that exact order .if defined(LDADD) .if ${LDADD:M-lpthread} LDADD:= ${LDADD:N-lpthread} -lpthread .endif .if ${LDADD:M-lc} LDADD:= ${LDADD:N-lc} -lc .endif .endif # Only do this for src builds. .if defined(SRCTOP) .if defined(_LIBRARIES) && defined(LIB) && \ ${_LIBRARIES:M${LIB}} != "" .if !defined(LIB${LIB:tu}) .error ${.CURDIR}: Missing value for LIB${LIB:tu} in ${_this:T}. Likely should be: LIB${LIB:tu}?= $${DESTDIR}$${LIBDIR}/lib${LIB}.a .endif .endif # Derive LIB*SRCDIR from LIB*DIR .for lib in ${_LIBRARIES} LIB${lib:tu}SRCDIR?= ${SRCTOP}/${LIB${lib:tu}DIR:S,^${OBJTOP}/,,} .endfor .endif Index: user/ngie/more-tests2/share/mk/src.libnames.mk =================================================================== --- user/ngie/more-tests2/share/mk/src.libnames.mk (revision 292053) +++ user/ngie/more-tests2/share/mk/src.libnames.mk (revision 292054) @@ -1,496 +1,527 @@ # $FreeBSD$ # # The include file define library names suitable # for INTERNALLIB and PRIVATELIB definition .if !target(____) .error src.libnames.mk cannot be included directly. .endif .if !target(____) ____: .include _PRIVATELIBS= \ atf_c \ atf_cxx \ bsdstat \ event \ heimipcc \ heimipcs \ ldns \ sqlite3 \ ssh \ ucl \ unbound _INTERNALLIBS= \ amu \ bsnmptools \ cron \ elftc \ fifolog \ ipf \ lpr \ netbsd \ ntp \ ntpevent \ openbsd \ opts \ parse \ readline \ sl \ sm \ smdb \ smutil \ telnet \ vers _LIBRARIES= \ ${_PRIVATELIBS} \ ${_INTERNALLIBS} \ ${LOCAL_LIBRARIES} \ 80211 \ alias \ archive \ asn1 \ auditd \ avl \ begemot \ bluetooth \ bsdxml \ bsm \ bsnmp \ bz2 \ c \ c_pic \ calendar \ cam \ capsicum \ casper \ com_err \ compiler_rt \ crypt \ crypto \ ctf \ cuse \ cxxrt \ devctl \ devinfo \ devstat \ dialog \ dpv \ dtrace \ dwarf \ edit \ elf \ execinfo \ fetch \ figpar \ geom \ gnuregex \ gpio \ gssapi \ gssapi_krb5 \ hdb \ heimbase \ heimntlm \ heimsqlite \ hx509 \ ipsec \ jail \ kadm5clnt \ kadm5srv \ kafs5 \ kdc \ kiconv \ krb5 \ kvm \ l \ lzma \ m \ magic \ md \ memstat \ mp \ mt \ nandfs \ ncurses \ ncursesw \ netgraph \ ngatm \ nv \ nvpair \ opie \ pam \ panel \ panelw \ pcap \ pcsclite \ pjdlog \ pmc \ proc \ procstat \ pthread \ radius \ readline \ roken \ rpcsec_gss \ rpcsvc \ rt \ rtld_db \ sbuf \ sdp \ sm \ smb \ ssl \ ssp_nonshared \ stdthreads \ supcplusplus \ tacplus \ termcapw \ ufs \ ugidfw \ ulog \ umem \ usb \ usbhid \ util \ uutil \ vmmapi \ wind \ wrap \ xo \ y \ ypclnt \ z \ zfs_core \ zfs \ zpool \ +.if ${MK_OFED} != "no" +_LIBRARIES+= \ + cxgb4 \ + ibcm \ + ibcommon \ + ibmad \ + ibsdp \ + ibumad \ + ibverbs \ + mlx4 \ + mthca \ + opensm \ + osmcomp \ + osmvendor \ + rdmacm \ +.endif + # Each library's LIBADD needs to be duplicated here for static linkage of # 2nd+ order consumers. Auto-generating this would be better. _DP_80211= sbuf bsdxml _DP_archive= z bz2 lzma bsdxml .if ${MK_OPENSSL} != "no" _DP_archive+= crypto .else _DP_archive+= md .endif _DP_sqlite3= pthread _DP_ssl= crypto _DP_ssh= crypto crypt z .if ${MK_LDNS} != "no" _DP_ssh+= ldns .endif _DP_edit= ncursesw .if ${MK_OPENSSL} != "no" _DP_bsnmp= crypto .endif _DP_geom= bsdxml sbuf _DP_cam= sbuf _DP_casper= capsicum nv pjdlog _DP_capsicum= nv _DP_kvm= elf _DP_pjdlog= util _DP_opie= md _DP_usb= pthread _DP_unbound= ssl crypto pthread _DP_rt= pthread .if ${MK_OPENSSL} == "no" _DP_radius= md .else _DP_radius= crypto .endif _DP_procstat= kvm util elf .if ${MK_CXX} == "yes" .if ${MK_LIBCPLUSPLUS} != "no" _DP_proc= cxxrt .else _DP_proc= supcplusplus .endif .endif .if ${MK_CDDL} != "no" _DP_proc+= ctf .endif _DP_proc+= elf rtld_db util _DP_mp= crypto _DP_memstat= kvm _DP_magic= z _DP_mt= sbuf bsdxml _DP_ldns= crypto .if ${MK_OPENSSL} != "no" _DP_fetch= ssl crypto .else _DP_fetch= md .endif _DP_execinfo= elf _DP_dwarf= elf _DP_dpv= dialog figpar util ncursesw _DP_dialog= ncursesw m _DP_cuse= pthread _DP_atf_cxx= atf_c _DP_devstat= kvm _DP_pam= radius tacplus opie md util .if ${MK_KERBEROS} != "no" _DP_pam+= krb5 .endif .if ${MK_OPENSSH} != "no" _DP_pam+= ssh .endif .if ${MK_NIS} != "no" _DP_pam+= ypclnt .endif _DP_readline= ncursesw _DP_roken= crypt _DP_kadm5clnt= com_err krb5 roken _DP_kadm5srv= com_err hdb krb5 roken _DP_heimntlm= crypto com_err krb5 roken _DP_hx509= asn1 com_err crypto roken wind _DP_hdb= asn1 com_err krb5 roken sqlite3 _DP_asn1= com_err roken _DP_kdc= roken hdb hx509 krb5 heimntlm asn1 crypto _DP_wind= com_err roken _DP_heimbase= pthread _DP_heimipcc= heimbase roken pthread _DP_heimipcs= heimbase roken pthread _DP_kafs5= asn1 krb5 roken _DP_krb5+= asn1 com_err crypt crypto hx509 roken wind heimbase heimipcc _DP_gssapi_krb5+= gssapi krb5 crypto roken asn1 com_err _DP_lzma= pthread _DP_ucl= m _DP_vmmapi= util _DP_ctf= z _DP_dtrace= ctf elf proc pthread rtld_db _DP_xo= util # The libc dependencies are not strictly needed but are defined to make the # assert happy. _DP_c= compiler_rt .if ${MK_SSP} != "no" _DP_c+= ssp_nonshared .endif _DP_stdthreads= pthread _DP_tacplus= md _DP_panel= ncurses _DP_panelw= ncursesw _DP_rpcsec_gss= gssapi _DP_smb= kiconv _DP_ulog= md _DP_fifolog= z _DP_ipf= kvm _DP_zfs= md pthread umem util uutil m nvpair avl bsdxml geom nvpair z \ zfs_core _DP_zfs_core= nvpair _DP_zpool= md pthread z nvpair avl umem # Define special cases LDADD_supcplusplus= -lsupc++ LIBATF_C= ${DESTDIR}${LIBDIR}/libprivateatf-c.a LIBATF_CXX= ${DESTDIR}${LIBDIR}/libprivateatf-c++.a LDADD_atf_c= -lprivateatf-c LDADD_atf_cxx= -lprivateatf-c++ .for _l in ${_PRIVATELIBS} LIB${_l:tu}?= ${DESTDIR}${LIBDIR}/libprivate${_l}.a .endfor .for _l in ${_LIBRARIES} .if ${_INTERNALLIBS:M${_l}} LDADD_${_l}_L+= -L${LIB${_l:tu}DIR} .endif DPADD_${_l}?= ${LIB${_l:tu}} .if ${_PRIVATELIBS:M${_l}} LDADD_${_l}?= -lprivate${_l} .else LDADD_${_l}?= ${LDADD_${_l}_L} -l${_l} .endif # Add in all dependencies for static linkage. .if defined(_DP_${_l}) && (${_INTERNALLIBS:M${_l}} || \ (defined(NO_SHARED) && (${NO_SHARED} != "no" && ${NO_SHARED} != "NO"))) .for _d in ${_DP_${_l}} DPADD_${_l}+= ${DPADD_${_d}} LDADD_${_l}+= ${LDADD_${_d}} .endfor .endif .endfor # These are special cases where the library is broken and anything that uses # it needs to add more dependencies. Broken usually means that it has a # cyclic dependency and cannot link its own dependencies. This is bad, please # fix the library instead. # Unless the library itself is broken then the proper place to define # dependencies is _DP_* above. # libatf-c++ exposes libatf-c abi hence we need to explicit link to atf_c for # atf_cxx DPADD_atf_cxx+= ${DPADD_atf_c} LDADD_atf_cxx+= ${LDADD_atf_c} # Detect LDADD/DPADD that should be LIBADD, before modifying LDADD here. .for _l in ${LDADD:M-l*:N-l*/*:C,^-l,,} .if ${_LIBRARIES:M${_l}} _BADLDADD+= ${_l} .endif .endfor .if !empty(_BADLDADD) .error ${.CURDIR}: These libraries should be LIBADD+=foo rather than DPADD/LDADD+=-lfoo: ${_BADLDADD} .endif .for _l in ${LIBADD} DPADD+= ${DPADD_${_l}} LDADD+= ${LDADD_${_l}} .endfor # INTERNALLIB definitions. LIBELFTCDIR= ${OBJTOP}/lib/libelftc LIBELFTC?= ${LIBELFTCDIR}/libelftc.a LIBREADLINEDIR= ${OBJTOP}/gnu/lib/libreadline/readline LIBREADLINE?= ${LIBREADLINEDIR}/libreadline.a LIBOPENBSDDIR= ${OBJTOP}/lib/libopenbsd LIBOPENBSD?= ${LIBOPENBSDDIR}/libopenbsd.a LIBSMDIR= ${OBJTOP}/lib/libsm LIBSM?= ${LIBSMDIR}/libsm.a LIBSMDBDIR= ${OBJTOP}/lib/libsmdb LIBSMDB?= ${LIBSMDBDIR}/libsmdb.a LIBSMUTILDIR= ${OBJTOP}/lib/libsmutil LIBSMUTIL?= ${LIBSMDBDIR}/libsmutil.a LIBNETBSDDIR?= ${OBJTOP}/lib/libnetbsd LIBNETBSD?= ${LIBNETBSDDIR}/libnetbsd.a LIBVERSDIR?= ${OBJTOP}/kerberos5/lib/libvers LIBVERS?= ${LIBVERSDIR}/libvers.a LIBSLDIR= ${OBJTOP}/kerberos5/lib/libsl LIBSL?= ${LIBSLDIR}/libsl.a LIBIPFDIR= ${OBJTOP}/sbin/ipf/libipf LIBIPF?= ${LIBIPFDIR}/libipf.a LIBTELNETDIR= ${OBJTOP}/lib/libtelnet LIBTELNET?= ${LIBTELNETDIR}/libtelnet.a LIBCRONDIR= ${OBJTOP}/usr.sbin/cron/lib LIBCRON?= ${LIBCRONDIR}/libcron.a LIBNTPDIR= ${OBJTOP}/usr.sbin/ntp/libntp LIBNTP?= ${LIBNTPDIR}/libntp.a LIBNTPEVENTDIR= ${OBJTOP}/usr.sbin/ntp/libntpevent LIBNTPEVENT?= ${LIBNTPEVENTDIR}/libntpevent.a LIBOPTSDIR= ${OBJTOP}/usr.sbin/ntp/libopts LIBOPTS?= ${LIBOPTSDIR}/libopts.a LIBPARSEDIR= ${OBJTOP}/usr.sbin/ntp/libparse LIBPARSE?= ${LIBPARSEDIR}/libparse.a LIBLPRDIR= ${OBJTOP}/usr.sbin/lpr/common_source LIBLPR?= ${LIBOPTSDIR}/liblpr.a LIBFIFOLOGDIR= ${OBJTOP}/usr.sbin/fifolog/lib LIBFIFOLOG?= ${LIBOPTSDIR}/libfifolog.a LIBBSNMPTOOLSDIR= ${OBJTOP}/usr.sbin/bsnmpd/tools/libbsnmptools LIBBSNMPTOOLS?= ${LIBBSNMPTOOLSDIR}/libbsnmptools.a LIBAMUDIR= ${OBJTOP}/usr.sbin/amd/libamu LIBAMU?= ${LIBAMUDIR}/libamu/libamu.a # Define a directory for each library. This is useful for adding -L in when # not using a --sysroot or for meta mode bootstrapping when there is no # Makefile.depend. These are sorted by directory. LIBAVLDIR= ${OBJTOP}/cddl/lib/libavl LIBCTFDIR= ${OBJTOP}/cddl/lib/libctf LIBDTRACEDIR= ${OBJTOP}/cddl/lib/libdtrace LIBNVPAIRDIR= ${OBJTOP}/cddl/lib/libnvpair LIBUMEMDIR= ${OBJTOP}/cddl/lib/libumem LIBUUTILDIR= ${OBJTOP}/cddl/lib/libuutil LIBZFSDIR= ${OBJTOP}/cddl/lib/libzfs LIBZFS_COREDIR= ${OBJTOP}/cddl/lib/libzfs_core LIBZPOOLDIR= ${OBJTOP}/cddl/lib/libzpool +LIBCXGB4DIR= ${OBJTOP}/contrib/ofed/usr.lib/libcxgb4 +LIBIBCMDIR= ${OBJTOP}/contrib/ofed/usr.lib/libibcm +LIBIBCOMMONDIR= ${OBJTOP}/contrib/ofed/usr.lib/libibcommon +LIBIBMADDIR= ${OBJTOP}/contrib/ofed/usr.lib/libibmad +LIBIBUMADDIR= ${OBJTOP}/contrib/ofed/usr.lib/libibumad +LIBIBVERBSDIR= ${OBJTOP}/contrib/ofed/usr.lib/libibverbs +LIBMLX4DIR= ${OBJTOP}/contrib/ofed/usr.lib/libmlx4 +LIBMTHCADIR= ${OBJTOP}/contrib/ofed/usr.lib/libmthca +LIBOPENSMDIR= ${OBJTOP}/contrib/ofed/usr.lib/libopensm +LIBOSMCOMPDIR= ${OBJTOP}/contrib/ofed/usr.lib/libosmcomp +LIBOSMVENDORDIR= ${OBJTOP}/contrib/ofed/usr.lib/libosmvendor +LIBRDMACMDIR= ${OBJTOP}/contrib/ofed/usr.lib/librdmacm +LIBIBSDPDIR= ${OBJTOP}/contrib/ofed/usr.lib/libsdp LIBDIALOGDIR= ${OBJTOP}/gnu/lib/libdialog LIBGCOVDIR= ${OBJTOP}/gnu/lib/libgcov LIBGOMPDIR= ${OBJTOP}/gnu/lib/libgomp LIBGNUREGEXDIR= ${OBJTOP}/gnu/lib/libregex LIBSSPDIR= ${OBJTOP}/gnu/lib/libssp LIBSSP_NONSHAREDDIR= ${OBJTOP}/gnu/lib/libssp/libssp_nonshared LIBSUPCPLUSPLUSDIR= ${OBJTOP}/gnu/lib/libsupc++ LIBASN1DIR= ${OBJTOP}/kerberos5/lib/libasn1 LIBGSSAPI_KRB5DIR= ${OBJTOP}/kerberos5/lib/libgssapi_krb5 LIBGSSAPI_NTLMDIR= ${OBJTOP}/kerberos5/lib/libgssapi_ntlm LIBGSSAPI_SPNEGODIR= ${OBJTOP}/kerberos5/lib/libgssapi_spnego LIBHDBDIR= ${OBJTOP}/kerberos5/lib/libhdb LIBHEIMBASEDIR= ${OBJTOP}/kerberos5/lib/libheimbase LIBHEIMIPCCDIR= ${OBJTOP}/kerberos5/lib/libheimipcc LIBHEIMIPCSDIR= ${OBJTOP}/kerberos5/lib/libheimipcs LIBHEIMNTLMDIR= ${OBJTOP}/kerberos5/lib/libheimntlm LIBHX509DIR= ${OBJTOP}/kerberos5/lib/libhx509 LIBKADM5CLNTDIR= ${OBJTOP}/kerberos5/lib/libkadm5clnt LIBKADM5SRVDIR= ${OBJTOP}/kerberos5/lib/libkadm5srv LIBKAFS5DIR= ${OBJTOP}/kerberos5/lib/libkafs5 LIBKDCDIR= ${OBJTOP}/kerberos5/lib/libkdc LIBKRB5DIR= ${OBJTOP}/kerberos5/lib/libkrb5 LIBROKENDIR= ${OBJTOP}/kerberos5/lib/libroken LIBWINDDIR= ${OBJTOP}/kerberos5/lib/libwind LIBALIASDIR= ${OBJTOP}/lib/libalias/libalias LIBBLOCKSRUNTIMEDIR= ${OBJTOP}/lib/libblocksruntime LIBBSNMPDIR= ${OBJTOP}/lib/libbsnmp/libbsnmp LIBBSDXMLDIR= ${OBJTOP}/lib/libexpat LIBKVMDIR= ${OBJTOP}/lib/libkvm LIBPTHREADDIR= ${OBJTOP}/lib/libthr LIBMDIR= ${OBJTOP}/lib/msun LIBFORMDIR= ${OBJTOP}/lib/ncurses/form LIBFORMLIBWDIR= ${OBJTOP}/lib/ncurses/formw LIBMENUDIR= ${OBJTOP}/lib/ncurses/menu LIBMENULIBWDIR= ${OBJTOP}/lib/ncurses/menuw LIBNCURSESDIR= ${OBJTOP}/lib/ncurses/ncurses LIBNCURSESWDIR= ${OBJTOP}/lib/ncurses/ncursesw -LIBTERMCAPDIR= ${LIBNCURSESDIR} -LIBTERMCAPWDIR= ${LIBNCURSESWDIR} LIBPANELDIR= ${OBJTOP}/lib/ncurses/panel LIBPANELWDIR= ${OBJTOP}/lib/ncurses/panelw LIBCRYPTODIR= ${OBJTOP}/secure/lib/libcrypto LIBSSHDIR= ${OBJTOP}/secure/lib/libssh LIBSSLDIR= ${OBJTOP}/secure/lib/libssl LIBTEKENDIR= ${OBJTOP}/sys/teken/libteken LIBEGACYDIR= ${OBJTOP}/tools/build LIBLNDIR= ${OBJTOP}/usr.bin/lex/lib + +LIBTERMCAPDIR= ${LIBNCURSESDIR} +LIBTERMCAPWDIR= ${LIBNCURSESWDIR} # Default other library directories to lib/libNAME. .for lib in ${_LIBRARIES} LIB${lib:tu}DIR?= ${OBJTOP}/lib/lib${lib} .endfor # Validate that listed LIBADD are valid. .for _l in ${LIBADD} .if empty(_LIBRARIES:M${_l}) _BADLIBADD+= ${_l} .endif .endfor .if !empty(_BADLIBADD) .error ${.CURDIR}: Invalid LIBADD used which may need to be added to ${_this:T}: ${_BADLIBADD} .endif # Sanity check that libraries are defined here properly when building them. .if defined(LIB) && ${_LIBRARIES:M${LIB}} != "" .if !empty(LIBADD) && \ (!defined(_DP_${LIB}) || ${LIBADD:O:u} != ${_DP_${LIB}:O:u}) .error ${.CURDIR}: Missing or incorrect _DP_${LIB} entry in ${_this:T}. Should match LIBADD for ${LIB} ('${LIBADD}' vs '${_DP_${LIB}}') .endif .if !defined(LIB${LIB:tu}DIR) || !exists(${SRCTOP}/${LIB${LIB:tu}DIR:S,^${OBJTOP}/,,}) .error ${.CURDIR}: Missing or incorrect value for LIB${LIB:tu}DIR in ${_this:T}: ${LIB${LIB:tu}DIR:S,^${OBJTOP}/,,} .endif .if ${_INTERNALLIBS:M${LIB}} != "" && !defined(LIB${LIB:tu}) .error ${.CURDIR}: Missing value for LIB${LIB:tu} in ${_this:T}. Likely should be: LIB${LIB:tu}?= $${LIB${LIB:tu}DIR}/lib${LIB}.a .endif .endif .endif # !target(____) Index: user/ngie/more-tests2/share =================================================================== --- user/ngie/more-tests2/share (revision 292053) +++ user/ngie/more-tests2/share (revision 292054) Property changes on: user/ngie/more-tests2/share ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,1 ## Merged /head/share:r292006-292053 Index: user/ngie/more-tests2/sys/dev/hwpmc/hwpmc_logging.c =================================================================== --- user/ngie/more-tests2/sys/dev/hwpmc/hwpmc_logging.c (revision 292053) +++ user/ngie/more-tests2/sys/dev/hwpmc/hwpmc_logging.c (revision 292054) @@ -1,1073 +1,1079 @@ /*- * Copyright (c) 2005-2007 Joseph Koshy * Copyright (c) 2007 The FreeBSD Foundation * All rights reserved. * * Portions of this software were developed by A. Joseph Koshy under * sponsorship from the FreeBSD Foundation and Google, Inc. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * */ /* * Logging code for hwpmc(4) */ #include __FBSDID("$FreeBSD$"); #include #if (__FreeBSD_version >= 1100000) #include #else #include #endif #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include /* * Sysctl tunables */ SYSCTL_DECL(_kern_hwpmc); /* * kern.hwpmc.logbuffersize -- size of the per-cpu owner buffers. */ static int pmclog_buffer_size = PMC_LOG_BUFFER_SIZE; +#if (__FreeBSD_version < 1100000) +TUNABLE_INT(PMC_SYSCTL_NAME_PREFIX "logbuffersize", &pmclog_buffer_size); +#endif SYSCTL_INT(_kern_hwpmc, OID_AUTO, logbuffersize, CTLFLAG_RDTUN, &pmclog_buffer_size, 0, "size of log buffers in kilobytes"); /* * kern.hwpmc.nbuffer -- number of global log buffers */ static int pmc_nlogbuffers = PMC_NLOGBUFFERS; +#if (__FreeBSD_version < 1100000) +TUNABLE_INT(PMC_SYSCTL_NAME_PREFIX "nbuffers", &pmc_nlogbuffers); +#endif SYSCTL_INT(_kern_hwpmc, OID_AUTO, nbuffers, CTLFLAG_RDTUN, &pmc_nlogbuffers, 0, "number of global log buffers"); /* * Global log buffer list and associated spin lock. */ TAILQ_HEAD(, pmclog_buffer) pmc_bufferlist = TAILQ_HEAD_INITIALIZER(pmc_bufferlist); static struct mtx pmc_bufferlist_mtx; /* spin lock */ static struct mtx pmc_kthread_mtx; /* sleep lock */ #define PMCLOG_INIT_BUFFER_DESCRIPTOR(D) do { \ const int __roundup = roundup(sizeof(*D), \ sizeof(uint32_t)); \ (D)->plb_fence = ((char *) (D)) + \ 1024*pmclog_buffer_size; \ (D)->plb_base = (D)->plb_ptr = ((char *) (D)) + \ __roundup; \ } while (0) /* * Log file record constructors. */ #define _PMCLOG_TO_HEADER(T,L) \ ((PMCLOG_HEADER_MAGIC << 24) | \ (PMCLOG_TYPE_ ## T << 16) | \ ((L) & 0xFFFF)) /* reserve LEN bytes of space and initialize the entry header */ #define _PMCLOG_RESERVE(PO,TYPE,LEN,ACTION) do { \ uint32_t *_le; \ int _len = roundup((LEN), sizeof(uint32_t)); \ if ((_le = pmclog_reserve((PO), _len)) == NULL) { \ ACTION; \ } \ *_le = _PMCLOG_TO_HEADER(TYPE,_len); \ _le += 3 /* skip over timestamp */ #define PMCLOG_RESERVE(P,T,L) _PMCLOG_RESERVE(P,T,L,return) #define PMCLOG_RESERVE_WITH_ERROR(P,T,L) _PMCLOG_RESERVE(P,T,L, \ error=ENOMEM;goto error) #define PMCLOG_EMIT32(V) do { *_le++ = (V); } while (0) #define PMCLOG_EMIT64(V) do { \ *_le++ = (uint32_t) ((V) & 0xFFFFFFFF); \ *_le++ = (uint32_t) (((V) >> 32) & 0xFFFFFFFF); \ } while (0) /* Emit a string. Caution: does NOT update _le, so needs to be last */ #define PMCLOG_EMITSTRING(S,L) do { bcopy((S), _le, (L)); } while (0) #define PMCLOG_EMITNULLSTRING(L) do { bzero(_le, (L)); } while (0) #define PMCLOG_DESPATCH(PO) \ pmclog_release((PO)); \ } while (0) /* * Assertions about the log file format. */ CTASSERT(sizeof(struct pmclog_callchain) == 6*4 + PMC_CALLCHAIN_DEPTH_MAX*sizeof(uintfptr_t)); CTASSERT(sizeof(struct pmclog_closelog) == 3*4); CTASSERT(sizeof(struct pmclog_dropnotify) == 3*4); CTASSERT(sizeof(struct pmclog_map_in) == PATH_MAX + 4*4 + sizeof(uintfptr_t)); CTASSERT(offsetof(struct pmclog_map_in,pl_pathname) == 4*4 + sizeof(uintfptr_t)); CTASSERT(sizeof(struct pmclog_map_out) == 4*4 + 2*sizeof(uintfptr_t)); CTASSERT(sizeof(struct pmclog_pcsample) == 6*4 + sizeof(uintfptr_t)); CTASSERT(sizeof(struct pmclog_pmcallocate) == 6*4); CTASSERT(sizeof(struct pmclog_pmcattach) == 5*4 + PATH_MAX); CTASSERT(offsetof(struct pmclog_pmcattach,pl_pathname) == 5*4); CTASSERT(sizeof(struct pmclog_pmcdetach) == 5*4); CTASSERT(sizeof(struct pmclog_proccsw) == 5*4 + 8); CTASSERT(sizeof(struct pmclog_procexec) == 5*4 + PATH_MAX + sizeof(uintfptr_t)); CTASSERT(offsetof(struct pmclog_procexec,pl_pathname) == 5*4 + sizeof(uintfptr_t)); CTASSERT(sizeof(struct pmclog_procexit) == 5*4 + 8); CTASSERT(sizeof(struct pmclog_procfork) == 5*4); CTASSERT(sizeof(struct pmclog_sysexit) == 4*4); CTASSERT(sizeof(struct pmclog_userdata) == 4*4); /* * Log buffer structure */ struct pmclog_buffer { TAILQ_ENTRY(pmclog_buffer) plb_next; char *plb_base; char *plb_ptr; char *plb_fence; }; /* * Prototypes */ static int pmclog_get_buffer(struct pmc_owner *po); static void pmclog_loop(void *arg); static void pmclog_release(struct pmc_owner *po); static uint32_t *pmclog_reserve(struct pmc_owner *po, int length); static void pmclog_schedule_io(struct pmc_owner *po); static void pmclog_stop_kthread(struct pmc_owner *po); /* * Helper functions */ /* * Get a log buffer */ static int pmclog_get_buffer(struct pmc_owner *po) { struct pmclog_buffer *plb; mtx_assert(&po->po_mtx, MA_OWNED); KASSERT(po->po_curbuf == NULL, ("[pmclog,%d] po=%p current buffer still valid", __LINE__, po)); mtx_lock_spin(&pmc_bufferlist_mtx); if ((plb = TAILQ_FIRST(&pmc_bufferlist)) != NULL) TAILQ_REMOVE(&pmc_bufferlist, plb, plb_next); mtx_unlock_spin(&pmc_bufferlist_mtx); PMCDBG2(LOG,GTB,1, "po=%p plb=%p", po, plb); #ifdef HWPMC_DEBUG if (plb) KASSERT(plb->plb_ptr == plb->plb_base && plb->plb_base < plb->plb_fence, ("[pmclog,%d] po=%p buffer invariants: ptr=%p " "base=%p fence=%p", __LINE__, po, plb->plb_ptr, plb->plb_base, plb->plb_fence)); #endif po->po_curbuf = plb; /* update stats */ atomic_add_int(&pmc_stats.pm_buffer_requests, 1); if (plb == NULL) atomic_add_int(&pmc_stats.pm_buffer_requests_failed, 1); return (plb ? 0 : ENOMEM); } /* * Log handler loop. * * This function is executed by each pmc owner's helper thread. */ static void pmclog_loop(void *arg) { int error; struct pmc_owner *po; struct pmclog_buffer *lb; struct proc *p; struct ucred *ownercred; struct ucred *mycred; struct thread *td; struct uio auio; struct iovec aiov; size_t nbytes; po = (struct pmc_owner *) arg; p = po->po_owner; td = curthread; mycred = td->td_ucred; PROC_LOCK(p); ownercred = crhold(p->p_ucred); PROC_UNLOCK(p); PMCDBG2(LOG,INI,1, "po=%p kt=%p", po, po->po_kthread); KASSERT(po->po_kthread == curthread->td_proc, ("[pmclog,%d] proc mismatch po=%p po/kt=%p curproc=%p", __LINE__, po, po->po_kthread, curthread->td_proc)); lb = NULL; /* * Loop waiting for I/O requests to be added to the owner * struct's queue. The loop is exited when the log file * is deconfigured. */ mtx_lock(&pmc_kthread_mtx); for (;;) { /* check if we've been asked to exit */ if ((po->po_flags & PMC_PO_OWNS_LOGFILE) == 0) break; if (lb == NULL) { /* look for a fresh buffer to write */ mtx_lock_spin(&po->po_mtx); if ((lb = TAILQ_FIRST(&po->po_logbuffers)) == NULL) { mtx_unlock_spin(&po->po_mtx); /* No more buffers and shutdown required. */ if (po->po_flags & PMC_PO_SHUTDOWN) { mtx_unlock(&pmc_kthread_mtx); /* * Close the file to get PMCLOG_EOF * error in pmclog(3). */ fo_close(po->po_file, curthread); mtx_lock(&pmc_kthread_mtx); break; } (void) msleep(po, &pmc_kthread_mtx, PWAIT, "pmcloop", 0); continue; } TAILQ_REMOVE(&po->po_logbuffers, lb, plb_next); mtx_unlock_spin(&po->po_mtx); } mtx_unlock(&pmc_kthread_mtx); /* process the request */ PMCDBG3(LOG,WRI,2, "po=%p base=%p ptr=%p", po, lb->plb_base, lb->plb_ptr); /* change our thread's credentials before issuing the I/O */ aiov.iov_base = lb->plb_base; aiov.iov_len = nbytes = lb->plb_ptr - lb->plb_base; auio.uio_iov = &aiov; auio.uio_iovcnt = 1; auio.uio_offset = -1; auio.uio_resid = nbytes; auio.uio_rw = UIO_WRITE; auio.uio_segflg = UIO_SYSSPACE; auio.uio_td = td; /* switch thread credentials -- see kern_ktrace.c */ td->td_ucred = ownercred; error = fo_write(po->po_file, &auio, ownercred, 0, td); td->td_ucred = mycred; if (error) { /* XXX some errors are recoverable */ /* send a SIGIO to the owner and exit */ PROC_LOCK(p); kern_psignal(p, SIGIO); PROC_UNLOCK(p); mtx_lock(&pmc_kthread_mtx); po->po_error = error; /* save for flush log */ PMCDBG2(LOG,WRI,2, "po=%p error=%d", po, error); break; } mtx_lock(&pmc_kthread_mtx); /* put the used buffer back into the global pool */ PMCLOG_INIT_BUFFER_DESCRIPTOR(lb); mtx_lock_spin(&pmc_bufferlist_mtx); TAILQ_INSERT_HEAD(&pmc_bufferlist, lb, plb_next); mtx_unlock_spin(&pmc_bufferlist_mtx); lb = NULL; } wakeup_one(po->po_kthread); po->po_kthread = NULL; mtx_unlock(&pmc_kthread_mtx); /* return the current I/O buffer to the global pool */ if (lb) { PMCLOG_INIT_BUFFER_DESCRIPTOR(lb); mtx_lock_spin(&pmc_bufferlist_mtx); TAILQ_INSERT_HEAD(&pmc_bufferlist, lb, plb_next); mtx_unlock_spin(&pmc_bufferlist_mtx); } /* * Exit this thread, signalling the waiter */ crfree(ownercred); kproc_exit(0); } /* * Release and log entry and schedule an I/O if needed. */ static void pmclog_release(struct pmc_owner *po) { KASSERT(po->po_curbuf->plb_ptr >= po->po_curbuf->plb_base, ("[pmclog,%d] buffer invariants po=%p ptr=%p base=%p", __LINE__, po, po->po_curbuf->plb_ptr, po->po_curbuf->plb_base)); KASSERT(po->po_curbuf->plb_ptr <= po->po_curbuf->plb_fence, ("[pmclog,%d] buffer invariants po=%p ptr=%p fenc=%p", __LINE__, po, po->po_curbuf->plb_ptr, po->po_curbuf->plb_fence)); /* schedule an I/O if we've filled a buffer */ if (po->po_curbuf->plb_ptr >= po->po_curbuf->plb_fence) pmclog_schedule_io(po); mtx_unlock_spin(&po->po_mtx); PMCDBG1(LOG,REL,1, "po=%p", po); } /* * Attempt to reserve 'length' bytes of space in an owner's log * buffer. The function returns a pointer to 'length' bytes of space * if there was enough space or returns NULL if no space was * available. Non-null returns do so with the po mutex locked. The * caller must invoke pmclog_release() on the pmc owner structure * when done. */ static uint32_t * pmclog_reserve(struct pmc_owner *po, int length) { uintptr_t newptr, oldptr; uint32_t *lh; struct timespec ts; PMCDBG2(LOG,ALL,1, "po=%p len=%d", po, length); KASSERT(length % sizeof(uint32_t) == 0, ("[pmclog,%d] length not a multiple of word size", __LINE__)); mtx_lock_spin(&po->po_mtx); /* No more data when shutdown in progress. */ if (po->po_flags & PMC_PO_SHUTDOWN) { mtx_unlock_spin(&po->po_mtx); return (NULL); } if (po->po_curbuf == NULL) if (pmclog_get_buffer(po) != 0) { mtx_unlock_spin(&po->po_mtx); return (NULL); } KASSERT(po->po_curbuf != NULL, ("[pmclog,%d] po=%p no current buffer", __LINE__, po)); KASSERT(po->po_curbuf->plb_ptr >= po->po_curbuf->plb_base && po->po_curbuf->plb_ptr <= po->po_curbuf->plb_fence, ("[pmclog,%d] po=%p buffer invariants: ptr=%p base=%p fence=%p", __LINE__, po, po->po_curbuf->plb_ptr, po->po_curbuf->plb_base, po->po_curbuf->plb_fence)); oldptr = (uintptr_t) po->po_curbuf->plb_ptr; newptr = oldptr + length; KASSERT(oldptr != (uintptr_t) NULL, ("[pmclog,%d] po=%p Null log buffer pointer", __LINE__, po)); /* * If we have space in the current buffer, return a pointer to * available space with the PO structure locked. */ if (newptr <= (uintptr_t) po->po_curbuf->plb_fence) { po->po_curbuf->plb_ptr = (char *) newptr; goto done; } /* * Otherwise, schedule the current buffer for output and get a * fresh buffer. */ pmclog_schedule_io(po); if (pmclog_get_buffer(po) != 0) { mtx_unlock_spin(&po->po_mtx); return (NULL); } KASSERT(po->po_curbuf != NULL, ("[pmclog,%d] po=%p no current buffer", __LINE__, po)); KASSERT(po->po_curbuf->plb_ptr != NULL, ("[pmclog,%d] null return from pmc_get_log_buffer", __LINE__)); KASSERT(po->po_curbuf->plb_ptr == po->po_curbuf->plb_base && po->po_curbuf->plb_ptr <= po->po_curbuf->plb_fence, ("[pmclog,%d] po=%p buffer invariants: ptr=%p base=%p fence=%p", __LINE__, po, po->po_curbuf->plb_ptr, po->po_curbuf->plb_base, po->po_curbuf->plb_fence)); oldptr = (uintptr_t) po->po_curbuf->plb_ptr; done: lh = (uint32_t *) oldptr; lh++; /* skip header */ getnanotime(&ts); /* fill in the timestamp */ *lh++ = ts.tv_sec & 0xFFFFFFFF; *lh++ = ts.tv_nsec & 0xFFFFFFF; return ((uint32_t *) oldptr); } /* * Schedule an I/O. * * Transfer the current buffer to the helper kthread. */ static void pmclog_schedule_io(struct pmc_owner *po) { KASSERT(po->po_curbuf != NULL, ("[pmclog,%d] schedule_io with null buffer po=%p", __LINE__, po)); KASSERT(po->po_curbuf->plb_ptr >= po->po_curbuf->plb_base, ("[pmclog,%d] buffer invariants po=%p ptr=%p base=%p", __LINE__, po, po->po_curbuf->plb_ptr, po->po_curbuf->plb_base)); KASSERT(po->po_curbuf->plb_ptr <= po->po_curbuf->plb_fence, ("[pmclog,%d] buffer invariants po=%p ptr=%p fenc=%p", __LINE__, po, po->po_curbuf->plb_ptr, po->po_curbuf->plb_fence)); PMCDBG1(LOG,SIO, 1, "po=%p", po); mtx_assert(&po->po_mtx, MA_OWNED); /* * Add the current buffer to the tail of the buffer list and * wakeup the helper. */ TAILQ_INSERT_TAIL(&po->po_logbuffers, po->po_curbuf, plb_next); po->po_curbuf = NULL; wakeup_one(po); } /* * Stop the helper kthread. */ static void pmclog_stop_kthread(struct pmc_owner *po) { /* * Close the file to force the thread out of fo_write, * unset flag, wakeup the helper thread, * wait for it to exit */ if (po->po_file != NULL) fo_close(po->po_file, curthread); mtx_lock(&pmc_kthread_mtx); po->po_flags &= ~PMC_PO_OWNS_LOGFILE; wakeup_one(po); if (po->po_kthread) msleep(po->po_kthread, &pmc_kthread_mtx, PPAUSE, "pmckstp", 0); mtx_unlock(&pmc_kthread_mtx); } /* * Public functions */ /* * Configure a log file for pmc owner 'po'. * * Parameter 'logfd' is a file handle referencing an open file in the * owner process. This file needs to have been opened for writing. */ int pmclog_configure_log(struct pmc_mdep *md, struct pmc_owner *po, int logfd) { int error; struct proc *p; cap_rights_t rights; /* * As long as it is possible to get a LOR between pmc_sx lock and * proctree/allproc sx locks used for adding a new process, assure * the former is not held here. */ sx_assert(&pmc_sx, SA_UNLOCKED); PMCDBG2(LOG,CFG,1, "config po=%p logfd=%d", po, logfd); p = po->po_owner; /* return EBUSY if a log file was already present */ if (po->po_flags & PMC_PO_OWNS_LOGFILE) return (EBUSY); KASSERT(po->po_kthread == NULL, ("[pmclog,%d] po=%p kthread (%p) already present", __LINE__, po, po->po_kthread)); KASSERT(po->po_file == NULL, ("[pmclog,%d] po=%p file (%p) already present", __LINE__, po, po->po_file)); /* get a reference to the file state */ error = fget_write(curthread, logfd, cap_rights_init(&rights, CAP_WRITE), &po->po_file); if (error) goto error; /* mark process as owning a log file */ po->po_flags |= PMC_PO_OWNS_LOGFILE; error = kproc_create(pmclog_loop, po, &po->po_kthread, RFHIGHPID, 0, "hwpmc: proc(%d)", p->p_pid); if (error) goto error; /* mark process as using HWPMCs */ PROC_LOCK(p); p->p_flag |= P_HWPMC; PROC_UNLOCK(p); /* create a log initialization entry */ PMCLOG_RESERVE_WITH_ERROR(po, INITIALIZE, sizeof(struct pmclog_initialize)); PMCLOG_EMIT32(PMC_VERSION); PMCLOG_EMIT32(md->pmd_cputype); PMCLOG_DESPATCH(po); return (0); error: /* shutdown the thread */ if (po->po_kthread) pmclog_stop_kthread(po); KASSERT(po->po_kthread == NULL, ("[pmclog,%d] po=%p kthread not " "stopped", __LINE__, po)); if (po->po_file) (void) fdrop(po->po_file, curthread); po->po_file = NULL; /* clear file and error state */ po->po_error = 0; return (error); } /* * De-configure a log file. This will throw away any buffers queued * for this owner process. */ int pmclog_deconfigure_log(struct pmc_owner *po) { int error; struct pmclog_buffer *lb; PMCDBG1(LOG,CFG,1, "de-config po=%p", po); if ((po->po_flags & PMC_PO_OWNS_LOGFILE) == 0) return (EINVAL); KASSERT(po->po_sscount == 0, ("[pmclog,%d] po=%p still owning SS PMCs", __LINE__, po)); KASSERT(po->po_file != NULL, ("[pmclog,%d] po=%p no log file", __LINE__, po)); /* stop the kthread, this will reset the 'OWNS_LOGFILE' flag */ pmclog_stop_kthread(po); KASSERT(po->po_kthread == NULL, ("[pmclog,%d] po=%p kthread not stopped", __LINE__, po)); /* return all queued log buffers to the global pool */ while ((lb = TAILQ_FIRST(&po->po_logbuffers)) != NULL) { TAILQ_REMOVE(&po->po_logbuffers, lb, plb_next); PMCLOG_INIT_BUFFER_DESCRIPTOR(lb); mtx_lock_spin(&pmc_bufferlist_mtx); TAILQ_INSERT_HEAD(&pmc_bufferlist, lb, plb_next); mtx_unlock_spin(&pmc_bufferlist_mtx); } /* return the 'current' buffer to the global pool */ if ((lb = po->po_curbuf) != NULL) { PMCLOG_INIT_BUFFER_DESCRIPTOR(lb); mtx_lock_spin(&pmc_bufferlist_mtx); TAILQ_INSERT_HEAD(&pmc_bufferlist, lb, plb_next); mtx_unlock_spin(&pmc_bufferlist_mtx); } /* drop a reference to the fd */ error = fdrop(po->po_file, curthread); po->po_file = NULL; po->po_error = 0; return (error); } /* * Flush a process' log buffer. */ int pmclog_flush(struct pmc_owner *po) { int error; struct pmclog_buffer *lb; PMCDBG1(LOG,FLS,1, "po=%p", po); /* * If there is a pending error recorded by the logger thread, * return that. */ if (po->po_error) return (po->po_error); error = 0; /* * Check that we do have an active log file. */ mtx_lock(&pmc_kthread_mtx); if ((po->po_flags & PMC_PO_OWNS_LOGFILE) == 0) { error = EINVAL; goto error; } /* * Schedule the current buffer if any and not empty. */ mtx_lock_spin(&po->po_mtx); lb = po->po_curbuf; if (lb && lb->plb_ptr != lb->plb_base) { pmclog_schedule_io(po); } else error = ENOBUFS; mtx_unlock_spin(&po->po_mtx); error: mtx_unlock(&pmc_kthread_mtx); return (error); } int pmclog_close(struct pmc_owner *po) { PMCDBG1(LOG,CLO,1, "po=%p", po); mtx_lock(&pmc_kthread_mtx); /* * Schedule the current buffer. */ mtx_lock_spin(&po->po_mtx); if (po->po_curbuf) pmclog_schedule_io(po); else wakeup_one(po); mtx_unlock_spin(&po->po_mtx); /* * Initiate shutdown: no new data queued, * thread will close file on last block. */ po->po_flags |= PMC_PO_SHUTDOWN; mtx_unlock(&pmc_kthread_mtx); return (0); } void pmclog_process_callchain(struct pmc *pm, struct pmc_sample *ps) { int n, recordlen; uint32_t flags; struct pmc_owner *po; PMCDBG3(LOG,SAM,1,"pm=%p pid=%d n=%d", pm, ps->ps_pid, ps->ps_nsamples); recordlen = offsetof(struct pmclog_callchain, pl_pc) + ps->ps_nsamples * sizeof(uintfptr_t); po = pm->pm_owner; flags = PMC_CALLCHAIN_TO_CPUFLAGS(ps->ps_cpu,ps->ps_flags); PMCLOG_RESERVE(po, CALLCHAIN, recordlen); PMCLOG_EMIT32(ps->ps_pid); PMCLOG_EMIT32(pm->pm_id); PMCLOG_EMIT32(flags); for (n = 0; n < ps->ps_nsamples; n++) PMCLOG_EMITADDR(ps->ps_pc[n]); PMCLOG_DESPATCH(po); } void pmclog_process_closelog(struct pmc_owner *po) { PMCLOG_RESERVE(po,CLOSELOG,sizeof(struct pmclog_closelog)); PMCLOG_DESPATCH(po); } void pmclog_process_dropnotify(struct pmc_owner *po) { PMCLOG_RESERVE(po,DROPNOTIFY,sizeof(struct pmclog_dropnotify)); PMCLOG_DESPATCH(po); } void pmclog_process_map_in(struct pmc_owner *po, pid_t pid, uintfptr_t start, const char *path) { int pathlen, recordlen; KASSERT(path != NULL, ("[pmclog,%d] map-in, null path", __LINE__)); pathlen = strlen(path) + 1; /* #bytes for path name */ recordlen = offsetof(struct pmclog_map_in, pl_pathname) + pathlen; PMCLOG_RESERVE(po, MAP_IN, recordlen); PMCLOG_EMIT32(pid); PMCLOG_EMITADDR(start); PMCLOG_EMITSTRING(path,pathlen); PMCLOG_DESPATCH(po); } void pmclog_process_map_out(struct pmc_owner *po, pid_t pid, uintfptr_t start, uintfptr_t end) { KASSERT(start <= end, ("[pmclog,%d] start > end", __LINE__)); PMCLOG_RESERVE(po, MAP_OUT, sizeof(struct pmclog_map_out)); PMCLOG_EMIT32(pid); PMCLOG_EMITADDR(start); PMCLOG_EMITADDR(end); PMCLOG_DESPATCH(po); } void pmclog_process_pmcallocate(struct pmc *pm) { struct pmc_owner *po; struct pmc_soft *ps; po = pm->pm_owner; PMCDBG1(LOG,ALL,1, "pm=%p", pm); if (PMC_TO_CLASS(pm) == PMC_CLASS_SOFT) { PMCLOG_RESERVE(po, PMCALLOCATEDYN, sizeof(struct pmclog_pmcallocatedyn)); PMCLOG_EMIT32(pm->pm_id); PMCLOG_EMIT32(pm->pm_event); PMCLOG_EMIT32(pm->pm_flags); ps = pmc_soft_ev_acquire(pm->pm_event); if (ps != NULL) PMCLOG_EMITSTRING(ps->ps_ev.pm_ev_name,PMC_NAME_MAX); else PMCLOG_EMITNULLSTRING(PMC_NAME_MAX); pmc_soft_ev_release(ps); PMCLOG_DESPATCH(po); } else { PMCLOG_RESERVE(po, PMCALLOCATE, sizeof(struct pmclog_pmcallocate)); PMCLOG_EMIT32(pm->pm_id); PMCLOG_EMIT32(pm->pm_event); PMCLOG_EMIT32(pm->pm_flags); PMCLOG_DESPATCH(po); } } void pmclog_process_pmcattach(struct pmc *pm, pid_t pid, char *path) { int pathlen, recordlen; struct pmc_owner *po; PMCDBG2(LOG,ATT,1,"pm=%p pid=%d", pm, pid); po = pm->pm_owner; pathlen = strlen(path) + 1; /* #bytes for the string */ recordlen = offsetof(struct pmclog_pmcattach, pl_pathname) + pathlen; PMCLOG_RESERVE(po, PMCATTACH, recordlen); PMCLOG_EMIT32(pm->pm_id); PMCLOG_EMIT32(pid); PMCLOG_EMITSTRING(path, pathlen); PMCLOG_DESPATCH(po); } void pmclog_process_pmcdetach(struct pmc *pm, pid_t pid) { struct pmc_owner *po; PMCDBG2(LOG,ATT,1,"!pm=%p pid=%d", pm, pid); po = pm->pm_owner; PMCLOG_RESERVE(po, PMCDETACH, sizeof(struct pmclog_pmcdetach)); PMCLOG_EMIT32(pm->pm_id); PMCLOG_EMIT32(pid); PMCLOG_DESPATCH(po); } /* * Log a context switch event to the log file. */ void pmclog_process_proccsw(struct pmc *pm, struct pmc_process *pp, pmc_value_t v) { struct pmc_owner *po; KASSERT(pm->pm_flags & PMC_F_LOG_PROCCSW, ("[pmclog,%d] log-process-csw called gratuitously", __LINE__)); PMCDBG3(LOG,SWO,1,"pm=%p pid=%d v=%jx", pm, pp->pp_proc->p_pid, v); po = pm->pm_owner; PMCLOG_RESERVE(po, PROCCSW, sizeof(struct pmclog_proccsw)); PMCLOG_EMIT32(pm->pm_id); PMCLOG_EMIT64(v); PMCLOG_EMIT32(pp->pp_proc->p_pid); PMCLOG_DESPATCH(po); } void pmclog_process_procexec(struct pmc_owner *po, pmc_id_t pmid, pid_t pid, uintfptr_t startaddr, char *path) { int pathlen, recordlen; PMCDBG3(LOG,EXC,1,"po=%p pid=%d path=\"%s\"", po, pid, path); pathlen = strlen(path) + 1; /* #bytes for the path */ recordlen = offsetof(struct pmclog_procexec, pl_pathname) + pathlen; PMCLOG_RESERVE(po, PROCEXEC, recordlen); PMCLOG_EMIT32(pid); PMCLOG_EMITADDR(startaddr); PMCLOG_EMIT32(pmid); PMCLOG_EMITSTRING(path,pathlen); PMCLOG_DESPATCH(po); } /* * Log a process exit event (and accumulated pmc value) to the log file. */ void pmclog_process_procexit(struct pmc *pm, struct pmc_process *pp) { int ri; struct pmc_owner *po; ri = PMC_TO_ROWINDEX(pm); PMCDBG3(LOG,EXT,1,"pm=%p pid=%d v=%jx", pm, pp->pp_proc->p_pid, pp->pp_pmcs[ri].pp_pmcval); po = pm->pm_owner; PMCLOG_RESERVE(po, PROCEXIT, sizeof(struct pmclog_procexit)); PMCLOG_EMIT32(pm->pm_id); PMCLOG_EMIT64(pp->pp_pmcs[ri].pp_pmcval); PMCLOG_EMIT32(pp->pp_proc->p_pid); PMCLOG_DESPATCH(po); } /* * Log a fork event. */ void pmclog_process_procfork(struct pmc_owner *po, pid_t oldpid, pid_t newpid) { PMCLOG_RESERVE(po, PROCFORK, sizeof(struct pmclog_procfork)); PMCLOG_EMIT32(oldpid); PMCLOG_EMIT32(newpid); PMCLOG_DESPATCH(po); } /* * Log a process exit event of the form suitable for system-wide PMCs. */ void pmclog_process_sysexit(struct pmc_owner *po, pid_t pid) { PMCLOG_RESERVE(po, SYSEXIT, sizeof(struct pmclog_sysexit)); PMCLOG_EMIT32(pid); PMCLOG_DESPATCH(po); } /* * Write a user log entry. */ int pmclog_process_userlog(struct pmc_owner *po, struct pmc_op_writelog *wl) { int error; PMCDBG2(LOG,WRI,1, "writelog po=%p ud=0x%x", po, wl->pm_userdata); error = 0; PMCLOG_RESERVE_WITH_ERROR(po, USERDATA, sizeof(struct pmclog_userdata)); PMCLOG_EMIT32(wl->pm_userdata); PMCLOG_DESPATCH(po); error: return (error); } /* * Initialization. * * Create a pool of log buffers and initialize mutexes. */ void pmclog_initialize() { int n; struct pmclog_buffer *plb; if (pmclog_buffer_size <= 0) { (void) printf("hwpmc: tunable logbuffersize=%d must be " "greater than zero.\n", pmclog_buffer_size); pmclog_buffer_size = PMC_LOG_BUFFER_SIZE; } if (pmc_nlogbuffers <= 0) { (void) printf("hwpmc: tunable nlogbuffers=%d must be greater " "than zero.\n", pmc_nlogbuffers); pmc_nlogbuffers = PMC_NLOGBUFFERS; } /* create global pool of log buffers */ for (n = 0; n < pmc_nlogbuffers; n++) { plb = malloc(1024 * pmclog_buffer_size, M_PMC, M_WAITOK|M_ZERO); PMCLOG_INIT_BUFFER_DESCRIPTOR(plb); TAILQ_INSERT_HEAD(&pmc_bufferlist, plb, plb_next); } mtx_init(&pmc_bufferlist_mtx, "pmc-buffer-list", "pmc-leaf", MTX_SPIN); mtx_init(&pmc_kthread_mtx, "pmc-kthread", "pmc-sleep", MTX_DEF); } /* * Shutdown logging. * * Destroy mutexes and release memory back the to free pool. */ void pmclog_shutdown() { struct pmclog_buffer *plb; mtx_destroy(&pmc_kthread_mtx); mtx_destroy(&pmc_bufferlist_mtx); while ((plb = TAILQ_FIRST(&pmc_bufferlist)) != NULL) { TAILQ_REMOVE(&pmc_bufferlist, plb, plb_next); free(plb, M_PMC); } } Index: user/ngie/more-tests2/sys/dev/ioat/ioat.c =================================================================== --- user/ngie/more-tests2/sys/dev/ioat/ioat.c (revision 292053) +++ user/ngie/more-tests2/sys/dev/ioat/ioat.c (revision 292054) @@ -1,1666 +1,1722 @@ /*- * Copyright (C) 2012 Intel Corporation * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "ioat.h" #include "ioat_hw.h" #include "ioat_internal.h" #define IOAT_INTR_TIMO (hz / 10) #define IOAT_REFLK (&ioat->submit_lock) static int ioat_probe(device_t device); static int ioat_attach(device_t device); static int ioat_detach(device_t device); static int ioat_setup_intr(struct ioat_softc *ioat); static int ioat_teardown_intr(struct ioat_softc *ioat); static int ioat3_attach(device_t device); static int ioat_start_channel(struct ioat_softc *ioat); static int ioat_map_pci_bar(struct ioat_softc *ioat); static void ioat_dmamap_cb(void *arg, bus_dma_segment_t *segs, int nseg, int error); static void ioat_interrupt_handler(void *arg); static boolean_t ioat_model_resets_msix(struct ioat_softc *ioat); static int chanerr_to_errno(uint32_t); static void ioat_process_events(struct ioat_softc *ioat); static inline uint32_t ioat_get_active(struct ioat_softc *ioat); static inline uint32_t ioat_get_ring_space(struct ioat_softc *ioat); static void ioat_free_ring(struct ioat_softc *, uint32_t size, struct ioat_descriptor **); static void ioat_free_ring_entry(struct ioat_softc *ioat, struct ioat_descriptor *desc); static struct ioat_descriptor *ioat_alloc_ring_entry(struct ioat_softc *, int mflags); static int ioat_reserve_space(struct ioat_softc *, uint32_t, int mflags); static struct ioat_descriptor *ioat_get_ring_entry(struct ioat_softc *ioat, uint32_t index); static struct ioat_descriptor **ioat_prealloc_ring(struct ioat_softc *, uint32_t size, boolean_t need_dscr, int mflags); static int ring_grow(struct ioat_softc *, uint32_t oldorder, struct ioat_descriptor **); static int ring_shrink(struct ioat_softc *, uint32_t oldorder, struct ioat_descriptor **); static void ioat_halted_debug(struct ioat_softc *, uint32_t); static void ioat_timer_callback(void *arg); static void dump_descriptor(void *hw_desc); static void ioat_submit_single(struct ioat_softc *ioat); static void ioat_comp_update_map(void *arg, bus_dma_segment_t *seg, int nseg, int error); static int ioat_reset_hw(struct ioat_softc *ioat); static void ioat_setup_sysctl(device_t device); static int sysctl_handle_reset(SYSCTL_HANDLER_ARGS); static inline struct ioat_softc *ioat_get(struct ioat_softc *, enum ioat_ref_kind); static inline void ioat_put(struct ioat_softc *, enum ioat_ref_kind); static inline void _ioat_putn(struct ioat_softc *, uint32_t, enum ioat_ref_kind, boolean_t); static inline void ioat_putn(struct ioat_softc *, uint32_t, enum ioat_ref_kind); static inline void ioat_putn_locked(struct ioat_softc *, uint32_t, enum ioat_ref_kind); static void ioat_drain_locked(struct ioat_softc *); #define ioat_log_message(v, ...) do { \ if ((v) <= g_ioat_debug_level) { \ device_printf(ioat->device, __VA_ARGS__); \ } \ } while (0) MALLOC_DEFINE(M_IOAT, "ioat", "ioat driver memory allocations"); SYSCTL_NODE(_hw, OID_AUTO, ioat, CTLFLAG_RD, 0, "ioat node"); static int g_force_legacy_interrupts; SYSCTL_INT(_hw_ioat, OID_AUTO, force_legacy_interrupts, CTLFLAG_RDTUN, &g_force_legacy_interrupts, 0, "Set to non-zero to force MSI-X disabled"); int g_ioat_debug_level = 0; SYSCTL_INT(_hw_ioat, OID_AUTO, debug_level, CTLFLAG_RWTUN, &g_ioat_debug_level, 0, "Set log level (0-3) for ioat(4). Higher is more verbose."); /* * OS <-> Driver interface structures */ static device_method_t ioat_pci_methods[] = { /* Device interface */ DEVMETHOD(device_probe, ioat_probe), DEVMETHOD(device_attach, ioat_attach), DEVMETHOD(device_detach, ioat_detach), { 0, 0 } }; static driver_t ioat_pci_driver = { "ioat", ioat_pci_methods, sizeof(struct ioat_softc), }; static devclass_t ioat_devclass; DRIVER_MODULE(ioat, pci, ioat_pci_driver, ioat_devclass, 0, 0); MODULE_VERSION(ioat, 1); /* * Private data structures */ static struct ioat_softc *ioat_channel[IOAT_MAX_CHANNELS]; static int ioat_channel_index = 0; SYSCTL_INT(_hw_ioat, OID_AUTO, channels, CTLFLAG_RD, &ioat_channel_index, 0, "Number of IOAT channels attached"); static struct _pcsid { u_int32_t type; const char *desc; } pci_ids[] = { { 0x34308086, "TBG IOAT Ch0" }, { 0x34318086, "TBG IOAT Ch1" }, { 0x34328086, "TBG IOAT Ch2" }, { 0x34338086, "TBG IOAT Ch3" }, { 0x34298086, "TBG IOAT Ch4" }, { 0x342a8086, "TBG IOAT Ch5" }, { 0x342b8086, "TBG IOAT Ch6" }, { 0x342c8086, "TBG IOAT Ch7" }, { 0x37108086, "JSF IOAT Ch0" }, { 0x37118086, "JSF IOAT Ch1" }, { 0x37128086, "JSF IOAT Ch2" }, { 0x37138086, "JSF IOAT Ch3" }, { 0x37148086, "JSF IOAT Ch4" }, { 0x37158086, "JSF IOAT Ch5" }, { 0x37168086, "JSF IOAT Ch6" }, { 0x37178086, "JSF IOAT Ch7" }, { 0x37188086, "JSF IOAT Ch0 (RAID)" }, { 0x37198086, "JSF IOAT Ch1 (RAID)" }, { 0x3c208086, "SNB IOAT Ch0" }, { 0x3c218086, "SNB IOAT Ch1" }, { 0x3c228086, "SNB IOAT Ch2" }, { 0x3c238086, "SNB IOAT Ch3" }, { 0x3c248086, "SNB IOAT Ch4" }, { 0x3c258086, "SNB IOAT Ch5" }, { 0x3c268086, "SNB IOAT Ch6" }, { 0x3c278086, "SNB IOAT Ch7" }, { 0x3c2e8086, "SNB IOAT Ch0 (RAID)" }, { 0x3c2f8086, "SNB IOAT Ch1 (RAID)" }, { 0x0e208086, "IVB IOAT Ch0" }, { 0x0e218086, "IVB IOAT Ch1" }, { 0x0e228086, "IVB IOAT Ch2" }, { 0x0e238086, "IVB IOAT Ch3" }, { 0x0e248086, "IVB IOAT Ch4" }, { 0x0e258086, "IVB IOAT Ch5" }, { 0x0e268086, "IVB IOAT Ch6" }, { 0x0e278086, "IVB IOAT Ch7" }, { 0x0e2e8086, "IVB IOAT Ch0 (RAID)" }, { 0x0e2f8086, "IVB IOAT Ch1 (RAID)" }, { 0x2f208086, "HSW IOAT Ch0" }, { 0x2f218086, "HSW IOAT Ch1" }, { 0x2f228086, "HSW IOAT Ch2" }, { 0x2f238086, "HSW IOAT Ch3" }, { 0x2f248086, "HSW IOAT Ch4" }, { 0x2f258086, "HSW IOAT Ch5" }, { 0x2f268086, "HSW IOAT Ch6" }, { 0x2f278086, "HSW IOAT Ch7" }, { 0x2f2e8086, "HSW IOAT Ch0 (RAID)" }, { 0x2f2f8086, "HSW IOAT Ch1 (RAID)" }, { 0x0c508086, "BWD IOAT Ch0" }, { 0x0c518086, "BWD IOAT Ch1" }, { 0x0c528086, "BWD IOAT Ch2" }, { 0x0c538086, "BWD IOAT Ch3" }, { 0x6f508086, "BDXDE IOAT Ch0" }, { 0x6f518086, "BDXDE IOAT Ch1" }, { 0x6f528086, "BDXDE IOAT Ch2" }, { 0x6f538086, "BDXDE IOAT Ch3" }, + { 0x6f208086, "BDX IOAT Ch0" }, + { 0x6f218086, "BDX IOAT Ch1" }, + { 0x6f228086, "BDX IOAT Ch2" }, + { 0x6f238086, "BDX IOAT Ch3" }, + { 0x6f248086, "BDX IOAT Ch4" }, + { 0x6f258086, "BDX IOAT Ch5" }, + { 0x6f268086, "BDX IOAT Ch6" }, + { 0x6f278086, "BDX IOAT Ch7" }, + { 0x6f2e8086, "BDX IOAT Ch0 (RAID)" }, + { 0x6f2f8086, "BDX IOAT Ch1 (RAID)" }, + { 0x00000000, NULL } }; /* * OS <-> Driver linkage functions */ static int ioat_probe(device_t device) { struct _pcsid *ep; u_int32_t type; type = pci_get_devid(device); for (ep = pci_ids; ep->type; ep++) { if (ep->type == type) { device_set_desc(device, ep->desc); return (0); } } return (ENXIO); } static int ioat_attach(device_t device) { struct ioat_softc *ioat; int error; ioat = DEVICE2SOFTC(device); ioat->device = device; error = ioat_map_pci_bar(ioat); if (error != 0) goto err; ioat->version = ioat_read_cbver(ioat); if (ioat->version < IOAT_VER_3_0) { error = ENODEV; goto err; } error = ioat3_attach(device); if (error != 0) goto err; error = pci_enable_busmaster(device); if (error != 0) goto err; error = ioat_setup_intr(ioat); if (error != 0) goto err; error = ioat_reset_hw(ioat); if (error != 0) goto err; ioat_process_events(ioat); ioat_setup_sysctl(device); ioat->chan_idx = ioat_channel_index; ioat_channel[ioat_channel_index++] = ioat; ioat_test_attach(); err: if (error != 0) ioat_detach(device); return (error); } static int ioat_detach(device_t device) { struct ioat_softc *ioat; ioat = DEVICE2SOFTC(device); ioat_test_detach(); mtx_lock(IOAT_REFLK); ioat->quiescing = TRUE; ioat_channel[ioat->chan_idx] = NULL; ioat_drain_locked(ioat); mtx_unlock(IOAT_REFLK); ioat_teardown_intr(ioat); callout_drain(&ioat->timer); pci_disable_busmaster(device); if (ioat->pci_resource != NULL) bus_release_resource(device, SYS_RES_MEMORY, ioat->pci_resource_id, ioat->pci_resource); if (ioat->ring != NULL) ioat_free_ring(ioat, 1 << ioat->ring_size_order, ioat->ring); if (ioat->comp_update != NULL) { bus_dmamap_unload(ioat->comp_update_tag, ioat->comp_update_map); bus_dmamem_free(ioat->comp_update_tag, ioat->comp_update, ioat->comp_update_map); bus_dma_tag_destroy(ioat->comp_update_tag); } bus_dma_tag_destroy(ioat->hw_desc_tag); return (0); } static int ioat_teardown_intr(struct ioat_softc *ioat) { if (ioat->tag != NULL) bus_teardown_intr(ioat->device, ioat->res, ioat->tag); if (ioat->res != NULL) bus_release_resource(ioat->device, SYS_RES_IRQ, rman_get_rid(ioat->res), ioat->res); pci_release_msi(ioat->device); return (0); } static int ioat_start_channel(struct ioat_softc *ioat) { uint64_t status; uint32_t chanerr; int i; ioat_acquire(&ioat->dmaengine); ioat_null(&ioat->dmaengine, NULL, NULL, 0); ioat_release(&ioat->dmaengine); for (i = 0; i < 100; i++) { DELAY(1); status = ioat_get_chansts(ioat); if (is_ioat_idle(status)) return (0); } chanerr = ioat_read_4(ioat, IOAT_CHANERR_OFFSET); ioat_log_message(0, "could not start channel: " "status = %#jx error = %b\n", (uintmax_t)status, (int)chanerr, IOAT_CHANERR_STR); return (ENXIO); } /* * Initialize Hardware */ static int ioat3_attach(device_t device) { struct ioat_softc *ioat; struct ioat_descriptor **ring; struct ioat_descriptor *next; struct ioat_dma_hw_descriptor *dma_hw_desc; int i, num_descriptors; int error; uint8_t xfercap; error = 0; ioat = DEVICE2SOFTC(device); ioat->capabilities = ioat_read_dmacapability(ioat); ioat_log_message(1, "Capabilities: %b\n", (int)ioat->capabilities, IOAT_DMACAP_STR); xfercap = ioat_read_xfercap(ioat); ioat->max_xfer_size = 1 << xfercap; /* TODO: need to check DCA here if we ever do XOR/PQ */ mtx_init(&ioat->submit_lock, "ioat_submit", NULL, MTX_DEF); mtx_init(&ioat->cleanup_lock, "ioat_cleanup", NULL, MTX_DEF); callout_init(&ioat->timer, 1); /* Establish lock order for Witness */ mtx_lock(&ioat->submit_lock); mtx_lock(&ioat->cleanup_lock); mtx_unlock(&ioat->cleanup_lock); mtx_unlock(&ioat->submit_lock); ioat->is_resize_pending = FALSE; ioat->is_completion_pending = FALSE; ioat->is_reset_pending = FALSE; ioat->is_channel_running = FALSE; bus_dma_tag_create(bus_get_dma_tag(ioat->device), sizeof(uint64_t), 0x0, BUS_SPACE_MAXADDR, BUS_SPACE_MAXADDR, NULL, NULL, sizeof(uint64_t), 1, sizeof(uint64_t), 0, NULL, NULL, &ioat->comp_update_tag); error = bus_dmamem_alloc(ioat->comp_update_tag, (void **)&ioat->comp_update, BUS_DMA_ZERO, &ioat->comp_update_map); if (ioat->comp_update == NULL) return (ENOMEM); error = bus_dmamap_load(ioat->comp_update_tag, ioat->comp_update_map, ioat->comp_update, sizeof(uint64_t), ioat_comp_update_map, ioat, 0); if (error != 0) return (error); ioat->ring_size_order = IOAT_MIN_ORDER; num_descriptors = 1 << ioat->ring_size_order; bus_dma_tag_create(bus_get_dma_tag(ioat->device), 0x40, 0x0, BUS_SPACE_MAXADDR, BUS_SPACE_MAXADDR, NULL, NULL, sizeof(struct ioat_dma_hw_descriptor), 1, sizeof(struct ioat_dma_hw_descriptor), 0, NULL, NULL, &ioat->hw_desc_tag); ioat->ring = malloc(num_descriptors * sizeof(*ring), M_IOAT, M_ZERO | M_WAITOK); if (ioat->ring == NULL) return (ENOMEM); ring = ioat->ring; for (i = 0; i < num_descriptors; i++) { ring[i] = ioat_alloc_ring_entry(ioat, M_WAITOK); if (ring[i] == NULL) return (ENOMEM); ring[i]->id = i; } for (i = 0; i < num_descriptors - 1; i++) { next = ring[i + 1]; dma_hw_desc = ring[i]->u.dma; dma_hw_desc->next = next->hw_desc_bus_addr; } ring[i]->u.dma->next = ring[0]->hw_desc_bus_addr; ioat->head = ioat->hw_head = 0; ioat->tail = 0; ioat->last_seen = 0; return (0); } static int ioat_map_pci_bar(struct ioat_softc *ioat) { ioat->pci_resource_id = PCIR_BAR(0); ioat->pci_resource = bus_alloc_resource_any(ioat->device, SYS_RES_MEMORY, &ioat->pci_resource_id, RF_ACTIVE); if (ioat->pci_resource == NULL) { ioat_log_message(0, "unable to allocate pci resource\n"); return (ENODEV); } ioat->pci_bus_tag = rman_get_bustag(ioat->pci_resource); ioat->pci_bus_handle = rman_get_bushandle(ioat->pci_resource); return (0); } static void ioat_comp_update_map(void *arg, bus_dma_segment_t *seg, int nseg, int error) { struct ioat_softc *ioat = arg; KASSERT(error == 0, ("%s: error:%d", __func__, error)); ioat->comp_update_bus_addr = seg[0].ds_addr; } static void ioat_dmamap_cb(void *arg, bus_dma_segment_t *segs, int nseg, int error) { bus_addr_t *baddr; KASSERT(error == 0, ("%s: error:%d", __func__, error)); baddr = arg; *baddr = segs->ds_addr; } /* * Interrupt setup and handlers */ static int ioat_setup_intr(struct ioat_softc *ioat) { uint32_t num_vectors; int error; boolean_t use_msix; boolean_t force_legacy_interrupts; use_msix = FALSE; force_legacy_interrupts = FALSE; if (!g_force_legacy_interrupts && pci_msix_count(ioat->device) >= 1) { num_vectors = 1; pci_alloc_msix(ioat->device, &num_vectors); if (num_vectors == 1) use_msix = TRUE; } if (use_msix) { ioat->rid = 1; ioat->res = bus_alloc_resource_any(ioat->device, SYS_RES_IRQ, &ioat->rid, RF_ACTIVE); } else { ioat->rid = 0; ioat->res = bus_alloc_resource_any(ioat->device, SYS_RES_IRQ, &ioat->rid, RF_SHAREABLE | RF_ACTIVE); } if (ioat->res == NULL) { ioat_log_message(0, "bus_alloc_resource failed\n"); return (ENOMEM); } ioat->tag = NULL; error = bus_setup_intr(ioat->device, ioat->res, INTR_MPSAFE | INTR_TYPE_MISC, NULL, ioat_interrupt_handler, ioat, &ioat->tag); if (error != 0) { ioat_log_message(0, "bus_setup_intr failed\n"); return (error); } ioat_write_intrctrl(ioat, IOAT_INTRCTRL_MASTER_INT_EN); return (0); } static boolean_t ioat_model_resets_msix(struct ioat_softc *ioat) { u_int32_t pciid; pciid = pci_get_devid(ioat->device); switch (pciid) { /* BWD: */ case 0x0c508086: case 0x0c518086: case 0x0c528086: case 0x0c538086: /* BDXDE: */ case 0x6f508086: case 0x6f518086: case 0x6f528086: case 0x6f538086: return (TRUE); } return (FALSE); } static void ioat_interrupt_handler(void *arg) { struct ioat_softc *ioat = arg; ioat_process_events(ioat); } static int chanerr_to_errno(uint32_t chanerr) { if (chanerr == 0) return (0); if ((chanerr & (IOAT_CHANERR_XSADDERR | IOAT_CHANERR_XDADDERR)) != 0) return (EFAULT); if ((chanerr & (IOAT_CHANERR_RDERR | IOAT_CHANERR_WDERR)) != 0) return (EIO); /* This one is probably our fault: */ if ((chanerr & IOAT_CHANERR_NDADDERR) != 0) return (EIO); return (EIO); } static void ioat_process_events(struct ioat_softc *ioat) { struct ioat_descriptor *desc; struct bus_dmadesc *dmadesc; uint64_t comp_update, status; uint32_t completed, chanerr; int error; mtx_lock(&ioat->cleanup_lock); completed = 0; comp_update = *ioat->comp_update; status = comp_update & IOAT_CHANSTS_COMPLETED_DESCRIPTOR_MASK; CTR0(KTR_IOAT, __func__); if (status == ioat->last_seen) goto out; while (1) { desc = ioat_get_ring_entry(ioat, ioat->tail); dmadesc = &desc->bus_dmadesc; CTR1(KTR_IOAT, "completing desc %d", ioat->tail); if (dmadesc->callback_fn != NULL) dmadesc->callback_fn(dmadesc->callback_arg, 0); completed++; ioat->tail++; if (desc->hw_desc_bus_addr == status) break; } ioat->last_seen = desc->hw_desc_bus_addr; if (ioat->head == ioat->tail) { ioat->is_completion_pending = FALSE; callout_reset(&ioat->timer, IOAT_INTR_TIMO, ioat_timer_callback, ioat); } out: ioat_write_chanctrl(ioat, IOAT_CHANCTRL_RUN); mtx_unlock(&ioat->cleanup_lock); ioat_putn(ioat, completed, IOAT_ACTIVE_DESCR_REF); wakeup(&ioat->tail); if (!is_ioat_halted(comp_update)) return; /* * Fatal programming error on this DMA channel. Flush any outstanding * work with error status and restart the engine. */ ioat_log_message(0, "Channel halted due to fatal programming error\n"); mtx_lock(&ioat->submit_lock); mtx_lock(&ioat->cleanup_lock); ioat->quiescing = TRUE; chanerr = ioat_read_4(ioat, IOAT_CHANERR_OFFSET); ioat_halted_debug(ioat, chanerr); while (ioat_get_active(ioat) > 0) { desc = ioat_get_ring_entry(ioat, ioat->tail); dmadesc = &desc->bus_dmadesc; CTR1(KTR_IOAT, "completing err desc %d", ioat->tail); if (dmadesc->callback_fn != NULL) dmadesc->callback_fn(dmadesc->callback_arg, chanerr_to_errno(chanerr)); ioat_putn_locked(ioat, 1, IOAT_ACTIVE_DESCR_REF); ioat->tail++; } /* Clear error status */ ioat_write_4(ioat, IOAT_CHANERR_OFFSET, chanerr); mtx_unlock(&ioat->cleanup_lock); mtx_unlock(&ioat->submit_lock); ioat_log_message(0, "Resetting channel to recover from error\n"); error = ioat_reset_hw(ioat); KASSERT(error == 0, ("%s: reset failed: %d", __func__, error)); } /* * User API functions */ bus_dmaengine_t ioat_get_dmaengine(uint32_t index) { struct ioat_softc *sc; if (index >= ioat_channel_index) return (NULL); sc = ioat_channel[index]; if (sc == NULL || sc->quiescing) return (NULL); return (&ioat_get(sc, IOAT_DMAENGINE_REF)->dmaengine); } void ioat_put_dmaengine(bus_dmaengine_t dmaengine) { struct ioat_softc *ioat; ioat = to_ioat_softc(dmaengine); ioat_put(ioat, IOAT_DMAENGINE_REF); } void ioat_acquire(bus_dmaengine_t dmaengine) { struct ioat_softc *ioat; ioat = to_ioat_softc(dmaengine); mtx_lock(&ioat->submit_lock); CTR0(KTR_IOAT, __func__); } void ioat_release(bus_dmaengine_t dmaengine) { struct ioat_softc *ioat; ioat = to_ioat_softc(dmaengine); CTR0(KTR_IOAT, __func__); ioat_write_2(ioat, IOAT_DMACOUNT_OFFSET, (uint16_t)ioat->hw_head); mtx_unlock(&ioat->submit_lock); } static struct ioat_descriptor * ioat_op_generic(struct ioat_softc *ioat, uint8_t op, uint32_t size, uint64_t src, uint64_t dst, bus_dmaengine_callback_t callback_fn, void *callback_arg, uint32_t flags) { struct ioat_generic_hw_descriptor *hw_desc; struct ioat_descriptor *desc; int mflags; mtx_assert(&ioat->submit_lock, MA_OWNED); KASSERT((flags & ~DMA_ALL_FLAGS) == 0, ("Unrecognized flag(s): %#x", flags & ~DMA_ALL_FLAGS)); if ((flags & DMA_NO_WAIT) != 0) mflags = M_NOWAIT; else mflags = M_WAITOK; if (size > ioat->max_xfer_size) { ioat_log_message(0, "%s: max_xfer_size = %d, requested = %u\n", __func__, ioat->max_xfer_size, (unsigned)size); return (NULL); } if (ioat_reserve_space(ioat, 1, mflags) != 0) return (NULL); desc = ioat_get_ring_entry(ioat, ioat->head); hw_desc = desc->u.generic; hw_desc->u.control_raw = 0; hw_desc->u.control_generic.op = op; hw_desc->u.control_generic.completion_update = 1; if ((flags & DMA_INT_EN) != 0) hw_desc->u.control_generic.int_enable = 1; hw_desc->size = size; hw_desc->src_addr = src; hw_desc->dest_addr = dst; desc->bus_dmadesc.callback_fn = callback_fn; desc->bus_dmadesc.callback_arg = callback_arg; return (desc); } struct bus_dmadesc * ioat_null(bus_dmaengine_t dmaengine, bus_dmaengine_callback_t callback_fn, void *callback_arg, uint32_t flags) { struct ioat_dma_hw_descriptor *hw_desc; struct ioat_descriptor *desc; struct ioat_softc *ioat; CTR0(KTR_IOAT, __func__); ioat = to_ioat_softc(dmaengine); desc = ioat_op_generic(ioat, IOAT_OP_COPY, 8, 0, 0, callback_fn, callback_arg, flags); if (desc == NULL) return (NULL); hw_desc = desc->u.dma; hw_desc->u.control.null = 1; ioat_submit_single(ioat); return (&desc->bus_dmadesc); } struct bus_dmadesc * ioat_copy(bus_dmaengine_t dmaengine, bus_addr_t dst, bus_addr_t src, bus_size_t len, bus_dmaengine_callback_t callback_fn, void *callback_arg, uint32_t flags) { struct ioat_dma_hw_descriptor *hw_desc; struct ioat_descriptor *desc; struct ioat_softc *ioat; CTR0(KTR_IOAT, __func__); ioat = to_ioat_softc(dmaengine); if (((src | dst) & (0xffffull << 48)) != 0) { ioat_log_message(0, "%s: High 16 bits of src/dst invalid\n", __func__); return (NULL); } desc = ioat_op_generic(ioat, IOAT_OP_COPY, len, src, dst, callback_fn, callback_arg, flags); if (desc == NULL) return (NULL); hw_desc = desc->u.dma; + if (g_ioat_debug_level >= 3) + dump_descriptor(hw_desc); + + ioat_submit_single(ioat); + return (&desc->bus_dmadesc); +} + +struct bus_dmadesc * +ioat_copy_8k_aligned(bus_dmaengine_t dmaengine, bus_addr_t dst1, + bus_addr_t dst2, bus_addr_t src1, bus_addr_t src2, + bus_dmaengine_callback_t callback_fn, void *callback_arg, uint32_t flags) +{ + struct ioat_dma_hw_descriptor *hw_desc; + struct ioat_descriptor *desc; + struct ioat_softc *ioat; + + CTR0(KTR_IOAT, __func__); + ioat = to_ioat_softc(dmaengine); + + if (((src1 | src2 | dst1 | dst2) & (0xffffull << 48)) != 0) { + ioat_log_message(0, "%s: High 16 bits of src/dst invalid\n", + __func__); + return (NULL); + } + if (((src1 | src2 | dst1 | dst2) & PAGE_MASK) != 0) { + ioat_log_message(0, "%s: Addresses must be page-aligned\n", + __func__); + return (NULL); + } + + desc = ioat_op_generic(ioat, IOAT_OP_COPY, 2 * PAGE_SIZE, src1, dst1, + callback_fn, callback_arg, flags); + if (desc == NULL) + return (NULL); + + hw_desc = desc->u.dma; + if (src2 != src1 + PAGE_SIZE) { + hw_desc->u.control.src_page_break = 1; + hw_desc->next_src_addr = src2; + } + if (dst2 != dst1 + PAGE_SIZE) { + hw_desc->u.control.dest_page_break = 1; + hw_desc->next_dest_addr = dst2; + } + if (g_ioat_debug_level >= 3) dump_descriptor(hw_desc); ioat_submit_single(ioat); return (&desc->bus_dmadesc); } struct bus_dmadesc * ioat_blockfill(bus_dmaengine_t dmaengine, bus_addr_t dst, uint64_t fillpattern, bus_size_t len, bus_dmaengine_callback_t callback_fn, void *callback_arg, uint32_t flags) { struct ioat_fill_hw_descriptor *hw_desc; struct ioat_descriptor *desc; struct ioat_softc *ioat; CTR0(KTR_IOAT, __func__); ioat = to_ioat_softc(dmaengine); if ((ioat->capabilities & IOAT_DMACAP_BFILL) == 0) { ioat_log_message(0, "%s: Device lacks BFILL capability\n", __func__); return (NULL); } if ((dst & (0xffffull << 48)) != 0) { ioat_log_message(0, "%s: High 16 bits of dst invalid\n", __func__); return (NULL); } desc = ioat_op_generic(ioat, IOAT_OP_FILL, len, fillpattern, dst, callback_fn, callback_arg, flags); if (desc == NULL) return (NULL); hw_desc = desc->u.fill; if (g_ioat_debug_level >= 3) dump_descriptor(hw_desc); ioat_submit_single(ioat); return (&desc->bus_dmadesc); } /* * Ring Management */ static inline uint32_t ioat_get_active(struct ioat_softc *ioat) { return ((ioat->head - ioat->tail) & ((1 << ioat->ring_size_order) - 1)); } static inline uint32_t ioat_get_ring_space(struct ioat_softc *ioat) { return ((1 << ioat->ring_size_order) - ioat_get_active(ioat) - 1); } static struct ioat_descriptor * ioat_alloc_ring_entry(struct ioat_softc *ioat, int mflags) { struct ioat_generic_hw_descriptor *hw_desc; struct ioat_descriptor *desc; int error, busdmaflag; error = ENOMEM; hw_desc = NULL; if ((mflags & M_WAITOK) != 0) busdmaflag = BUS_DMA_WAITOK; else busdmaflag = BUS_DMA_NOWAIT; desc = malloc(sizeof(*desc), M_IOAT, mflags); if (desc == NULL) goto out; bus_dmamem_alloc(ioat->hw_desc_tag, (void **)&hw_desc, BUS_DMA_ZERO | busdmaflag, &ioat->hw_desc_map); if (hw_desc == NULL) goto out; memset(&desc->bus_dmadesc, 0, sizeof(desc->bus_dmadesc)); desc->u.generic = hw_desc; error = bus_dmamap_load(ioat->hw_desc_tag, ioat->hw_desc_map, hw_desc, sizeof(*hw_desc), ioat_dmamap_cb, &desc->hw_desc_bus_addr, busdmaflag); if (error) goto out; out: if (error) { ioat_free_ring_entry(ioat, desc); return (NULL); } return (desc); } static void ioat_free_ring_entry(struct ioat_softc *ioat, struct ioat_descriptor *desc) { if (desc == NULL) return; if (desc->u.generic) bus_dmamem_free(ioat->hw_desc_tag, desc->u.generic, ioat->hw_desc_map); free(desc, M_IOAT); } /* * Reserves space in this IOAT descriptor ring by ensuring enough slots remain * for 'num_descs'. * * If mflags contains M_WAITOK, blocks until enough space is available. * * Returns zero on success, or an errno on error. If num_descs is beyond the * maximum ring size, returns EINVAl; if allocation would block and mflags * contains M_NOWAIT, returns EAGAIN. * * Must be called with the submit_lock held; returns with the lock held. The * lock may be dropped to allocate the ring. * * (The submit_lock is needed to add any entries to the ring, so callers are * assured enough room is available.) */ static int ioat_reserve_space(struct ioat_softc *ioat, uint32_t num_descs, int mflags) { struct ioat_descriptor **new_ring; uint32_t order; int error; mtx_assert(&ioat->submit_lock, MA_OWNED); error = 0; if (num_descs < 1 || num_descs > (1 << IOAT_MAX_ORDER)) { error = EINVAL; goto out; } if (ioat->quiescing) { error = ENXIO; goto out; } for (;;) { if (ioat_get_ring_space(ioat) >= num_descs) goto out; order = ioat->ring_size_order; if (ioat->is_resize_pending || order == IOAT_MAX_ORDER) { if ((mflags & M_WAITOK) != 0) { msleep(&ioat->tail, &ioat->submit_lock, 0, "ioat_rsz", 0); continue; } error = EAGAIN; break; } ioat->is_resize_pending = TRUE; for (;;) { mtx_unlock(&ioat->submit_lock); new_ring = ioat_prealloc_ring(ioat, 1 << (order + 1), TRUE, mflags); mtx_lock(&ioat->submit_lock); KASSERT(ioat->ring_size_order == order, ("is_resize_pending should protect order")); if (new_ring == NULL) { KASSERT((mflags & M_WAITOK) == 0, ("allocation failed")); error = EAGAIN; break; } error = ring_grow(ioat, order, new_ring); if (error == 0) break; } ioat->is_resize_pending = FALSE; wakeup(&ioat->tail); if (error) break; } out: mtx_assert(&ioat->submit_lock, MA_OWNED); return (error); } static struct ioat_descriptor ** ioat_prealloc_ring(struct ioat_softc *ioat, uint32_t size, boolean_t need_dscr, int mflags) { struct ioat_descriptor **ring; uint32_t i; int error; KASSERT(size > 0 && powerof2(size), ("bogus size")); ring = malloc(size * sizeof(*ring), M_IOAT, M_ZERO | mflags); if (ring == NULL) return (NULL); if (need_dscr) { error = ENOMEM; for (i = size / 2; i < size; i++) { ring[i] = ioat_alloc_ring_entry(ioat, mflags); if (ring[i] == NULL) goto out; ring[i]->id = i; } } error = 0; out: if (error != 0 && ring != NULL) { ioat_free_ring(ioat, size, ring); ring = NULL; } return (ring); } static void ioat_free_ring(struct ioat_softc *ioat, uint32_t size, struct ioat_descriptor **ring) { uint32_t i; for (i = 0; i < size; i++) { if (ring[i] != NULL) ioat_free_ring_entry(ioat, ring[i]); } free(ring, M_IOAT); } static struct ioat_descriptor * ioat_get_ring_entry(struct ioat_softc *ioat, uint32_t index) { return (ioat->ring[index % (1 << ioat->ring_size_order)]); } static int ring_grow(struct ioat_softc *ioat, uint32_t oldorder, struct ioat_descriptor **newring) { struct ioat_descriptor *tmp, *next; struct ioat_dma_hw_descriptor *hw; uint32_t oldsize, newsize, head, tail, i, end; int error; CTR0(KTR_IOAT, __func__); mtx_assert(&ioat->submit_lock, MA_OWNED); if (oldorder != ioat->ring_size_order || oldorder >= IOAT_MAX_ORDER) { error = EINVAL; goto out; } oldsize = (1 << oldorder); newsize = (1 << (oldorder + 1)); mtx_lock(&ioat->cleanup_lock); head = ioat->head & (oldsize - 1); tail = ioat->tail & (oldsize - 1); /* Copy old descriptors to new ring */ for (i = 0; i < oldsize; i++) newring[i] = ioat->ring[i]; /* * If head has wrapped but tail hasn't, we must swap some descriptors * around so that tail can increment directly to head. */ if (head < tail) { for (i = 0; i <= head; i++) { tmp = newring[oldsize + i]; newring[oldsize + i] = newring[i]; newring[oldsize + i]->id = oldsize + i; newring[i] = tmp; newring[i]->id = i; } head += oldsize; } KASSERT(head >= tail, ("invariants")); /* Head didn't wrap; we only need to link in oldsize..newsize */ if (head < oldsize) { i = oldsize - 1; end = newsize; } else { /* Head did wrap; link newhead..newsize and 0..oldhead */ i = head; end = newsize + (head - oldsize) + 1; } /* * Fix up hardware ring, being careful not to trample the active * section (tail -> head). */ for (; i < end; i++) { KASSERT((i & (newsize - 1)) < tail || (i & (newsize - 1)) >= head, ("trampling snake")); next = newring[(i + 1) & (newsize - 1)]; hw = newring[i & (newsize - 1)]->u.dma; hw->next = next->hw_desc_bus_addr; } free(ioat->ring, M_IOAT); ioat->ring = newring; ioat->ring_size_order = oldorder + 1; ioat->tail = tail; ioat->head = head; error = 0; mtx_unlock(&ioat->cleanup_lock); out: if (error) ioat_free_ring(ioat, (1 << (oldorder + 1)), newring); return (error); } static int ring_shrink(struct ioat_softc *ioat, uint32_t oldorder, struct ioat_descriptor **newring) { struct ioat_dma_hw_descriptor *hw; struct ioat_descriptor *ent, *next; uint32_t oldsize, newsize, current_idx, new_idx, i; int error; CTR0(KTR_IOAT, __func__); mtx_assert(&ioat->submit_lock, MA_OWNED); if (oldorder != ioat->ring_size_order || oldorder <= IOAT_MIN_ORDER) { error = EINVAL; goto out_unlocked; } oldsize = (1 << oldorder); newsize = (1 << (oldorder - 1)); mtx_lock(&ioat->cleanup_lock); /* Can't shrink below current active set! */ if (ioat_get_active(ioat) >= newsize) { error = ENOMEM; goto out; } /* * Copy current descriptors to the new ring, dropping the removed * descriptors. */ for (i = 0; i < newsize; i++) { current_idx = (ioat->tail + i) & (oldsize - 1); new_idx = (ioat->tail + i) & (newsize - 1); newring[new_idx] = ioat->ring[current_idx]; newring[new_idx]->id = new_idx; } /* Free deleted descriptors */ for (i = newsize; i < oldsize; i++) { ent = ioat_get_ring_entry(ioat, ioat->tail + i); ioat_free_ring_entry(ioat, ent); } /* Fix up hardware ring. */ hw = newring[(ioat->tail + newsize - 1) & (newsize - 1)]->u.dma; next = newring[(ioat->tail + newsize) & (newsize - 1)]; hw->next = next->hw_desc_bus_addr; free(ioat->ring, M_IOAT); ioat->ring = newring; ioat->ring_size_order = oldorder - 1; error = 0; out: mtx_unlock(&ioat->cleanup_lock); out_unlocked: if (error) ioat_free_ring(ioat, (1 << (oldorder - 1)), newring); return (error); } static void ioat_halted_debug(struct ioat_softc *ioat, uint32_t chanerr) { struct ioat_descriptor *desc; ioat_log_message(0, "Channel halted (%b)\n", (int)chanerr, IOAT_CHANERR_STR); if (chanerr == 0) return; mtx_assert(&ioat->cleanup_lock, MA_OWNED); desc = ioat_get_ring_entry(ioat, ioat->tail + 0); dump_descriptor(desc->u.raw); desc = ioat_get_ring_entry(ioat, ioat->tail + 1); dump_descriptor(desc->u.raw); } static void ioat_timer_callback(void *arg) { struct ioat_descriptor **newring; struct ioat_softc *ioat; uint32_t order; ioat = arg; ioat_log_message(1, "%s\n", __func__); if (ioat->is_completion_pending) { ioat_process_events(ioat); return; } /* Slowly scale the ring down if idle. */ mtx_lock(&ioat->submit_lock); order = ioat->ring_size_order; if (ioat->is_resize_pending || order == IOAT_MIN_ORDER) { mtx_unlock(&ioat->submit_lock); goto out; } ioat->is_resize_pending = TRUE; mtx_unlock(&ioat->submit_lock); newring = ioat_prealloc_ring(ioat, 1 << (order - 1), FALSE, M_NOWAIT); mtx_lock(&ioat->submit_lock); KASSERT(ioat->ring_size_order == order, ("resize_pending protects order")); if (newring != NULL) ring_shrink(ioat, order, newring); ioat->is_resize_pending = FALSE; mtx_unlock(&ioat->submit_lock); out: if (ioat->ring_size_order > IOAT_MIN_ORDER) callout_reset(&ioat->timer, 10 * hz, ioat_timer_callback, ioat); } /* * Support Functions */ static void ioat_submit_single(struct ioat_softc *ioat) { ioat_get(ioat, IOAT_ACTIVE_DESCR_REF); atomic_add_rel_int(&ioat->head, 1); atomic_add_rel_int(&ioat->hw_head, 1); if (!ioat->is_completion_pending) { ioat->is_completion_pending = TRUE; callout_reset(&ioat->timer, IOAT_INTR_TIMO, ioat_timer_callback, ioat); } } static int ioat_reset_hw(struct ioat_softc *ioat) { uint64_t status; uint32_t chanerr; unsigned timeout; int error; mtx_lock(IOAT_REFLK); ioat->quiescing = TRUE; ioat_drain_locked(ioat); mtx_unlock(IOAT_REFLK); status = ioat_get_chansts(ioat); if (is_ioat_active(status) || is_ioat_idle(status)) ioat_suspend(ioat); /* Wait at most 20 ms */ for (timeout = 0; (is_ioat_active(status) || is_ioat_idle(status)) && timeout < 20; timeout++) { DELAY(1000); status = ioat_get_chansts(ioat); } if (timeout == 20) { error = ETIMEDOUT; goto out; } KASSERT(ioat_get_active(ioat) == 0, ("active after quiesce")); chanerr = ioat_read_4(ioat, IOAT_CHANERR_OFFSET); ioat_write_4(ioat, IOAT_CHANERR_OFFSET, chanerr); /* * IOAT v3 workaround - CHANERRMSK_INT with 3E07h to masks out errors * that can cause stability issues for IOAT v3. */ pci_write_config(ioat->device, IOAT_CFG_CHANERRMASK_INT_OFFSET, 0x3e07, 4); chanerr = pci_read_config(ioat->device, IOAT_CFG_CHANERR_INT_OFFSET, 4); pci_write_config(ioat->device, IOAT_CFG_CHANERR_INT_OFFSET, chanerr, 4); /* * BDXDE and BWD models reset MSI-X registers on device reset. * Save/restore their contents manually. */ if (ioat_model_resets_msix(ioat)) { ioat_log_message(1, "device resets MSI-X registers; saving\n"); pci_save_state(ioat->device); } ioat_reset(ioat); /* Wait at most 20 ms */ for (timeout = 0; ioat_reset_pending(ioat) && timeout < 20; timeout++) DELAY(1000); if (timeout == 20) { error = ETIMEDOUT; goto out; } if (ioat_model_resets_msix(ioat)) { ioat_log_message(1, "device resets registers; restored\n"); pci_restore_state(ioat->device); } /* Reset attempts to return the hardware to "halted." */ status = ioat_get_chansts(ioat); if (is_ioat_active(status) || is_ioat_idle(status)) { /* So this really shouldn't happen... */ ioat_log_message(0, "Device is active after a reset?\n"); ioat_write_chanctrl(ioat, IOAT_CHANCTRL_RUN); error = 0; goto out; } chanerr = ioat_read_4(ioat, IOAT_CHANERR_OFFSET); if (chanerr != 0) { mtx_lock(&ioat->cleanup_lock); ioat_halted_debug(ioat, chanerr); mtx_unlock(&ioat->cleanup_lock); error = EIO; goto out; } /* * Bring device back online after reset. Writing CHAINADDR brings the * device back to active. * * The internal ring counter resets to zero, so we have to start over * at zero as well. */ ioat->tail = ioat->head = ioat->hw_head = 0; ioat->last_seen = 0; ioat_write_chanctrl(ioat, IOAT_CHANCTRL_RUN); ioat_write_chancmp(ioat, ioat->comp_update_bus_addr); ioat_write_chainaddr(ioat, ioat->ring[0]->hw_desc_bus_addr); error = 0; out: mtx_lock(IOAT_REFLK); ioat->quiescing = FALSE; mtx_unlock(IOAT_REFLK); if (error == 0) error = ioat_start_channel(ioat); return (error); } static int sysctl_handle_chansts(SYSCTL_HANDLER_ARGS) { struct ioat_softc *ioat; struct sbuf sb; uint64_t status; int error; ioat = arg1; status = ioat_get_chansts(ioat) & IOAT_CHANSTS_STATUS; sbuf_new_for_sysctl(&sb, NULL, 256, req); switch (status) { case IOAT_CHANSTS_ACTIVE: sbuf_printf(&sb, "ACTIVE"); break; case IOAT_CHANSTS_IDLE: sbuf_printf(&sb, "IDLE"); break; case IOAT_CHANSTS_SUSPENDED: sbuf_printf(&sb, "SUSPENDED"); break; case IOAT_CHANSTS_HALTED: sbuf_printf(&sb, "HALTED"); break; case IOAT_CHANSTS_ARMED: sbuf_printf(&sb, "ARMED"); break; default: sbuf_printf(&sb, "UNKNOWN"); break; } error = sbuf_finish(&sb); sbuf_delete(&sb); if (error != 0 || req->newptr == NULL) return (error); return (EINVAL); } static int sysctl_handle_error(SYSCTL_HANDLER_ARGS) { struct ioat_descriptor *desc; struct ioat_softc *ioat; int error, arg; ioat = arg1; arg = 0; error = SYSCTL_OUT(req, &arg, sizeof(arg)); if (error != 0 || req->newptr == NULL) return (error); error = SYSCTL_IN(req, &arg, sizeof(arg)); if (error != 0) return (error); if (arg != 0) { ioat_acquire(&ioat->dmaengine); desc = ioat_op_generic(ioat, IOAT_OP_COPY, 1, 0xffff000000000000ull, 0xffff000000000000ull, NULL, NULL, 0); if (desc == NULL) error = ENOMEM; else ioat_submit_single(ioat); ioat_release(&ioat->dmaengine); } return (error); } static int sysctl_handle_reset(SYSCTL_HANDLER_ARGS) { struct ioat_softc *ioat; int error, arg; ioat = arg1; arg = 0; error = SYSCTL_OUT(req, &arg, sizeof(arg)); if (error != 0 || req->newptr == NULL) return (error); error = SYSCTL_IN(req, &arg, sizeof(arg)); if (error != 0) return (error); if (arg != 0) error = ioat_reset_hw(ioat); return (error); } static void dump_descriptor(void *hw_desc) { int i, j; for (i = 0; i < 2; i++) { for (j = 0; j < 8; j++) printf("%08x ", ((uint32_t *)hw_desc)[i * 8 + j]); printf("\n"); } } static void ioat_setup_sysctl(device_t device) { struct sysctl_oid_list *par; struct sysctl_ctx_list *ctx; struct sysctl_oid *tree; struct ioat_softc *ioat; ioat = DEVICE2SOFTC(device); ctx = device_get_sysctl_ctx(device); tree = device_get_sysctl_tree(device); par = SYSCTL_CHILDREN(tree); SYSCTL_ADD_INT(ctx, par, OID_AUTO, "version", CTLFLAG_RD, &ioat->version, 0, "HW version (0xMM form)"); SYSCTL_ADD_UINT(ctx, par, OID_AUTO, "max_xfer_size", CTLFLAG_RD, &ioat->max_xfer_size, 0, "HW maximum transfer size"); SYSCTL_ADD_UINT(ctx, par, OID_AUTO, "ring_size_order", CTLFLAG_RD, &ioat->ring_size_order, 0, "SW descriptor ring size order"); SYSCTL_ADD_UINT(ctx, par, OID_AUTO, "head", CTLFLAG_RD, &ioat->head, 0, "SW descriptor head pointer index"); SYSCTL_ADD_UINT(ctx, par, OID_AUTO, "tail", CTLFLAG_RD, &ioat->tail, 0, "SW descriptor tail pointer index"); SYSCTL_ADD_UINT(ctx, par, OID_AUTO, "hw_head", CTLFLAG_RD, &ioat->hw_head, 0, "HW DMACOUNT"); SYSCTL_ADD_UQUAD(ctx, par, OID_AUTO, "last_completion", CTLFLAG_RD, ioat->comp_update, "HW addr of last completion"); SYSCTL_ADD_INT(ctx, par, OID_AUTO, "is_resize_pending", CTLFLAG_RD, &ioat->is_resize_pending, 0, "resize pending"); SYSCTL_ADD_INT(ctx, par, OID_AUTO, "is_completion_pending", CTLFLAG_RD, &ioat->is_completion_pending, 0, "completion pending"); SYSCTL_ADD_INT(ctx, par, OID_AUTO, "is_reset_pending", CTLFLAG_RD, &ioat->is_reset_pending, 0, "reset pending"); SYSCTL_ADD_INT(ctx, par, OID_AUTO, "is_channel_running", CTLFLAG_RD, &ioat->is_channel_running, 0, "channel running"); SYSCTL_ADD_PROC(ctx, par, OID_AUTO, "force_hw_reset", CTLTYPE_INT | CTLFLAG_RW, ioat, 0, sysctl_handle_reset, "I", "Set to non-zero to reset the hardware"); SYSCTL_ADD_PROC(ctx, par, OID_AUTO, "force_hw_error", CTLTYPE_INT | CTLFLAG_RW, ioat, 0, sysctl_handle_error, "I", "Set to non-zero to inject a recoverable hardware error"); SYSCTL_ADD_PROC(ctx, par, OID_AUTO, "chansts", CTLTYPE_STRING | CTLFLAG_RD, ioat, 0, sysctl_handle_chansts, "A", "String of the channel status"); } static inline struct ioat_softc * ioat_get(struct ioat_softc *ioat, enum ioat_ref_kind kind) { uint32_t old; KASSERT(kind < IOAT_NUM_REF_KINDS, ("bogus")); old = atomic_fetchadd_32(&ioat->refcnt, 1); KASSERT(old < UINT32_MAX, ("refcnt overflow")); #ifdef INVARIANTS old = atomic_fetchadd_32(&ioat->refkinds[kind], 1); KASSERT(old < UINT32_MAX, ("refcnt kind overflow")); #endif return (ioat); } static inline void ioat_putn(struct ioat_softc *ioat, uint32_t n, enum ioat_ref_kind kind) { _ioat_putn(ioat, n, kind, FALSE); } static inline void ioat_putn_locked(struct ioat_softc *ioat, uint32_t n, enum ioat_ref_kind kind) { _ioat_putn(ioat, n, kind, TRUE); } static inline void _ioat_putn(struct ioat_softc *ioat, uint32_t n, enum ioat_ref_kind kind, boolean_t locked) { uint32_t old; KASSERT(kind < IOAT_NUM_REF_KINDS, ("bogus")); if (n == 0) return; #ifdef INVARIANTS old = atomic_fetchadd_32(&ioat->refkinds[kind], -n); KASSERT(old >= n, ("refcnt kind underflow")); #endif /* Skip acquiring the lock if resulting refcnt > 0. */ for (;;) { old = ioat->refcnt; if (old <= n) break; if (atomic_cmpset_32(&ioat->refcnt, old, old - n)) return; } if (locked) mtx_assert(IOAT_REFLK, MA_OWNED); else mtx_lock(IOAT_REFLK); old = atomic_fetchadd_32(&ioat->refcnt, -n); KASSERT(old >= n, ("refcnt error")); if (old == n) wakeup(IOAT_REFLK); if (!locked) mtx_unlock(IOAT_REFLK); } static inline void ioat_put(struct ioat_softc *ioat, enum ioat_ref_kind kind) { ioat_putn(ioat, 1, kind); } static void ioat_drain_locked(struct ioat_softc *ioat) { mtx_assert(IOAT_REFLK, MA_OWNED); while (ioat->refcnt > 0) msleep(IOAT_REFLK, IOAT_REFLK, 0, "ioat_drain", 0); } Index: user/ngie/more-tests2/sys/dev/ioat/ioat.h =================================================================== --- user/ngie/more-tests2/sys/dev/ioat/ioat.h (revision 292053) +++ user/ngie/more-tests2/sys/dev/ioat/ioat.h (revision 292054) @@ -1,95 +1,108 @@ /*- * Copyright (C) 2012 Intel Corporation * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ __FBSDID("$FreeBSD$"); #ifndef __IOAT_H__ #define __IOAT_H__ #include #include /* * This file defines the public interface to the IOAT driver. */ /* * Enables an interrupt for this operation. Typically, you would only enable * this on the last operation in a group */ #define DMA_INT_EN 0x1 /* * Like M_NOWAIT. Operations will return NULL if they cannot allocate a * descriptor without blocking. */ #define DMA_NO_WAIT 0x2 #define DMA_ALL_FLAGS (DMA_INT_EN | DMA_NO_WAIT) typedef void *bus_dmaengine_t; struct bus_dmadesc; typedef void (*bus_dmaengine_callback_t)(void *arg, int error); /* * Called first to acquire a reference to the DMA channel */ bus_dmaengine_t ioat_get_dmaengine(uint32_t channel_index); /* Release the DMA channel */ void ioat_put_dmaengine(bus_dmaengine_t dmaengine); /* * Acquire must be called before issuing an operation to perform. Release is * called after. Multiple operations can be issued within the context of one * acquire and release */ void ioat_acquire(bus_dmaengine_t dmaengine); void ioat_release(bus_dmaengine_t dmaengine); /* * Issue a blockfill operation. The 64-bit pattern 'fillpattern' is written to * 'len' physically contiguous bytes at 'dst'. * * Only supported on devices with the BFILL capability. */ struct bus_dmadesc *ioat_blockfill(bus_dmaengine_t dmaengine, bus_addr_t dst, uint64_t fillpattern, bus_size_t len, bus_dmaengine_callback_t callback_fn, void *callback_arg, uint32_t flags); /* Issues the copy data operation */ struct bus_dmadesc *ioat_copy(bus_dmaengine_t dmaengine, bus_addr_t dst, bus_addr_t src, bus_size_t len, bus_dmaengine_callback_t callback_fn, void *callback_arg, uint32_t flags); /* + * Issue a copy data operation, with constraints: + * - src1, src2, dst1, dst2 are all page-aligned addresses + * - The quantity to copy is exactly 2 pages; + * - src1 -> dst1, src2 -> dst2 + * + * Why use this instead of normal _copy()? You can copy two non-contiguous + * pages (src, dst, or both) with one descriptor. + */ +struct bus_dmadesc *ioat_copy_8k_aligned(bus_dmaengine_t dmaengine, + bus_addr_t dst1, bus_addr_t dst2, bus_addr_t src1, bus_addr_t src2, + bus_dmaengine_callback_t callback_fn, void *callback_arg, uint32_t flags); + +/* * Issues a null operation. This issues the operation to the hardware, but the * hardware doesn't do anything with it. */ struct bus_dmadesc *ioat_null(bus_dmaengine_t dmaengine, bus_dmaengine_callback_t callback_fn, void *callback_arg, uint32_t flags); #endif /* __IOAT_H__ */ Index: user/ngie/more-tests2/sys/dev/ioat/ioat_internal.h =================================================================== --- user/ngie/more-tests2/sys/dev/ioat/ioat_internal.h (revision 292053) +++ user/ngie/more-tests2/sys/dev/ioat/ioat_internal.h (revision 292054) @@ -1,493 +1,493 @@ /*- * Copyright (C) 2012 Intel Corporation * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ __FBSDID("$FreeBSD$"); #ifndef __IOAT_INTERNAL_H__ #define __IOAT_INTERNAL_H__ #define DEVICE2SOFTC(dev) ((struct ioat_softc *) device_get_softc(dev)) #define KTR_IOAT KTR_SPARE3 #define ioat_read_chancnt(ioat) \ ioat_read_1((ioat), IOAT_CHANCNT_OFFSET) #define ioat_read_xfercap(ioat) \ (ioat_read_1((ioat), IOAT_XFERCAP_OFFSET) & IOAT_XFERCAP_VALID_MASK) #define ioat_write_intrctrl(ioat, value) \ ioat_write_1((ioat), IOAT_INTRCTRL_OFFSET, (value)) #define ioat_read_cbver(ioat) \ (ioat_read_1((ioat), IOAT_CBVER_OFFSET) & 0xFF) #define ioat_read_dmacapability(ioat) \ ioat_read_4((ioat), IOAT_DMACAPABILITY_OFFSET) #define ioat_write_chanctrl(ioat, value) \ ioat_write_2((ioat), IOAT_CHANCTRL_OFFSET, (value)) static __inline uint64_t ioat_bus_space_read_8_lower_first(bus_space_tag_t tag, bus_space_handle_t handle, bus_size_t offset) { return (bus_space_read_4(tag, handle, offset) | ((uint64_t)bus_space_read_4(tag, handle, offset + 4)) << 32); } static __inline void ioat_bus_space_write_8_lower_first(bus_space_tag_t tag, bus_space_handle_t handle, bus_size_t offset, uint64_t val) { bus_space_write_4(tag, handle, offset, val); bus_space_write_4(tag, handle, offset + 4, val >> 32); } #ifdef __i386__ #define ioat_bus_space_read_8 ioat_bus_space_read_8_lower_first #define ioat_bus_space_write_8 ioat_bus_space_write_8_lower_first #else #define ioat_bus_space_read_8(tag, handle, offset) \ bus_space_read_8((tag), (handle), (offset)) #define ioat_bus_space_write_8(tag, handle, offset, val) \ bus_space_write_8((tag), (handle), (offset), (val)) #endif #define ioat_read_1(ioat, offset) \ bus_space_read_1((ioat)->pci_bus_tag, (ioat)->pci_bus_handle, \ (offset)) #define ioat_read_2(ioat, offset) \ bus_space_read_2((ioat)->pci_bus_tag, (ioat)->pci_bus_handle, \ (offset)) #define ioat_read_4(ioat, offset) \ bus_space_read_4((ioat)->pci_bus_tag, (ioat)->pci_bus_handle, \ (offset)) #define ioat_read_8(ioat, offset) \ ioat_bus_space_read_8((ioat)->pci_bus_tag, (ioat)->pci_bus_handle, \ (offset)) #define ioat_read_double_4(ioat, offset) \ ioat_bus_space_read_8_lower_first((ioat)->pci_bus_tag, \ (ioat)->pci_bus_handle, (offset)) #define ioat_write_1(ioat, offset, value) \ bus_space_write_1((ioat)->pci_bus_tag, (ioat)->pci_bus_handle, \ (offset), (value)) #define ioat_write_2(ioat, offset, value) \ bus_space_write_2((ioat)->pci_bus_tag, (ioat)->pci_bus_handle, \ (offset), (value)) #define ioat_write_4(ioat, offset, value) \ bus_space_write_4((ioat)->pci_bus_tag, (ioat)->pci_bus_handle, \ (offset), (value)) #define ioat_write_8(ioat, offset, value) \ ioat_bus_space_write_8((ioat)->pci_bus_tag, (ioat)->pci_bus_handle, \ (offset), (value)) #define ioat_write_double_4(ioat, offset, value) \ ioat_bus_space_write_8_lower_first((ioat)->pci_bus_tag, \ (ioat)->pci_bus_handle, (offset), (value)) MALLOC_DECLARE(M_IOAT); SYSCTL_DECL(_hw_ioat); extern int g_ioat_debug_level; struct generic_dma_control { uint32_t int_enable:1; uint32_t src_snoop_disable:1; uint32_t dest_snoop_disable:1; uint32_t completion_update:1; uint32_t fence:1; uint32_t reserved1:1; uint32_t src_page_break:1; uint32_t dest_page_break:1; uint32_t bundle:1; uint32_t dest_dca:1; uint32_t hint:1; uint32_t reserved2:13; uint32_t op:8; }; struct ioat_generic_hw_descriptor { uint32_t size; union { uint32_t control_raw; struct generic_dma_control control_generic; } u; uint64_t src_addr; uint64_t dest_addr; uint64_t next; uint64_t reserved[4]; }; struct ioat_dma_hw_descriptor { uint32_t size; union { uint32_t control_raw; struct generic_dma_control control_generic; struct { uint32_t int_enable:1; uint32_t src_snoop_disable:1; uint32_t dest_snoop_disable:1; uint32_t completion_update:1; uint32_t fence:1; uint32_t null:1; uint32_t src_page_break:1; uint32_t dest_page_break:1; uint32_t bundle:1; uint32_t dest_dca:1; uint32_t hint:1; uint32_t reserved:13; #define IOAT_OP_COPY 0x00 uint32_t op:8; } control; } u; uint64_t src_addr; uint64_t dest_addr; uint64_t next; - uint64_t reserved; - uint64_t reserved2; + uint64_t next_src_addr; + uint64_t next_dest_addr; uint64_t user1; uint64_t user2; }; struct ioat_fill_hw_descriptor { uint32_t size; union { uint32_t control_raw; struct generic_dma_control control_generic; struct { uint32_t int_enable:1; uint32_t reserved:1; uint32_t dest_snoop_disable:1; uint32_t completion_update:1; uint32_t fence:1; uint32_t reserved2:2; uint32_t dest_page_break:1; uint32_t bundle:1; uint32_t reserved3:15; #define IOAT_OP_FILL 0x01 uint32_t op:8; } control; } u; uint64_t src_data; uint64_t dest_addr; uint64_t next; uint64_t reserved; uint64_t next_dest_addr; uint64_t user1; uint64_t user2; }; struct ioat_xor_hw_descriptor { uint32_t size; union { uint32_t control_raw; struct generic_dma_control control_generic; struct { uint32_t int_enable:1; uint32_t src_snoop_disable:1; uint32_t dest_snoop_disable:1; uint32_t completion_update:1; uint32_t fence:1; uint32_t src_count:3; uint32_t bundle:1; uint32_t dest_dca:1; uint32_t hint:1; uint32_t reserved:13; #define IOAT_OP_XOR 0x87 #define IOAT_OP_XOR_VAL 0x88 uint32_t op:8; } control; } u; uint64_t src_addr; uint64_t dest_addr; uint64_t next; uint64_t src_addr2; uint64_t src_addr3; uint64_t src_addr4; uint64_t src_addr5; }; struct ioat_xor_ext_hw_descriptor { uint64_t src_addr6; uint64_t src_addr7; uint64_t src_addr8; uint64_t next; uint64_t reserved[4]; }; struct ioat_pq_hw_descriptor { uint32_t size; union { uint32_t control_raw; struct generic_dma_control control_generic; struct { uint32_t int_enable:1; uint32_t src_snoop_disable:1; uint32_t dest_snoop_disable:1; uint32_t completion_update:1; uint32_t fence:1; uint32_t src_count:3; uint32_t bundle:1; uint32_t dest_dca:1; uint32_t hint:1; uint32_t p_disable:1; uint32_t q_disable:1; uint32_t reserved:11; #define IOAT_OP_PQ 0x89 #define IOAT_OP_PQ_VAL 0x8a uint32_t op:8; } control; } u; uint64_t src_addr; uint64_t p_addr; uint64_t next; uint64_t src_addr2; uint64_t src_addr3; uint8_t coef[8]; uint64_t q_addr; }; struct ioat_pq_ext_hw_descriptor { uint64_t src_addr4; uint64_t src_addr5; uint64_t src_addr6; uint64_t next; uint64_t src_addr7; uint64_t src_addr8; uint64_t reserved[2]; }; struct ioat_pq_update_hw_descriptor { uint32_t size; union { uint32_t control_raw; struct generic_dma_control control_generic; struct { uint32_t int_enable:1; uint32_t src_snoop_disable:1; uint32_t dest_snoop_disable:1; uint32_t completion_update:1; uint32_t fence:1; uint32_t src_cnt:3; uint32_t bundle:1; uint32_t dest_dca:1; uint32_t hint:1; uint32_t p_disable:1; uint32_t q_disable:1; uint32_t reserved:3; uint32_t coef:8; #define IOAT_OP_PQ_UP 0x8b uint32_t op:8; } control; } u; uint64_t src_addr; uint64_t p_addr; uint64_t next; uint64_t src_addr2; uint64_t p_src; uint64_t q_src; uint64_t q_addr; }; struct ioat_raw_hw_descriptor { uint64_t field[8]; }; struct bus_dmadesc { bus_dmaengine_callback_t callback_fn; void *callback_arg; }; struct ioat_descriptor { struct bus_dmadesc bus_dmadesc; union { struct ioat_generic_hw_descriptor *generic; struct ioat_dma_hw_descriptor *dma; struct ioat_fill_hw_descriptor *fill; struct ioat_xor_hw_descriptor *xor; struct ioat_xor_ext_hw_descriptor *xor_ext; struct ioat_pq_hw_descriptor *pq; struct ioat_pq_ext_hw_descriptor *pq_ext; struct ioat_raw_hw_descriptor *raw; } u; uint32_t id; bus_addr_t hw_desc_bus_addr; }; enum ioat_ref_kind { IOAT_DMAENGINE_REF = 0, IOAT_ACTIVE_DESCR_REF, IOAT_NUM_REF_KINDS }; /* One of these per allocated PCI device. */ struct ioat_softc { bus_dmaengine_t dmaengine; #define to_ioat_softc(_dmaeng) \ ({ \ bus_dmaengine_t *_p = (_dmaeng); \ (struct ioat_softc *)((char *)_p - \ offsetof(struct ioat_softc, dmaengine)); \ }) int version; int chan_idx; struct mtx submit_lock; device_t device; bus_space_tag_t pci_bus_tag; bus_space_handle_t pci_bus_handle; int pci_resource_id; struct resource *pci_resource; uint32_t max_xfer_size; uint32_t capabilities; struct resource *res; int rid; void *tag; bus_dma_tag_t hw_desc_tag; bus_dmamap_t hw_desc_map; bus_dma_tag_t comp_update_tag; bus_dmamap_t comp_update_map; uint64_t *comp_update; bus_addr_t comp_update_bus_addr; struct callout timer; boolean_t quiescing; boolean_t is_resize_pending; boolean_t is_completion_pending; boolean_t is_reset_pending; boolean_t is_channel_running; uint32_t head; uint32_t tail; uint32_t hw_head; uint32_t ring_size_order; bus_addr_t last_seen; struct ioat_descriptor **ring; struct mtx cleanup_lock; volatile uint32_t refcnt; #ifdef INVARIANTS volatile uint32_t refkinds[IOAT_NUM_REF_KINDS]; #endif }; void ioat_test_attach(void); void ioat_test_detach(void); static inline uint64_t ioat_get_chansts(struct ioat_softc *ioat) { uint64_t status; if (ioat->version >= IOAT_VER_3_3) status = ioat_read_8(ioat, IOAT_CHANSTS_OFFSET); else /* Must read lower 4 bytes before upper 4 bytes. */ status = ioat_read_double_4(ioat, IOAT_CHANSTS_OFFSET); return (status); } static inline void ioat_write_chancmp(struct ioat_softc *ioat, uint64_t addr) { if (ioat->version >= IOAT_VER_3_3) ioat_write_8(ioat, IOAT_CHANCMP_OFFSET_LOW, addr); else ioat_write_double_4(ioat, IOAT_CHANCMP_OFFSET_LOW, addr); } static inline void ioat_write_chainaddr(struct ioat_softc *ioat, uint64_t addr) { if (ioat->version >= IOAT_VER_3_3) ioat_write_8(ioat, IOAT_CHAINADDR_OFFSET_LOW, addr); else ioat_write_double_4(ioat, IOAT_CHAINADDR_OFFSET_LOW, addr); } static inline boolean_t is_ioat_active(uint64_t status) { return ((status & IOAT_CHANSTS_STATUS) == IOAT_CHANSTS_ACTIVE); } static inline boolean_t is_ioat_idle(uint64_t status) { return ((status & IOAT_CHANSTS_STATUS) == IOAT_CHANSTS_IDLE); } static inline boolean_t is_ioat_halted(uint64_t status) { return ((status & IOAT_CHANSTS_STATUS) == IOAT_CHANSTS_HALTED); } static inline boolean_t is_ioat_suspended(uint64_t status) { return ((status & IOAT_CHANSTS_STATUS) == IOAT_CHANSTS_SUSPENDED); } static inline void ioat_suspend(struct ioat_softc *ioat) { ioat_write_1(ioat, IOAT_CHANCMD_OFFSET, IOAT_CHANCMD_SUSPEND); } static inline void ioat_reset(struct ioat_softc *ioat) { ioat_write_1(ioat, IOAT_CHANCMD_OFFSET, IOAT_CHANCMD_RESET); } static inline boolean_t ioat_reset_pending(struct ioat_softc *ioat) { uint8_t cmd; cmd = ioat_read_1(ioat, IOAT_CHANCMD_OFFSET); return ((cmd & IOAT_CHANCMD_RESET) != 0); } #endif /* __IOAT_INTERNAL_H__ */ Index: user/ngie/more-tests2/sys/dev/ioat/ioat_test.c =================================================================== --- user/ngie/more-tests2/sys/dev/ioat/ioat_test.c (revision 292053) +++ user/ngie/more-tests2/sys/dev/ioat/ioat_test.c (revision 292054) @@ -1,537 +1,572 @@ /*- * Copyright (C) 2012 Intel Corporation * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "ioat.h" #include "ioat_hw.h" #include "ioat_internal.h" #include "ioat_test.h" #ifndef time_after #define time_after(a,b) ((long)(b) - (long)(a) < 0) #endif MALLOC_DEFINE(M_IOAT_TEST, "ioat_test", "ioat test allocations"); #define IOAT_MAX_BUFS 256 struct test_transaction { void *buf[IOAT_MAX_BUFS]; uint32_t length; uint32_t depth; struct ioat_test *test; TAILQ_ENTRY(test_transaction) entry; }; #define IT_LOCK() mtx_lock(&ioat_test_lk) #define IT_UNLOCK() mtx_unlock(&ioat_test_lk) #define IT_ASSERT() mtx_assert(&ioat_test_lk, MA_OWNED) static struct mtx ioat_test_lk; MTX_SYSINIT(ioat_test_lk, &ioat_test_lk, "test coordination mtx", MTX_DEF); static int g_thread_index = 1; static struct cdev *g_ioat_cdev = NULL; #define ioat_test_log(v, ...) _ioat_test_log((v), "ioat_test: " __VA_ARGS__) static inline void _ioat_test_log(int verbosity, const char *fmt, ...); static void ioat_test_transaction_destroy(struct test_transaction *tx) { + struct ioat_test *test; int i; + test = tx->test; + for (i = 0; i < IOAT_MAX_BUFS; i++) { if (tx->buf[i] != NULL) { - contigfree(tx->buf[i], tx->length, M_IOAT_TEST); + if (test->testkind == IOAT_TEST_DMA_8K) + free(tx->buf[i], M_IOAT_TEST); + else + contigfree(tx->buf[i], tx->length, M_IOAT_TEST); tx->buf[i] = NULL; } } free(tx, M_IOAT_TEST); } static struct -test_transaction *ioat_test_transaction_create(unsigned num_buffers, - uint32_t buffer_size) +test_transaction *ioat_test_transaction_create(struct ioat_test *test, + unsigned num_buffers) { struct test_transaction *tx; unsigned i; tx = malloc(sizeof(*tx), M_IOAT_TEST, M_NOWAIT | M_ZERO); if (tx == NULL) return (NULL); - tx->length = buffer_size; + tx->length = test->buffer_size; for (i = 0; i < num_buffers; i++) { - tx->buf[i] = contigmalloc(buffer_size, M_IOAT_TEST, M_NOWAIT, - 0, BUS_SPACE_MAXADDR, PAGE_SIZE, 0); + if (test->testkind == IOAT_TEST_DMA_8K) + tx->buf[i] = malloc(test->buffer_size, M_IOAT_TEST, + M_NOWAIT); + else + tx->buf[i] = contigmalloc(test->buffer_size, + M_IOAT_TEST, M_NOWAIT, 0, BUS_SPACE_MAXADDR, + PAGE_SIZE, 0); if (tx->buf[i] == NULL) { ioat_test_transaction_destroy(tx); return (NULL); } } return (tx); } static void dump_hex(void *p, size_t chunks) { size_t i, j; for (i = 0; i < chunks; i++) { for (j = 0; j < 8; j++) printf("%08x ", ((uint32_t *)p)[i * 8 + j]); printf("\n"); } } static bool ioat_compare_ok(struct test_transaction *tx) { struct ioat_test *test; char *dst, *src; uint32_t i, j; test = tx->test; for (i = 0; i < tx->depth; i++) { dst = tx->buf[2 * i + 1]; src = tx->buf[2 * i]; if (test->testkind == IOAT_TEST_FILL) { for (j = 0; j < tx->length; j += sizeof(uint64_t)) { if (memcmp(src, &dst[j], MIN(sizeof(uint64_t), tx->length - j)) != 0) return (false); } } else if (test->testkind == IOAT_TEST_DMA) { if (memcmp(src, dst, tx->length) != 0) return (false); } else if (test->testkind == IOAT_TEST_RAW_DMA) { if (test->raw_write) dst = test->raw_vtarget; dump_hex(dst, tx->length / 32); } } return (true); } static void ioat_dma_test_callback(void *arg, int error) { struct test_transaction *tx; struct ioat_test *test; if (error != 0) ioat_test_log(0, "%s: Got error: %d\n", __func__, error); tx = arg; test = tx->test; if (test->verify && !ioat_compare_ok(tx)) { ioat_test_log(0, "miscompare found\n"); atomic_add_32(&test->status[IOAT_TEST_MISCOMPARE], tx->depth); } else if (!test->too_late) atomic_add_32(&test->status[IOAT_TEST_OK], tx->depth); IT_LOCK(); TAILQ_REMOVE(&test->pend_q, tx, entry); TAILQ_INSERT_TAIL(&test->free_q, tx, entry); wakeup(&test->free_q); IT_UNLOCK(); } static int ioat_test_prealloc_memory(struct ioat_test *test, int index) { uint32_t i, j, k; struct test_transaction *tx; for (i = 0; i < test->transactions; i++) { - tx = ioat_test_transaction_create(test->chain_depth * 2, - test->buffer_size); + tx = ioat_test_transaction_create(test, test->chain_depth * 2); if (tx == NULL) { ioat_test_log(0, "tx == NULL - memory exhausted\n"); test->status[IOAT_TEST_NO_MEMORY]++; return (ENOMEM); } TAILQ_INSERT_HEAD(&test->free_q, tx, entry); tx->test = test; tx->depth = test->chain_depth; /* fill in source buffers */ for (j = 0; j < (tx->length / sizeof(uint32_t)); j++) { uint32_t val = j + (index << 28); for (k = 0; k < test->chain_depth; k++) { ((uint32_t *)tx->buf[2*k])[j] = ~val; ((uint32_t *)tx->buf[2*k+1])[j] = val; } } } return (0); } static void ioat_test_release_memory(struct ioat_test *test) { struct test_transaction *tx, *s; TAILQ_FOREACH_SAFE(tx, &test->free_q, entry, s) ioat_test_transaction_destroy(tx); TAILQ_INIT(&test->free_q); TAILQ_FOREACH_SAFE(tx, &test->pend_q, entry, s) ioat_test_transaction_destroy(tx); TAILQ_INIT(&test->pend_q); } static void ioat_test_submit_1_tx(struct ioat_test *test, bus_dmaengine_t dma) { struct test_transaction *tx; struct bus_dmadesc *desc; bus_dmaengine_callback_t cb; bus_addr_t src, dest; uint64_t fillpattern; uint32_t i, flags; desc = NULL; IT_LOCK(); while (TAILQ_EMPTY(&test->free_q)) msleep(&test->free_q, &ioat_test_lk, 0, "test_submit", 0); tx = TAILQ_FIRST(&test->free_q); TAILQ_REMOVE(&test->free_q, tx, entry); TAILQ_INSERT_HEAD(&test->pend_q, tx, entry); IT_UNLOCK(); - ioat_acquire(dma); + if (test->testkind != IOAT_TEST_MEMCPY) + ioat_acquire(dma); for (i = 0; i < tx->depth; i++) { + if (test->testkind == IOAT_TEST_MEMCPY) { + memcpy(tx->buf[2 * i + 1], tx->buf[2 * i], tx->length); + if (i == tx->depth - 1) + ioat_dma_test_callback(tx, 0); + continue; + } + src = vtophys((vm_offset_t)tx->buf[2*i]); dest = vtophys((vm_offset_t)tx->buf[2*i+1]); if (test->testkind == IOAT_TEST_RAW_DMA) { if (test->raw_write) dest = test->raw_target; else src = test->raw_target; } if (i == tx->depth - 1) { cb = ioat_dma_test_callback; flags = DMA_INT_EN; } else { cb = NULL; flags = 0; } if (test->testkind == IOAT_TEST_DMA || test->testkind == IOAT_TEST_RAW_DMA) desc = ioat_copy(dma, dest, src, tx->length, cb, tx, flags); else if (test->testkind == IOAT_TEST_FILL) { fillpattern = *(uint64_t *)tx->buf[2*i]; desc = ioat_blockfill(dma, dest, fillpattern, tx->length, cb, tx, flags); + } else if (test->testkind == IOAT_TEST_DMA_8K) { + bus_addr_t src2, dst2; + + src2 = vtophys((vm_offset_t)tx->buf[2*i] + PAGE_SIZE); + dst2 = vtophys((vm_offset_t)tx->buf[2*i+1] + PAGE_SIZE); + + desc = ioat_copy_8k_aligned(dma, dest, dst2, src, src2, + cb, tx, flags); } if (desc == NULL) break; } + if (test->testkind == IOAT_TEST_MEMCPY) + return; ioat_release(dma); /* * We couldn't issue an IO -- either the device is being detached or * the HW reset. Essentially spin until the device comes back up or * our timer expires. */ if (desc == NULL && tx->depth > 0) { atomic_add_32(&test->status[IOAT_TEST_NO_DMA_ENGINE], tx->depth); IT_LOCK(); TAILQ_REMOVE(&test->pend_q, tx, entry); TAILQ_INSERT_HEAD(&test->free_q, tx, entry); IT_UNLOCK(); } } static void ioat_dma_test(void *arg) { struct ioat_test *test; bus_dmaengine_t dmaengine; uint32_t loops; int index, rc, start, end; test = arg; memset(__DEVOLATILE(void *, test->status), 0, sizeof(test->status)); + + if (test->testkind == IOAT_TEST_DMA_8K && + test->buffer_size != 2 * PAGE_SIZE) { + ioat_test_log(0, "Asked for 8k test and buffer size isn't 8k\n"); + test->status[IOAT_TEST_INVALID_INPUT]++; + return; + } if (test->buffer_size > 1024 * 1024) { ioat_test_log(0, "Buffer size too large >1MB\n"); test->status[IOAT_TEST_NO_MEMORY]++; return; } if (test->chain_depth * 2 > IOAT_MAX_BUFS) { ioat_test_log(0, "Depth too large (> %u)\n", (unsigned)IOAT_MAX_BUFS / 2); test->status[IOAT_TEST_NO_MEMORY]++; return; } if (btoc((uint64_t)test->buffer_size * test->chain_depth * test->transactions) > (physmem / 4)) { ioat_test_log(0, "Sanity check failed -- test would " "use more than 1/4 of phys mem.\n"); test->status[IOAT_TEST_NO_MEMORY]++; return; } if ((uint64_t)test->transactions * test->chain_depth > (1<<16)) { ioat_test_log(0, "Sanity check failed -- test would " "use more than available IOAT ring space.\n"); test->status[IOAT_TEST_NO_MEMORY]++; return; } if (test->testkind >= IOAT_NUM_TESTKINDS) { ioat_test_log(0, "Invalid kind %u\n", (unsigned)test->testkind); test->status[IOAT_TEST_INVALID_INPUT]++; return; } dmaengine = ioat_get_dmaengine(test->channel_index); if (dmaengine == NULL) { ioat_test_log(0, "Couldn't acquire dmaengine\n"); test->status[IOAT_TEST_NO_DMA_ENGINE]++; return; } if (test->testkind == IOAT_TEST_FILL && (to_ioat_softc(dmaengine)->capabilities & IOAT_DMACAP_BFILL) == 0) { ioat_test_log(0, "Hardware doesn't support block fill, aborting test\n"); test->status[IOAT_TEST_INVALID_INPUT]++; goto out; } if (test->testkind == IOAT_TEST_RAW_DMA) { if (test->raw_is_virtual) { test->raw_vtarget = (void *)test->raw_target; test->raw_target = vtophys(test->raw_vtarget); } else { test->raw_vtarget = pmap_mapdev(test->raw_target, test->buffer_size); } } index = g_thread_index++; TAILQ_INIT(&test->free_q); TAILQ_INIT(&test->pend_q); if (test->duration == 0) ioat_test_log(1, "Thread %d: num_loops remaining: 0x%08x\n", index, test->transactions); else ioat_test_log(1, "Thread %d: starting\n", index); rc = ioat_test_prealloc_memory(test, index); if (rc != 0) { ioat_test_log(0, "prealloc_memory: %d\n", rc); goto out; } wmb(); test->too_late = false; start = ticks; end = start + (((sbintime_t)test->duration * hz) / 1000); for (loops = 0;; loops++) { if (test->duration == 0 && loops >= test->transactions) break; else if (test->duration != 0 && time_after(ticks, end)) { test->too_late = true; break; } ioat_test_submit_1_tx(test, dmaengine); } ioat_test_log(1, "Test Elapsed: %d ticks (overrun %d), %d sec.\n", ticks - start, ticks - end, (ticks - start) / hz); IT_LOCK(); while (!TAILQ_EMPTY(&test->pend_q)) msleep(&test->free_q, &ioat_test_lk, 0, "ioattestcompl", hz); IT_UNLOCK(); ioat_test_log(1, "Test Elapsed2: %d ticks (overrun %d), %d sec.\n", ticks - start, ticks - end, (ticks - start) / hz); ioat_test_release_memory(test); out: if (test->testkind == IOAT_TEST_RAW_DMA && !test->raw_is_virtual) pmap_unmapdev((vm_offset_t)test->raw_vtarget, test->buffer_size); ioat_put_dmaengine(dmaengine); } static int ioat_test_open(struct cdev *dev, int flags, int fmt, struct thread *td) { return (0); } static int ioat_test_close(struct cdev *dev, int flags, int fmt, struct thread *td) { return (0); } static int ioat_test_ioctl(struct cdev *dev, unsigned long cmd, caddr_t arg, int flag, struct thread *td) { switch (cmd) { case IOAT_DMATEST: ioat_dma_test(arg); break; default: return (EINVAL); } return (0); } static struct cdevsw ioat_cdevsw = { .d_version = D_VERSION, .d_flags = 0, .d_open = ioat_test_open, .d_close = ioat_test_close, .d_ioctl = ioat_test_ioctl, .d_name = "ioat_test", }; static int enable_ioat_test(bool enable) { mtx_assert(&Giant, MA_OWNED); if (enable && g_ioat_cdev == NULL) { g_ioat_cdev = make_dev(&ioat_cdevsw, 0, UID_ROOT, GID_WHEEL, 0600, "ioat_test"); } else if (!enable && g_ioat_cdev != NULL) { destroy_dev(g_ioat_cdev); g_ioat_cdev = NULL; } return (0); } static int sysctl_enable_ioat_test(SYSCTL_HANDLER_ARGS) { int error, enabled; enabled = (g_ioat_cdev != NULL); error = sysctl_handle_int(oidp, &enabled, 0, req); if (error != 0 || req->newptr == NULL) return (error); enable_ioat_test(enabled); return (0); } SYSCTL_PROC(_hw_ioat, OID_AUTO, enable_ioat_test, CTLTYPE_INT | CTLFLAG_RW, 0, 0, sysctl_enable_ioat_test, "I", "Non-zero: Enable the /dev/ioat_test device"); void ioat_test_attach(void) { char *val; val = kern_getenv("hw.ioat.enable_ioat_test"); if (val != NULL && strcmp(val, "0") != 0) { mtx_lock(&Giant); enable_ioat_test(true); mtx_unlock(&Giant); } freeenv(val); } void ioat_test_detach(void) { mtx_lock(&Giant); enable_ioat_test(false); mtx_unlock(&Giant); } static inline void _ioat_test_log(int verbosity, const char *fmt, ...) { va_list argp; if (verbosity > g_ioat_debug_level) return; va_start(argp, fmt); vprintf(fmt, argp); va_end(argp); } Index: user/ngie/more-tests2/sys/dev/ioat/ioat_test.h =================================================================== --- user/ngie/more-tests2/sys/dev/ioat/ioat_test.h (revision 292053) +++ user/ngie/more-tests2/sys/dev/ioat/ioat_test.h (revision 292054) @@ -1,84 +1,86 @@ /*- * Copyright (C) 2012 Intel Corporation * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ __FBSDID("$FreeBSD$"); #ifndef __IOAT_TEST_H__ #define __IOAT_TEST_H__ enum ioat_res { IOAT_TEST_OK = 0, IOAT_TEST_NO_DMA_ENGINE, IOAT_TEST_NO_MEMORY, IOAT_TEST_MISCOMPARE, IOAT_TEST_INVALID_INPUT, IOAT_NUM_RES }; enum ioat_test_kind { IOAT_TEST_FILL = 0, IOAT_TEST_DMA, IOAT_TEST_RAW_DMA, + IOAT_TEST_DMA_8K, + IOAT_TEST_MEMCPY, IOAT_NUM_TESTKINDS }; struct test_transaction; struct ioat_test { volatile uint32_t status[IOAT_NUM_RES]; uint32_t channel_index; enum ioat_test_kind testkind; /* HW max of 1MB */ uint32_t buffer_size; uint32_t chain_depth; uint32_t transactions; /* * If non-zero, duration is time in ms; * If zero, bounded by 'transactions' above. */ uint32_t duration; /* If true, check for miscompares after a copy. */ bool verify; /* DMA directly to/from some memory address */ uint64_t raw_target; void *raw_vtarget; bool raw_write; bool raw_is_virtual; /* Internal usage -- not test inputs */ TAILQ_HEAD(, test_transaction) free_q; TAILQ_HEAD(, test_transaction) pend_q; volatile bool too_late; }; #define IOAT_DMATEST _IOWR('i', 0, struct ioat_test) #endif /* __IOAT_TEST_H__ */ Index: user/ngie/more-tests2/sys/dev/isp/isp.c =================================================================== --- user/ngie/more-tests2/sys/dev/isp/isp.c (revision 292053) +++ user/ngie/more-tests2/sys/dev/isp/isp.c (revision 292054) @@ -1,8336 +1,8338 @@ /*- * Copyright (c) 1997-2009 by Matthew Jacob * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * */ /* * Machine and OS Independent (well, as best as possible) * code for the Qlogic ISP SCSI and FC-SCSI adapters. */ /* * Inspiration and ideas about this driver are from Erik Moe's Linux driver * (qlogicisp.c) and Dave Miller's SBus version of same (qlogicisp.c). Some * ideas dredged from the Solaris driver. */ /* * Include header file appropriate for platform we're building on. */ #ifdef __NetBSD__ #include __KERNEL_RCSID(0, "$NetBSD$"); #include #endif #ifdef __FreeBSD__ #include __FBSDID("$FreeBSD$"); #include #endif #ifdef __OpenBSD__ #include #endif #ifdef __linux__ #include "isp_linux.h" #endif #ifdef __svr4__ #include "isp_solaris.h" #endif /* * General defines */ #define MBOX_DELAY_COUNT 1000000 / 100 /* * Local static data */ static const char notresp[] = "Not RESPONSE in RESPONSE Queue (type 0x%x) @ idx %d (next %d) nlooked %d"; static const char bun[] = "bad underrun (count %d, resid %d, status %s)"; static const char lipd[] = "Chan %d LIP destroyed %d active commands"; static const char sacq[] = "unable to acquire scratch area"; static const uint8_t alpa_map[] = { 0xef, 0xe8, 0xe4, 0xe2, 0xe1, 0xe0, 0xdc, 0xda, 0xd9, 0xd6, 0xd5, 0xd4, 0xd3, 0xd2, 0xd1, 0xce, 0xcd, 0xcc, 0xcb, 0xca, 0xc9, 0xc7, 0xc6, 0xc5, 0xc3, 0xbc, 0xba, 0xb9, 0xb6, 0xb5, 0xb4, 0xb3, 0xb2, 0xb1, 0xae, 0xad, 0xac, 0xab, 0xaa, 0xa9, 0xa7, 0xa6, 0xa5, 0xa3, 0x9f, 0x9e, 0x9d, 0x9b, 0x98, 0x97, 0x90, 0x8f, 0x88, 0x84, 0x82, 0x81, 0x80, 0x7c, 0x7a, 0x79, 0x76, 0x75, 0x74, 0x73, 0x72, 0x71, 0x6e, 0x6d, 0x6c, 0x6b, 0x6a, 0x69, 0x67, 0x66, 0x65, 0x63, 0x5c, 0x5a, 0x59, 0x56, 0x55, 0x54, 0x53, 0x52, 0x51, 0x4e, 0x4d, 0x4c, 0x4b, 0x4a, 0x49, 0x47, 0x46, 0x45, 0x43, 0x3c, 0x3a, 0x39, 0x36, 0x35, 0x34, 0x33, 0x32, 0x31, 0x2e, 0x2d, 0x2c, 0x2b, 0x2a, 0x29, 0x27, 0x26, 0x25, 0x23, 0x1f, 0x1e, 0x1d, 0x1b, 0x18, 0x17, 0x10, 0x0f, 0x08, 0x04, 0x02, 0x01, 0x00 }; /* * Local function prototypes. */ static int isp_parse_async(ispsoftc_t *, uint16_t); static int isp_parse_async_fc(ispsoftc_t *, uint16_t); static int isp_handle_other_response(ispsoftc_t *, int, isphdr_t *, uint32_t *); static void isp_parse_status(ispsoftc_t *, ispstatusreq_t *, XS_T *, long *); static void isp_parse_status_24xx(ispsoftc_t *, isp24xx_statusreq_t *, XS_T *, long *); static void isp_fastpost_complete(ispsoftc_t *, uint32_t); static int isp_mbox_continue(ispsoftc_t *); static void isp_scsi_init(ispsoftc_t *); static void isp_scsi_channel_init(ispsoftc_t *, int); static void isp_fibre_init(ispsoftc_t *); static void isp_fibre_init_2400(ispsoftc_t *); static void isp_clear_portdb(ispsoftc_t *, int); static void isp_mark_portdb(ispsoftc_t *, int); static int isp_plogx(ispsoftc_t *, int, uint16_t, uint32_t, int, int); static int isp_port_login(ispsoftc_t *, uint16_t, uint32_t); static int isp_port_logout(ispsoftc_t *, uint16_t, uint32_t); static int isp_getpdb(ispsoftc_t *, int, uint16_t, isp_pdb_t *, int); static int isp_gethandles(ispsoftc_t *, int, uint16_t *, int *, int, int); static void isp_dump_chip_portdb(ispsoftc_t *, int, int); static uint64_t isp_get_wwn(ispsoftc_t *, int, int, int); static int isp_fclink_test(ispsoftc_t *, int, int); static int isp_pdb_sync(ispsoftc_t *, int); static int isp_scan_loop(ispsoftc_t *, int); static int isp_gid_ft_sns(ispsoftc_t *, int); static int isp_gid_ft_ct_passthru(ispsoftc_t *, int); static int isp_scan_fabric(ispsoftc_t *, int); static int isp_login_device(ispsoftc_t *, int, uint32_t, isp_pdb_t *, uint16_t *); static int isp_send_change_request(ispsoftc_t *, int); static int isp_register_fc4_type(ispsoftc_t *, int); static int isp_register_fc4_type_24xx(ispsoftc_t *, int); static int isp_register_fc4_features_24xx(ispsoftc_t *, int); static uint16_t isp_next_handle(ispsoftc_t *, uint16_t *); static int isp_fw_state(ispsoftc_t *, int); static void isp_mboxcmd_qnw(ispsoftc_t *, mbreg_t *, int); static void isp_mboxcmd(ispsoftc_t *, mbreg_t *); static void isp_spi_update(ispsoftc_t *, int); static void isp_setdfltsdparm(ispsoftc_t *); static void isp_setdfltfcparm(ispsoftc_t *, int); static int isp_read_nvram(ispsoftc_t *, int); static int isp_read_nvram_2400(ispsoftc_t *, uint8_t *); static void isp_rdnvram_word(ispsoftc_t *, int, uint16_t *); static void isp_rd_2400_nvram(ispsoftc_t *, uint32_t, uint32_t *); static void isp_parse_nvram_1020(ispsoftc_t *, uint8_t *); static void isp_parse_nvram_1080(ispsoftc_t *, int, uint8_t *); static void isp_parse_nvram_12160(ispsoftc_t *, int, uint8_t *); static void isp_parse_nvram_2100(ispsoftc_t *, uint8_t *); static void isp_parse_nvram_2400(ispsoftc_t *, uint8_t *); static void isp_change_fw_state(ispsoftc_t *isp, int chan, int state) { fcparam *fcp = FCPARAM(isp, chan); if (fcp->isp_fwstate == state) return; isp_prt(isp, ISP_LOGCONFIG|ISP_LOG_SANCFG, "Chan %d Firmware state <%s->%s>", chan, isp_fc_fw_statename(fcp->isp_fwstate), isp_fc_fw_statename(state)); fcp->isp_fwstate = state; } /* * Reset Hardware. * * Hit the chip over the head, download new f/w if available and set it running. * * Locking done elsewhere. */ void isp_reset(ispsoftc_t *isp, int do_load_defaults) { mbreg_t mbs; char *buf; uint64_t fwt; uint32_t code_org, val; int loops, i, dodnld = 1; const char *btype = "????"; static const char dcrc[] = "Downloaded RISC Code Checksum Failure"; isp->isp_state = ISP_NILSTATE; if (isp->isp_dead) { isp_shutdown(isp); ISP_DISABLE_INTS(isp); return; } /* * Basic types (SCSI, FibreChannel and PCI or SBus) * have been set in the MD code. We figure out more * here. Possibly more refined types based upon PCI * identification. Chip revision has been gathered. * * After we've fired this chip up, zero out the conf1 register * for SCSI adapters and do other settings for the 2100. */ ISP_DISABLE_INTS(isp); /* * Pick an initial maxcmds value which will be used * to allocate xflist pointer space. It may be changed * later by the firmware. */ if (IS_24XX(isp)) { isp->isp_maxcmds = 4096; } else if (IS_2322(isp)) { isp->isp_maxcmds = 2048; } else if (IS_23XX(isp) || IS_2200(isp)) { isp->isp_maxcmds = 1024; } else { isp->isp_maxcmds = 512; } /* * Set up DMA for the request and response queues. * * We do this now so we can use the request queue * for dma to load firmware from. */ if (ISP_MBOXDMASETUP(isp) != 0) { isp_prt(isp, ISP_LOGERR, "Cannot setup DMA"); return; } /* * Set up default request/response queue in-pointer/out-pointer * register indices. */ if (IS_24XX(isp)) { isp->isp_rqstinrp = BIU2400_REQINP; isp->isp_rqstoutrp = BIU2400_REQOUTP; isp->isp_respinrp = BIU2400_RSPINP; isp->isp_respoutrp = BIU2400_RSPOUTP; } else if (IS_23XX(isp)) { isp->isp_rqstinrp = BIU_REQINP; isp->isp_rqstoutrp = BIU_REQOUTP; isp->isp_respinrp = BIU_RSPINP; isp->isp_respoutrp = BIU_RSPOUTP; } else { isp->isp_rqstinrp = INMAILBOX4; isp->isp_rqstoutrp = OUTMAILBOX4; isp->isp_respinrp = OUTMAILBOX5; isp->isp_respoutrp = INMAILBOX5; } /* * Put the board into PAUSE mode (so we can read the SXP registers * or write FPM/FBM registers). */ if (IS_24XX(isp)) { ISP_WRITE(isp, BIU2400_HCCR, HCCR_2400_CMD_CLEAR_HOST_INT); ISP_WRITE(isp, BIU2400_HCCR, HCCR_2400_CMD_CLEAR_RISC_INT); ISP_WRITE(isp, BIU2400_HCCR, HCCR_2400_CMD_PAUSE); } else { ISP_WRITE(isp, HCCR, HCCR_CMD_PAUSE); } if (IS_FC(isp)) { switch (isp->isp_type) { case ISP_HA_FC_2100: btype = "2100"; break; case ISP_HA_FC_2200: btype = "2200"; break; case ISP_HA_FC_2300: btype = "2300"; break; case ISP_HA_FC_2312: btype = "2312"; break; case ISP_HA_FC_2322: btype = "2322"; break; case ISP_HA_FC_2400: btype = "2422"; break; case ISP_HA_FC_2500: btype = "2532"; break; case ISP_HA_FC_2600: btype = "2031"; break; default: break; } if (!IS_24XX(isp)) { /* * While we're paused, reset the FPM module and FBM * fifos. */ ISP_WRITE(isp, BIU2100_CSR, BIU2100_FPM0_REGS); ISP_WRITE(isp, FPM_DIAG_CONFIG, FPM_SOFT_RESET); ISP_WRITE(isp, BIU2100_CSR, BIU2100_FB_REGS); ISP_WRITE(isp, FBM_CMD, FBMCMD_FIFO_RESET_ALL); ISP_WRITE(isp, BIU2100_CSR, BIU2100_RISC_REGS); } } else if (IS_1240(isp)) { sdparam *sdp; btype = "1240"; isp->isp_clock = 60; sdp = SDPARAM(isp, 0); sdp->isp_ultramode = 1; sdp = SDPARAM(isp, 1); sdp->isp_ultramode = 1; /* * XXX: Should probably do some bus sensing. */ } else if (IS_ULTRA3(isp)) { sdparam *sdp = isp->isp_param; isp->isp_clock = 100; if (IS_10160(isp)) btype = "10160"; else if (IS_12160(isp)) btype = "12160"; else btype = ""; sdp->isp_lvdmode = 1; if (IS_DUALBUS(isp)) { sdp++; sdp->isp_lvdmode = 1; } } else if (IS_ULTRA2(isp)) { static const char m[] = "bus %d is in %s Mode"; uint16_t l; sdparam *sdp = SDPARAM(isp, 0); isp->isp_clock = 100; if (IS_1280(isp)) btype = "1280"; else if (IS_1080(isp)) btype = "1080"; else btype = ""; l = ISP_READ(isp, SXP_PINS_DIFF) & ISP1080_MODE_MASK; switch (l) { case ISP1080_LVD_MODE: sdp->isp_lvdmode = 1; isp_prt(isp, ISP_LOGCONFIG, m, 0, "LVD"); break; case ISP1080_HVD_MODE: sdp->isp_diffmode = 1; isp_prt(isp, ISP_LOGCONFIG, m, 0, "Differential"); break; case ISP1080_SE_MODE: sdp->isp_ultramode = 1; isp_prt(isp, ISP_LOGCONFIG, m, 0, "Single-Ended"); break; default: isp_prt(isp, ISP_LOGERR, "unknown mode on bus %d (0x%x)", 0, l); break; } if (IS_DUALBUS(isp)) { sdp = SDPARAM(isp, 1); l = ISP_READ(isp, SXP_PINS_DIFF|SXP_BANK1_SELECT); l &= ISP1080_MODE_MASK; switch (l) { case ISP1080_LVD_MODE: sdp->isp_lvdmode = 1; isp_prt(isp, ISP_LOGCONFIG, m, 1, "LVD"); break; case ISP1080_HVD_MODE: sdp->isp_diffmode = 1; isp_prt(isp, ISP_LOGCONFIG, m, 1, "Differential"); break; case ISP1080_SE_MODE: sdp->isp_ultramode = 1; isp_prt(isp, ISP_LOGCONFIG, m, 1, "Single-Ended"); break; default: isp_prt(isp, ISP_LOGERR, "unknown mode on bus %d (0x%x)", 1, l); break; } } } else { sdparam *sdp = SDPARAM(isp, 0); i = ISP_READ(isp, BIU_CONF0) & BIU_CONF0_HW_MASK; switch (i) { default: isp_prt(isp, ISP_LOGALL, "Unknown Chip Type 0x%x", i); /* FALLTHROUGH */ case 1: btype = "1020"; isp->isp_type = ISP_HA_SCSI_1020; isp->isp_clock = 40; break; case 2: /* * Some 1020A chips are Ultra Capable, but don't * run the clock rate up for that unless told to * do so by the Ultra Capable bits being set. */ btype = "1020A"; isp->isp_type = ISP_HA_SCSI_1020A; isp->isp_clock = 40; break; case 3: btype = "1040"; isp->isp_type = ISP_HA_SCSI_1040; isp->isp_clock = 60; break; case 4: btype = "1040A"; isp->isp_type = ISP_HA_SCSI_1040A; isp->isp_clock = 60; break; case 5: btype = "1040B"; isp->isp_type = ISP_HA_SCSI_1040B; isp->isp_clock = 60; break; case 6: btype = "1040C"; isp->isp_type = ISP_HA_SCSI_1040C; isp->isp_clock = 60; break; } /* * Now, while we're at it, gather info about ultra * and/or differential mode. */ if (ISP_READ(isp, SXP_PINS_DIFF) & SXP_PINS_DIFF_MODE) { isp_prt(isp, ISP_LOGCONFIG, "Differential Mode"); sdp->isp_diffmode = 1; } else { sdp->isp_diffmode = 0; } i = ISP_READ(isp, RISC_PSR); if (isp->isp_bustype == ISP_BT_SBUS) { i &= RISC_PSR_SBUS_ULTRA; } else { i &= RISC_PSR_PCI_ULTRA; } if (i != 0) { isp_prt(isp, ISP_LOGCONFIG, "Ultra Mode Capable"); sdp->isp_ultramode = 1; /* * If we're in Ultra Mode, we have to be 60MHz clock- * even for the SBus version. */ isp->isp_clock = 60; } else { sdp->isp_ultramode = 0; /* * Clock is known. Gronk. */ } /* * Machine dependent clock (if set) overrides * our generic determinations. */ if (isp->isp_mdvec->dv_clock) { if (isp->isp_mdvec->dv_clock < isp->isp_clock) { isp->isp_clock = isp->isp_mdvec->dv_clock; } } } /* * Clear instrumentation */ isp->isp_intcnt = isp->isp_intbogus = 0; /* * Do MD specific pre initialization */ ISP_RESET0(isp); /* * Hit the chip over the head with hammer, * and give it a chance to recover. */ if (IS_SCSI(isp)) { ISP_WRITE(isp, BIU_ICR, BIU_ICR_SOFT_RESET); /* * A slight delay... */ ISP_DELAY(100); /* * Clear data && control DMA engines. */ ISP_WRITE(isp, CDMA_CONTROL, DMA_CNTRL_CLEAR_CHAN | DMA_CNTRL_RESET_INT); ISP_WRITE(isp, DDMA_CONTROL, DMA_CNTRL_CLEAR_CHAN | DMA_CNTRL_RESET_INT); } else if (IS_24XX(isp)) { /* * Stop DMA and wait for it to stop. */ ISP_WRITE(isp, BIU2400_CSR, BIU2400_DMA_STOP|(3 << 4)); for (val = loops = 0; loops < 30000; loops++) { ISP_DELAY(10); val = ISP_READ(isp, BIU2400_CSR); if ((val & BIU2400_DMA_ACTIVE) == 0) { break; } } if (val & BIU2400_DMA_ACTIVE) { ISP_RESET0(isp); isp_prt(isp, ISP_LOGERR, "DMA Failed to Stop on Reset"); return; } /* * Hold it in SOFT_RESET and STOP state for 100us. */ ISP_WRITE(isp, BIU2400_CSR, BIU2400_SOFT_RESET|BIU2400_DMA_STOP|(3 << 4)); ISP_DELAY(100); for (loops = 0; loops < 10000; loops++) { ISP_DELAY(5); val = ISP_READ(isp, OUTMAILBOX0); } for (val = loops = 0; loops < 500000; loops ++) { val = ISP_READ(isp, BIU2400_CSR); if ((val & BIU2400_SOFT_RESET) == 0) { break; } } if (val & BIU2400_SOFT_RESET) { ISP_RESET0(isp); isp_prt(isp, ISP_LOGERR, "Failed to come out of reset"); return; } } else { ISP_WRITE(isp, BIU2100_CSR, BIU2100_SOFT_RESET); /* * A slight delay... */ ISP_DELAY(100); /* * Clear data && control DMA engines. */ ISP_WRITE(isp, CDMA2100_CONTROL, DMA_CNTRL2100_CLEAR_CHAN | DMA_CNTRL2100_RESET_INT); ISP_WRITE(isp, TDMA2100_CONTROL, DMA_CNTRL2100_CLEAR_CHAN | DMA_CNTRL2100_RESET_INT); ISP_WRITE(isp, RDMA2100_CONTROL, DMA_CNTRL2100_CLEAR_CHAN | DMA_CNTRL2100_RESET_INT); } /* * Wait for ISP to be ready to go... */ loops = MBOX_DELAY_COUNT; for (;;) { if (IS_SCSI(isp)) { if (!(ISP_READ(isp, BIU_ICR) & BIU_ICR_SOFT_RESET)) { break; } } else if (IS_24XX(isp)) { if (ISP_READ(isp, OUTMAILBOX0) == 0) { break; } } else { if (!(ISP_READ(isp, BIU2100_CSR) & BIU2100_SOFT_RESET)) break; } ISP_DELAY(100); if (--loops < 0) { ISP_DUMPREGS(isp, "chip reset timed out"); ISP_RESET0(isp); return; } } /* * After we've fired this chip up, zero out the conf1 register * for SCSI adapters and other settings for the 2100. */ if (IS_SCSI(isp)) { ISP_WRITE(isp, BIU_CONF1, 0); } else if (!IS_24XX(isp)) { ISP_WRITE(isp, BIU2100_CSR, 0); } /* * Reset RISC Processor */ if (IS_24XX(isp)) { ISP_WRITE(isp, BIU2400_HCCR, HCCR_2400_CMD_RESET); ISP_WRITE(isp, BIU2400_HCCR, HCCR_2400_CMD_RELEASE); ISP_WRITE(isp, BIU2400_HCCR, HCCR_2400_CMD_CLEAR_RESET); } else { ISP_WRITE(isp, HCCR, HCCR_CMD_RESET); ISP_DELAY(100); ISP_WRITE(isp, BIU_SEMA, 0); } /* * Post-RISC Reset stuff. */ if (IS_24XX(isp)) { for (val = loops = 0; loops < 5000000; loops++) { ISP_DELAY(5); val = ISP_READ(isp, OUTMAILBOX0); if (val == 0) { break; } } if (val != 0) { ISP_RESET0(isp); isp_prt(isp, ISP_LOGERR, "reset didn't clear"); return; } } else if (IS_SCSI(isp)) { uint16_t tmp = isp->isp_mdvec->dv_conf1; /* * Busted FIFO. Turn off all but burst enables. */ if (isp->isp_type == ISP_HA_SCSI_1040A) { tmp &= BIU_BURST_ENABLE; } ISP_SETBITS(isp, BIU_CONF1, tmp); if (tmp & BIU_BURST_ENABLE) { ISP_SETBITS(isp, CDMA_CONF, DMA_ENABLE_BURST); ISP_SETBITS(isp, DDMA_CONF, DMA_ENABLE_BURST); } if (SDPARAM(isp, 0)->isp_ptisp) { if (SDPARAM(isp, 0)->isp_ultramode) { while (ISP_READ(isp, RISC_MTR) != 0x1313) { ISP_WRITE(isp, RISC_MTR, 0x1313); ISP_WRITE(isp, HCCR, HCCR_CMD_STEP); } } else { ISP_WRITE(isp, RISC_MTR, 0x1212); } /* * PTI specific register */ ISP_WRITE(isp, RISC_EMB, DUAL_BANK); } else { ISP_WRITE(isp, RISC_MTR, 0x1212); } ISP_WRITE(isp, HCCR, HCCR_CMD_RELEASE); } else { ISP_WRITE(isp, RISC_MTR2100, 0x1212); if (IS_2200(isp) || IS_23XX(isp)) { ISP_WRITE(isp, HCCR, HCCR_2X00_DISABLE_PARITY_PAUSE); } ISP_WRITE(isp, HCCR, HCCR_CMD_RELEASE); } ISP_WRITE(isp, isp->isp_rqstinrp, 0); ISP_WRITE(isp, isp->isp_rqstoutrp, 0); ISP_WRITE(isp, isp->isp_respinrp, 0); ISP_WRITE(isp, isp->isp_respoutrp, 0); if (IS_24XX(isp)) { if (!IS_26XX(isp)) { ISP_WRITE(isp, BIU2400_PRI_REQINP, 0); ISP_WRITE(isp, BIU2400_PRI_REQOUTP, 0); } ISP_WRITE(isp, BIU2400_ATIO_RSPINP, 0); ISP_WRITE(isp, BIU2400_ATIO_RSPOUTP, 0); } /* * Do MD specific post initialization */ ISP_RESET1(isp); /* * Wait for everything to finish firing up. * * Avoid doing this on early 2312s because you can generate a PCI * parity error (chip breakage). */ if (IS_2312(isp) && isp->isp_revision < 2) { ISP_DELAY(100); } else { loops = MBOX_DELAY_COUNT; while (ISP_READ(isp, OUTMAILBOX0) == MBOX_BUSY) { ISP_DELAY(100); if (--loops < 0) { ISP_RESET0(isp); isp_prt(isp, ISP_LOGERR, "MBOX_BUSY never cleared on reset"); return; } } } /* * Up until this point we've done everything by just reading or * setting registers. From this point on we rely on at least *some* * kind of firmware running in the card. */ /* * Do some sanity checking by running a NOP command. * If it succeeds, the ROM firmware is now running. */ MBSINIT(&mbs, MBOX_NO_OP, MBLOGALL, 0); isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { isp_prt(isp, ISP_LOGERR, "NOP command failed (%x)", mbs.param[0]); ISP_RESET0(isp); return; } /* * Do some operational tests */ if (IS_SCSI(isp) || IS_24XX(isp)) { static const uint16_t patterns[MAX_MAILBOX] = { 0x0000, 0xdead, 0xbeef, 0xffff, 0xa5a5, 0x5a5a, 0x7f7f, 0x7ff7, 0x3421, 0xabcd, 0xdcba, 0xfeef, 0xbead, 0xdebe, 0x2222, 0x3333, 0x5555, 0x6666, 0x7777, 0xaaaa, 0xffff, 0xdddd, 0x9999, 0x1fbc, 0x6666, 0x6677, 0x1122, 0x33ff, 0x0000, 0x0001, 0x1000, 0x1010, }; int nmbox = ISP_NMBOX(isp); if (IS_SCSI(isp)) nmbox = 6; MBSINIT(&mbs, MBOX_MAILBOX_REG_TEST, MBLOGALL, 0); for (i = 1; i < nmbox; i++) { mbs.param[i] = patterns[i]; } isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { ISP_RESET0(isp); return; } for (i = 1; i < nmbox; i++) { if (mbs.param[i] != patterns[i]) { ISP_RESET0(isp); isp_prt(isp, ISP_LOGERR, "Register Test Failed at Register %d: should have 0x%04x but got 0x%04x", i, patterns[i], mbs.param[i]); return; } } } /* * Download new Firmware, unless requested not to do so. * This is made slightly trickier in some cases where the * firmware of the ROM revision is newer than the revision * compiled into the driver. So, where we used to compare * versions of our f/w and the ROM f/w, now we just see * whether we have f/w at all and whether a config flag * has disabled our download. */ if ((isp->isp_mdvec->dv_ispfw == NULL) || (isp->isp_confopts & ISP_CFG_NORELOAD)) { dodnld = 0; } if (IS_24XX(isp)) { code_org = ISP_CODE_ORG_2400; } else if (IS_23XX(isp)) { code_org = ISP_CODE_ORG_2300; } else { code_org = ISP_CODE_ORG; } isp->isp_loaded_fw = 0; if (dodnld && IS_24XX(isp)) { const uint32_t *ptr = isp->isp_mdvec->dv_ispfw; int wordload; /* * Keep loading until we run out of f/w. */ code_org = ptr[2]; /* 1st load address is our start addr */ wordload = 0; for (;;) { uint32_t la, wi, wl; isp_prt(isp, ISP_LOGDEBUG0, "load 0x%x words of code at load address 0x%x", ptr[3], ptr[2]); wi = 0; la = ptr[2]; wl = ptr[3]; while (wi < ptr[3]) { uint32_t *cp; uint32_t nw; nw = ISP_QUEUE_SIZE(RQUEST_QUEUE_LEN(isp)) >> 2; if (nw > wl) { nw = wl; } cp = isp->isp_rquest; for (i = 0; i < nw; i++) { ISP_IOXPUT_32(isp, ptr[wi++], &cp[i]); wl--; } MEMORYBARRIER(isp, SYNC_REQUEST, 0, ISP_QUEUE_SIZE(RQUEST_QUEUE_LEN(isp)), -1); again: MBSINIT(&mbs, 0, MBLOGALL, 0); if (la < 0x10000 && nw < 0x10000) { mbs.param[0] = MBOX_LOAD_RISC_RAM_2100; mbs.param[1] = la; mbs.param[2] = DMA_WD1(isp->isp_rquest_dma); mbs.param[3] = DMA_WD0(isp->isp_rquest_dma); mbs.param[4] = nw; mbs.param[6] = DMA_WD3(isp->isp_rquest_dma); mbs.param[7] = DMA_WD2(isp->isp_rquest_dma); isp_prt(isp, ISP_LOGDEBUG0, "LOAD RISC RAM 2100 %u words at load address 0x%x", nw, la); } else if (wordload) { union { const uint32_t *cp; uint32_t *np; } ucd; ucd.cp = (const uint32_t *)cp; mbs.param[0] = MBOX_WRITE_RAM_WORD_EXTENDED; mbs.param[1] = la; mbs.param[2] = (*ucd.np); mbs.param[3] = (*ucd.np) >> 16; mbs.param[8] = la >> 16; isp->isp_mbxwrk0 = nw - 1; isp->isp_mbxworkp = ucd.np+1; isp->isp_mbxwrk1 = (la + 1); isp->isp_mbxwrk8 = (la + 1) >> 16; isp_prt(isp, ISP_LOGDEBUG0, "WRITE RAM WORD EXTENDED %u words at load address 0x%x", nw, la); } else { mbs.param[0] = MBOX_LOAD_RISC_RAM; mbs.param[1] = la; mbs.param[2] = DMA_WD1(isp->isp_rquest_dma); mbs.param[3] = DMA_WD0(isp->isp_rquest_dma); mbs.param[4] = nw >> 16; mbs.param[5] = nw; mbs.param[6] = DMA_WD3(isp->isp_rquest_dma); mbs.param[7] = DMA_WD2(isp->isp_rquest_dma); mbs.param[8] = la >> 16; isp_prt(isp, ISP_LOGDEBUG0, "LOAD RISC RAM %u words at load address 0x%x", nw, la); } isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { if (mbs.param[0] == MBOX_HOST_INTERFACE_ERROR) { isp_prt(isp, ISP_LOGERR, "switching to word load"); wordload = 1; goto again; } isp_prt(isp, ISP_LOGERR, "F/W Risc Ram Load Failed"); ISP_RESET0(isp); return; } la += nw; } if (ptr[1] == 0) { break; } ptr += ptr[3]; } isp->isp_loaded_fw = 1; } else if (dodnld && IS_23XX(isp)) { const uint16_t *ptr = isp->isp_mdvec->dv_ispfw; uint16_t wi, wl, segno; uint32_t la; la = code_org; segno = 0; for (;;) { uint32_t nxtaddr; isp_prt(isp, ISP_LOGDEBUG0, "load 0x%x words of code at load address 0x%x", ptr[3], la); wi = 0; wl = ptr[3]; while (wi < ptr[3]) { uint16_t *cp; uint16_t nw; nw = ISP_QUEUE_SIZE(RQUEST_QUEUE_LEN(isp)) >> 1; if (nw > wl) { nw = wl; } if (nw > (1 << 15)) { nw = 1 << 15; } cp = isp->isp_rquest; for (i = 0; i < nw; i++) { ISP_IOXPUT_16(isp, ptr[wi++], &cp[i]); wl--; } MEMORYBARRIER(isp, SYNC_REQUEST, 0, ISP_QUEUE_SIZE(RQUEST_QUEUE_LEN(isp)), -1); MBSINIT(&mbs, 0, MBLOGALL, 0); if (la < 0x10000) { mbs.param[0] = MBOX_LOAD_RISC_RAM_2100; mbs.param[1] = la; mbs.param[2] = DMA_WD1(isp->isp_rquest_dma); mbs.param[3] = DMA_WD0(isp->isp_rquest_dma); mbs.param[4] = nw; mbs.param[6] = DMA_WD3(isp->isp_rquest_dma); mbs.param[7] = DMA_WD2(isp->isp_rquest_dma); isp_prt(isp, ISP_LOGDEBUG1, "LOAD RISC RAM 2100 %u words at load address 0x%x\n", nw, la); } else { mbs.param[0] = MBOX_LOAD_RISC_RAM; mbs.param[1] = la; mbs.param[2] = DMA_WD1(isp->isp_rquest_dma); mbs.param[3] = DMA_WD0(isp->isp_rquest_dma); mbs.param[4] = nw; mbs.param[6] = DMA_WD3(isp->isp_rquest_dma); mbs.param[7] = DMA_WD2(isp->isp_rquest_dma); mbs.param[8] = la >> 16; isp_prt(isp, ISP_LOGDEBUG1, "LOAD RISC RAM %u words at load address 0x%x\n", nw, la); } isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { isp_prt(isp, ISP_LOGERR, "F/W Risc Ram Load Failed"); ISP_RESET0(isp); return; } la += nw; } if (!IS_2322(isp)) { break; } if (++segno == 3) { break; } /* * If we're a 2322, the firmware actually comes in * three chunks. We loaded the first at the code_org * address. The other two chunks, which follow right * after each other in memory here, get loaded at * addresses specfied at offset 0x9..0xB. */ nxtaddr = ptr[3]; ptr = &ptr[nxtaddr]; la = ptr[5] | ((ptr[4] & 0x3f) << 16); } isp->isp_loaded_fw = 1; } else if (dodnld) { union { const uint16_t *cp; uint16_t *np; } ucd; ucd.cp = isp->isp_mdvec->dv_ispfw; isp->isp_mbxworkp = &ucd.np[1]; isp->isp_mbxwrk0 = ucd.np[3] - 1; isp->isp_mbxwrk1 = code_org + 1; MBSINIT(&mbs, MBOX_WRITE_RAM_WORD, MBLOGNONE, 0); mbs.param[1] = code_org; mbs.param[2] = ucd.np[0]; isp_prt(isp, ISP_LOGDEBUG1, "WRITE RAM %u words at load address 0x%x", ucd.np[3], code_org); isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { isp_prt(isp, ISP_LOGERR, "F/W download failed at word %d", isp->isp_mbxwrk1 - code_org); ISP_RESET0(isp); return; } } else if (IS_26XX(isp)) { MBSINIT(&mbs, MBOX_LOAD_FLASH_FIRMWARE, MBLOGALL, 5000000); mbs.ibitm = 0x01; mbs.obitm = 0x07; isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { isp_prt(isp, ISP_LOGERR, "Flash F/W load failed"); ISP_RESET0(isp); return; } } else { isp_prt(isp, ISP_LOGDEBUG2, "skipping f/w download"); } /* * If we loaded firmware, verify its checksum */ if (isp->isp_loaded_fw) { MBSINIT(&mbs, MBOX_VERIFY_CHECKSUM, MBLOGNONE, 0); if (IS_24XX(isp)) { mbs.param[1] = code_org >> 16; mbs.param[2] = code_org; } else { mbs.param[1] = code_org; } isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { isp_prt(isp, ISP_LOGERR, dcrc); ISP_RESET0(isp); return; } } /* * Now start it rolling. * * If we didn't actually download f/w, * we still need to (re)start it. */ MBSINIT(&mbs, MBOX_EXEC_FIRMWARE, MBLOGALL, 5000000); if (IS_24XX(isp)) { mbs.param[1] = code_org >> 16; mbs.param[2] = code_org; if (isp->isp_loaded_fw) { mbs.param[3] = 0; } else { mbs.param[3] = 1; } } else if (IS_2322(isp)) { mbs.param[1] = code_org; if (isp->isp_loaded_fw) { mbs.param[2] = 0; } else { mbs.param[2] = 1; } } else { mbs.param[1] = code_org; } isp_mboxcmd(isp, &mbs); if (IS_2322(isp) || IS_24XX(isp)) { if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { ISP_RESET0(isp); return; } } if (IS_SCSI(isp)) { /* * Set CLOCK RATE, but only if asked to. */ if (isp->isp_clock) { MBSINIT(&mbs, MBOX_SET_CLOCK_RATE, MBLOGALL, 0); mbs.param[1] = isp->isp_clock; isp_mboxcmd(isp, &mbs); /* we will try not to care if this fails */ } } /* * Ask the chip for the current firmware version. * This should prove that the new firmware is working. */ MBSINIT(&mbs, MBOX_ABOUT_FIRMWARE, MBLOGALL, 0); isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { ISP_RESET0(isp); return; } /* * The SBus firmware that we are using apparently does not return * major, minor, micro revisions in the mailbox registers, which * is really, really, annoying. */ if (ISP_SBUS_SUPPORTED && isp->isp_bustype == ISP_BT_SBUS) { if (dodnld) { #ifdef ISP_TARGET_MODE isp->isp_fwrev[0] = 7; isp->isp_fwrev[1] = 55; #else isp->isp_fwrev[0] = 1; isp->isp_fwrev[1] = 37; #endif isp->isp_fwrev[2] = 0; } } else { isp->isp_fwrev[0] = mbs.param[1]; isp->isp_fwrev[1] = mbs.param[2]; isp->isp_fwrev[2] = mbs.param[3]; } if (IS_FC(isp)) { /* * We do not believe firmware attributes for 2100 code less * than 1.17.0, unless it's the firmware we specifically * are loading. * * Note that all 22XX and later f/w is greater than 1.X.0. */ if ((ISP_FW_OLDER_THAN(isp, 1, 17, 1))) { #ifdef USE_SMALLER_2100_FIRMWARE isp->isp_fwattr = ISP_FW_ATTR_SCCLUN; #else isp->isp_fwattr = 0; #endif } else { isp->isp_fwattr = mbs.param[6]; } if (IS_24XX(isp)) { isp->isp_fwattr |= ((uint64_t) mbs.param[15]) << 16; if (isp->isp_fwattr & ISP2400_FW_ATTR_EXTNDED) { isp->isp_fwattr |= (((uint64_t) mbs.param[16]) << 32) | (((uint64_t) mbs.param[17]) << 48); } } } else { isp->isp_fwattr = 0; } isp_prt(isp, ISP_LOGCONFIG, "Board Type %s, Chip Revision 0x%x, %s F/W Revision %d.%d.%d", btype, isp->isp_revision, dodnld? "loaded" : "resident", isp->isp_fwrev[0], isp->isp_fwrev[1], isp->isp_fwrev[2]); fwt = isp->isp_fwattr; if (IS_24XX(isp)) { buf = FCPARAM(isp, 0)->isp_scratch; ISP_SNPRINTF(buf, ISP_FC_SCRLEN, "Attributes:"); if (fwt & ISP2400_FW_ATTR_CLASS2) { fwt ^=ISP2400_FW_ATTR_CLASS2; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s Class2", buf); } if (fwt & ISP2400_FW_ATTR_IP) { fwt ^=ISP2400_FW_ATTR_IP; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s IP", buf); } if (fwt & ISP2400_FW_ATTR_MULTIID) { fwt ^=ISP2400_FW_ATTR_MULTIID; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s MultiID", buf); } if (fwt & ISP2400_FW_ATTR_SB2) { fwt ^=ISP2400_FW_ATTR_SB2; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s SB2", buf); } if (fwt & ISP2400_FW_ATTR_T10CRC) { fwt ^=ISP2400_FW_ATTR_T10CRC; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s T10CRC", buf); } if (fwt & ISP2400_FW_ATTR_VI) { fwt ^=ISP2400_FW_ATTR_VI; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s VI", buf); } if (fwt & ISP2400_FW_ATTR_MQ) { fwt ^=ISP2400_FW_ATTR_MQ; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s MQ", buf); } if (fwt & ISP2400_FW_ATTR_MSIX) { fwt ^=ISP2400_FW_ATTR_MSIX; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s MSIX", buf); } if (fwt & ISP2400_FW_ATTR_FCOE) { fwt ^=ISP2400_FW_ATTR_FCOE; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s FCOE", buf); } if (fwt & ISP2400_FW_ATTR_VP0) { fwt ^= ISP2400_FW_ATTR_VP0; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s VP0_Decoupling", buf); } if (fwt & ISP2400_FW_ATTR_EXPFW) { fwt ^= ISP2400_FW_ATTR_EXPFW; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s (Experimental)", buf); } if (fwt & ISP2400_FW_ATTR_HOTFW) { fwt ^= ISP2400_FW_ATTR_HOTFW; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s HotFW", buf); } fwt &= ~ISP2400_FW_ATTR_EXTNDED; if (fwt & ISP2400_FW_ATTR_EXTVP) { fwt ^= ISP2400_FW_ATTR_EXTVP; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s ExtVP", buf); } if (fwt & ISP2400_FW_ATTR_VN2VN) { fwt ^= ISP2400_FW_ATTR_VN2VN; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s VN2VN", buf); } if (fwt & ISP2400_FW_ATTR_EXMOFF) { fwt ^= ISP2400_FW_ATTR_EXMOFF; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s EXMOFF", buf); } if (fwt & ISP2400_FW_ATTR_NPMOFF) { fwt ^= ISP2400_FW_ATTR_NPMOFF; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s NPMOFF", buf); } if (fwt & ISP2400_FW_ATTR_DIFCHOP) { fwt ^= ISP2400_FW_ATTR_DIFCHOP; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s DIFCHOP", buf); } if (fwt & ISP2400_FW_ATTR_SRIOV) { fwt ^= ISP2400_FW_ATTR_SRIOV; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s SRIOV", buf); } if (fwt & ISP2400_FW_ATTR_ASICTMP) { fwt ^= ISP2400_FW_ATTR_ASICTMP; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s ASICTMP", buf); } if (fwt & ISP2400_FW_ATTR_ATIOMQ) { fwt ^= ISP2400_FW_ATTR_ATIOMQ; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s ATIOMQ", buf); } if (fwt) { ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s (unknown 0x%08x%08x)", buf, (uint32_t) (fwt >> 32), (uint32_t) fwt); } isp_prt(isp, ISP_LOGCONFIG, "%s", buf); } else if (IS_FC(isp)) { buf = FCPARAM(isp, 0)->isp_scratch; ISP_SNPRINTF(buf, ISP_FC_SCRLEN, "Attributes:"); if (fwt & ISP_FW_ATTR_TMODE) { fwt ^=ISP_FW_ATTR_TMODE; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s TargetMode", buf); } if (fwt & ISP_FW_ATTR_SCCLUN) { fwt ^=ISP_FW_ATTR_SCCLUN; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s SCC-Lun", buf); } if (fwt & ISP_FW_ATTR_FABRIC) { fwt ^=ISP_FW_ATTR_FABRIC; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s Fabric", buf); } if (fwt & ISP_FW_ATTR_CLASS2) { fwt ^=ISP_FW_ATTR_CLASS2; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s Class2", buf); } if (fwt & ISP_FW_ATTR_FCTAPE) { fwt ^=ISP_FW_ATTR_FCTAPE; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s FC-Tape", buf); } if (fwt & ISP_FW_ATTR_IP) { fwt ^=ISP_FW_ATTR_IP; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s IP", buf); } if (fwt & ISP_FW_ATTR_VI) { fwt ^=ISP_FW_ATTR_VI; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s VI", buf); } if (fwt & ISP_FW_ATTR_VI_SOLARIS) { fwt ^=ISP_FW_ATTR_VI_SOLARIS; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s VI_SOLARIS", buf); } if (fwt & ISP_FW_ATTR_2KLOGINS) { fwt ^=ISP_FW_ATTR_2KLOGINS; ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s 2K-Login", buf); } if (fwt != 0) { ISP_SNPRINTF(buf, ISP_FC_SCRLEN - strlen(buf), "%s (unknown 0x%08x%08x)", buf, (uint32_t) (fwt >> 32), (uint32_t) fwt); } isp_prt(isp, ISP_LOGCONFIG, "%s", buf); } if (IS_24XX(isp)) { MBSINIT(&mbs, MBOX_GET_RESOURCE_COUNT, MBLOGALL, 0); isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { ISP_RESET0(isp); return; } if (isp->isp_maxcmds >= mbs.param[3]) { isp->isp_maxcmds = mbs.param[3]; } } else { MBSINIT(&mbs, MBOX_GET_FIRMWARE_STATUS, MBLOGALL, 0); isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { ISP_RESET0(isp); return; } if (isp->isp_maxcmds >= mbs.param[2]) { isp->isp_maxcmds = mbs.param[2]; } } isp_prt(isp, ISP_LOGCONFIG, "%d max I/O command limit set", isp->isp_maxcmds); /* * If we don't have Multi-ID f/w loaded, we need to restrict channels to one. * Only make this check for non-SCSI cards (I'm not sure firmware attributes * work for them). */ if (IS_FC(isp) && isp->isp_nchan > 1) { if (!ISP_CAP_MULTI_ID(isp)) { isp_prt(isp, ISP_LOGWARN, "non-MULTIID f/w loaded, " "only can enable 1 of %d channels", isp->isp_nchan); isp->isp_nchan = 1; } else if (!ISP_CAP_VP0(isp)) { isp_prt(isp, ISP_LOGWARN, "We can not use MULTIID " "feature properly without VP0_Decoupling"); isp->isp_nchan = 1; } } if (IS_FC(isp)) { for (i = 0; i < isp->isp_nchan; i++) isp_change_fw_state(isp, i, FW_CONFIG_WAIT); } if (isp->isp_dead) { isp_shutdown(isp); ISP_DISABLE_INTS(isp); return; } isp->isp_state = ISP_RESETSTATE; /* * Okay- now that we have new firmware running, we now (re)set our * notion of how many luns we support. This is somewhat tricky because * if we haven't loaded firmware, we sometimes do not have an easy way * of knowing how many luns we support. * * Expanded lun firmware gives you 32 luns for SCSI cards and * 16384 luns for Fibre Channel cards. * * It turns out that even for QLogic 2100s with ROM 1.10 and above * we do get a firmware attributes word returned in mailbox register 6. * * Because the lun is in a different position in the Request Queue * Entry structure for Fibre Channel with expanded lun firmware, we * can only support one lun (lun zero) when we don't know what kind * of firmware we're running. */ if (IS_SCSI(isp)) { if (dodnld) { if (IS_ULTRA2(isp) || IS_ULTRA3(isp)) { isp->isp_maxluns = 32; } else { isp->isp_maxluns = 8; } } else { isp->isp_maxluns = 8; } } else { if (ISP_CAP_SCCFW(isp)) { isp->isp_maxluns = 0; /* No limit -- 2/8 bytes */ } else { isp->isp_maxluns = 16; } } /* * We get some default values established. As a side * effect, NVRAM is read here (unless overriden by * a configuration flag). */ if (do_load_defaults) { if (IS_SCSI(isp)) { isp_setdfltsdparm(isp); } else { for (i = 0; i < isp->isp_nchan; i++) { isp_setdfltfcparm(isp, i); } } } } /* * Clean firmware shutdown. */ static int isp_deinit(ispsoftc_t *isp) { mbreg_t mbs; isp->isp_state = ISP_NILSTATE; MBSINIT(&mbs, MBOX_STOP_FIRMWARE, MBLOGALL, 500000); mbs.param[1] = 0; mbs.param[2] = 0; mbs.param[3] = 0; mbs.param[4] = 0; mbs.param[5] = 0; mbs.param[6] = 0; mbs.param[7] = 0; mbs.param[8] = 0; isp_mboxcmd(isp, &mbs); return (mbs.param[0] == MBOX_COMMAND_COMPLETE ? 0 : mbs.param[0]); } /* * Initialize Parameters of Hardware to a known state. * * Locks are held before coming here. */ void isp_init(ispsoftc_t *isp) { if (IS_FC(isp)) { if (IS_24XX(isp)) { isp_fibre_init_2400(isp); } else { isp_fibre_init(isp); } } else { isp_scsi_init(isp); } GET_NANOTIME(&isp->isp_init_time); } static void isp_scsi_init(ispsoftc_t *isp) { sdparam *sdp_chan0, *sdp_chan1; mbreg_t mbs; isp->isp_state = ISP_INITSTATE; sdp_chan0 = SDPARAM(isp, 0); sdp_chan1 = sdp_chan0; if (IS_DUALBUS(isp)) { sdp_chan1 = SDPARAM(isp, 1); } /* First do overall per-card settings. */ /* * If we have fast memory timing enabled, turn it on. */ if (sdp_chan0->isp_fast_mttr) { ISP_WRITE(isp, RISC_MTR, 0x1313); } /* * Set Retry Delay and Count. * You set both channels at the same time. */ MBSINIT(&mbs, MBOX_SET_RETRY_COUNT, MBLOGALL, 0); mbs.param[1] = sdp_chan0->isp_retry_count; mbs.param[2] = sdp_chan0->isp_retry_delay; mbs.param[6] = sdp_chan1->isp_retry_count; mbs.param[7] = sdp_chan1->isp_retry_delay; isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { return; } /* * Set ASYNC DATA SETUP time. This is very important. */ MBSINIT(&mbs, MBOX_SET_ASYNC_DATA_SETUP_TIME, MBLOGALL, 0); mbs.param[1] = sdp_chan0->isp_async_data_setup; mbs.param[2] = sdp_chan1->isp_async_data_setup; isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { return; } /* * Set ACTIVE Negation State. */ MBSINIT(&mbs, MBOX_SET_ACT_NEG_STATE, MBLOGNONE, 0); mbs.param[1] = (sdp_chan0->isp_req_ack_active_neg << 4) | (sdp_chan0->isp_data_line_active_neg << 5); mbs.param[2] = (sdp_chan1->isp_req_ack_active_neg << 4) | (sdp_chan1->isp_data_line_active_neg << 5); isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { isp_prt(isp, ISP_LOGERR, "failed to set active negation state (%d,%d), (%d,%d)", sdp_chan0->isp_req_ack_active_neg, sdp_chan0->isp_data_line_active_neg, sdp_chan1->isp_req_ack_active_neg, sdp_chan1->isp_data_line_active_neg); /* * But don't return. */ } /* * Set the Tag Aging limit */ MBSINIT(&mbs, MBOX_SET_TAG_AGE_LIMIT, MBLOGALL, 0); mbs.param[1] = sdp_chan0->isp_tag_aging; mbs.param[2] = sdp_chan1->isp_tag_aging; isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { isp_prt(isp, ISP_LOGERR, "failed to set tag age limit (%d,%d)", sdp_chan0->isp_tag_aging, sdp_chan1->isp_tag_aging); return; } /* * Set selection timeout. */ MBSINIT(&mbs, MBOX_SET_SELECT_TIMEOUT, MBLOGALL, 0); mbs.param[1] = sdp_chan0->isp_selection_timeout; mbs.param[2] = sdp_chan1->isp_selection_timeout; isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { return; } /* now do per-channel settings */ isp_scsi_channel_init(isp, 0); if (IS_DUALBUS(isp)) isp_scsi_channel_init(isp, 1); /* * Now enable request/response queues */ if (IS_ULTRA2(isp) || IS_1240(isp)) { MBSINIT(&mbs, MBOX_INIT_RES_QUEUE_A64, MBLOGALL, 0); mbs.param[1] = RESULT_QUEUE_LEN(isp); mbs.param[2] = DMA_WD1(isp->isp_result_dma); mbs.param[3] = DMA_WD0(isp->isp_result_dma); mbs.param[4] = 0; mbs.param[6] = DMA_WD3(isp->isp_result_dma); mbs.param[7] = DMA_WD2(isp->isp_result_dma); isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { return; } isp->isp_residx = isp->isp_resodx = mbs.param[5]; MBSINIT(&mbs, MBOX_INIT_REQ_QUEUE_A64, MBLOGALL, 0); mbs.param[1] = RQUEST_QUEUE_LEN(isp); mbs.param[2] = DMA_WD1(isp->isp_rquest_dma); mbs.param[3] = DMA_WD0(isp->isp_rquest_dma); mbs.param[5] = 0; mbs.param[6] = DMA_WD3(isp->isp_result_dma); mbs.param[7] = DMA_WD2(isp->isp_result_dma); isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { return; } isp->isp_reqidx = isp->isp_reqodx = mbs.param[4]; } else { MBSINIT(&mbs, MBOX_INIT_RES_QUEUE, MBLOGALL, 0); mbs.param[1] = RESULT_QUEUE_LEN(isp); mbs.param[2] = DMA_WD1(isp->isp_result_dma); mbs.param[3] = DMA_WD0(isp->isp_result_dma); mbs.param[4] = 0; isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { return; } isp->isp_residx = isp->isp_resodx = mbs.param[5]; MBSINIT(&mbs, MBOX_INIT_REQ_QUEUE, MBLOGALL, 0); mbs.param[1] = RQUEST_QUEUE_LEN(isp); mbs.param[2] = DMA_WD1(isp->isp_rquest_dma); mbs.param[3] = DMA_WD0(isp->isp_rquest_dma); mbs.param[5] = 0; isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { return; } isp->isp_reqidx = isp->isp_reqodx = mbs.param[4]; } /* * Turn on LVD transitions for ULTRA2 or better and other features * * Now that we have 32 bit handles, don't do any fast posting * any more. For Ultra2/Ultra3 cards, we can turn on 32 bit RIO * operation or use fast posting. To be conservative, we'll only * do this for Ultra3 cards now because the other cards are so * rare for this author to find and test with. */ MBSINIT(&mbs, MBOX_SET_FW_FEATURES, MBLOGALL, 0); if (IS_ULTRA2(isp)) mbs.param[1] |= FW_FEATURE_LVD_NOTIFY; #ifdef ISP_NO_RIO if (IS_ULTRA3(isp)) mbs.param[1] |= FW_FEATURE_FAST_POST; #else if (IS_ULTRA3(isp)) mbs.param[1] |= FW_FEATURE_RIO_32BIT; #endif if (mbs.param[1] != 0) { uint16_t sfeat = mbs.param[1]; isp_mboxcmd(isp, &mbs); if (mbs.param[0] == MBOX_COMMAND_COMPLETE) { isp_prt(isp, ISP_LOGINFO, "Enabled FW features (0x%x)", sfeat); } } isp->isp_state = ISP_RUNSTATE; } static void isp_scsi_channel_init(ispsoftc_t *isp, int chan) { sdparam *sdp; mbreg_t mbs; int tgt; sdp = SDPARAM(isp, chan); /* * Set (possibly new) Initiator ID. */ MBSINIT(&mbs, MBOX_SET_INIT_SCSI_ID, MBLOGALL, 0); mbs.param[1] = (chan << 7) | sdp->isp_initiator_id; isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { return; } isp_prt(isp, ISP_LOGINFO, "Chan %d Initiator ID is %d", chan, sdp->isp_initiator_id); /* * Set current per-target parameters to an initial safe minimum. */ for (tgt = 0; tgt < MAX_TARGETS; tgt++) { int lun; uint16_t sdf; if (sdp->isp_devparam[tgt].dev_enable == 0) { continue; } #ifndef ISP_TARGET_MODE sdf = sdp->isp_devparam[tgt].goal_flags; sdf &= DPARM_SAFE_DFLT; /* * It is not quite clear when this changed over so that * we could force narrow and async for 1000/1020 cards, * but assume that this is only the case for loaded * firmware. */ if (isp->isp_loaded_fw) { sdf |= DPARM_NARROW | DPARM_ASYNC; } #else /* * The !$*!)$!$)* f/w uses the same index into some * internal table to decide how to respond to negotiations, * so if we've said "let's be safe" for ID X, and ID X * selects *us*, the negotiations will back to 'safe' * (as in narrow/async). What the f/w *should* do is * use the initiator id settings to decide how to respond. */ sdp->isp_devparam[tgt].goal_flags = sdf = DPARM_DEFAULT; #endif MBSINIT(&mbs, MBOX_SET_TARGET_PARAMS, MBLOGNONE, 0); mbs.param[1] = (chan << 15) | (tgt << 8); mbs.param[2] = sdf; if ((sdf & DPARM_SYNC) == 0) { mbs.param[3] = 0; } else { mbs.param[3] = (sdp->isp_devparam[tgt].goal_offset << 8) | (sdp->isp_devparam[tgt].goal_period); } isp_prt(isp, ISP_LOGDEBUG0, "Initial Settings bus%d tgt%d flags 0x%x off 0x%x per 0x%x", chan, tgt, mbs.param[2], mbs.param[3] >> 8, mbs.param[3] & 0xff); isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { sdf = DPARM_SAFE_DFLT; MBSINIT(&mbs, MBOX_SET_TARGET_PARAMS, MBLOGALL, 0); mbs.param[1] = (tgt << 8) | (chan << 15); mbs.param[2] = sdf; mbs.param[3] = 0; isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { continue; } } /* * We don't update any information directly from the f/w * because we need to run at least one command to cause a * new state to be latched up. So, we just assume that we * converge to the values we just had set. * * Ensure that we don't believe tagged queuing is enabled yet. * It turns out that sometimes the ISP just ignores our * attempts to set parameters for devices that it hasn't * seen yet. */ sdp->isp_devparam[tgt].actv_flags = sdf & ~DPARM_TQING; for (lun = 0; lun < (int) isp->isp_maxluns; lun++) { MBSINIT(&mbs, MBOX_SET_DEV_QUEUE_PARAMS, MBLOGALL, 0); mbs.param[1] = (chan << 15) | (tgt << 8) | lun; mbs.param[2] = sdp->isp_max_queue_depth; mbs.param[3] = sdp->isp_devparam[tgt].exc_throttle; isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { break; } } } for (tgt = 0; tgt < MAX_TARGETS; tgt++) { if (sdp->isp_devparam[tgt].dev_refresh) { sdp->sendmarker = 1; sdp->update = 1; break; } } } /* * Fibre Channel specific initialization. */ static void isp_fibre_init(ispsoftc_t *isp) { fcparam *fcp; isp_icb_t local, *icbp = &local; mbreg_t mbs; /* * We only support one channel on non-24XX cards */ fcp = FCPARAM(isp, 0); if (fcp->role == ISP_ROLE_NONE) return; isp->isp_state = ISP_INITSTATE; ISP_MEMZERO(icbp, sizeof (*icbp)); icbp->icb_version = ICB_VERSION1; icbp->icb_fwoptions = fcp->isp_fwoptions; /* * Firmware Options are either retrieved from NVRAM or * are patched elsewhere. We check them for sanity here * and make changes based on board revision, but otherwise * let others decide policy. */ /* * If this is a 2100 < revision 5, we have to turn off FAIRNESS. */ if (IS_2100(isp) && isp->isp_revision < 5) { icbp->icb_fwoptions &= ~ICBOPT_FAIRNESS; } /* * We have to use FULL LOGIN even though it resets the loop too much * because otherwise port database entries don't get updated after * a LIP- this is a known f/w bug for 2100 f/w less than 1.17.0. */ if (!ISP_FW_NEWER_THAN(isp, 1, 17, 0)) { icbp->icb_fwoptions |= ICBOPT_FULL_LOGIN; } /* * Insist on Port Database Update Async notifications */ icbp->icb_fwoptions |= ICBOPT_PDBCHANGE_AE; /* * Make sure that target role reflects into fwoptions. */ if (fcp->role & ISP_ROLE_TARGET) { icbp->icb_fwoptions |= ICBOPT_TGT_ENABLE; } else { icbp->icb_fwoptions &= ~ICBOPT_TGT_ENABLE; } /* * For some reason my 2200 does not generate ATIOs in target mode * if initiator is disabled. Extra logins are better then target * not working at all. */ if ((fcp->role & ISP_ROLE_INITIATOR) || IS_2100(isp) || IS_2200(isp)) { icbp->icb_fwoptions &= ~ICBOPT_INI_DISABLE; } else { icbp->icb_fwoptions |= ICBOPT_INI_DISABLE; } icbp->icb_maxfrmlen = DEFAULT_FRAMESIZE(isp); if (icbp->icb_maxfrmlen < ICB_MIN_FRMLEN || icbp->icb_maxfrmlen > ICB_MAX_FRMLEN) { isp_prt(isp, ISP_LOGERR, "bad frame length (%d) from NVRAM- using %d", DEFAULT_FRAMESIZE(isp), ICB_DFLT_FRMLEN); icbp->icb_maxfrmlen = ICB_DFLT_FRMLEN; } icbp->icb_maxalloc = fcp->isp_maxalloc; if (icbp->icb_maxalloc < 1) { isp_prt(isp, ISP_LOGERR, "bad maximum allocation (%d)- using 16", fcp->isp_maxalloc); icbp->icb_maxalloc = 16; } icbp->icb_execthrottle = DEFAULT_EXEC_THROTTLE(isp); if (icbp->icb_execthrottle < 1) { isp_prt(isp, ISP_LOGERR, "bad execution throttle of %d- using %d", DEFAULT_EXEC_THROTTLE(isp), ICB_DFLT_THROTTLE); icbp->icb_execthrottle = ICB_DFLT_THROTTLE; } icbp->icb_retry_delay = fcp->isp_retry_delay; icbp->icb_retry_count = fcp->isp_retry_count; if (fcp->isp_loopid < LOCAL_LOOP_LIM) { icbp->icb_hardaddr = fcp->isp_loopid; if (isp->isp_confopts & ISP_CFG_OWNLOOPID) icbp->icb_fwoptions |= ICBOPT_HARD_ADDRESS; else icbp->icb_fwoptions |= ICBOPT_PREV_ADDRESS; } /* * Right now we just set extended options to prefer point-to-point * over loop based upon some soft config options. * * NB: for the 2300, ICBOPT_EXTENDED is required. */ if (IS_2100(isp)) { /* * We can't have Fast Posting any more- we now * have 32 bit handles. */ icbp->icb_fwoptions &= ~ICBOPT_FAST_POST; } else if (IS_2200(isp) || IS_23XX(isp)) { icbp->icb_fwoptions |= ICBOPT_EXTENDED; icbp->icb_xfwoptions = fcp->isp_xfwoptions; if (ISP_CAP_FCTAPE(isp)) { if (isp->isp_confopts & ISP_CFG_NOFCTAPE) icbp->icb_xfwoptions &= ~ICBXOPT_FCTAPE; if (isp->isp_confopts & ISP_CFG_FCTAPE) icbp->icb_xfwoptions |= ICBXOPT_FCTAPE; if (icbp->icb_xfwoptions & ICBXOPT_FCTAPE) { icbp->icb_fwoptions &= ~ICBOPT_FULL_LOGIN; /* per documents */ icbp->icb_xfwoptions |= ICBXOPT_FCTAPE_CCQ|ICBXOPT_FCTAPE_CONFIRM; FCPARAM(isp, 0)->fctape_enabled = 1; } else { FCPARAM(isp, 0)->fctape_enabled = 0; } } else { icbp->icb_xfwoptions &= ~ICBXOPT_FCTAPE; FCPARAM(isp, 0)->fctape_enabled = 0; } /* * Prefer or force Point-To-Point instead Loop? */ switch (isp->isp_confopts & ISP_CFG_PORT_PREF) { case ISP_CFG_NPORT: icbp->icb_xfwoptions &= ~ICBXOPT_TOPO_MASK; icbp->icb_xfwoptions |= ICBXOPT_PTP_2_LOOP; break; case ISP_CFG_NPORT_ONLY: icbp->icb_xfwoptions &= ~ICBXOPT_TOPO_MASK; icbp->icb_xfwoptions |= ICBXOPT_PTP_ONLY; break; case ISP_CFG_LPORT_ONLY: icbp->icb_xfwoptions &= ~ICBXOPT_TOPO_MASK; icbp->icb_xfwoptions |= ICBXOPT_LOOP_ONLY; break; default: /* * Let NVRAM settings define it if they are sane */ switch (icbp->icb_xfwoptions & ICBXOPT_TOPO_MASK) { case ICBXOPT_PTP_2_LOOP: case ICBXOPT_PTP_ONLY: case ICBXOPT_LOOP_ONLY: case ICBXOPT_LOOP_2_PTP: break; default: icbp->icb_xfwoptions &= ~ICBXOPT_TOPO_MASK; icbp->icb_xfwoptions |= ICBXOPT_LOOP_2_PTP; } break; } if (IS_2200(isp)) { /* * We can't have Fast Posting any more- we now * have 32 bit handles. * * RIO seemed to have to much breakage. * * Just opt for safety. */ icbp->icb_xfwoptions &= ~ICBXOPT_RIO_16BIT; icbp->icb_fwoptions &= ~ICBOPT_FAST_POST; } else { /* * QLogic recommends that FAST Posting be turned * off for 23XX cards and instead allow the HBA * to write response queue entries and interrupt * after a delay (ZIO). */ icbp->icb_fwoptions &= ~ICBOPT_FAST_POST; if ((fcp->isp_xfwoptions & ICBXOPT_TIMER_MASK) == ICBXOPT_ZIO) { icbp->icb_xfwoptions |= ICBXOPT_ZIO; icbp->icb_idelaytimer = 10; } icbp->icb_zfwoptions = fcp->isp_zfwoptions; if (isp->isp_confopts & ISP_CFG_1GB) { icbp->icb_zfwoptions &= ~ICBZOPT_RATE_MASK; icbp->icb_zfwoptions |= ICBZOPT_RATE_1GB; } else if (isp->isp_confopts & ISP_CFG_2GB) { icbp->icb_zfwoptions &= ~ICBZOPT_RATE_MASK; icbp->icb_zfwoptions |= ICBZOPT_RATE_2GB; } else { switch (icbp->icb_zfwoptions & ICBZOPT_RATE_MASK) { case ICBZOPT_RATE_1GB: case ICBZOPT_RATE_2GB: case ICBZOPT_RATE_AUTO: break; default: icbp->icb_zfwoptions &= ~ICBZOPT_RATE_MASK; icbp->icb_zfwoptions |= ICBZOPT_RATE_AUTO; break; } } } } /* * For 22XX > 2.1.26 && 23XX, set some options. */ if (ISP_FW_NEWER_THAN(isp, 2, 26, 0)) { MBSINIT(&mbs, MBOX_SET_FIRMWARE_OPTIONS, MBLOGALL, 0); mbs.param[1] = IFCOPT1_DISF7SWTCH|IFCOPT1_LIPASYNC|IFCOPT1_LIPF8; mbs.param[2] = 0; mbs.param[3] = 0; if (ISP_FW_NEWER_THAN(isp, 3, 16, 0)) { mbs.param[1] |= IFCOPT1_EQFQASYNC|IFCOPT1_CTIO_RETRY; if (fcp->role & ISP_ROLE_TARGET) { if (ISP_FW_NEWER_THAN(isp, 3, 25, 0)) { mbs.param[1] |= IFCOPT1_ENAPURE; } mbs.param[3] = IFCOPT3_NOPRLI; } } isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { return; } } icbp->icb_logintime = ICB_LOGIN_TOV; #ifdef ISP_TARGET_MODE if (icbp->icb_fwoptions & ICBOPT_TGT_ENABLE) { icbp->icb_lunenables = 0xffff; icbp->icb_ccnt = 0xff; icbp->icb_icnt = 0xff; icbp->icb_lunetimeout = ICB_LUN_ENABLE_TOV; } #endif if (fcp->isp_wwnn && fcp->isp_wwpn) { icbp->icb_fwoptions |= ICBOPT_BOTH_WWNS; MAKE_NODE_NAME_FROM_WWN(icbp->icb_nodename, fcp->isp_wwnn); MAKE_NODE_NAME_FROM_WWN(icbp->icb_portname, fcp->isp_wwpn); isp_prt(isp, ISP_LOGDEBUG1, "Setting ICB Node 0x%08x%08x Port 0x%08x%08x", ((uint32_t) (fcp->isp_wwnn >> 32)), ((uint32_t) (fcp->isp_wwnn)), ((uint32_t) (fcp->isp_wwpn >> 32)), ((uint32_t) (fcp->isp_wwpn))); } else if (fcp->isp_wwpn) { icbp->icb_fwoptions &= ~ICBOPT_BOTH_WWNS; MAKE_NODE_NAME_FROM_WWN(icbp->icb_portname, fcp->isp_wwpn); isp_prt(isp, ISP_LOGDEBUG1, "Setting ICB Port 0x%08x%08x", ((uint32_t) (fcp->isp_wwpn >> 32)), ((uint32_t) (fcp->isp_wwpn))); } else { isp_prt(isp, ISP_LOGERR, "No valid WWNs to use"); return; } icbp->icb_rqstqlen = RQUEST_QUEUE_LEN(isp); if (icbp->icb_rqstqlen < 1) { isp_prt(isp, ISP_LOGERR, "bad request queue length"); } icbp->icb_rsltqlen = RESULT_QUEUE_LEN(isp); if (icbp->icb_rsltqlen < 1) { isp_prt(isp, ISP_LOGERR, "bad result queue length"); } icbp->icb_rqstaddr[RQRSP_ADDR0015] = DMA_WD0(isp->isp_rquest_dma); icbp->icb_rqstaddr[RQRSP_ADDR1631] = DMA_WD1(isp->isp_rquest_dma); icbp->icb_rqstaddr[RQRSP_ADDR3247] = DMA_WD2(isp->isp_rquest_dma); icbp->icb_rqstaddr[RQRSP_ADDR4863] = DMA_WD3(isp->isp_rquest_dma); icbp->icb_respaddr[RQRSP_ADDR0015] = DMA_WD0(isp->isp_result_dma); icbp->icb_respaddr[RQRSP_ADDR1631] = DMA_WD1(isp->isp_result_dma); icbp->icb_respaddr[RQRSP_ADDR3247] = DMA_WD2(isp->isp_result_dma); icbp->icb_respaddr[RQRSP_ADDR4863] = DMA_WD3(isp->isp_result_dma); if (FC_SCRATCH_ACQUIRE(isp, 0)) { isp_prt(isp, ISP_LOGERR, sacq); return; } isp_prt(isp, ISP_LOGDEBUG0, "isp_fibre_init: fwopt 0x%x xfwopt 0x%x zfwopt 0x%x", icbp->icb_fwoptions, icbp->icb_xfwoptions, icbp->icb_zfwoptions); if (isp->isp_dblev & ISP_LOGDEBUG1) isp_print_bytes(isp, "isp_fibre_init", sizeof (*icbp), icbp); isp_put_icb(isp, icbp, (isp_icb_t *)fcp->isp_scratch); /* * Init the firmware */ MBSINIT(&mbs, MBOX_INIT_FIRMWARE, MBLOGALL, 30000000); mbs.param[1] = 0; mbs.param[2] = DMA_WD1(fcp->isp_scdma); mbs.param[3] = DMA_WD0(fcp->isp_scdma); mbs.param[6] = DMA_WD3(fcp->isp_scdma); mbs.param[7] = DMA_WD2(fcp->isp_scdma); isp_prt(isp, ISP_LOGDEBUG0, "INIT F/W from %p (%08x%08x)", fcp->isp_scratch, (uint32_t) ((uint64_t)fcp->isp_scdma >> 32), (uint32_t) fcp->isp_scdma); MEMORYBARRIER(isp, SYNC_SFORDEV, 0, sizeof (*icbp), 0); isp_mboxcmd(isp, &mbs); FC_SCRATCH_RELEASE(isp, 0); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) return; isp->isp_reqidx = 0; isp->isp_reqodx = 0; isp->isp_residx = 0; isp->isp_resodx = 0; /* * Whatever happens, we're now committed to being here. */ isp->isp_state = ISP_RUNSTATE; } static void isp_fibre_init_2400(ispsoftc_t *isp) { fcparam *fcp; isp_icb_2400_t local, *icbp = &local; mbreg_t mbs; int chan; /* * Check to see whether all channels have *some* kind of role */ for (chan = 0; chan < isp->isp_nchan; chan++) { fcp = FCPARAM(isp, chan); if (fcp->role != ISP_ROLE_NONE) { break; } } if (chan == isp->isp_nchan) { isp_prt(isp, ISP_LOG_WARN1, "all %d channels with role 'none'", chan); return; } isp->isp_state = ISP_INITSTATE; /* * Start with channel 0. */ fcp = FCPARAM(isp, 0); /* * Turn on LIP F8 async event (1) */ MBSINIT(&mbs, MBOX_SET_FIRMWARE_OPTIONS, MBLOGALL, 0); mbs.param[1] = 1; isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { return; } ISP_MEMZERO(icbp, sizeof (*icbp)); icbp->icb_fwoptions1 = fcp->isp_fwoptions; icbp->icb_fwoptions2 = fcp->isp_xfwoptions; icbp->icb_fwoptions3 = fcp->isp_zfwoptions; if (isp->isp_nchan > 1 && ISP_CAP_VP0(isp)) { icbp->icb_fwoptions1 &= ~ICB2400_OPT1_INI_DISABLE; icbp->icb_fwoptions1 |= ICB2400_OPT1_TGT_ENABLE; } else { if (fcp->role & ISP_ROLE_TARGET) icbp->icb_fwoptions1 |= ICB2400_OPT1_TGT_ENABLE; else icbp->icb_fwoptions1 &= ~ICB2400_OPT1_TGT_ENABLE; if (fcp->role & ISP_ROLE_INITIATOR) icbp->icb_fwoptions1 &= ~ICB2400_OPT1_INI_DISABLE; else icbp->icb_fwoptions1 |= ICB2400_OPT1_INI_DISABLE; } icbp->icb_version = ICB_VERSION1; icbp->icb_maxfrmlen = DEFAULT_FRAMESIZE(isp); if (icbp->icb_maxfrmlen < ICB_MIN_FRMLEN || icbp->icb_maxfrmlen > ICB_MAX_FRMLEN) { isp_prt(isp, ISP_LOGERR, "bad frame length (%d) from NVRAM- using %d", DEFAULT_FRAMESIZE(isp), ICB_DFLT_FRMLEN); icbp->icb_maxfrmlen = ICB_DFLT_FRMLEN; } icbp->icb_execthrottle = DEFAULT_EXEC_THROTTLE(isp); if (icbp->icb_execthrottle < 1) { isp_prt(isp, ISP_LOGERR, "bad execution throttle of %d- using %d", DEFAULT_EXEC_THROTTLE(isp), ICB_DFLT_THROTTLE); icbp->icb_execthrottle = ICB_DFLT_THROTTLE; } /* * Set target exchange count. Take half if we are supporting both roles. */ if (icbp->icb_fwoptions1 & ICB2400_OPT1_TGT_ENABLE) { icbp->icb_xchgcnt = isp->isp_maxcmds; if ((icbp->icb_fwoptions1 & ICB2400_OPT1_INI_DISABLE) == 0) icbp->icb_xchgcnt >>= 1; } if (fcp->isp_loopid < LOCAL_LOOP_LIM) { icbp->icb_hardaddr = fcp->isp_loopid; if (isp->isp_confopts & ISP_CFG_OWNLOOPID) icbp->icb_fwoptions1 |= ICB2400_OPT1_HARD_ADDRESS; else icbp->icb_fwoptions1 |= ICB2400_OPT1_PREV_ADDRESS; } if (isp->isp_confopts & ISP_CFG_NOFCTAPE) { icbp->icb_fwoptions2 &= ~ICB2400_OPT2_FCTAPE; } if (isp->isp_confopts & ISP_CFG_FCTAPE) { icbp->icb_fwoptions2 |= ICB2400_OPT2_FCTAPE; } for (chan = 0; chan < isp->isp_nchan; chan++) { if (icbp->icb_fwoptions2 & ICB2400_OPT2_FCTAPE) FCPARAM(isp, chan)->fctape_enabled = 1; else FCPARAM(isp, chan)->fctape_enabled = 0; } switch (isp->isp_confopts & ISP_CFG_PORT_PREF) { case ISP_CFG_NPORT_ONLY: icbp->icb_fwoptions2 &= ~ICB2400_OPT2_TOPO_MASK; icbp->icb_fwoptions2 |= ICB2400_OPT2_PTP_ONLY; break; case ISP_CFG_LPORT_ONLY: icbp->icb_fwoptions2 &= ~ICB2400_OPT2_TOPO_MASK; icbp->icb_fwoptions2 |= ICB2400_OPT2_LOOP_ONLY; break; default: /* ISP_CFG_PTP_2_LOOP not available in 24XX/25XX */ icbp->icb_fwoptions2 &= ~ICB2400_OPT2_TOPO_MASK; icbp->icb_fwoptions2 |= ICB2400_OPT2_LOOP_2_PTP; break; } switch (icbp->icb_fwoptions2 & ICB2400_OPT2_TIMER_MASK) { case ICB2400_OPT2_ZIO: case ICB2400_OPT2_ZIO1: icbp->icb_idelaytimer = 0; break; case 0: break; default: isp_prt(isp, ISP_LOGWARN, "bad value %x in fwopt2 timer field", icbp->icb_fwoptions2 & ICB2400_OPT2_TIMER_MASK); icbp->icb_fwoptions2 &= ~ICB2400_OPT2_TIMER_MASK; break; } if (IS_26XX(isp)) { /* We don't support MSI-X yet, so set this unconditionally. */ icbp->icb_fwoptions2 |= ICB2400_OPT2_ENA_IHR; icbp->icb_fwoptions2 |= ICB2400_OPT2_ENA_IHA; } if ((icbp->icb_fwoptions3 & ICB2400_OPT3_RSPSZ_MASK) == 0) { icbp->icb_fwoptions3 |= ICB2400_OPT3_RSPSZ_24; } icbp->icb_fwoptions3 &= ~ICB2400_OPT3_RATE_AUTO; if (isp->isp_confopts & ISP_CFG_1GB) { icbp->icb_fwoptions3 |= ICB2400_OPT3_RATE_1GB; } else if (isp->isp_confopts & ISP_CFG_2GB) { icbp->icb_fwoptions3 |= ICB2400_OPT3_RATE_2GB; } else if (isp->isp_confopts & ISP_CFG_4GB) { icbp->icb_fwoptions3 |= ICB2400_OPT3_RATE_4GB; } else if (isp->isp_confopts & ISP_CFG_8GB) { icbp->icb_fwoptions3 |= ICB2400_OPT3_RATE_8GB; } else if (isp->isp_confopts & ISP_CFG_16GB) { icbp->icb_fwoptions3 |= ICB2400_OPT3_RATE_16GB; } else { icbp->icb_fwoptions3 |= ICB2400_OPT3_RATE_AUTO; } icbp->icb_logintime = ICB_LOGIN_TOV; if (fcp->isp_wwnn && fcp->isp_wwpn) { icbp->icb_fwoptions1 |= ICB2400_OPT1_BOTH_WWNS; MAKE_NODE_NAME_FROM_WWN(icbp->icb_portname, fcp->isp_wwpn); MAKE_NODE_NAME_FROM_WWN(icbp->icb_nodename, fcp->isp_wwnn); isp_prt(isp, ISP_LOGDEBUG1, "Setting ICB Node 0x%08x%08x Port 0x%08x%08x", ((uint32_t) (fcp->isp_wwnn >> 32)), ((uint32_t) (fcp->isp_wwnn)), ((uint32_t) (fcp->isp_wwpn >> 32)), ((uint32_t) (fcp->isp_wwpn))); } else if (fcp->isp_wwpn) { icbp->icb_fwoptions1 &= ~ICB2400_OPT1_BOTH_WWNS; MAKE_NODE_NAME_FROM_WWN(icbp->icb_portname, fcp->isp_wwpn); isp_prt(isp, ISP_LOGDEBUG1, "Setting ICB Node to be same as Port 0x%08x%08x", ((uint32_t) (fcp->isp_wwpn >> 32)), ((uint32_t) (fcp->isp_wwpn))); } else { isp_prt(isp, ISP_LOGERR, "No valid WWNs to use"); return; } icbp->icb_retry_count = fcp->isp_retry_count; icbp->icb_rqstqlen = RQUEST_QUEUE_LEN(isp); if (icbp->icb_rqstqlen < 8) { isp_prt(isp, ISP_LOGERR, "bad request queue length %d", icbp->icb_rqstqlen); return; } icbp->icb_rsltqlen = RESULT_QUEUE_LEN(isp); if (icbp->icb_rsltqlen < 8) { isp_prt(isp, ISP_LOGERR, "bad result queue length %d", icbp->icb_rsltqlen); return; } icbp->icb_rqstaddr[RQRSP_ADDR0015] = DMA_WD0(isp->isp_rquest_dma); icbp->icb_rqstaddr[RQRSP_ADDR1631] = DMA_WD1(isp->isp_rquest_dma); icbp->icb_rqstaddr[RQRSP_ADDR3247] = DMA_WD2(isp->isp_rquest_dma); icbp->icb_rqstaddr[RQRSP_ADDR4863] = DMA_WD3(isp->isp_rquest_dma); icbp->icb_respaddr[RQRSP_ADDR0015] = DMA_WD0(isp->isp_result_dma); icbp->icb_respaddr[RQRSP_ADDR1631] = DMA_WD1(isp->isp_result_dma); icbp->icb_respaddr[RQRSP_ADDR3247] = DMA_WD2(isp->isp_result_dma); icbp->icb_respaddr[RQRSP_ADDR4863] = DMA_WD3(isp->isp_result_dma); #ifdef ISP_TARGET_MODE /* unconditionally set up the ATIO queue if we support target mode */ icbp->icb_atioqlen = RESULT_QUEUE_LEN(isp); if (icbp->icb_atioqlen < 8) { isp_prt(isp, ISP_LOGERR, "bad ATIO queue length %d", icbp->icb_atioqlen); return; } icbp->icb_atioqaddr[RQRSP_ADDR0015] = DMA_WD0(isp->isp_atioq_dma); icbp->icb_atioqaddr[RQRSP_ADDR1631] = DMA_WD1(isp->isp_atioq_dma); icbp->icb_atioqaddr[RQRSP_ADDR3247] = DMA_WD2(isp->isp_atioq_dma); icbp->icb_atioqaddr[RQRSP_ADDR4863] = DMA_WD3(isp->isp_atioq_dma); isp_prt(isp, ISP_LOGDEBUG0, "isp_fibre_init_2400: atioq %04x%04x%04x%04x", DMA_WD3(isp->isp_atioq_dma), DMA_WD2(isp->isp_atioq_dma), DMA_WD1(isp->isp_atioq_dma), DMA_WD0(isp->isp_atioq_dma)); #endif isp_prt(isp, ISP_LOGDEBUG0, "isp_fibre_init_2400: fwopt1 0x%x fwopt2 0x%x fwopt3 0x%x", icbp->icb_fwoptions1, icbp->icb_fwoptions2, icbp->icb_fwoptions3); isp_prt(isp, ISP_LOGDEBUG0, "isp_fibre_init_2400: rqst %04x%04x%04x%04x rsp %04x%04x%04x%04x", DMA_WD3(isp->isp_rquest_dma), DMA_WD2(isp->isp_rquest_dma), DMA_WD1(isp->isp_rquest_dma), DMA_WD0(isp->isp_rquest_dma), DMA_WD3(isp->isp_result_dma), DMA_WD2(isp->isp_result_dma), DMA_WD1(isp->isp_result_dma), DMA_WD0(isp->isp_result_dma)); if (isp->isp_dblev & ISP_LOGDEBUG1) { isp_print_bytes(isp, "isp_fibre_init_2400", sizeof (*icbp), icbp); } if (FC_SCRATCH_ACQUIRE(isp, 0)) { isp_prt(isp, ISP_LOGERR, sacq); return; } ISP_MEMZERO(fcp->isp_scratch, ISP_FC_SCRLEN); isp_put_icb_2400(isp, icbp, fcp->isp_scratch); /* * Now fill in information about any additional channels */ if (isp->isp_nchan > 1) { isp_icb_2400_vpinfo_t vpinfo, *vdst; vp_port_info_t pi, *pdst; size_t amt = 0; uint8_t *off; vpinfo.vp_global_options = ICB2400_VPGOPT_GEN_RIDA; if (ISP_CAP_VP0(isp)) { vpinfo.vp_global_options |= ICB2400_VPGOPT_VP0_DECOUPLE; vpinfo.vp_count = isp->isp_nchan; chan = 0; } else { vpinfo.vp_count = isp->isp_nchan - 1; chan = 1; } off = fcp->isp_scratch; off += ICB2400_VPINFO_OFF; vdst = (isp_icb_2400_vpinfo_t *) off; isp_put_icb_2400_vpinfo(isp, &vpinfo, vdst); amt = ICB2400_VPINFO_OFF + sizeof (isp_icb_2400_vpinfo_t); for (; chan < isp->isp_nchan; chan++) { fcparam *fcp2; ISP_MEMZERO(&pi, sizeof (pi)); fcp2 = FCPARAM(isp, chan); if (fcp2->role != ISP_ROLE_NONE) { pi.vp_port_options = ICB2400_VPOPT_ENABLED | ICB2400_VPOPT_ENA_SNSLOGIN; if (fcp2->role & ISP_ROLE_INITIATOR) pi.vp_port_options |= ICB2400_VPOPT_INI_ENABLE; if ((fcp2->role & ISP_ROLE_TARGET) == 0) pi.vp_port_options |= ICB2400_VPOPT_TGT_DISABLE; } if (fcp2->isp_loopid < LOCAL_LOOP_LIM) { pi.vp_port_loopid = fcp2->isp_loopid; if (isp->isp_confopts & ISP_CFG_OWNLOOPID) pi.vp_port_options |= ICB2400_VPOPT_HARD_ADDRESS; else pi.vp_port_options |= ICB2400_VPOPT_PREV_ADDRESS; } MAKE_NODE_NAME_FROM_WWN(pi.vp_port_portname, fcp2->isp_wwpn); MAKE_NODE_NAME_FROM_WWN(pi.vp_port_nodename, fcp2->isp_wwnn); off = fcp->isp_scratch; if (ISP_CAP_VP0(isp)) off += ICB2400_VPINFO_PORT_OFF(chan); else off += ICB2400_VPINFO_PORT_OFF(chan - 1); pdst = (vp_port_info_t *) off; isp_put_vp_port_info(isp, &pi, pdst); amt += ICB2400_VPOPT_WRITE_SIZE; } if (isp->isp_dblev & ISP_LOGDEBUG1) { isp_print_bytes(isp, "isp_fibre_init_2400", amt - ICB2400_VPINFO_OFF, (char *)fcp->isp_scratch + ICB2400_VPINFO_OFF); } } /* * Init the firmware */ MBSINIT(&mbs, 0, MBLOGALL, 30000000); if (isp->isp_nchan > 1) { mbs.param[0] = MBOX_INIT_FIRMWARE_MULTI_ID; } else { mbs.param[0] = MBOX_INIT_FIRMWARE; } mbs.param[1] = 0; mbs.param[2] = DMA_WD1(fcp->isp_scdma); mbs.param[3] = DMA_WD0(fcp->isp_scdma); mbs.param[6] = DMA_WD3(fcp->isp_scdma); mbs.param[7] = DMA_WD2(fcp->isp_scdma); isp_prt(isp, ISP_LOGDEBUG0, "INIT F/W from %04x%04x%04x%04x", DMA_WD3(fcp->isp_scdma), DMA_WD2(fcp->isp_scdma), DMA_WD1(fcp->isp_scdma), DMA_WD0(fcp->isp_scdma)); MEMORYBARRIER(isp, SYNC_SFORDEV, 0, sizeof (*icbp), 0); isp_mboxcmd(isp, &mbs); FC_SCRATCH_RELEASE(isp, 0); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { return; } isp->isp_reqidx = 0; isp->isp_reqodx = 0; isp->isp_residx = 0; isp->isp_resodx = 0; isp->isp_atioodx = 0; /* * Whatever happens, we're now committed to being here. */ isp->isp_state = ISP_RUNSTATE; } static void isp_clear_portdb(ispsoftc_t *isp, int chan) { fcparam *fcp = FCPARAM(isp, chan); fcportdb_t *lp; int i; for (i = 0; i < MAX_FC_TARG; i++) { lp = &fcp->portdb[i]; switch (lp->state) { case FC_PORTDB_STATE_DEAD: case FC_PORTDB_STATE_CHANGED: case FC_PORTDB_STATE_VALID: lp->state = FC_PORTDB_STATE_NIL; isp_async(isp, ISPASYNC_DEV_GONE, chan, lp); break; case FC_PORTDB_STATE_NIL: case FC_PORTDB_STATE_NEW: lp->state = FC_PORTDB_STATE_NIL; break; case FC_PORTDB_STATE_ZOMBIE: break; default: panic("Don't know how to clear state %d\n", lp->state); } } } static void isp_mark_portdb(ispsoftc_t *isp, int chan) { fcparam *fcp = FCPARAM(isp, chan); fcportdb_t *lp; int i; for (i = 0; i < MAX_FC_TARG; i++) { lp = &fcp->portdb[i]; if (lp->state == FC_PORTDB_STATE_NIL) continue; if (lp->portid >= DOMAIN_CONTROLLER_BASE && lp->portid <= DOMAIN_CONTROLLER_END) continue; fcp->portdb[i].probational = 1; } } /* * Perform an IOCB PLOGI or LOGO via EXECUTE IOCB A64 for 24XX cards * or via FABRIC LOGIN/FABRIC LOGOUT for other cards. */ static int isp_plogx(ispsoftc_t *isp, int chan, uint16_t handle, uint32_t portid, int flags, int gs) { mbreg_t mbs; uint8_t q[QENTRY_LEN]; isp_plogx_t *plp; fcparam *fcp; uint8_t *scp; uint32_t sst, parm1; int rval, lev; const char *msg; char buf[64]; isp_prt(isp, ISP_LOG_SANCFG, "Chan %d PLOGX %s PortID 0x%06x nphdl 0x%x", chan, (flags & PLOGX_FLG_CMD_MASK) == PLOGX_FLG_CMD_PLOGI ? "Login":"Logout", portid, handle); if (!IS_24XX(isp)) { int action = flags & PLOGX_FLG_CMD_MASK; if (action == PLOGX_FLG_CMD_PLOGI) { return (isp_port_login(isp, handle, portid)); } else if (action == PLOGX_FLG_CMD_LOGO) { return (isp_port_logout(isp, handle, portid)); } else { return (MBOX_INVALID_COMMAND); } } ISP_MEMZERO(q, QENTRY_LEN); plp = (isp_plogx_t *) q; plp->plogx_header.rqs_entry_count = 1; plp->plogx_header.rqs_entry_type = RQSTYPE_LOGIN; plp->plogx_handle = 0xffffffff; plp->plogx_nphdl = handle; plp->plogx_vphdl = chan; plp->plogx_portlo = portid; plp->plogx_rspsz_porthi = (portid >> 16) & 0xff; plp->plogx_flags = flags; if (isp->isp_dblev & ISP_LOGDEBUG1) { isp_print_bytes(isp, "IOCB LOGX", QENTRY_LEN, plp); } if (gs == 0) { if (FC_SCRATCH_ACQUIRE(isp, chan)) { isp_prt(isp, ISP_LOGERR, sacq); return (-1); } } fcp = FCPARAM(isp, chan); scp = fcp->isp_scratch; isp_put_plogx(isp, plp, (isp_plogx_t *) scp); MBSINIT(&mbs, MBOX_EXEC_COMMAND_IOCB_A64, MBLOGALL, 500000); mbs.param[1] = QENTRY_LEN; mbs.param[2] = DMA_WD1(fcp->isp_scdma); mbs.param[3] = DMA_WD0(fcp->isp_scdma); mbs.param[6] = DMA_WD3(fcp->isp_scdma); mbs.param[7] = DMA_WD2(fcp->isp_scdma); MEMORYBARRIER(isp, SYNC_SFORDEV, 0, QENTRY_LEN, chan); isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { rval = mbs.param[0]; goto out; } MEMORYBARRIER(isp, SYNC_SFORCPU, QENTRY_LEN, QENTRY_LEN, chan); scp += QENTRY_LEN; isp_get_plogx(isp, (isp_plogx_t *) scp, plp); if (isp->isp_dblev & ISP_LOGDEBUG1) { isp_print_bytes(isp, "IOCB LOGX response", QENTRY_LEN, plp); } if (plp->plogx_status == PLOGX_STATUS_OK) { rval = 0; goto out; } else if (plp->plogx_status != PLOGX_STATUS_IOCBERR) { isp_prt(isp, ISP_LOGWARN, "status 0x%x on port login IOCB channel %d", plp->plogx_status, chan); rval = -1; goto out; } sst = plp->plogx_ioparm[0].lo16 | (plp->plogx_ioparm[0].hi16 << 16); parm1 = plp->plogx_ioparm[1].lo16 | (plp->plogx_ioparm[1].hi16 << 16); rval = -1; lev = ISP_LOGERR; msg = NULL; switch (sst) { case PLOGX_IOCBERR_NOLINK: msg = "no link"; break; case PLOGX_IOCBERR_NOIOCB: msg = "no IOCB buffer"; break; case PLOGX_IOCBERR_NOXGHG: msg = "no Exchange Control Block"; break; case PLOGX_IOCBERR_FAILED: ISP_SNPRINTF(buf, sizeof (buf), "reason 0x%x (last LOGIN state 0x%x)", parm1 & 0xff, (parm1 >> 8) & 0xff); msg = buf; break; case PLOGX_IOCBERR_NOFABRIC: msg = "no fabric"; break; case PLOGX_IOCBERR_NOTREADY: msg = "firmware not ready"; break; case PLOGX_IOCBERR_NOLOGIN: ISP_SNPRINTF(buf, sizeof (buf), "not logged in (last state 0x%x)", parm1); msg = buf; rval = MBOX_NOT_LOGGED_IN; break; case PLOGX_IOCBERR_REJECT: ISP_SNPRINTF(buf, sizeof (buf), "LS_RJT = 0x%x", parm1); msg = buf; break; case PLOGX_IOCBERR_NOPCB: msg = "no PCB allocated"; break; case PLOGX_IOCBERR_EINVAL: ISP_SNPRINTF(buf, sizeof (buf), "invalid parameter at offset 0x%x", parm1); msg = buf; break; case PLOGX_IOCBERR_PORTUSED: lev = ISP_LOG_SANCFG|ISP_LOG_WARN1; ISP_SNPRINTF(buf, sizeof (buf), "already logged in with N-Port handle 0x%x", parm1); msg = buf; rval = MBOX_PORT_ID_USED | (parm1 << 16); break; case PLOGX_IOCBERR_HNDLUSED: lev = ISP_LOG_SANCFG|ISP_LOG_WARN1; ISP_SNPRINTF(buf, sizeof (buf), "handle already used for PortID 0x%06x", parm1); msg = buf; rval = MBOX_LOOP_ID_USED; break; case PLOGX_IOCBERR_NOHANDLE: msg = "no handle allocated"; break; case PLOGX_IOCBERR_NOFLOGI: msg = "no FLOGI_ACC"; break; default: ISP_SNPRINTF(buf, sizeof (buf), "status %x from %x", plp->plogx_status, flags); msg = buf; break; } if (msg) { isp_prt(isp, ISP_LOGERR, "Chan %d PLOGX PortID 0x%06x to N-Port handle 0x%x: %s", chan, portid, handle, msg); } out: if (gs == 0) { FC_SCRATCH_RELEASE(isp, chan); } return (rval); } static int isp_port_login(ispsoftc_t *isp, uint16_t handle, uint32_t portid) { mbreg_t mbs; MBSINIT(&mbs, MBOX_FABRIC_LOGIN, MBLOGNONE, 500000); if (ISP_CAP_2KLOGIN(isp)) { mbs.param[1] = handle; mbs.ibits = (1 << 10); } else { mbs.param[1] = handle << 8; } mbs.param[2] = portid >> 16; mbs.param[3] = portid; mbs.logval = MBLOGNONE; mbs.timeout = 500000; isp_mboxcmd(isp, &mbs); switch (mbs.param[0]) { case MBOX_PORT_ID_USED: isp_prt(isp, ISP_LOG_SANCFG|ISP_LOG_WARN1, "isp_port_login: portid 0x%06x already logged in as 0x%x", portid, mbs.param[1]); return (MBOX_PORT_ID_USED | (mbs.param[1] << 16)); case MBOX_LOOP_ID_USED: isp_prt(isp, ISP_LOG_SANCFG|ISP_LOG_WARN1, "isp_port_login: handle 0x%x in use for port id 0x%02xXXXX", handle, mbs.param[1] & 0xff); return (MBOX_LOOP_ID_USED); case MBOX_COMMAND_COMPLETE: return (0); case MBOX_COMMAND_ERROR: isp_prt(isp, ISP_LOG_SANCFG|ISP_LOG_WARN1, "isp_port_login: error 0x%x in PLOGI to port 0x%06x", mbs.param[1], portid); return (MBOX_COMMAND_ERROR); case MBOX_ALL_IDS_USED: isp_prt(isp, ISP_LOG_SANCFG|ISP_LOG_WARN1, "isp_port_login: all IDs used for fabric login"); return (MBOX_ALL_IDS_USED); default: isp_prt(isp, ISP_LOG_SANCFG, "isp_port_login: error 0x%x on port login of 0x%06x@0x%0x", mbs.param[0], portid, handle); return (mbs.param[0]); } } /* * Pre-24XX fabric port logout * * Note that portid is not used */ static int isp_port_logout(ispsoftc_t *isp, uint16_t handle, uint32_t portid) { mbreg_t mbs; MBSINIT(&mbs, MBOX_FABRIC_LOGOUT, MBLOGNONE, 500000); if (ISP_CAP_2KLOGIN(isp)) { mbs.param[1] = handle; mbs.ibits = (1 << 10); } else { mbs.param[1] = handle << 8; } isp_mboxcmd(isp, &mbs); return (mbs.param[0] == MBOX_COMMAND_COMPLETE? 0 : mbs.param[0]); } static int isp_getpdb(ispsoftc_t *isp, int chan, uint16_t id, isp_pdb_t *pdb, int dolock) { fcparam *fcp = FCPARAM(isp, chan); mbreg_t mbs; union { isp_pdb_21xx_t fred; isp_pdb_24xx_t bill; } un; MBSINIT(&mbs, MBOX_GET_PORT_DB, MBLOGALL & ~MBLOGMASK(MBOX_COMMAND_PARAM_ERROR), 250000); if (IS_24XX(isp)) { mbs.ibits = (1 << 9)|(1 << 10); mbs.param[1] = id; mbs.param[9] = chan; } else if (ISP_CAP_2KLOGIN(isp)) { mbs.param[1] = id; } else { mbs.param[1] = id << 8; } mbs.param[2] = DMA_WD1(fcp->isp_scdma); mbs.param[3] = DMA_WD0(fcp->isp_scdma); mbs.param[6] = DMA_WD3(fcp->isp_scdma); mbs.param[7] = DMA_WD2(fcp->isp_scdma); if (dolock) { if (FC_SCRATCH_ACQUIRE(isp, chan)) { isp_prt(isp, ISP_LOGERR, sacq); return (-1); } } MEMORYBARRIER(isp, SYNC_SFORDEV, 0, sizeof (un), chan); isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { if (dolock) { FC_SCRATCH_RELEASE(isp, chan); } return (mbs.param[0] | (mbs.param[1] << 16)); } if (IS_24XX(isp)) { isp_get_pdb_24xx(isp, fcp->isp_scratch, &un.bill); pdb->handle = un.bill.pdb_handle; pdb->prli_word3 = un.bill.pdb_prli_svc3; pdb->portid = BITS2WORD_24XX(un.bill.pdb_portid_bits); ISP_MEMCPY(pdb->portname, un.bill.pdb_portname, 8); ISP_MEMCPY(pdb->nodename, un.bill.pdb_nodename, 8); isp_prt(isp, ISP_LOGDEBUG1, "Chan %d handle 0x%x Port 0x%06x flags 0x%x curstate %x", chan, id, pdb->portid, un.bill.pdb_flags, un.bill.pdb_curstate); if (un.bill.pdb_curstate < PDB2400_STATE_PLOGI_DONE || un.bill.pdb_curstate > PDB2400_STATE_LOGGED_IN) { mbs.param[0] = MBOX_NOT_LOGGED_IN; if (dolock) { FC_SCRATCH_RELEASE(isp, chan); } return (mbs.param[0]); } } else { isp_get_pdb_21xx(isp, fcp->isp_scratch, &un.fred); pdb->handle = un.fred.pdb_loopid; pdb->prli_word3 = un.fred.pdb_prli_svc3; pdb->portid = BITS2WORD(un.fred.pdb_portid_bits); ISP_MEMCPY(pdb->portname, un.fred.pdb_portname, 8); ISP_MEMCPY(pdb->nodename, un.fred.pdb_nodename, 8); isp_prt(isp, ISP_LOGDEBUG1, "Chan %d handle 0x%x Port 0x%06x", chan, id, pdb->portid); } if (dolock) { FC_SCRATCH_RELEASE(isp, chan); } return (0); } static int isp_gethandles(ispsoftc_t *isp, int chan, uint16_t *handles, int *num, int dolock, int loop) { fcparam *fcp = FCPARAM(isp, chan); mbreg_t mbs; isp_pnhle_21xx_t el1, *elp1; isp_pnhle_23xx_t el3, *elp3; isp_pnhle_24xx_t el4, *elp4; int i, j; uint32_t p; uint16_t h; MBSINIT(&mbs, MBOX_GET_ID_LIST, MBLOGALL, 250000); if (IS_24XX(isp)) { mbs.param[2] = DMA_WD1(fcp->isp_scdma); mbs.param[3] = DMA_WD0(fcp->isp_scdma); mbs.param[6] = DMA_WD3(fcp->isp_scdma); mbs.param[7] = DMA_WD2(fcp->isp_scdma); mbs.param[8] = ISP_FC_SCRLEN; mbs.param[9] = chan; } else { mbs.ibits = (1 << 1)|(1 << 2)|(1 << 3)|(1 << 6); mbs.param[1] = DMA_WD1(fcp->isp_scdma); mbs.param[2] = DMA_WD0(fcp->isp_scdma); mbs.param[3] = DMA_WD3(fcp->isp_scdma); mbs.param[6] = DMA_WD2(fcp->isp_scdma); } if (dolock) { if (FC_SCRATCH_ACQUIRE(isp, chan)) { isp_prt(isp, ISP_LOGERR, sacq); return (-1); } } MEMORYBARRIER(isp, SYNC_SFORDEV, 0, ISP_FC_SCRLEN, chan); isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { if (dolock) { FC_SCRATCH_RELEASE(isp, chan); } return (mbs.param[0] | (mbs.param[1] << 16)); } elp1 = fcp->isp_scratch; elp3 = fcp->isp_scratch; elp4 = fcp->isp_scratch; for (i = 0, j = 0; i < mbs.param[1] && j < *num; i++) { if (IS_24XX(isp)) { isp_get_pnhle_24xx(isp, &elp4[i], &el4); p = el4.pnhle_port_id_lo | (el4.pnhle_port_id_hi << 16); h = el4.pnhle_handle; } else if (IS_23XX(isp)) { isp_get_pnhle_23xx(isp, &elp3[i], &el3); p = el3.pnhle_port_id_lo | (el3.pnhle_port_id_hi << 16); h = el3.pnhle_handle; } else { /* 21xx */ isp_get_pnhle_21xx(isp, &elp1[i], &el1); p = el1.pnhle_port_id_lo | ((el1.pnhle_port_id_hi_handle & 0xff) << 16); h = el1.pnhle_port_id_hi_handle >> 8; } if (loop && (p >> 8) != (fcp->isp_portid >> 8)) continue; handles[j++] = h; } *num = j; if (dolock) FC_SCRATCH_RELEASE(isp, chan); return (0); } static void isp_dump_chip_portdb(ispsoftc_t *isp, int chan, int dolock) { isp_pdb_t pdb; uint16_t lim, nphdl; isp_prt(isp, ISP_LOG_SANCFG|ISP_LOGINFO, "Chan %d chip port dump", chan); if (ISP_CAP_2KLOGIN(isp)) { lim = NPH_MAX_2K; } else { lim = NPH_MAX; } for (nphdl = 0; nphdl != lim; nphdl++) { if (isp_getpdb(isp, chan, nphdl, &pdb, dolock)) { continue; } isp_prt(isp, ISP_LOG_SANCFG|ISP_LOGINFO, "Chan %d Handle 0x%04x " "PortID 0x%06x WWPN 0x%02x%02x%02x%02x%02x%02x%02x%02x", chan, nphdl, pdb.portid, pdb.portname[0], pdb.portname[1], pdb.portname[2], pdb.portname[3], pdb.portname[4], pdb.portname[5], pdb.portname[6], pdb.portname[7]); } } static uint64_t isp_get_wwn(ispsoftc_t *isp, int chan, int nphdl, int nodename) { uint64_t wwn = INI_NONE; mbreg_t mbs; MBSINIT(&mbs, MBOX_GET_PORT_NAME, MBLOGALL & ~MBLOGMASK(MBOX_COMMAND_PARAM_ERROR), 500000); if (ISP_CAP_2KLOGIN(isp)) { mbs.param[1] = nphdl; if (nodename) { mbs.param[10] = 1; } mbs.param[9] = chan; } else { mbs.ibitm = 3; mbs.param[1] = nphdl << 8; if (nodename) { mbs.param[1] |= 1; } } isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { return (wwn); } if (IS_24XX(isp)) { wwn = (((uint64_t)(mbs.param[2] >> 8)) << 56) | (((uint64_t)(mbs.param[2] & 0xff)) << 48) | (((uint64_t)(mbs.param[3] >> 8)) << 40) | (((uint64_t)(mbs.param[3] & 0xff)) << 32) | (((uint64_t)(mbs.param[6] >> 8)) << 24) | (((uint64_t)(mbs.param[6] & 0xff)) << 16) | (((uint64_t)(mbs.param[7] >> 8)) << 8) | (((uint64_t)(mbs.param[7] & 0xff))); } else { wwn = (((uint64_t)(mbs.param[2] & 0xff)) << 56) | (((uint64_t)(mbs.param[2] >> 8)) << 48) | (((uint64_t)(mbs.param[3] & 0xff)) << 40) | (((uint64_t)(mbs.param[3] >> 8)) << 32) | (((uint64_t)(mbs.param[6] & 0xff)) << 24) | (((uint64_t)(mbs.param[6] >> 8)) << 16) | (((uint64_t)(mbs.param[7] & 0xff)) << 8) | (((uint64_t)(mbs.param[7] >> 8))); } return (wwn); } /* * Make sure we have good FC link. */ static int isp_fclink_test(ispsoftc_t *isp, int chan, int usdelay) { mbreg_t mbs; int i, r; uint16_t nphdl; fcparam *fcp; isp_pdb_t pdb; NANOTIME_T hra, hrb; fcp = FCPARAM(isp, chan); if (fcp->isp_loopstate < LOOP_HAVE_LINK) return (-1); if (fcp->isp_loopstate >= LOOP_LTEST_DONE) return (0); isp_prt(isp, ISP_LOG_SANCFG, "Chan %d FC link test", chan); fcp->isp_loopstate = LOOP_TESTING_LINK; /* * Wait up to N microseconds for F/W to go to a ready state. */ GET_NANOTIME(&hra); while (1) { isp_change_fw_state(isp, chan, isp_fw_state(isp, chan)); if (fcp->isp_fwstate == FW_READY) { break; } if (fcp->isp_loopstate < LOOP_TESTING_LINK) goto abort; GET_NANOTIME(&hrb); if ((NANOTIME_SUB(&hrb, &hra) / 1000 + 1000 >= usdelay)) break; ISP_SLEEP(isp, 1000); } if (fcp->isp_fwstate != FW_READY) { isp_prt(isp, ISP_LOG_SANCFG, "Chan %d Firmware is not ready (%s)", chan, isp_fc_fw_statename(fcp->isp_fwstate)); return (-1); } /* * Get our Loop ID and Port ID. */ MBSINIT(&mbs, MBOX_GET_LOOP_ID, MBLOGALL, 0); mbs.param[9] = chan; isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { return (-1); } if (IS_2100(isp)) { /* * Don't bother with fabric if we are using really old * 2100 firmware. It's just not worth it. */ if (ISP_FW_NEWER_THAN(isp, 1, 15, 37)) fcp->isp_topo = TOPO_FL_PORT; else fcp->isp_topo = TOPO_NL_PORT; } else { int topo = (int) mbs.param[6]; if (topo < TOPO_NL_PORT || topo > TOPO_PTP_STUB) { topo = TOPO_PTP_STUB; } fcp->isp_topo = topo; } fcp->isp_portid = mbs.param[2] | (mbs.param[3] << 16); if (!TOPO_IS_FABRIC(fcp->isp_topo)) { fcp->isp_loopid = mbs.param[1] & 0xff; } else if (fcp->isp_topo != TOPO_F_PORT) { uint8_t alpa = fcp->isp_portid; for (i = 0; alpa_map[i]; i++) { if (alpa_map[i] == alpa) break; } if (alpa_map[i]) fcp->isp_loopid = i; } if (fcp->isp_topo == TOPO_F_PORT || fcp->isp_topo == TOPO_FL_PORT) { nphdl = IS_24XX(isp) ? NPH_FL_ID : FL_ID; r = isp_getpdb(isp, chan, nphdl, &pdb, 1); if (r != 0 || pdb.portid == 0) { if (IS_2100(isp)) { fcp->isp_topo = TOPO_NL_PORT; } else { isp_prt(isp, ISP_LOGWARN, "fabric topology, but cannot get info about fabric controller (0x%x)", r); fcp->isp_topo = TOPO_PTP_STUB; } goto not_on_fabric; } if (IS_24XX(isp)) { fcp->isp_fabric_params = mbs.param[7]; fcp->isp_sns_hdl = NPH_SNS_ID; r = isp_register_fc4_type_24xx(isp, chan); if (r == 0) isp_register_fc4_features_24xx(isp, chan); } else { fcp->isp_sns_hdl = SNS_ID; r = isp_register_fc4_type(isp, chan); if (r == 0 && fcp->role == ISP_ROLE_TARGET) isp_send_change_request(isp, chan); } if (r) { isp_prt(isp, ISP_LOGWARN|ISP_LOG_SANCFG, "%s: register fc4 type failed", __func__); return (-1); } } not_on_fabric: /* Get link speed. */ fcp->isp_gbspeed = 1; if (IS_23XX(isp) || IS_24XX(isp)) { MBSINIT(&mbs, MBOX_GET_SET_DATA_RATE, MBLOGALL, 3000000); mbs.param[1] = MBGSD_GET_RATE; /* mbs.param[2] undefined if we're just getting rate */ isp_mboxcmd(isp, &mbs); if (mbs.param[0] == MBOX_COMMAND_COMPLETE) { if (mbs.param[1] == MBGSD_10GB) fcp->isp_gbspeed = 10; else if (mbs.param[1] == MBGSD_16GB) fcp->isp_gbspeed = 16; else if (mbs.param[1] == MBGSD_8GB) fcp->isp_gbspeed = 8; else if (mbs.param[1] == MBGSD_4GB) fcp->isp_gbspeed = 4; else if (mbs.param[1] == MBGSD_2GB) fcp->isp_gbspeed = 2; else if (mbs.param[1] == MBGSD_1GB) fcp->isp_gbspeed = 1; } } if (fcp->isp_loopstate < LOOP_TESTING_LINK) { abort: isp_prt(isp, ISP_LOG_SANCFG, "Chan %d FC link test aborted", chan); return (1); } fcp->isp_loopstate = LOOP_LTEST_DONE; isp_prt(isp, ISP_LOG_SANCFG|ISP_LOGCONFIG, "Chan %d WWPN %016jx WWNN %016jx", chan, (uintmax_t)fcp->isp_wwpn, (uintmax_t)fcp->isp_wwnn); isp_prt(isp, ISP_LOG_SANCFG|ISP_LOGCONFIG, "Chan %d %dGb %s PortID 0x%06x LoopID 0x%02x", chan, fcp->isp_gbspeed, isp_fc_toponame(fcp), fcp->isp_portid, fcp->isp_loopid); isp_prt(isp, ISP_LOG_SANCFG, "Chan %d FC link test done", chan); return (0); } /* * Complete the synchronization of our Port Database. * * At this point, we've scanned the local loop (if any) and the fabric * and performed fabric logins on all new devices. * * Our task here is to go through our port database removing any entities * that are still marked probational (issuing PLOGO for ones which we had * PLOGI'd into) or are dead, and notifying upper layers about new/changed * devices. */ static int isp_pdb_sync(ispsoftc_t *isp, int chan) { fcparam *fcp = FCPARAM(isp, chan); fcportdb_t *lp; uint16_t dbidx; if (fcp->isp_loopstate < LOOP_FSCAN_DONE) return (-1); if (fcp->isp_loopstate >= LOOP_READY) return (0); isp_prt(isp, ISP_LOG_SANCFG, "Chan %d FC PDB sync", chan); fcp->isp_loopstate = LOOP_SYNCING_PDB; for (dbidx = 0; dbidx < MAX_FC_TARG; dbidx++) { lp = &fcp->portdb[dbidx]; if (lp->state == FC_PORTDB_STATE_NIL) continue; if (lp->probational && lp->state != FC_PORTDB_STATE_ZOMBIE) lp->state = FC_PORTDB_STATE_DEAD; switch (lp->state) { case FC_PORTDB_STATE_DEAD: lp->state = FC_PORTDB_STATE_NIL; isp_async(isp, ISPASYNC_DEV_GONE, chan, lp); if (lp->autologin == 0) { (void) isp_plogx(isp, chan, lp->handle, lp->portid, PLOGX_FLG_CMD_LOGO | PLOGX_FLG_IMPLICIT | PLOGX_FLG_FREE_NPHDL, 0); } /* * Note that we might come out of this with our state * set to FC_PORTDB_STATE_ZOMBIE. */ break; case FC_PORTDB_STATE_NEW: lp->state = FC_PORTDB_STATE_VALID; isp_async(isp, ISPASYNC_DEV_ARRIVED, chan, lp); break; case FC_PORTDB_STATE_CHANGED: lp->state = FC_PORTDB_STATE_VALID; isp_async(isp, ISPASYNC_DEV_CHANGED, chan, lp); lp->portid = lp->new_portid; lp->prli_word3 = lp->new_prli_word3; break; case FC_PORTDB_STATE_VALID: isp_async(isp, ISPASYNC_DEV_STAYED, chan, lp); break; case FC_PORTDB_STATE_ZOMBIE: break; default: isp_prt(isp, ISP_LOGWARN, "isp_pdb_sync: state %d for idx %d", lp->state, dbidx); isp_dump_portdb(isp, chan); } } if (fcp->isp_loopstate < LOOP_SYNCING_PDB) { isp_prt(isp, ISP_LOG_SANCFG, "Chan %d FC PDB sync aborted", chan); return (1); } fcp->isp_loopstate = LOOP_READY; isp_prt(isp, ISP_LOG_SANCFG, "Chan %d FC PDB sync done", chan); return (0); } static void isp_pdb_add_update(ispsoftc_t *isp, int chan, isp_pdb_t *pdb) { fcportdb_t *lp; uint64_t wwnn, wwpn; MAKE_WWN_FROM_NODE_NAME(wwnn, pdb->nodename); MAKE_WWN_FROM_NODE_NAME(wwpn, pdb->portname); /* Search port database for the same WWPN. */ if (isp_find_pdb_by_wwpn(isp, chan, wwpn, &lp)) { if (!lp->probational) { isp_prt(isp, ISP_LOGERR, "Chan %d Port 0x%06x@0x%04x [%d] is not probational (0x%x)", chan, lp->portid, lp->handle, FC_PORTDB_TGT(isp, chan, lp), lp->state); isp_dump_portdb(isp, chan); return; } lp->probational = 0; lp->node_wwn = wwnn; /* Old device, nothing new. */ if (lp->portid == pdb->portid && lp->handle == pdb->handle && lp->prli_word3 == pdb->prli_word3) { if (lp->state != FC_PORTDB_STATE_NEW) lp->state = FC_PORTDB_STATE_VALID; isp_prt(isp, ISP_LOG_SANCFG, "Chan %d Port 0x%06x@0x%04x is valid", chan, pdb->portid, pdb->handle); return; } /* Something has changed. */ lp->state = FC_PORTDB_STATE_CHANGED; lp->handle = pdb->handle; lp->new_portid = pdb->portid; lp->new_prli_word3 = pdb->prli_word3; isp_prt(isp, ISP_LOG_SANCFG, "Chan %d Port 0x%06x@0x%04x is changed", chan, pdb->portid, pdb->handle); return; } /* It seems like a new port. Find an empty slot for it. */ if (!isp_find_pdb_empty(isp, chan, &lp)) { isp_prt(isp, ISP_LOGERR, "Chan %d out of portdb entries", chan); return; } ISP_MEMZERO(lp, sizeof (fcportdb_t)); lp->autologin = 1; lp->probational = 0; lp->state = FC_PORTDB_STATE_NEW; lp->portid = lp->new_portid = pdb->portid; lp->prli_word3 = lp->new_prli_word3 = pdb->prli_word3; lp->handle = pdb->handle; lp->port_wwn = wwpn; lp->node_wwn = wwnn; isp_prt(isp, ISP_LOG_SANCFG, "Chan %d Port 0x%06x@0x%04x is new", chan, pdb->portid, pdb->handle); } /* * Fix port IDs for logged-in initiators on pre-2400 chips. * For those chips we are not receiving login events, adding initiators * based on ATIO requests, but there is no port ID in that structure. */ static void isp_fix_portids(ispsoftc_t *isp, int chan) { fcparam *fcp = FCPARAM(isp, chan); isp_pdb_t pdb; uint64_t wwpn; int i, r; for (i = 0; i < MAX_FC_TARG; i++) { fcportdb_t *lp = &fcp->portdb[i]; if (lp->state == FC_PORTDB_STATE_NIL || lp->state == FC_PORTDB_STATE_ZOMBIE) continue; if (VALID_PORT(lp->portid)) continue; r = isp_getpdb(isp, chan, lp->handle, &pdb, 1); if (fcp->isp_loopstate < LOOP_SCANNING_LOOP) return; if (r != 0) { isp_prt(isp, ISP_LOGDEBUG1, "Chan %d FC Scan Loop handle %d returned %x", chan, lp->handle, r); continue; } MAKE_WWN_FROM_NODE_NAME(wwpn, pdb.portname); if (lp->port_wwn != wwpn) continue; lp->portid = lp->new_portid = pdb.portid; isp_prt(isp, ISP_LOG_SANCFG, "Chan %d Port 0x%06x@0x%04x is fixed", chan, pdb.portid, pdb.handle); } } /* * Scan local loop for devices. */ static int isp_scan_loop(ispsoftc_t *isp, int chan) { fcparam *fcp = FCPARAM(isp, chan); int idx, lim, r; isp_pdb_t pdb; uint16_t handles[LOCAL_LOOP_LIM]; uint16_t handle; if (fcp->isp_loopstate < LOOP_LTEST_DONE) return (-1); if (fcp->isp_loopstate >= LOOP_LSCAN_DONE) return (0); isp_prt(isp, ISP_LOG_SANCFG, "Chan %d FC loop scan", chan); fcp->isp_loopstate = LOOP_SCANNING_LOOP; if (TOPO_IS_FABRIC(fcp->isp_topo)) { if (!IS_24XX(isp)) { isp_fix_portids(isp, chan); if (fcp->isp_loopstate < LOOP_SCANNING_LOOP) goto abort; } isp_prt(isp, ISP_LOG_SANCFG, "Chan %d FC loop scan done (no loop)", chan); fcp->isp_loopstate = LOOP_LSCAN_DONE; return (0); } lim = LOCAL_LOOP_LIM; r = isp_gethandles(isp, chan, handles, &lim, 1, 1); if (r != 0) { isp_prt(isp, ISP_LOG_SANCFG, "Chan %d Getting list of handles failed with %x", chan, r); isp_prt(isp, ISP_LOG_SANCFG, "Chan %d FC loop scan done (bad)", chan); return (-1); } isp_prt(isp, ISP_LOG_SANCFG, "Chan %d Got %d handles", chan, lim); /* * Run through the list and get the port database info for each one. */ isp_mark_portdb(isp, chan); for (idx = 0; idx < lim; idx++) { handle = handles[idx]; /* * Don't scan "special" ids. */ if (ISP_CAP_2KLOGIN(isp)) { if (handle >= NPH_RESERVED) continue; } else { if (handle >= FL_ID && handle <= SNS_ID) continue; } /* * In older cards with older f/w GET_PORT_DATABASE has been * known to hang. This trick gets around that problem. */ if (IS_2100(isp) || IS_2200(isp)) { uint64_t node_wwn = isp_get_wwn(isp, chan, handle, 1); if (fcp->isp_loopstate < LOOP_SCANNING_LOOP) { abort: isp_prt(isp, ISP_LOG_SANCFG, "Chan %d FC loop scan aborted", chan); return (1); } if (node_wwn == INI_NONE) { continue; } } /* * Get the port database entity for this index. */ r = isp_getpdb(isp, chan, handle, &pdb, 1); if (fcp->isp_loopstate < LOOP_SCANNING_LOOP) goto abort; if (r != 0) { isp_prt(isp, ISP_LOGDEBUG1, "Chan %d FC Scan Loop handle %d returned %x", chan, handle, r); continue; } isp_pdb_add_update(isp, chan, &pdb); } if (fcp->isp_loopstate < LOOP_SCANNING_LOOP) goto abort; fcp->isp_loopstate = LOOP_LSCAN_DONE; isp_prt(isp, ISP_LOG_SANCFG, "Chan %d FC loop scan done", chan); return (0); } /* * Scan the fabric for devices and add them to our port database. * * Use the GID_FT command to get all Port IDs for FC4 SCSI devices it knows. * * For 2100-23XX cards, we can use the SNS mailbox command to pass simple * name server commands to the switch management server via the QLogic f/w. * * For the 24XX card, we have to use CT-Pass through run via the Execute IOCB * mailbox command. * * The net result is to leave the list of Port IDs setting untranslated in * offset IGPOFF of the FC scratch area, whereupon we'll canonicalize it to * host order at OGPOFF. */ /* * Take half of our scratch area to store Port IDs */ #define GIDLEN (ISP_FC_SCRLEN >> 1) #define NGENT ((GIDLEN - 16) >> 2) #define IGPOFF (0) #define OGPOFF (ISP_FC_SCRLEN >> 1) #define XTXOFF (ISP_FC_SCRLEN - (3 * QENTRY_LEN)) /* CT request */ #define CTXOFF (ISP_FC_SCRLEN - (2 * QENTRY_LEN)) /* Request IOCB */ #define ZTXOFF (ISP_FC_SCRLEN - (1 * QENTRY_LEN)) /* Response IOCB */ static int isp_gid_ft_sns(ispsoftc_t *isp, int chan) { union { sns_gid_ft_req_t _x; uint8_t _y[SNS_GID_FT_REQ_SIZE]; } un; fcparam *fcp = FCPARAM(isp, chan); sns_gid_ft_req_t *rq = &un._x; uint8_t *scp = fcp->isp_scratch; mbreg_t mbs; isp_prt(isp, ISP_LOGDEBUG0, "Chan %d scanning fabric (GID_FT) via SNS", chan); ISP_MEMZERO(rq, SNS_GID_FT_REQ_SIZE); rq->snscb_rblen = GIDLEN >> 1; rq->snscb_addr[RQRSP_ADDR0015] = DMA_WD0(fcp->isp_scdma + IGPOFF); rq->snscb_addr[RQRSP_ADDR1631] = DMA_WD1(fcp->isp_scdma + IGPOFF); rq->snscb_addr[RQRSP_ADDR3247] = DMA_WD2(fcp->isp_scdma + IGPOFF); rq->snscb_addr[RQRSP_ADDR4863] = DMA_WD3(fcp->isp_scdma + IGPOFF); rq->snscb_sblen = 6; rq->snscb_cmd = SNS_GID_FT; rq->snscb_mword_div_2 = NGENT; rq->snscb_fc4_type = FC4_SCSI; isp_put_gid_ft_request(isp, rq, (sns_gid_ft_req_t *)&scp[CTXOFF]); MEMORYBARRIER(isp, SYNC_SFORDEV, 0, SNS_GID_FT_REQ_SIZE, chan); MBSINIT(&mbs, MBOX_SEND_SNS, MBLOGALL, 10000000); mbs.param[0] = MBOX_SEND_SNS; mbs.param[1] = SNS_GID_FT_REQ_SIZE >> 1; mbs.param[2] = DMA_WD1(fcp->isp_scdma + CTXOFF); mbs.param[3] = DMA_WD0(fcp->isp_scdma + CTXOFF); mbs.param[6] = DMA_WD3(fcp->isp_scdma + CTXOFF); mbs.param[7] = DMA_WD2(fcp->isp_scdma + CTXOFF); isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { if (mbs.param[0] == MBOX_INVALID_COMMAND) { return (1); } else { return (-1); } } return (0); } static int isp_gid_ft_ct_passthru(ispsoftc_t *isp, int chan) { mbreg_t mbs; fcparam *fcp = FCPARAM(isp, chan); union { isp_ct_pt_t plocal; ct_hdr_t clocal; uint8_t q[QENTRY_LEN]; } un; isp_ct_pt_t *pt; ct_hdr_t *ct; uint32_t *rp; uint8_t *scp = fcp->isp_scratch; isp_prt(isp, ISP_LOGDEBUG0, "Chan %d scanning fabric (GID_FT) via CT", chan); /* * Build a Passthrough IOCB in memory. */ pt = &un.plocal; ISP_MEMZERO(un.q, QENTRY_LEN); pt->ctp_header.rqs_entry_count = 1; pt->ctp_header.rqs_entry_type = RQSTYPE_CT_PASSTHRU; pt->ctp_handle = 0xffffffff; pt->ctp_nphdl = fcp->isp_sns_hdl; pt->ctp_cmd_cnt = 1; pt->ctp_vpidx = ISP_GET_VPIDX(isp, chan); pt->ctp_time = 30; pt->ctp_rsp_cnt = 1; pt->ctp_rsp_bcnt = GIDLEN; pt->ctp_cmd_bcnt = sizeof (*ct) + sizeof (uint32_t); pt->ctp_dataseg[0].ds_base = DMA_LO32(fcp->isp_scdma+XTXOFF); pt->ctp_dataseg[0].ds_basehi = DMA_HI32(fcp->isp_scdma+XTXOFF); pt->ctp_dataseg[0].ds_count = sizeof (*ct) + sizeof (uint32_t); pt->ctp_dataseg[1].ds_base = DMA_LO32(fcp->isp_scdma+IGPOFF); pt->ctp_dataseg[1].ds_basehi = DMA_HI32(fcp->isp_scdma+IGPOFF); pt->ctp_dataseg[1].ds_count = GIDLEN; if (isp->isp_dblev & ISP_LOGDEBUG1) { isp_print_bytes(isp, "ct IOCB", QENTRY_LEN, pt); } isp_put_ct_pt(isp, pt, (isp_ct_pt_t *) &scp[CTXOFF]); /* * Build the CT header and command in memory. * * Note that the CT header has to end up as Big Endian format in memory. */ ct = &un.clocal; ISP_MEMZERO(ct, sizeof (*ct)); ct->ct_revision = CT_REVISION; ct->ct_fcs_type = CT_FC_TYPE_FC; ct->ct_fcs_subtype = CT_FC_SUBTYPE_NS; ct->ct_cmd_resp = SNS_GID_FT; ct->ct_bcnt_resid = (GIDLEN - 16) >> 2; isp_put_ct_hdr(isp, ct, (ct_hdr_t *) &scp[XTXOFF]); rp = (uint32_t *) &scp[XTXOFF+sizeof (*ct)]; ISP_IOZPUT_32(isp, FC4_SCSI, rp); if (isp->isp_dblev & ISP_LOGDEBUG1) { isp_print_bytes(isp, "CT HDR + payload after put", sizeof (*ct) + sizeof (uint32_t), &scp[XTXOFF]); } ISP_MEMZERO(&scp[ZTXOFF], QENTRY_LEN); MBSINIT(&mbs, MBOX_EXEC_COMMAND_IOCB_A64, MBLOGALL, 500000); mbs.param[1] = QENTRY_LEN; mbs.param[2] = DMA_WD1(fcp->isp_scdma + CTXOFF); mbs.param[3] = DMA_WD0(fcp->isp_scdma + CTXOFF); mbs.param[6] = DMA_WD3(fcp->isp_scdma + CTXOFF); mbs.param[7] = DMA_WD2(fcp->isp_scdma + CTXOFF); MEMORYBARRIER(isp, SYNC_SFORDEV, XTXOFF, 2 * QENTRY_LEN, chan); isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { return (-1); } MEMORYBARRIER(isp, SYNC_SFORCPU, ZTXOFF, QENTRY_LEN, chan); pt = &un.plocal; isp_get_ct_pt(isp, (isp_ct_pt_t *) &scp[ZTXOFF], pt); if (isp->isp_dblev & ISP_LOGDEBUG1) { isp_print_bytes(isp, "IOCB response", QENTRY_LEN, pt); } if (pt->ctp_status && pt->ctp_status != RQCS_DATA_UNDERRUN) { isp_prt(isp, ISP_LOGWARN, "Chan %d ISP GID FT CT Passthrough returned 0x%x", chan, pt->ctp_status); return (-1); } MEMORYBARRIER(isp, SYNC_SFORCPU, IGPOFF, GIDLEN, chan); if (isp->isp_dblev & ISP_LOGDEBUG1) { isp_print_bytes(isp, "CT response", GIDLEN, &scp[IGPOFF]); } return (0); } static int isp_scan_fabric(ispsoftc_t *isp, int chan) { fcparam *fcp = FCPARAM(isp, chan); fcportdb_t *lp; uint32_t portid; uint16_t nphdl; isp_pdb_t pdb; int portidx, portlim, r; sns_gid_ft_rsp_t *rs0, *rs1; if (fcp->isp_loopstate < LOOP_LSCAN_DONE) return (-1); if (fcp->isp_loopstate >= LOOP_FSCAN_DONE) return (0); isp_prt(isp, ISP_LOG_SANCFG, "Chan %d FC fabric scan", chan); fcp->isp_loopstate = LOOP_SCANNING_FABRIC; if (!TOPO_IS_FABRIC(fcp->isp_topo)) { fcp->isp_loopstate = LOOP_FSCAN_DONE; isp_prt(isp, ISP_LOG_SANCFG, "Chan %d FC fabric scan done (no fabric)", chan); return (0); } if (FC_SCRATCH_ACQUIRE(isp, chan)) { isp_prt(isp, ISP_LOGERR, sacq); fail: isp_prt(isp, ISP_LOG_SANCFG, "Chan %d FC fabric scan done (bad)", chan); return (-1); } if (fcp->isp_loopstate < LOOP_SCANNING_FABRIC) { abort: FC_SCRATCH_RELEASE(isp, chan); isp_prt(isp, ISP_LOG_SANCFG, "Chan %d FC fabric scan aborted", chan); return (1); } /* * Make sure we still are logged into the fabric controller. */ nphdl = IS_24XX(isp) ? NPH_FL_ID : FL_ID; r = isp_getpdb(isp, chan, nphdl, &pdb, 0); if ((r & 0xffff) == MBOX_NOT_LOGGED_IN) { isp_dump_chip_portdb(isp, chan, 0); } if (r) { fcp->isp_loopstate = LOOP_LTEST_DONE; FC_SCRATCH_RELEASE(isp, chan); goto fail; } /* Get list of port IDs from SNS. */ if (IS_24XX(isp)) r = isp_gid_ft_ct_passthru(isp, chan); else r = isp_gid_ft_sns(isp, chan); if (fcp->isp_loopstate < LOOP_SCANNING_FABRIC) goto abort; if (r > 0) { fcp->isp_loopstate = LOOP_FSCAN_DONE; FC_SCRATCH_RELEASE(isp, chan); return (-1); } else if (r < 0) { fcp->isp_loopstate = LOOP_LTEST_DONE; /* try again */ FC_SCRATCH_RELEASE(isp, chan); return (-1); } MEMORYBARRIER(isp, SYNC_SFORCPU, IGPOFF, GIDLEN, chan); rs0 = (sns_gid_ft_rsp_t *) ((uint8_t *)fcp->isp_scratch+IGPOFF); rs1 = (sns_gid_ft_rsp_t *) ((uint8_t *)fcp->isp_scratch+OGPOFF); isp_get_gid_ft_response(isp, rs0, rs1, NGENT); if (fcp->isp_loopstate < LOOP_SCANNING_FABRIC) goto abort; if (rs1->snscb_cthdr.ct_cmd_resp != LS_ACC) { int level; if (rs1->snscb_cthdr.ct_reason == 9 && rs1->snscb_cthdr.ct_explanation == 7) { level = ISP_LOG_SANCFG; } else { level = ISP_LOGWARN; } isp_prt(isp, level, "Chan %d Fabric Nameserver rejected GID_FT" " (Reason=0x%x Expl=0x%x)", chan, rs1->snscb_cthdr.ct_reason, rs1->snscb_cthdr.ct_explanation); FC_SCRATCH_RELEASE(isp, chan); fcp->isp_loopstate = LOOP_FSCAN_DONE; return (-1); } /* Check our buffer was big enough to get the full list. */ for (portidx = 0; portidx < NGENT-1; portidx++) { if (rs1->snscb_ports[portidx].control & 0x80) break; } if ((rs1->snscb_ports[portidx].control & 0x80) == 0) { isp_prt(isp, ISP_LOGWARN, "fabric too big for scratch area: increase ISP_FC_SCRLEN"); } portlim = portidx + 1; isp_prt(isp, ISP_LOG_SANCFG, "Chan %d Got %d ports back from name server", chan, portlim); /* Go through the list and remove duplicate port ids. */ for (portidx = 0; portidx < portlim; portidx++) { int npidx; portid = ((rs1->snscb_ports[portidx].portid[0]) << 16) | ((rs1->snscb_ports[portidx].portid[1]) << 8) | ((rs1->snscb_ports[portidx].portid[2])); for (npidx = portidx + 1; npidx < portlim; npidx++) { uint32_t new_portid = ((rs1->snscb_ports[npidx].portid[0]) << 16) | ((rs1->snscb_ports[npidx].portid[1]) << 8) | ((rs1->snscb_ports[npidx].portid[2])); if (new_portid == portid) { break; } } if (npidx < portlim) { rs1->snscb_ports[npidx].portid[0] = 0; rs1->snscb_ports[npidx].portid[1] = 0; rs1->snscb_ports[npidx].portid[2] = 0; isp_prt(isp, ISP_LOG_SANCFG, "Chan %d removing duplicate PortID 0x%06x entry from list", chan, portid); } } /* * We now have a list of Port IDs for all FC4 SCSI devices * that the Fabric Name server knows about. * * For each entry on this list go through our port database looking * for probational entries- if we find one, then an old entry is * maybe still this one. We get some information to find out. * * Otherwise, it's a new fabric device, and we log into it * (unconditionally). After searching the entire database * again to make sure that we never ever ever ever have more * than one entry that has the same PortID or the same * WWNN/WWPN duple, we enter the device into our database. */ isp_mark_portdb(isp, chan); for (portidx = 0; portidx < portlim; portidx++) { portid = ((rs1->snscb_ports[portidx].portid[0]) << 16) | ((rs1->snscb_ports[portidx].portid[1]) << 8) | ((rs1->snscb_ports[portidx].portid[2])); isp_prt(isp, ISP_LOG_SANCFG, "Chan %d Checking fabric port 0x%06x", chan, portid); if (portid == 0) { isp_prt(isp, ISP_LOG_SANCFG, "Chan %d Port at idx %d is zero", chan, portidx); continue; } if (portid == fcp->isp_portid) { isp_prt(isp, ISP_LOG_SANCFG, "Chan %d Port 0x%06x is our", chan, portid); continue; } /* Now search the entire port database for the same portid. */ if (isp_find_pdb_by_portid(isp, chan, portid, &lp)) { if (!lp->probational) { isp_prt(isp, ISP_LOGERR, "Chan %d Port 0x%06x@0x%04x [%d] is not probational (0x%x)", chan, lp->portid, lp->handle, FC_PORTDB_TGT(isp, chan, lp), lp->state); FC_SCRATCH_RELEASE(isp, chan); isp_dump_portdb(isp, chan); goto fail; } /* * See if we're still logged into it. * * If we aren't, mark it as a dead device and * leave the new portid in the database entry * for somebody further along to decide what to * do (policy choice). * * If we are, check to see if it's the same * device still (it should be). If for some * reason it isn't, mark it as a changed device * and leave the new portid and role in the * database entry for somebody further along to * decide what to do (policy choice). */ r = isp_getpdb(isp, chan, lp->handle, &pdb, 0); if (fcp->isp_loopstate < LOOP_SCANNING_FABRIC) goto abort; if (r != 0) { lp->state = FC_PORTDB_STATE_DEAD; isp_prt(isp, ISP_LOG_SANCFG, "Chan %d Port 0x%06x handle 0x%x is dead (%d)", chan, portid, lp->handle, r); goto relogin; } isp_pdb_add_update(isp, chan, &pdb); continue; } relogin: if ((fcp->role & ISP_ROLE_INITIATOR) == 0) { isp_prt(isp, ISP_LOG_SANCFG, "Chan %d Port 0x%06x is not logged in", chan, portid); continue; } if (isp_login_device(isp, chan, portid, &pdb, &FCPARAM(isp, 0)->isp_lasthdl)) { if (fcp->isp_loopstate < LOOP_SCANNING_FABRIC) goto abort; continue; } isp_pdb_add_update(isp, chan, &pdb); } if (fcp->isp_loopstate < LOOP_SCANNING_FABRIC) goto abort; FC_SCRATCH_RELEASE(isp, chan); fcp->isp_loopstate = LOOP_FSCAN_DONE; isp_prt(isp, ISP_LOG_SANCFG, "Chan %d FC fabric scan done", chan); return (0); } /* * Find an unused handle and try and use to login to a port. */ static int isp_login_device(ispsoftc_t *isp, int chan, uint32_t portid, isp_pdb_t *p, uint16_t *ohp) { int lim, i, r; uint16_t handle; if (ISP_CAP_2KLOGIN(isp)) { lim = NPH_MAX_2K; } else { lim = NPH_MAX; } handle = isp_next_handle(isp, ohp); for (i = 0; i < lim; i++) { if (FCPARAM(isp, chan)->isp_loopstate != LOOP_SCANNING_FABRIC) return (-1); /* Check if this handle is free. */ r = isp_getpdb(isp, chan, handle, p, 0); if (r == 0) { if (p->portid != portid) { /* This handle is busy, try next one. */ handle = isp_next_handle(isp, ohp); continue; } break; } if (FCPARAM(isp, chan)->isp_loopstate != LOOP_SCANNING_FABRIC) return (-1); /* * Now try and log into the device */ r = isp_plogx(isp, chan, handle, portid, PLOGX_FLG_CMD_PLOGI, 1); if (r == 0) { break; } else if ((r & 0xffff) == MBOX_PORT_ID_USED) { /* * If we get here, then the firmwware still thinks we're logged into this device, but with a different * handle. We need to break that association. We used to try and just substitute the handle, but then * failed to get any data via isp_getpdb (below). */ if (isp_plogx(isp, chan, r >> 16, portid, PLOGX_FLG_CMD_LOGO | PLOGX_FLG_IMPLICIT | PLOGX_FLG_FREE_NPHDL, 1)) { isp_prt(isp, ISP_LOGERR, "baw... logout of %x failed", r >> 16); } if (FCPARAM(isp, chan)->isp_loopstate != LOOP_SCANNING_FABRIC) return (-1); r = isp_plogx(isp, chan, handle, portid, PLOGX_FLG_CMD_PLOGI, 1); if (r != 0) i = lim; break; } else if ((r & 0xffff) == MBOX_LOOP_ID_USED) { /* Try the next handle. */ handle = isp_next_handle(isp, ohp); } else { /* Give up. */ i = lim; break; } } if (i == lim) { isp_prt(isp, ISP_LOGWARN, "Chan %d PLOGI 0x%06x failed", chan, portid); return (-1); } /* * If we successfully logged into it, get the PDB for it * so we can crosscheck that it is still what we think it * is and that we also have the role it plays */ r = isp_getpdb(isp, chan, handle, p, 0); if (r != 0) { isp_prt(isp, ISP_LOGERR, "Chan %d new device 0x%06x@0x%x disappeared", chan, portid, handle); return (-1); } if (p->handle != handle || p->portid != portid) { isp_prt(isp, ISP_LOGERR, "Chan %d new device 0x%06x@0x%x changed (0x%06x@0x%0x)", chan, portid, handle, p->portid, p->handle); return (-1); } return (0); } static int isp_send_change_request(ispsoftc_t *isp, int chan) { mbreg_t mbs; MBSINIT(&mbs, MBOX_SEND_CHANGE_REQUEST, MBLOGALL, 500000); mbs.param[1] = 0x03; mbs.param[9] = chan; isp_mboxcmd(isp, &mbs); return (mbs.param[0] == MBOX_COMMAND_COMPLETE ? 0 : -1); } static int isp_register_fc4_type(ispsoftc_t *isp, int chan) { fcparam *fcp = FCPARAM(isp, chan); uint8_t local[SNS_RFT_ID_REQ_SIZE]; sns_screq_t *reqp = (sns_screq_t *) local; mbreg_t mbs; ISP_MEMZERO((void *) reqp, SNS_RFT_ID_REQ_SIZE); reqp->snscb_rblen = SNS_RFT_ID_RESP_SIZE >> 1; reqp->snscb_addr[RQRSP_ADDR0015] = DMA_WD0(fcp->isp_scdma + 0x100); reqp->snscb_addr[RQRSP_ADDR1631] = DMA_WD1(fcp->isp_scdma + 0x100); reqp->snscb_addr[RQRSP_ADDR3247] = DMA_WD2(fcp->isp_scdma + 0x100); reqp->snscb_addr[RQRSP_ADDR4863] = DMA_WD3(fcp->isp_scdma + 0x100); reqp->snscb_sblen = 22; reqp->snscb_data[0] = SNS_RFT_ID; reqp->snscb_data[4] = fcp->isp_portid & 0xffff; reqp->snscb_data[5] = (fcp->isp_portid >> 16) & 0xff; reqp->snscb_data[6] = (1 << FC4_SCSI); if (FC_SCRATCH_ACQUIRE(isp, chan)) { isp_prt(isp, ISP_LOGERR, sacq); return (-1); } isp_put_sns_request(isp, reqp, (sns_screq_t *) fcp->isp_scratch); MBSINIT(&mbs, MBOX_SEND_SNS, MBLOGALL, 1000000); mbs.param[1] = SNS_RFT_ID_REQ_SIZE >> 1; mbs.param[2] = DMA_WD1(fcp->isp_scdma); mbs.param[3] = DMA_WD0(fcp->isp_scdma); mbs.param[6] = DMA_WD3(fcp->isp_scdma); mbs.param[7] = DMA_WD2(fcp->isp_scdma); MEMORYBARRIER(isp, SYNC_SFORDEV, 0, SNS_RFT_ID_REQ_SIZE, chan); isp_mboxcmd(isp, &mbs); FC_SCRATCH_RELEASE(isp, chan); if (mbs.param[0] == MBOX_COMMAND_COMPLETE) { return (0); } else { return (-1); } } static int isp_register_fc4_type_24xx(ispsoftc_t *isp, int chan) { mbreg_t mbs; fcparam *fcp = FCPARAM(isp, chan); union { isp_ct_pt_t plocal; rft_id_t clocal; uint8_t q[QENTRY_LEN]; } un; isp_ct_pt_t *pt; ct_hdr_t *ct; rft_id_t *rp; uint8_t *scp = fcp->isp_scratch; if (FC_SCRATCH_ACQUIRE(isp, chan)) { isp_prt(isp, ISP_LOGERR, sacq); return (-1); } /* * Build a Passthrough IOCB in memory. */ ISP_MEMZERO(un.q, QENTRY_LEN); pt = &un.plocal; pt->ctp_header.rqs_entry_count = 1; pt->ctp_header.rqs_entry_type = RQSTYPE_CT_PASSTHRU; pt->ctp_handle = 0xffffffff; pt->ctp_nphdl = fcp->isp_sns_hdl; pt->ctp_cmd_cnt = 1; pt->ctp_vpidx = ISP_GET_VPIDX(isp, chan); pt->ctp_time = 1; pt->ctp_rsp_cnt = 1; pt->ctp_rsp_bcnt = sizeof (ct_hdr_t); pt->ctp_cmd_bcnt = sizeof (rft_id_t); pt->ctp_dataseg[0].ds_base = DMA_LO32(fcp->isp_scdma+XTXOFF); pt->ctp_dataseg[0].ds_basehi = DMA_HI32(fcp->isp_scdma+XTXOFF); pt->ctp_dataseg[0].ds_count = sizeof (rft_id_t); pt->ctp_dataseg[1].ds_base = DMA_LO32(fcp->isp_scdma+IGPOFF); pt->ctp_dataseg[1].ds_basehi = DMA_HI32(fcp->isp_scdma+IGPOFF); pt->ctp_dataseg[1].ds_count = sizeof (ct_hdr_t); isp_put_ct_pt(isp, pt, (isp_ct_pt_t *) &scp[CTXOFF]); if (isp->isp_dblev & ISP_LOGDEBUG1) { isp_print_bytes(isp, "IOCB CT Request", QENTRY_LEN, pt); } /* * Build the CT header and command in memory. * * Note that the CT header has to end up as Big Endian format in memory. */ ISP_MEMZERO(&un.clocal, sizeof (un.clocal)); ct = &un.clocal.rftid_hdr; ct->ct_revision = CT_REVISION; ct->ct_fcs_type = CT_FC_TYPE_FC; ct->ct_fcs_subtype = CT_FC_SUBTYPE_NS; ct->ct_cmd_resp = SNS_RFT_ID; ct->ct_bcnt_resid = (sizeof (rft_id_t) - sizeof (ct_hdr_t)) >> 2; rp = &un.clocal; rp->rftid_portid[0] = fcp->isp_portid >> 16; rp->rftid_portid[1] = fcp->isp_portid >> 8; rp->rftid_portid[2] = fcp->isp_portid; rp->rftid_fc4types[FC4_SCSI >> 5] = 1 << (FC4_SCSI & 0x1f); isp_put_rft_id(isp, rp, (rft_id_t *) &scp[XTXOFF]); if (isp->isp_dblev & ISP_LOGDEBUG1) { isp_print_bytes(isp, "CT Header", QENTRY_LEN, &scp[XTXOFF]); } ISP_MEMZERO(&scp[ZTXOFF], sizeof (ct_hdr_t)); MBSINIT(&mbs, MBOX_EXEC_COMMAND_IOCB_A64, MBLOGALL, 1000000); mbs.param[1] = QENTRY_LEN; mbs.param[2] = DMA_WD1(fcp->isp_scdma + CTXOFF); mbs.param[3] = DMA_WD0(fcp->isp_scdma + CTXOFF); mbs.param[6] = DMA_WD3(fcp->isp_scdma + CTXOFF); mbs.param[7] = DMA_WD2(fcp->isp_scdma + CTXOFF); MEMORYBARRIER(isp, SYNC_SFORDEV, XTXOFF, 2 * QENTRY_LEN, chan); isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { FC_SCRATCH_RELEASE(isp, chan); return (-1); } MEMORYBARRIER(isp, SYNC_SFORCPU, ZTXOFF, QENTRY_LEN, chan); pt = &un.plocal; isp_get_ct_pt(isp, (isp_ct_pt_t *) &scp[ZTXOFF], pt); if (isp->isp_dblev & ISP_LOGDEBUG1) { isp_print_bytes(isp, "IOCB response", QENTRY_LEN, pt); } if (pt->ctp_status) { FC_SCRATCH_RELEASE(isp, chan); isp_prt(isp, ISP_LOGWARN, "Chan %d Register FC4 Type CT Passthrough returned 0x%x", chan, pt->ctp_status); return (1); } isp_get_ct_hdr(isp, (ct_hdr_t *) &scp[IGPOFF], ct); FC_SCRATCH_RELEASE(isp, chan); if (ct->ct_cmd_resp == LS_RJT) { isp_prt(isp, ISP_LOG_SANCFG|ISP_LOG_WARN1, "Chan %d Register FC4 Type rejected", chan); return (-1); } else if (ct->ct_cmd_resp == LS_ACC) { isp_prt(isp, ISP_LOG_SANCFG, "Chan %d Register FC4 Type accepted", chan); return (0); } else { isp_prt(isp, ISP_LOGWARN, "Chan %d Register FC4 Type: 0x%x", chan, ct->ct_cmd_resp); return (-1); } } static int isp_register_fc4_features_24xx(ispsoftc_t *isp, int chan) { mbreg_t mbs; fcparam *fcp = FCPARAM(isp, chan); union { isp_ct_pt_t plocal; rff_id_t clocal; uint8_t q[QENTRY_LEN]; } un; isp_ct_pt_t *pt; ct_hdr_t *ct; rff_id_t *rp; uint8_t *scp = fcp->isp_scratch; if (FC_SCRATCH_ACQUIRE(isp, chan)) { isp_prt(isp, ISP_LOGERR, sacq); return (-1); } /* * Build a Passthrough IOCB in memory. */ ISP_MEMZERO(un.q, QENTRY_LEN); pt = &un.plocal; pt->ctp_header.rqs_entry_count = 1; pt->ctp_header.rqs_entry_type = RQSTYPE_CT_PASSTHRU; pt->ctp_handle = 0xffffffff; pt->ctp_nphdl = fcp->isp_sns_hdl; pt->ctp_cmd_cnt = 1; pt->ctp_vpidx = ISP_GET_VPIDX(isp, chan); pt->ctp_time = 1; pt->ctp_rsp_cnt = 1; pt->ctp_rsp_bcnt = sizeof (ct_hdr_t); pt->ctp_cmd_bcnt = sizeof (rff_id_t); pt->ctp_dataseg[0].ds_base = DMA_LO32(fcp->isp_scdma+XTXOFF); pt->ctp_dataseg[0].ds_basehi = DMA_HI32(fcp->isp_scdma+XTXOFF); pt->ctp_dataseg[0].ds_count = sizeof (rff_id_t); pt->ctp_dataseg[1].ds_base = DMA_LO32(fcp->isp_scdma+IGPOFF); pt->ctp_dataseg[1].ds_basehi = DMA_HI32(fcp->isp_scdma+IGPOFF); pt->ctp_dataseg[1].ds_count = sizeof (ct_hdr_t); isp_put_ct_pt(isp, pt, (isp_ct_pt_t *) &scp[CTXOFF]); if (isp->isp_dblev & ISP_LOGDEBUG1) { isp_print_bytes(isp, "IOCB CT Request", QENTRY_LEN, pt); } /* * Build the CT header and command in memory. * * Note that the CT header has to end up as Big Endian format in memory. */ ISP_MEMZERO(&un.clocal, sizeof (un.clocal)); ct = &un.clocal.rffid_hdr; ct->ct_revision = CT_REVISION; ct->ct_fcs_type = CT_FC_TYPE_FC; ct->ct_fcs_subtype = CT_FC_SUBTYPE_NS; ct->ct_cmd_resp = SNS_RFF_ID; ct->ct_bcnt_resid = (sizeof (rff_id_t) - sizeof (ct_hdr_t)) >> 2; rp = &un.clocal; rp->rffid_portid[0] = fcp->isp_portid >> 16; rp->rffid_portid[1] = fcp->isp_portid >> 8; rp->rffid_portid[2] = fcp->isp_portid; rp->rffid_fc4features = 0; if (fcp->role & ISP_ROLE_TARGET) rp->rffid_fc4features |= 1; if (fcp->role & ISP_ROLE_INITIATOR) rp->rffid_fc4features |= 2; rp->rffid_fc4type = FC4_SCSI; isp_put_rff_id(isp, rp, (rff_id_t *) &scp[XTXOFF]); if (isp->isp_dblev & ISP_LOGDEBUG1) { isp_print_bytes(isp, "CT Header", QENTRY_LEN, &scp[XTXOFF]); } ISP_MEMZERO(&scp[ZTXOFF], sizeof (ct_hdr_t)); MBSINIT(&mbs, MBOX_EXEC_COMMAND_IOCB_A64, MBLOGALL, 1000000); mbs.param[1] = QENTRY_LEN; mbs.param[2] = DMA_WD1(fcp->isp_scdma + CTXOFF); mbs.param[3] = DMA_WD0(fcp->isp_scdma + CTXOFF); mbs.param[6] = DMA_WD3(fcp->isp_scdma + CTXOFF); mbs.param[7] = DMA_WD2(fcp->isp_scdma + CTXOFF); MEMORYBARRIER(isp, SYNC_SFORDEV, XTXOFF, 2 * QENTRY_LEN, chan); isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { FC_SCRATCH_RELEASE(isp, chan); return (-1); } MEMORYBARRIER(isp, SYNC_SFORCPU, ZTXOFF, QENTRY_LEN, chan); pt = &un.plocal; isp_get_ct_pt(isp, (isp_ct_pt_t *) &scp[ZTXOFF], pt); if (isp->isp_dblev & ISP_LOGDEBUG1) { isp_print_bytes(isp, "IOCB response", QENTRY_LEN, pt); } if (pt->ctp_status) { FC_SCRATCH_RELEASE(isp, chan); isp_prt(isp, ISP_LOGWARN, "Chan %d Register FC4 Features CT Passthrough returned 0x%x", chan, pt->ctp_status); return (1); } isp_get_ct_hdr(isp, (ct_hdr_t *) &scp[IGPOFF], ct); FC_SCRATCH_RELEASE(isp, chan); if (ct->ct_cmd_resp == LS_RJT) { isp_prt(isp, ISP_LOG_SANCFG|ISP_LOG_WARN1, "Chan %d Register FC4 Features rejected", chan); return (-1); } else if (ct->ct_cmd_resp == LS_ACC) { isp_prt(isp, ISP_LOG_SANCFG, "Chan %d Register FC4 Features accepted", chan); return (0); } else { isp_prt(isp, ISP_LOGWARN, "Chan %d Register FC4 Features: 0x%x", chan, ct->ct_cmd_resp); return (-1); } } static uint16_t isp_next_handle(ispsoftc_t *isp, uint16_t *ohp) { fcparam *fcp; int i, chan, wrap; uint16_t handle, minh, maxh; handle = *ohp; if (ISP_CAP_2KLOGIN(isp)) { minh = 0; maxh = NPH_RESERVED - 1; } else { minh = SNS_ID + 1; maxh = NPH_MAX - 1; } wrap = 0; next: if (handle == NIL_HANDLE) { handle = minh; } else { handle++; if (handle > maxh) { if (++wrap >= 2) { isp_prt(isp, ISP_LOGERR, "Out of port handles!"); return (NIL_HANDLE); } handle = minh; } } for (chan = 0; chan < isp->isp_nchan; chan++) { fcp = FCPARAM(isp, chan); if (fcp->role == ISP_ROLE_NONE) continue; for (i = 0; i < MAX_FC_TARG; i++) { if (fcp->portdb[i].state != FC_PORTDB_STATE_NIL && fcp->portdb[i].handle == handle) goto next; } } *ohp = handle; return (handle); } /* * Start a command. Locking is assumed done in the caller. */ int isp_start(XS_T *xs) { ispsoftc_t *isp; uint32_t handle, cdblen; uint8_t local[QENTRY_LEN]; ispreq_t *reqp; void *cdbp, *qep; uint16_t *tptr; fcportdb_t *lp; int target, dmaresult; XS_INITERR(xs); isp = XS_ISP(xs); /* * Check command CDB length, etc.. We really are limited to 16 bytes * for Fibre Channel, but can do up to 44 bytes in parallel SCSI, * but probably only if we're running fairly new firmware (we'll * let the old f/w choke on an extended command queue entry). */ if (XS_CDBLEN(xs) > (IS_FC(isp)? 16 : 44) || XS_CDBLEN(xs) == 0) { isp_prt(isp, ISP_LOGERR, "unsupported cdb length (%d, CDB[0]=0x%x)", XS_CDBLEN(xs), XS_CDBP(xs)[0] & 0xff); XS_SETERR(xs, HBA_BOTCH); return (CMD_COMPLETE); } /* * Translate the target to device handle as appropriate, checking * for correct device state as well. */ target = XS_TGT(xs); if (IS_FC(isp)) { fcparam *fcp = FCPARAM(isp, XS_CHANNEL(xs)); if ((fcp->role & ISP_ROLE_INITIATOR) == 0) { isp_prt(isp, ISP_LOG_WARN1, "%d.%d.%jx I am not an initiator", XS_CHANNEL(xs), target, (uintmax_t)XS_LUN(xs)); XS_SETERR(xs, HBA_SELTIMEOUT); return (CMD_COMPLETE); } if (isp->isp_state != ISP_RUNSTATE) { isp_prt(isp, ISP_LOGERR, "Adapter not at RUNSTATE"); XS_SETERR(xs, HBA_BOTCH); return (CMD_COMPLETE); } /* * Try again later. */ if (fcp->isp_loopstate != LOOP_READY) { return (CMD_RQLATER); } isp_prt(isp, ISP_LOGDEBUG2, "XS_TGT(xs)=%d", target); lp = &fcp->portdb[target]; if (target < 0 || target >= MAX_FC_TARG || lp->is_target == 0) { XS_SETERR(xs, HBA_SELTIMEOUT); return (CMD_COMPLETE); } if (lp->state == FC_PORTDB_STATE_ZOMBIE) { isp_prt(isp, ISP_LOGDEBUG1, "%d.%d.%jx target zombie", XS_CHANNEL(xs), target, (uintmax_t)XS_LUN(xs)); return (CMD_RQLATER); } if (lp->state != FC_PORTDB_STATE_VALID) { isp_prt(isp, ISP_LOGDEBUG1, "%d.%d.%jx bad db port state 0x%x", XS_CHANNEL(xs), target, (uintmax_t)XS_LUN(xs), lp->state); XS_SETERR(xs, HBA_SELTIMEOUT); return (CMD_COMPLETE); } } else { sdparam *sdp = SDPARAM(isp, XS_CHANNEL(xs)); if (isp->isp_state != ISP_RUNSTATE) { isp_prt(isp, ISP_LOGERR, "Adapter not at RUNSTATE"); XS_SETERR(xs, HBA_BOTCH); return (CMD_COMPLETE); } if (sdp->update) { isp_spi_update(isp, XS_CHANNEL(xs)); } lp = NULL; } start_again: qep = isp_getrqentry(isp); if (qep == NULL) { isp_prt(isp, ISP_LOG_WARN1, "Request Queue Overflow"); XS_SETERR(xs, HBA_BOTCH); return (CMD_EAGAIN); } XS_SETERR(xs, HBA_NOERROR); /* * Now see if we need to synchronize the ISP with respect to anything. * We do dual duty here (cough) for synchronizing for busses other * than which we got here to send a command to. */ reqp = (ispreq_t *) local; ISP_MEMZERO(local, QENTRY_LEN); if (ISP_TST_SENDMARKER(isp, XS_CHANNEL(xs))) { if (IS_24XX(isp)) { isp_marker_24xx_t *m = (isp_marker_24xx_t *) reqp; m->mrk_header.rqs_entry_count = 1; m->mrk_header.rqs_entry_type = RQSTYPE_MARKER; m->mrk_modifier = SYNC_ALL; m->mrk_vphdl = XS_CHANNEL(xs); isp_put_marker_24xx(isp, m, qep); } else { isp_marker_t *m = (isp_marker_t *) reqp; m->mrk_header.rqs_entry_count = 1; m->mrk_header.rqs_entry_type = RQSTYPE_MARKER; m->mrk_target = (XS_CHANNEL(xs) << 7); /* bus # */ m->mrk_modifier = SYNC_ALL; isp_put_marker(isp, m, qep); } ISP_SYNC_REQUEST(isp); ISP_SET_SENDMARKER(isp, XS_CHANNEL(xs), 0); goto start_again; } reqp->req_header.rqs_entry_count = 1; /* * Select and install Header Code. * Note that it might be overridden before going out * if we're on a 64 bit platform. The lower level * code (isp_send_cmd) will select the appropriate * 64 bit variant if it needs to. */ if (IS_24XX(isp)) { reqp->req_header.rqs_entry_type = RQSTYPE_T7RQS; } else if (IS_FC(isp)) { reqp->req_header.rqs_entry_type = RQSTYPE_T2RQS; } else { if (XS_CDBLEN(xs) > 12) { reqp->req_header.rqs_entry_type = RQSTYPE_CMDONLY; } else { reqp->req_header.rqs_entry_type = RQSTYPE_REQUEST; } } /* * Set task attributes */ if (IS_24XX(isp)) { int ttype; if (XS_TAG_P(xs)) { ttype = XS_TAG_TYPE(xs); } else { if (XS_CDBP(xs)[0] == 0x3) { ttype = REQFLAG_HTAG; } else { ttype = REQFLAG_STAG; } } if (ttype == REQFLAG_OTAG) { ttype = FCP_CMND_TASK_ATTR_ORDERED; } else if (ttype == REQFLAG_HTAG) { ttype = FCP_CMND_TASK_ATTR_HEAD; } else { ttype = FCP_CMND_TASK_ATTR_SIMPLE; } ((ispreqt7_t *)reqp)->req_task_attribute = ttype; } else if (IS_FC(isp)) { /* * See comment in isp_intr */ /* XS_SET_RESID(xs, 0); */ /* * Fibre Channel always requires some kind of tag. * The Qlogic drivers seem be happy not to use a tag, * but this breaks for some devices (IBM drives). */ if (XS_TAG_P(xs)) { ((ispreqt2_t *)reqp)->req_flags = XS_TAG_TYPE(xs); } else { /* * If we don't know what tag to use, use HEAD OF QUEUE * for Request Sense or Simple. */ if (XS_CDBP(xs)[0] == 0x3) /* REQUEST SENSE */ ((ispreqt2_t *)reqp)->req_flags = REQFLAG_HTAG; else ((ispreqt2_t *)reqp)->req_flags = REQFLAG_STAG; } } else { sdparam *sdp = SDPARAM(isp, XS_CHANNEL(xs)); if ((sdp->isp_devparam[target].actv_flags & DPARM_TQING) && XS_TAG_P(xs)) { reqp->req_flags = XS_TAG_TYPE(xs); } } tptr = &reqp->req_time; /* * NB: we do not support long CDBs (yet) */ cdblen = XS_CDBLEN(xs); if (IS_SCSI(isp)) { if (cdblen > sizeof (reqp->req_cdb)) { isp_prt(isp, ISP_LOGERR, "Command Length %u too long for this chip", cdblen); XS_SETERR(xs, HBA_BOTCH); return (CMD_COMPLETE); } reqp->req_target = target | (XS_CHANNEL(xs) << 7); reqp->req_lun_trn = XS_LUN(xs); cdbp = reqp->req_cdb; reqp->req_cdblen = cdblen; } else if (IS_24XX(isp)) { ispreqt7_t *t7 = (ispreqt7_t *)local; if (cdblen > sizeof (t7->req_cdb)) { isp_prt(isp, ISP_LOGERR, "Command Length %u too long for this chip", cdblen); XS_SETERR(xs, HBA_BOTCH); return (CMD_COMPLETE); } t7->req_nphdl = lp->handle; t7->req_tidlo = lp->portid; t7->req_tidhi = lp->portid >> 16; t7->req_vpidx = ISP_GET_VPIDX(isp, XS_CHANNEL(xs)); #if __FreeBSD_version >= 1000700 be64enc(t7->req_lun, CAM_EXTLUN_BYTE_SWIZZLE(XS_LUN(xs))); #else if (XS_LUN(xs) >= 256) { t7->req_lun[0] = XS_LUN(xs) >> 8; t7->req_lun[0] |= 0x40; } t7->req_lun[1] = XS_LUN(xs); #endif if (FCPARAM(isp, XS_CHANNEL(xs))->fctape_enabled && (lp->prli_word3 & PRLI_WD3_RETRY)) { if (FCP_NEXT_CRN(isp, &t7->req_crn, xs)) { isp_prt(isp, ISP_LOG_WARN1, "%d.%d.%jx cannot generate next CRN", XS_CHANNEL(xs), target, (uintmax_t)XS_LUN(xs)); XS_SETERR(xs, HBA_BOTCH); return (CMD_EAGAIN); } } tptr = &t7->req_time; cdbp = t7->req_cdb; } else { ispreqt2_t *t2 = (ispreqt2_t *)local; if (cdblen > sizeof t2->req_cdb) { isp_prt(isp, ISP_LOGERR, "Command Length %u too long for this chip", cdblen); XS_SETERR(xs, HBA_BOTCH); return (CMD_COMPLETE); } if (FCPARAM(isp, XS_CHANNEL(xs))->fctape_enabled && (lp->prli_word3 & PRLI_WD3_RETRY)) { if (FCP_NEXT_CRN(isp, &t2->req_crn, xs)) { isp_prt(isp, ISP_LOG_WARN1, "%d.%d.%jx cannot generate next CRN", XS_CHANNEL(xs), target, (uintmax_t)XS_LUN(xs)); XS_SETERR(xs, HBA_BOTCH); return (CMD_EAGAIN); } } if (ISP_CAP_2KLOGIN(isp)) { ispreqt2e_t *t2e = (ispreqt2e_t *)local; t2e->req_target = lp->handle; t2e->req_scclun = XS_LUN(xs); #if __FreeBSD_version < 1000700 if (XS_LUN(xs) >= 256) t2e->req_scclun |= 0x4000; #endif cdbp = t2e->req_cdb; } else if (ISP_CAP_SCCFW(isp)) { ispreqt2_t *t2 = (ispreqt2_t *)local; t2->req_target = lp->handle; t2->req_scclun = XS_LUN(xs); #if __FreeBSD_version < 1000700 if (XS_LUN(xs) >= 256) t2->req_scclun |= 0x4000; #endif cdbp = t2->req_cdb; } else { t2->req_target = lp->handle; t2->req_lun_trn = XS_LUN(xs); cdbp = t2->req_cdb; } } ISP_MEMCPY(cdbp, XS_CDBP(xs), cdblen); *tptr = XS_TIME(xs) / 1000; if (*tptr == 0 && XS_TIME(xs)) { *tptr = 1; } if (IS_24XX(isp) && *tptr > 0x1999) { *tptr = 0x1999; } if (isp_allocate_xs(isp, xs, &handle)) { isp_prt(isp, ISP_LOG_WARN1, "out of xflist pointers"); XS_SETERR(xs, HBA_BOTCH); return (CMD_EAGAIN); } /* Whew. Thankfully the same for type 7 requests */ reqp->req_handle = handle; /* * Set up DMA and/or do any platform dependent swizzling of the request entry * so that the Qlogic F/W understands what is being asked of it. * * The callee is responsible for adding all requests at this point. */ dmaresult = ISP_DMASETUP(isp, xs, reqp); if (dmaresult != CMD_QUEUED) { isp_destroy_handle(isp, handle); /* * dmasetup sets actual error in packet, and * return what we were given to return. */ return (dmaresult); } isp_xs_prt(isp, xs, ISP_LOGDEBUG0, "START cmd cdb[0]=0x%x datalen %ld", XS_CDBP(xs)[0], (long) XS_XFRLEN(xs)); isp->isp_nactive++; return (CMD_QUEUED); } /* * isp control * Locks (ints blocked) assumed held. */ int isp_control(ispsoftc_t *isp, ispctl_t ctl, ...) { XS_T *xs; mbreg_t *mbr, mbs; int chan, tgt; uint32_t handle; va_list ap; switch (ctl) { case ISPCTL_RESET_BUS: /* * Issue a bus reset. */ if (IS_24XX(isp)) { isp_prt(isp, ISP_LOGERR, "BUS RESET NOT IMPLEMENTED"); break; } else if (IS_FC(isp)) { mbs.param[1] = 10; chan = 0; } else { va_start(ap, ctl); chan = va_arg(ap, int); va_end(ap); mbs.param[1] = SDPARAM(isp, chan)->isp_bus_reset_delay; if (mbs.param[1] < 2) { mbs.param[1] = 2; } mbs.param[2] = chan; } MBSINIT(&mbs, MBOX_BUS_RESET, MBLOGALL, 0); ISP_SET_SENDMARKER(isp, chan, 1); isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { break; } isp_prt(isp, ISP_LOGINFO, "driver initiated bus reset of bus %d", chan); return (0); case ISPCTL_RESET_DEV: va_start(ap, ctl); chan = va_arg(ap, int); tgt = va_arg(ap, int); va_end(ap); if (IS_24XX(isp)) { uint8_t local[QENTRY_LEN]; isp24xx_tmf_t *tmf; isp24xx_statusreq_t *sp; fcparam *fcp = FCPARAM(isp, chan); fcportdb_t *lp; if (tgt < 0 || tgt >= MAX_FC_TARG) { isp_prt(isp, ISP_LOGWARN, "Chan %d trying to reset bad target %d", chan, tgt); break; } lp = &fcp->portdb[tgt]; if (lp->is_target == 0 || lp->state != FC_PORTDB_STATE_VALID) { isp_prt(isp, ISP_LOGWARN, "Chan %d abort of no longer valid target %d", chan, tgt); break; } tmf = (isp24xx_tmf_t *) local; ISP_MEMZERO(tmf, QENTRY_LEN); tmf->tmf_header.rqs_entry_type = RQSTYPE_TSK_MGMT; tmf->tmf_header.rqs_entry_count = 1; tmf->tmf_nphdl = lp->handle; tmf->tmf_delay = 2; tmf->tmf_timeout = 2; tmf->tmf_flags = ISP24XX_TMF_TARGET_RESET; tmf->tmf_tidlo = lp->portid; tmf->tmf_tidhi = lp->portid >> 16; tmf->tmf_vpidx = ISP_GET_VPIDX(isp, chan); isp_prt(isp, ISP_LOGALL, "Chan %d Reset N-Port Handle 0x%04x @ Port 0x%06x", chan, lp->handle, lp->portid); MBSINIT(&mbs, MBOX_EXEC_COMMAND_IOCB_A64, MBLOGALL, 5000000); mbs.param[1] = QENTRY_LEN; mbs.param[2] = DMA_WD1(fcp->isp_scdma); mbs.param[3] = DMA_WD0(fcp->isp_scdma); mbs.param[6] = DMA_WD3(fcp->isp_scdma); mbs.param[7] = DMA_WD2(fcp->isp_scdma); if (FC_SCRATCH_ACQUIRE(isp, chan)) { isp_prt(isp, ISP_LOGERR, sacq); break; } isp_put_24xx_tmf(isp, tmf, fcp->isp_scratch); MEMORYBARRIER(isp, SYNC_SFORDEV, 0, QENTRY_LEN, chan); fcp->sendmarker = 1; isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { FC_SCRATCH_RELEASE(isp, chan); break; } MEMORYBARRIER(isp, SYNC_SFORCPU, QENTRY_LEN, QENTRY_LEN, chan); sp = (isp24xx_statusreq_t *) local; isp_get_24xx_response(isp, &((isp24xx_statusreq_t *)fcp->isp_scratch)[1], sp); FC_SCRATCH_RELEASE(isp, chan); if (sp->req_completion_status == 0) { return (0); } isp_prt(isp, ISP_LOGWARN, "Chan %d reset of target %d returned 0x%x", chan, tgt, sp->req_completion_status); break; } else if (IS_FC(isp)) { if (ISP_CAP_2KLOGIN(isp)) { mbs.param[1] = tgt; mbs.ibits = (1 << 10); } else { mbs.param[1] = (tgt << 8); } } else { mbs.param[1] = (chan << 15) | (tgt << 8); } MBSINIT(&mbs, MBOX_ABORT_TARGET, MBLOGALL, 0); mbs.param[2] = 3; /* 'delay', in seconds */ isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { break; } isp_prt(isp, ISP_LOGINFO, "Target %d on Bus %d Reset Succeeded", tgt, chan); ISP_SET_SENDMARKER(isp, chan, 1); return (0); case ISPCTL_ABORT_CMD: va_start(ap, ctl); xs = va_arg(ap, XS_T *); va_end(ap); tgt = XS_TGT(xs); chan = XS_CHANNEL(xs); handle = isp_find_handle(isp, xs); if (handle == 0) { isp_prt(isp, ISP_LOGWARN, "cannot find handle for command to abort"); break; } if (IS_24XX(isp)) { isp24xx_abrt_t local, *ab = &local, *ab2; fcparam *fcp; fcportdb_t *lp; fcp = FCPARAM(isp, chan); if (tgt < 0 || tgt >= MAX_FC_TARG) { isp_prt(isp, ISP_LOGWARN, "Chan %d trying to abort bad target %d", chan, tgt); break; } lp = &fcp->portdb[tgt]; if (lp->is_target == 0 || lp->state != FC_PORTDB_STATE_VALID) { isp_prt(isp, ISP_LOGWARN, "Chan %d abort of no longer valid target %d", chan, tgt); break; } isp_prt(isp, ISP_LOGALL, "Chan %d Abort Cmd for N-Port 0x%04x @ Port 0x%06x", chan, lp->handle, lp->portid); ISP_MEMZERO(ab, QENTRY_LEN); ab->abrt_header.rqs_entry_type = RQSTYPE_ABORT_IO; ab->abrt_header.rqs_entry_count = 1; ab->abrt_handle = lp->handle; ab->abrt_cmd_handle = handle; ab->abrt_tidlo = lp->portid; ab->abrt_tidhi = lp->portid >> 16; ab->abrt_vpidx = ISP_GET_VPIDX(isp, chan); ISP_MEMZERO(&mbs, sizeof (mbs)); MBSINIT(&mbs, MBOX_EXEC_COMMAND_IOCB_A64, MBLOGALL, 5000000); mbs.param[1] = QENTRY_LEN; mbs.param[2] = DMA_WD1(fcp->isp_scdma); mbs.param[3] = DMA_WD0(fcp->isp_scdma); mbs.param[6] = DMA_WD3(fcp->isp_scdma); mbs.param[7] = DMA_WD2(fcp->isp_scdma); if (FC_SCRATCH_ACQUIRE(isp, chan)) { isp_prt(isp, ISP_LOGERR, sacq); break; } isp_put_24xx_abrt(isp, ab, fcp->isp_scratch); ab2 = (isp24xx_abrt_t *) &((uint8_t *)fcp->isp_scratch)[QENTRY_LEN]; ab2->abrt_nphdl = 0xdeaf; MEMORYBARRIER(isp, SYNC_SFORDEV, 0, 2 * QENTRY_LEN, chan); isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { FC_SCRATCH_RELEASE(isp, chan); break; } MEMORYBARRIER(isp, SYNC_SFORCPU, QENTRY_LEN, QENTRY_LEN, chan); isp_get_24xx_abrt(isp, ab2, ab); FC_SCRATCH_RELEASE(isp, chan); if (ab->abrt_nphdl == ISP24XX_ABRT_OKAY) { return (0); } isp_prt(isp, ISP_LOGWARN, "Chan %d handle %d abort returned 0x%x", chan, tgt, ab->abrt_nphdl); break; } else if (IS_FC(isp)) { if (ISP_CAP_SCCFW(isp)) { if (ISP_CAP_2KLOGIN(isp)) { mbs.param[1] = tgt; } else { mbs.param[1] = tgt << 8; } mbs.param[6] = XS_LUN(xs); } else { mbs.param[1] = tgt << 8 | XS_LUN(xs); } } else { mbs.param[1] = (chan << 15) | (tgt << 8) | XS_LUN(xs); } MBSINIT(&mbs, MBOX_ABORT, MBLOGALL & ~MBLOGMASK(MBOX_COMMAND_ERROR), 0); mbs.param[2] = handle; isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { break; } return (0); case ISPCTL_UPDATE_PARAMS: va_start(ap, ctl); chan = va_arg(ap, int); va_end(ap); isp_spi_update(isp, chan); return (0); case ISPCTL_FCLINK_TEST: if (IS_FC(isp)) { int usdelay; va_start(ap, ctl); chan = va_arg(ap, int); usdelay = va_arg(ap, int); va_end(ap); if (usdelay == 0) { usdelay = 250000; } return (isp_fclink_test(isp, chan, usdelay)); } break; case ISPCTL_SCAN_FABRIC: if (IS_FC(isp)) { va_start(ap, ctl); chan = va_arg(ap, int); va_end(ap); return (isp_scan_fabric(isp, chan)); } break; case ISPCTL_SCAN_LOOP: if (IS_FC(isp)) { va_start(ap, ctl); chan = va_arg(ap, int); va_end(ap); return (isp_scan_loop(isp, chan)); } break; case ISPCTL_PDB_SYNC: if (IS_FC(isp)) { va_start(ap, ctl); chan = va_arg(ap, int); va_end(ap); return (isp_pdb_sync(isp, chan)); } break; case ISPCTL_SEND_LIP: if (IS_FC(isp) && !IS_24XX(isp)) { MBSINIT(&mbs, MBOX_INIT_LIP, MBLOGALL, 0); if (ISP_CAP_2KLOGIN(isp)) { mbs.ibits = (1 << 10); } isp_mboxcmd(isp, &mbs); if (mbs.param[0] == MBOX_COMMAND_COMPLETE) { return (0); } } break; case ISPCTL_GET_PDB: if (IS_FC(isp)) { isp_pdb_t *pdb; va_start(ap, ctl); chan = va_arg(ap, int); tgt = va_arg(ap, int); pdb = va_arg(ap, isp_pdb_t *); va_end(ap); return (isp_getpdb(isp, chan, tgt, pdb, 1)); } break; case ISPCTL_GET_NAMES: { uint64_t *wwnn, *wwnp; va_start(ap, ctl); chan = va_arg(ap, int); tgt = va_arg(ap, int); wwnn = va_arg(ap, uint64_t *); wwnp = va_arg(ap, uint64_t *); va_end(ap); if (wwnn == NULL && wwnp == NULL) { break; } if (wwnn) { *wwnn = isp_get_wwn(isp, chan, tgt, 1); if (*wwnn == INI_NONE) { break; } } if (wwnp) { *wwnp = isp_get_wwn(isp, chan, tgt, 0); if (*wwnp == INI_NONE) { break; } } return (0); } case ISPCTL_RUN_MBOXCMD: { va_start(ap, ctl); mbr = va_arg(ap, mbreg_t *); va_end(ap); isp_mboxcmd(isp, mbr); return (0); } case ISPCTL_PLOGX: { isp_plcmd_t *p; int r; va_start(ap, ctl); p = va_arg(ap, isp_plcmd_t *); va_end(ap); if ((p->flags & PLOGX_FLG_CMD_MASK) != PLOGX_FLG_CMD_PLOGI || (p->handle != NIL_HANDLE)) { return (isp_plogx(isp, p->channel, p->handle, p->portid, p->flags, 0)); } do { isp_next_handle(isp, &p->handle); r = isp_plogx(isp, p->channel, p->handle, p->portid, p->flags, 0); if ((r & 0xffff) == MBOX_PORT_ID_USED) { p->handle = r >> 16; r = 0; break; } } while ((r & 0xffff) == MBOX_LOOP_ID_USED); return (r); } case ISPCTL_CHANGE_ROLE: if (IS_FC(isp)) { int role, r; va_start(ap, ctl); chan = va_arg(ap, int); role = va_arg(ap, int); va_end(ap); r = isp_fc_change_role(isp, chan, role); return (r); } break; default: isp_prt(isp, ISP_LOGERR, "Unknown Control Opcode 0x%x", ctl); break; } return (-1); } /* * Interrupt Service Routine(s). * * External (OS) framework has done the appropriate locking, * and the locking will be held throughout this function. */ /* * Limit our stack depth by sticking with the max likely number * of completions on a request queue at any one time. */ #ifndef MAX_REQUESTQ_COMPLETIONS #define MAX_REQUESTQ_COMPLETIONS 32 #endif void isp_intr(ispsoftc_t *isp, uint16_t isr, uint16_t sema, uint16_t info) { XS_T *complist[MAX_REQUESTQ_COMPLETIONS], *xs; uint32_t iptr, optr, junk; int i, nlooked = 0, ndone = 0, continuations_expected = 0; int etype, last_etype = 0; again: /* * Is this a mailbox related interrupt? * The mailbox semaphore will be nonzero if so. */ if (sema) { fmbox: if (info & MBOX_COMMAND_COMPLETE) { isp->isp_intmboxc++; if (isp->isp_mboxbsy) { int obits = isp->isp_obits; isp->isp_mboxtmp[0] = info; for (i = 1; i < ISP_NMBOX(isp); i++) { if ((obits & (1 << i)) == 0) { continue; } isp->isp_mboxtmp[i] = ISP_READ(isp, MBOX_OFF(i)); } if (isp->isp_mbxwrk0) { if (isp_mbox_continue(isp) == 0) { return; } } MBOX_NOTIFY_COMPLETE(isp); } else { isp_prt(isp, ISP_LOGWARN, "mailbox cmd (0x%x) with no waiters", info); } } else { i = IS_FC(isp)? isp_parse_async_fc(isp, info) : isp_parse_async(isp, info); if (i < 0) { return; } } if ((IS_FC(isp) && info != ASYNC_RIOZIO_STALL) || isp->isp_state != ISP_RUNSTATE) { goto out; } } /* * We can't be getting this now. */ if (isp->isp_state != ISP_RUNSTATE) { /* * This seems to happen to 23XX and 24XX cards- don't know why. */ if (isp->isp_mboxbsy && isp->isp_lastmbxcmd == MBOX_ABOUT_FIRMWARE) { goto fmbox; } isp_prt(isp, ISP_LOGINFO, "interrupt (ISR=%x SEMA=%x INFO=%x) " "when not ready", isr, sema, info); /* * Thank you very much! *Burrrp*! */ isp->isp_residx = ISP_READ(isp, isp->isp_respinrp); isp->isp_resodx = isp->isp_residx; ISP_WRITE(isp, isp->isp_respoutrp, isp->isp_resodx); if (IS_24XX(isp)) { ISP_DISABLE_INTS(isp); } goto out; } #ifdef ISP_TARGET_MODE /* * Check for ATIO Queue entries. */ if (IS_24XX(isp) && (isr == ISPR2HST_ATIO_UPDATE || isr == ISPR2HST_ATIO_RSPQ_UPDATE || isr == ISPR2HST_ATIO_UPDATE2)) { iptr = ISP_READ(isp, BIU2400_ATIO_RSPINP); optr = isp->isp_atioodx; while (optr != iptr) { uint8_t qe[QENTRY_LEN]; isphdr_t *hp; uint32_t oop; void *addr; oop = optr; MEMORYBARRIER(isp, SYNC_ATIOQ, oop, QENTRY_LEN, -1); addr = ISP_QUEUE_ENTRY(isp->isp_atioq, oop); isp_get_hdr(isp, addr, (isphdr_t *)qe); hp = (isphdr_t *)qe; switch (hp->rqs_entry_type) { case RQSTYPE_NOTIFY: case RQSTYPE_ATIO: (void) isp_target_notify(isp, addr, &oop); break; default: isp_print_qentry(isp, "?ATIOQ entry?", oop, addr); break; } optr = ISP_NXT_QENTRY(oop, RESULT_QUEUE_LEN(isp)); } if (isp->isp_atioodx != optr) { ISP_WRITE(isp, BIU2400_ATIO_RSPOUTP, optr); isp->isp_atioodx = optr; } } #endif /* * You *must* read the Response Queue In Pointer * prior to clearing the RISC interrupt. * * Debounce the 2300 if revision less than 2. */ if (IS_2100(isp) || (IS_2300(isp) && isp->isp_revision < 2)) { i = 0; do { iptr = ISP_READ(isp, isp->isp_respinrp); junk = ISP_READ(isp, isp->isp_respinrp); } while (junk != iptr && ++i < 1000); if (iptr != junk) { isp_prt(isp, ISP_LOGWARN, "Response Queue Out Pointer Unstable (%x, %x)", iptr, junk); goto out; } } else { iptr = ISP_READ(isp, isp->isp_respinrp); } optr = isp->isp_resodx; if (optr == iptr && sema == 0) { /* * There are a lot of these- reasons unknown- mostly on * faster Alpha machines. * * I tried delaying after writing HCCR_CMD_CLEAR_RISC_INT to * make sure the old interrupt went away (to avoid 'ringing' * effects), but that didn't stop this from occurring. */ if (IS_24XX(isp)) { junk = 0; } else if (IS_23XX(isp)) { ISP_DELAY(100); iptr = ISP_READ(isp, isp->isp_respinrp); junk = ISP_READ(isp, BIU_R2HSTSLO); } else { junk = ISP_READ(isp, BIU_ISR); } if (optr == iptr) { if (IS_23XX(isp) || IS_24XX(isp)) { ; } else { sema = ISP_READ(isp, BIU_SEMA); info = ISP_READ(isp, OUTMAILBOX0); if ((sema & 0x3) && (info & 0x8000)) { goto again; } } isp->isp_intbogus++; isp_prt(isp, ISP_LOGDEBUG1, "bogus intr- isr %x (%x) iptr %x optr %x", isr, junk, iptr, optr); } } isp->isp_residx = iptr; while (optr != iptr) { uint8_t qe[QENTRY_LEN]; ispstatusreq_t *sp = (ispstatusreq_t *) qe; isphdr_t *hp; int buddaboom, scsi_status, completion_status; int req_status_flags, req_state_flags; uint8_t *snsp, *resp; uint32_t rlen, slen, totslen; long resid; uint16_t oop; hp = (isphdr_t *) ISP_QUEUE_ENTRY(isp->isp_result, optr); oop = optr; optr = ISP_NXT_QENTRY(optr, RESULT_QUEUE_LEN(isp)); nlooked++; read_again: buddaboom = req_status_flags = req_state_flags = 0; resid = 0L; /* * Synchronize our view of this response queue entry. */ MEMORYBARRIER(isp, SYNC_RESULT, oop, QENTRY_LEN, -1); isp_get_hdr(isp, hp, &sp->req_header); etype = sp->req_header.rqs_entry_type; if (IS_24XX(isp) && etype == RQSTYPE_RESPONSE) { isp24xx_statusreq_t *sp2 = (isp24xx_statusreq_t *)qe; isp_get_24xx_response(isp, (isp24xx_statusreq_t *)hp, sp2); if (isp->isp_dblev & ISP_LOGDEBUG1) { isp_print_bytes(isp, "Response Queue Entry", QENTRY_LEN, sp2); } scsi_status = sp2->req_scsi_status; completion_status = sp2->req_completion_status; if ((scsi_status & 0xff) != 0) req_state_flags = RQSF_GOT_STATUS; else req_state_flags = 0; resid = sp2->req_resid; } else if (etype == RQSTYPE_RESPONSE) { isp_get_response(isp, (ispstatusreq_t *) hp, sp); if (isp->isp_dblev & ISP_LOGDEBUG1) { isp_print_bytes(isp, "Response Queue Entry", QENTRY_LEN, sp); } scsi_status = sp->req_scsi_status; completion_status = sp->req_completion_status; req_status_flags = sp->req_status_flags; req_state_flags = sp->req_state_flags; resid = sp->req_resid; } else if (etype == RQSTYPE_RIO1) { isp_rio1_t *rio = (isp_rio1_t *) qe; isp_get_rio1(isp, (isp_rio1_t *) hp, rio); if (isp->isp_dblev & ISP_LOGDEBUG1) { isp_print_bytes(isp, "Response Queue Entry", QENTRY_LEN, rio); } for (i = 0; i < rio->req_header.rqs_seqno; i++) { isp_fastpost_complete(isp, rio->req_handles[i]); } if (isp->isp_fpcchiwater < rio->req_header.rqs_seqno) { isp->isp_fpcchiwater = rio->req_header.rqs_seqno; } ISP_MEMZERO(hp, QENTRY_LEN); /* PERF */ last_etype = etype; continue; } else if (etype == RQSTYPE_RIO2) { isp_prt(isp, ISP_LOGERR, "dropping RIO2 response"); ISP_MEMZERO(hp, QENTRY_LEN); /* PERF */ last_etype = etype; continue; } else if (etype == RQSTYPE_STATUS_CONT) { isp_get_cont_response(isp, (ispstatus_cont_t *) hp, (ispstatus_cont_t *) sp); if (last_etype == RQSTYPE_RESPONSE && continuations_expected && ndone > 0 && (xs = complist[ndone-1]) != NULL) { ispstatus_cont_t *scp = (ispstatus_cont_t *) sp; XS_SENSE_APPEND(xs, scp->req_sense_data, sizeof (scp->req_sense_data)); isp_prt(isp, ISP_LOGDEBUG0|ISP_LOG_CWARN, "%d more Status Continuations expected", --continuations_expected); } else { isp_prt(isp, ISP_LOG_WARN1, "Ignored Continuation Response"); } ISP_MEMZERO(hp, QENTRY_LEN); /* PERF */ continue; } else { /* * Somebody reachable via isp_handle_other_response * may have updated the response queue pointers for * us, so we reload our goal index. */ int r; uint32_t tsto = oop; r = isp_handle_other_response(isp, etype, hp, &tsto); if (r < 0) { goto read_again; } /* * If somebody updated the output pointer, then reset * optr to be one more than the updated amount. */ while (tsto != oop) { optr = ISP_NXT_QENTRY(tsto, RESULT_QUEUE_LEN(isp)); } if (r > 0) { ISP_MEMZERO(hp, QENTRY_LEN); /* PERF */ last_etype = etype; continue; } /* * After this point, we'll just look at the header as * we don't know how to deal with the rest of the * response. */ /* * It really has to be a bounced request just copied * from the request queue to the response queue. If * not, something bad has happened. */ if (etype != RQSTYPE_REQUEST) { isp_prt(isp, ISP_LOGERR, notresp, etype, oop, optr, nlooked); isp_print_bytes(isp, "Request Queue Entry", QENTRY_LEN, sp); ISP_MEMZERO(hp, QENTRY_LEN); /* PERF */ last_etype = etype; continue; } buddaboom = 1; scsi_status = sp->req_scsi_status; completion_status = sp->req_completion_status; req_status_flags = sp->req_status_flags; req_state_flags = sp->req_state_flags; resid = sp->req_resid; } if (sp->req_header.rqs_flags & RQSFLAG_MASK) { if (sp->req_header.rqs_flags & RQSFLAG_CONTINUATION) { isp_print_bytes(isp, "unexpected continuation segment", QENTRY_LEN, sp); last_etype = etype; continue; } if (sp->req_header.rqs_flags & RQSFLAG_FULL) { isp_prt(isp, ISP_LOG_WARN1, "internal queues full"); /* * We'll synthesize a QUEUE FULL message below. */ } if (sp->req_header.rqs_flags & RQSFLAG_BADHEADER) { isp_print_bytes(isp, "bad header flag", QENTRY_LEN, sp); buddaboom++; } if (sp->req_header.rqs_flags & RQSFLAG_BADPACKET) { isp_print_bytes(isp, "bad request packet", QENTRY_LEN, sp); buddaboom++; } if (sp->req_header.rqs_flags & RQSFLAG_BADCOUNT) { isp_print_bytes(isp, "invalid entry count", QENTRY_LEN, sp); buddaboom++; } if (sp->req_header.rqs_flags & RQSFLAG_BADORDER) { isp_print_bytes(isp, "invalid IOCB ordering", QENTRY_LEN, sp); last_etype = etype; continue; } } if (!ISP_VALID_HANDLE(isp, sp->req_handle)) { isp_prt(isp, ISP_LOGERR, "bad request handle 0x%x (iocb type 0x%x)", sp->req_handle, etype); ISP_MEMZERO(hp, QENTRY_LEN); /* PERF */ last_etype = etype; continue; } xs = isp_find_xs(isp, sp->req_handle); if (xs == NULL) { uint8_t ts = completion_status & 0xff; /* * Only whine if this isn't the expected fallout of * aborting the command or resetting the target. */ if (etype != RQSTYPE_RESPONSE) { isp_prt(isp, ISP_LOGERR, "cannot find handle 0x%x (type 0x%x)", sp->req_handle, etype); } else if (ts != RQCS_ABORTED && ts != RQCS_RESET_OCCURRED) { isp_prt(isp, ISP_LOGERR, "cannot find handle 0x%x (status 0x%x)", sp->req_handle, ts); } ISP_MEMZERO(hp, QENTRY_LEN); /* PERF */ last_etype = etype; continue; } if (req_status_flags & RQSTF_BUS_RESET) { isp_prt(isp, ISP_LOG_WARN1, "%d.%d.%jx bus was reset", XS_CHANNEL(xs), XS_TGT(xs), (uintmax_t)XS_LUN(xs)); XS_SETERR(xs, HBA_BUSRESET); ISP_SET_SENDMARKER(isp, XS_CHANNEL(xs), 1); } if (buddaboom) { isp_prt(isp, ISP_LOG_WARN1, "%d.%d.%jx buddaboom", XS_CHANNEL(xs), XS_TGT(xs), (uintmax_t)XS_LUN(xs)); XS_SETERR(xs, HBA_BOTCH); } resp = NULL; rlen = 0; snsp = NULL; totslen = slen = 0; if (IS_24XX(isp) && (scsi_status & (RQCS_RV|RQCS_SV)) != 0) { resp = ((isp24xx_statusreq_t *)sp)->req_rsp_sense; rlen = ((isp24xx_statusreq_t *)sp)->req_response_len; } else if (IS_FC(isp) && (scsi_status & RQCS_RV) != 0) { resp = sp->req_response; rlen = sp->req_response_len; } if (IS_FC(isp) && (scsi_status & RQCS_SV) != 0) { /* * Fibre Channel F/W doesn't say we got status * if there's Sense Data instead. I guess they * think it goes w/o saying. */ req_state_flags |= RQSF_GOT_STATUS|RQSF_GOT_SENSE; if (IS_24XX(isp)) { snsp = ((isp24xx_statusreq_t *)sp)->req_rsp_sense; snsp += rlen; totslen = ((isp24xx_statusreq_t *)sp)->req_sense_len; slen = (sizeof (((isp24xx_statusreq_t *)sp)->req_rsp_sense)) - rlen; if (totslen < slen) slen = totslen; } else { snsp = sp->req_sense_data; totslen = sp->req_sense_len; slen = sizeof (sp->req_sense_data); if (totslen < slen) slen = totslen; } } else if (IS_SCSI(isp) && (req_state_flags & RQSF_GOT_SENSE)) { snsp = sp->req_sense_data; totslen = sp->req_sense_len; slen = sizeof (sp->req_sense_data); if (totslen < slen) slen = totslen; } if (req_state_flags & RQSF_GOT_STATUS) { *XS_STSP(xs) = scsi_status & 0xff; } switch (etype) { case RQSTYPE_RESPONSE: if (resp && rlen >= 4 && resp[FCP_RSPNS_CODE_OFFSET] != 0) { const char *ptr; char lb[64]; const char *rnames[10] = { "Task Management function complete", "FCP_DATA length different than FCP_BURST_LEN", "FCP_CMND fields invalid", "FCP_DATA parameter mismatch with FCP_DATA_RO", "Task Management function rejected", "Task Management function failed", NULL, NULL, "Task Management function succeeded", "Task Management function incorrect logical unit number", }; uint8_t code = resp[FCP_RSPNS_CODE_OFFSET]; if (code >= 10 || rnames[code] == NULL) { ISP_SNPRINTF(lb, sizeof(lb), "Unknown FCP Response Code 0x%x", code); ptr = lb; } else { ptr = rnames[code]; } isp_xs_prt(isp, xs, ISP_LOGWARN, "FCP RESPONSE, LENGTH %u: %s CDB0=0x%02x", rlen, ptr, XS_CDBP(xs)[0] & 0xff); if (code != 0 && code != 8) XS_SETERR(xs, HBA_BOTCH); } if (IS_24XX(isp)) { isp_parse_status_24xx(isp, (isp24xx_statusreq_t *)sp, xs, &resid); } else { isp_parse_status(isp, (void *)sp, xs, &resid); } if ((XS_NOERR(xs) || XS_ERR(xs) == HBA_NOERROR) && (*XS_STSP(xs) == SCSI_BUSY)) { XS_SETERR(xs, HBA_TGTBSY); } if (IS_SCSI(isp)) { XS_SET_RESID(xs, resid); /* * A new synchronous rate was negotiated for * this target. Mark state such that we'll go * look up that which has changed later. */ if (req_status_flags & RQSTF_NEGOTIATION) { int t = XS_TGT(xs); sdparam *sdp = SDPARAM(isp, XS_CHANNEL(xs)); sdp->isp_devparam[t].dev_refresh = 1; sdp->update = 1; } } else { if (req_status_flags & RQSF_XFER_COMPLETE) { XS_SET_RESID(xs, 0); } else if (scsi_status & RQCS_RESID) { XS_SET_RESID(xs, resid); } else { XS_SET_RESID(xs, 0); } } if (snsp && slen) { if (totslen > slen) { continuations_expected += ((totslen - slen + QENTRY_LEN - 5) / (QENTRY_LEN - 4)); if (ndone > (MAX_REQUESTQ_COMPLETIONS - continuations_expected - 1)) { /* we'll lose some stats, but that's a small price to pay */ for (i = 0; i < ndone; i++) { if (complist[i]) { isp->isp_rsltccmplt++; isp_done(complist[i]); } } ndone = 0; } isp_prt(isp, ISP_LOGDEBUG0|ISP_LOG_CWARN, "Expecting %d more Status Continuations for total sense length of %u", continuations_expected, totslen); } XS_SAVE_SENSE(xs, snsp, totslen, slen); } else if ((req_status_flags & RQSF_GOT_STATUS) && (scsi_status & 0xff) == SCSI_CHECK && IS_FC(isp)) { isp_prt(isp, ISP_LOGWARN, "CHECK CONDITION w/o sense data for CDB=0x%x", XS_CDBP(xs)[0] & 0xff); isp_print_bytes(isp, "CC with no Sense", QENTRY_LEN, qe); } isp_prt(isp, ISP_LOGDEBUG2, "asked for %ld got raw resid %ld settled for %ld", (long) XS_XFRLEN(xs), resid, (long) XS_GET_RESID(xs)); break; case RQSTYPE_REQUEST: case RQSTYPE_A64: case RQSTYPE_T2RQS: case RQSTYPE_T3RQS: case RQSTYPE_T7RQS: if (!IS_24XX(isp) && (sp->req_header.rqs_flags & RQSFLAG_FULL)) { /* * Force Queue Full status. */ *XS_STSP(xs) = SCSI_QFULL; XS_SETERR(xs, HBA_NOERROR); } else if (XS_NOERR(xs)) { isp_prt(isp, ISP_LOG_WARN1, "%d.%d.%jx badness at %s:%u", XS_CHANNEL(xs), XS_TGT(xs), (uintmax_t)XS_LUN(xs), __func__, __LINE__); XS_SETERR(xs, HBA_BOTCH); } XS_SET_RESID(xs, XS_XFRLEN(xs)); break; default: isp_print_bytes(isp, "Unhandled Response Type", QENTRY_LEN, qe); if (XS_NOERR(xs)) { XS_SETERR(xs, HBA_BOTCH); } break; } /* * Free any DMA resources. As a side effect, this may * also do any cache flushing necessary for data coherence. */ if (XS_XFRLEN(xs)) { ISP_DMAFREE(isp, xs, sp->req_handle); } isp_destroy_handle(isp, sp->req_handle); if (isp->isp_nactive > 0) { isp->isp_nactive--; } complist[ndone++] = xs; /* defer completion call until later */ ISP_MEMZERO(hp, QENTRY_LEN); /* PERF */ last_etype = etype; if (ndone == MAX_REQUESTQ_COMPLETIONS) { break; } } /* * If we looked at any commands, then it's valid to find out * what the outpointer is. It also is a trigger to update the * ISP's notion of what we've seen so far. */ if (nlooked) { ISP_WRITE(isp, isp->isp_respoutrp, optr); isp->isp_resodx = optr; if (isp->isp_rscchiwater < ndone) isp->isp_rscchiwater = ndone; } out: if (IS_24XX(isp)) { ISP_WRITE(isp, BIU2400_HCCR, HCCR_2400_CMD_CLEAR_RISC_INT); } else { ISP_WRITE(isp, HCCR, HCCR_CMD_CLEAR_RISC_INT); ISP_WRITE(isp, BIU_SEMA, 0); } for (i = 0; i < ndone; i++) { xs = complist[i]; if (xs) { if (((isp->isp_dblev & (ISP_LOGDEBUG1|ISP_LOGDEBUG2|ISP_LOGDEBUG3))) || ((isp->isp_dblev & (ISP_LOGDEBUG0|ISP_LOG_CWARN) && ((!XS_NOERR(xs)) || (*XS_STSP(xs) != SCSI_GOOD))))) { isp_prt_endcmd(isp, xs); } isp->isp_rsltccmplt++; isp_done(xs); } } } /* * Support routines. */ void isp_prt_endcmd(ispsoftc_t *isp, XS_T *xs) { char cdbstr[16 * 5 + 1]; int i, lim; lim = XS_CDBLEN(xs) > 16? 16 : XS_CDBLEN(xs); ISP_SNPRINTF(cdbstr, sizeof (cdbstr), "0x%02x ", XS_CDBP(xs)[0]); for (i = 1; i < lim; i++) { ISP_SNPRINTF(cdbstr, sizeof (cdbstr), "%s0x%02x ", cdbstr, XS_CDBP(xs)[i]); } if (XS_SENSE_VALID(xs)) { isp_xs_prt(isp, xs, ISP_LOGALL, "FIN dl%d resid %ld CDB=%s SenseLength=%u/%u KEY/ASC/ASCQ=0x%02x/0x%02x/0x%02x", XS_XFRLEN(xs), (long) XS_GET_RESID(xs), cdbstr, XS_CUR_SNSLEN(xs), XS_TOT_SNSLEN(xs), XS_SNSKEY(xs), XS_SNSASC(xs), XS_SNSASCQ(xs)); } else { isp_xs_prt(isp, xs, ISP_LOGALL, "FIN dl%d resid %ld CDB=%s STS 0x%x XS_ERR=0x%x", XS_XFRLEN(xs), (long) XS_GET_RESID(xs), cdbstr, *XS_STSP(xs), XS_ERR(xs)); } } /* * Parse an ASYNC mailbox complete * * Return non-zero if the event has been acknowledged. */ static int isp_parse_async(ispsoftc_t *isp, uint16_t mbox) { int acked = 0; uint32_t h1 = 0, h2 = 0; uint16_t chan = 0; /* * Pick up the channel, but not if this is a ASYNC_RIO32_2, * where Mailboxes 6/7 have the second handle. */ if (mbox != ASYNC_RIO32_2) { if (IS_DUALBUS(isp)) { chan = ISP_READ(isp, OUTMAILBOX6); } } isp_prt(isp, ISP_LOGDEBUG2, "Async Mbox 0x%x", mbox); switch (mbox) { case ASYNC_BUS_RESET: ISP_SET_SENDMARKER(isp, chan, 1); #ifdef ISP_TARGET_MODE if (isp_target_async(isp, chan, mbox)) { acked = 1; } #endif isp_async(isp, ISPASYNC_BUS_RESET, chan); break; case ASYNC_SYSTEM_ERROR: isp->isp_dead = 1; isp->isp_state = ISP_CRASHED; /* * Were we waiting for a mailbox command to complete? * If so, it's dead, so wake up the waiter. */ if (isp->isp_mboxbsy) { isp->isp_obits = 1; isp->isp_mboxtmp[0] = MBOX_HOST_INTERFACE_ERROR; MBOX_NOTIFY_COMPLETE(isp); } /* * It's up to the handler for isp_async to reinit stuff and * restart the firmware */ isp_async(isp, ISPASYNC_FW_CRASH); acked = 1; break; case ASYNC_RQS_XFER_ERR: isp_prt(isp, ISP_LOGERR, "Request Queue Transfer Error"); break; case ASYNC_RSP_XFER_ERR: isp_prt(isp, ISP_LOGERR, "Response Queue Transfer Error"); break; case ASYNC_QWAKEUP: /* * We've just been notified that the Queue has woken up. * We don't need to be chatty about this- just unlatch things * and move on. */ mbox = ISP_READ(isp, isp->isp_rqstoutrp); break; case ASYNC_TIMEOUT_RESET: isp_prt(isp, ISP_LOGWARN, "timeout initiated SCSI bus reset of chan %d", chan); ISP_SET_SENDMARKER(isp, chan, 1); #ifdef ISP_TARGET_MODE if (isp_target_async(isp, chan, mbox)) { acked = 1; } #endif break; case ASYNC_DEVICE_RESET: isp_prt(isp, ISP_LOGINFO, "device reset on chan %d", chan); ISP_SET_SENDMARKER(isp, chan, 1); #ifdef ISP_TARGET_MODE if (isp_target_async(isp, chan, mbox)) { acked = 1; } #endif break; case ASYNC_EXTMSG_UNDERRUN: isp_prt(isp, ISP_LOGWARN, "extended message underrun"); break; case ASYNC_SCAM_INT: isp_prt(isp, ISP_LOGINFO, "SCAM interrupt"); break; case ASYNC_HUNG_SCSI: isp_prt(isp, ISP_LOGERR, "stalled SCSI Bus after DATA Overrun"); /* XXX: Need to issue SCSI reset at this point */ break; case ASYNC_KILLED_BUS: isp_prt(isp, ISP_LOGERR, "SCSI Bus reset after DATA Overrun"); break; case ASYNC_BUS_TRANSIT: mbox = ISP_READ(isp, OUTMAILBOX2); switch (mbox & SXP_PINS_MODE_MASK) { case SXP_PINS_LVD_MODE: isp_prt(isp, ISP_LOGINFO, "Transition to LVD mode"); SDPARAM(isp, chan)->isp_diffmode = 0; SDPARAM(isp, chan)->isp_ultramode = 0; SDPARAM(isp, chan)->isp_lvdmode = 1; break; case SXP_PINS_HVD_MODE: isp_prt(isp, ISP_LOGINFO, "Transition to Differential mode"); SDPARAM(isp, chan)->isp_diffmode = 1; SDPARAM(isp, chan)->isp_ultramode = 0; SDPARAM(isp, chan)->isp_lvdmode = 0; break; case SXP_PINS_SE_MODE: isp_prt(isp, ISP_LOGINFO, "Transition to Single Ended mode"); SDPARAM(isp, chan)->isp_diffmode = 0; SDPARAM(isp, chan)->isp_ultramode = 1; SDPARAM(isp, chan)->isp_lvdmode = 0; break; default: isp_prt(isp, ISP_LOGWARN, "Transition to Unknown Mode 0x%x", mbox); break; } /* * XXX: Set up to renegotiate again! */ /* Can only be for a 1080... */ ISP_SET_SENDMARKER(isp, chan, 1); break; case ASYNC_CMD_CMPLT: case ASYNC_RIO32_1: if (!IS_ULTRA3(isp)) { isp_prt(isp, ISP_LOGERR, "unexpected fast posting completion"); break; } /* FALLTHROUGH */ h1 = (ISP_READ(isp, OUTMAILBOX2) << 16) | ISP_READ(isp, OUTMAILBOX1); break; case ASYNC_RIO32_2: h1 = (ISP_READ(isp, OUTMAILBOX2) << 16) | ISP_READ(isp, OUTMAILBOX1); h2 = (ISP_READ(isp, OUTMAILBOX7) << 16) | ISP_READ(isp, OUTMAILBOX6); break; case ASYNC_RIO16_5: case ASYNC_RIO16_4: case ASYNC_RIO16_3: case ASYNC_RIO16_2: case ASYNC_RIO16_1: isp_prt(isp, ISP_LOGERR, "unexpected 16 bit RIO handle"); break; default: isp_prt(isp, ISP_LOGWARN, "%s: unhandled async code 0x%x", __func__, mbox); break; } if (h1 || h2) { isp_prt(isp, ISP_LOGDEBUG3, "fast post/rio completion of 0x%08x", h1); isp_fastpost_complete(isp, h1); if (h2) { isp_prt(isp, ISP_LOGDEBUG3, "fast post/rio completion of 0x%08x", h2); isp_fastpost_complete(isp, h2); if (isp->isp_fpcchiwater < 2) { isp->isp_fpcchiwater = 2; } } else { if (isp->isp_fpcchiwater < 1) { isp->isp_fpcchiwater = 1; } } } else { isp->isp_intoasync++; } return (acked); } static int isp_parse_async_fc(ispsoftc_t *isp, uint16_t mbox) { fcparam *fcp; int acked = 0; uint16_t chan; if (IS_DUALBUS(isp)) { chan = ISP_READ(isp, OUTMAILBOX6); } else { chan = 0; } isp_prt(isp, ISP_LOGDEBUG2, "Async Mbox 0x%x", mbox); switch (mbox) { case ASYNC_SYSTEM_ERROR: isp->isp_dead = 1; isp->isp_state = ISP_CRASHED; FCPARAM(isp, chan)->isp_loopstate = LOOP_NIL; isp_change_fw_state(isp, chan, FW_CONFIG_WAIT); /* * Were we waiting for a mailbox command to complete? * If so, it's dead, so wake up the waiter. */ if (isp->isp_mboxbsy) { isp->isp_obits = 1; isp->isp_mboxtmp[0] = MBOX_HOST_INTERFACE_ERROR; MBOX_NOTIFY_COMPLETE(isp); } /* * It's up to the handler for isp_async to reinit stuff and * restart the firmware */ isp_async(isp, ISPASYNC_FW_CRASH); acked = 1; break; case ASYNC_RQS_XFER_ERR: isp_prt(isp, ISP_LOGERR, "Request Queue Transfer Error"); break; case ASYNC_RSP_XFER_ERR: isp_prt(isp, ISP_LOGERR, "Response Queue Transfer Error"); break; case ASYNC_QWAKEUP: #ifdef ISP_TARGET_MODE if (IS_24XX(isp)) { isp_prt(isp, ISP_LOGERR, "ATIO Queue Transfer Error"); break; } #endif isp_prt(isp, ISP_LOGERR, "%s: unexpected ASYNC_QWAKEUP code", __func__); break; case ASYNC_CMD_CMPLT: isp_fastpost_complete(isp, (ISP_READ(isp, OUTMAILBOX2) << 16) | ISP_READ(isp, OUTMAILBOX1)); if (isp->isp_fpcchiwater < 1) { isp->isp_fpcchiwater = 1; } break; case ASYNC_RIOZIO_STALL: break; case ASYNC_CTIO_DONE: #ifdef ISP_TARGET_MODE if (isp_target_async(isp, (ISP_READ(isp, OUTMAILBOX2) << 16) | ISP_READ(isp, OUTMAILBOX1), mbox)) { acked = 1; } else { isp->isp_fphccmplt++; } #else isp_prt(isp, ISP_LOGWARN, "unexpected ASYNC CTIO done"); #endif break; case ASYNC_LIP_ERROR: case ASYNC_LIP_NOS_OLS_RECV: case ASYNC_LIP_OCCURRED: case ASYNC_PTPMODE: /* * These are broadcast events that have to be sent across * all active channels. */ for (chan = 0; chan < isp->isp_nchan; chan++) { fcp = FCPARAM(isp, chan); int topo = fcp->isp_topo; if (fcp->role == ISP_ROLE_NONE) continue; if (fcp->isp_loopstate > LOOP_HAVE_LINK) fcp->isp_loopstate = LOOP_HAVE_LINK; ISP_SET_SENDMARKER(isp, chan, 1); isp_async(isp, ISPASYNC_LIP, chan); #ifdef ISP_TARGET_MODE if (isp_target_async(isp, chan, mbox)) { acked = 1; } #endif /* * We've had problems with data corruption occuring on * commands that complete (with no apparent error) after * we receive a LIP. This has been observed mostly on * Local Loop topologies. To be safe, let's just mark * all active initiator commands as dead. */ if (topo == TOPO_NL_PORT || topo == TOPO_FL_PORT) { int i, j; for (i = j = 0; i < isp->isp_maxcmds; i++) { XS_T *xs; isp_hdl_t *hdp; hdp = &isp->isp_xflist[i]; if (ISP_H2HT(hdp->handle) != ISP_HANDLE_INITIATOR) { continue; } xs = hdp->cmd; if (XS_CHANNEL(xs) != chan) { continue; } j++; isp_prt(isp, ISP_LOG_WARN1, "%d.%d.%jx bus reset set at %s:%u", XS_CHANNEL(xs), XS_TGT(xs), (uintmax_t)XS_LUN(xs), __func__, __LINE__); XS_SETERR(xs, HBA_BUSRESET); } if (j) { isp_prt(isp, ISP_LOGERR, lipd, chan, j); } } } break; case ASYNC_LOOP_UP: /* * This is a broadcast event that has to be sent across * all active channels. */ for (chan = 0; chan < isp->isp_nchan; chan++) { fcp = FCPARAM(isp, chan); if (fcp->role == ISP_ROLE_NONE) continue; fcp->isp_linkstate = 1; if (fcp->isp_loopstate < LOOP_HAVE_LINK) fcp->isp_loopstate = LOOP_HAVE_LINK; ISP_SET_SENDMARKER(isp, chan, 1); isp_async(isp, ISPASYNC_LOOP_UP, chan); #ifdef ISP_TARGET_MODE if (isp_target_async(isp, chan, mbox)) { acked = 1; } #endif } break; case ASYNC_LOOP_DOWN: /* * This is a broadcast event that has to be sent across * all active channels. */ for (chan = 0; chan < isp->isp_nchan; chan++) { fcp = FCPARAM(isp, chan); if (fcp->role == ISP_ROLE_NONE) continue; ISP_SET_SENDMARKER(isp, chan, 1); fcp->isp_linkstate = 0; fcp->isp_loopstate = LOOP_NIL; isp_async(isp, ISPASYNC_LOOP_DOWN, chan); #ifdef ISP_TARGET_MODE if (isp_target_async(isp, chan, mbox)) { acked = 1; } #endif } break; case ASYNC_LOOP_RESET: /* * This is a broadcast event that has to be sent across * all active channels. */ for (chan = 0; chan < isp->isp_nchan; chan++) { fcp = FCPARAM(isp, chan); if (fcp->role == ISP_ROLE_NONE) continue; ISP_SET_SENDMARKER(isp, chan, 1); if (fcp->isp_loopstate > LOOP_HAVE_LINK) fcp->isp_loopstate = LOOP_HAVE_LINK; isp_async(isp, ISPASYNC_LOOP_RESET, chan); #ifdef ISP_TARGET_MODE if (isp_target_async(isp, chan, mbox)) { acked = 1; } #endif } break; case ASYNC_PDB_CHANGED: { int echan, nphdl, nlstate, reason; if (IS_23XX(isp) || IS_24XX(isp)) { nphdl = ISP_READ(isp, OUTMAILBOX1); nlstate = ISP_READ(isp, OUTMAILBOX2); } else { nphdl = nlstate = 0xffff; } if (IS_24XX(isp)) reason = ISP_READ(isp, OUTMAILBOX3) >> 8; else reason = 0xff; if (ISP_CAP_MULTI_ID(isp)) { chan = ISP_READ(isp, OUTMAILBOX3) & 0xff; if (chan == 0xff || nphdl == NIL_HANDLE) { chan = 0; echan = isp->isp_nchan - 1; } else if (chan >= isp->isp_nchan) { break; } else { echan = chan; } } else { chan = echan = 0; } for (; chan <= echan; chan++) { fcp = FCPARAM(isp, chan); if (fcp->role == ISP_ROLE_NONE) continue; if (fcp->isp_loopstate > LOOP_LTEST_DONE) fcp->isp_loopstate = LOOP_LTEST_DONE; else if (fcp->isp_loopstate < LOOP_HAVE_LINK) fcp->isp_loopstate = LOOP_HAVE_LINK; isp_async(isp, ISPASYNC_CHANGE_NOTIFY, chan, ISPASYNC_CHANGE_PDB, nphdl, nlstate, reason); } break; } case ASYNC_CHANGE_NOTIFY: { int portid; portid = ((ISP_READ(isp, OUTMAILBOX1) & 0xff) << 16) | ISP_READ(isp, OUTMAILBOX2); if (ISP_CAP_MULTI_ID(isp)) { chan = ISP_READ(isp, OUTMAILBOX3) & 0xff; if (chan >= isp->isp_nchan) break; } else { chan = 0; } fcp = FCPARAM(isp, chan); if (fcp->role == ISP_ROLE_NONE) break; if (fcp->isp_loopstate > LOOP_LTEST_DONE) fcp->isp_loopstate = LOOP_LTEST_DONE; else if (fcp->isp_loopstate < LOOP_HAVE_LINK) fcp->isp_loopstate = LOOP_HAVE_LINK; isp_async(isp, ISPASYNC_CHANGE_NOTIFY, chan, ISPASYNC_CHANGE_SNS, portid); break; } case ASYNC_ERR_LOGGING_DISABLED: isp_prt(isp, ISP_LOGWARN, "Error logging disabled (reason 0x%x)", ISP_READ(isp, OUTMAILBOX1)); break; case ASYNC_CONNMODE: /* * This only applies to 2100 amd 2200 cards */ if (!IS_2200(isp) && !IS_2100(isp)) { isp_prt(isp, ISP_LOGWARN, "bad card for ASYNC_CONNMODE event"); break; } chan = 0; mbox = ISP_READ(isp, OUTMAILBOX1); switch (mbox) { case ISP_CONN_LOOP: isp_prt(isp, ISP_LOGINFO, "Point-to-Point -> Loop mode"); break; case ISP_CONN_PTP: isp_prt(isp, ISP_LOGINFO, "Loop -> Point-to-Point mode"); break; case ISP_CONN_BADLIP: isp_prt(isp, ISP_LOGWARN, "Point-to-Point -> Loop mode (BAD LIP)"); break; case ISP_CONN_FATAL: isp->isp_dead = 1; isp->isp_state = ISP_CRASHED; isp_prt(isp, ISP_LOGERR, "FATAL CONNECTION ERROR"); isp_async(isp, ISPASYNC_FW_CRASH); return (-1); case ISP_CONN_LOOPBACK: isp_prt(isp, ISP_LOGWARN, "Looped Back in Point-to-Point mode"); break; default: isp_prt(isp, ISP_LOGWARN, "Unknown connection mode (0x%x)", mbox); break; } ISP_SET_SENDMARKER(isp, chan, 1); FCPARAM(isp, chan)->isp_loopstate = LOOP_HAVE_LINK; isp_async(isp, ISPASYNC_CHANGE_NOTIFY, chan, ISPASYNC_CHANGE_OTHER); break; case ASYNC_P2P_INIT_ERR: isp_prt(isp, ISP_LOGWARN, "P2P init error (reason 0x%x)", ISP_READ(isp, OUTMAILBOX1)); break; case ASYNC_RCV_ERR: if (IS_24XX(isp)) { isp_prt(isp, ISP_LOGWARN, "Receive Error"); } else { isp_prt(isp, ISP_LOGWARN, "unexpected ASYNC_RCV_ERR"); } break; case ASYNC_RJT_SENT: /* same as ASYNC_QFULL_SENT */ if (IS_24XX(isp)) { isp_prt(isp, ISP_LOGTDEBUG0, "LS_RJT sent"); break; } else { isp_prt(isp, ISP_LOGTDEBUG0, "QFULL sent"); break; } case ASYNC_FW_RESTART_COMPLETE: isp_prt(isp, ISP_LOGDEBUG0, "FW restart complete"); break; case ASYNC_TEMPERATURE_ALERT: isp_prt(isp, ISP_LOGERR, "Temperature alert (subcode 0x%x)", ISP_READ(isp, OUTMAILBOX1)); break; case ASYNC_AUTOLOAD_FW_COMPLETE: isp_prt(isp, ISP_LOGDEBUG0, "Autoload FW init complete"); break; case ASYNC_AUTOLOAD_FW_FAILURE: isp_prt(isp, ISP_LOGERR, "Autoload FW init failure"); break; default: isp_prt(isp, ISP_LOGWARN, "Unknown Async Code 0x%x", mbox); break; } if (mbox != ASYNC_CTIO_DONE && mbox != ASYNC_CMD_CMPLT) { isp->isp_intoasync++; } return (acked); } /* * Handle other response entries. A pointer to the request queue output * index is here in case we want to eat several entries at once, although * this is not used currently. */ static int isp_handle_other_response(ispsoftc_t *isp, int type, isphdr_t *hp, uint32_t *optrp) { isp_ridacq_t rid; int chan, c; switch (type) { case RQSTYPE_STATUS_CONT: isp_prt(isp, ISP_LOG_WARN1, "Ignored Continuation Response"); return (1); case RQSTYPE_MARKER: isp_prt(isp, ISP_LOG_WARN1, "Marker Response"); return (1); case RQSTYPE_RPT_ID_ACQ: isp_get_ridacq(isp, (isp_ridacq_t *)hp, &rid); if (rid.ridacq_format == 0) { for (chan = 0; chan < isp->isp_nchan; chan++) { fcparam *fcp = FCPARAM(isp, chan); if (fcp->role == ISP_ROLE_NONE) continue; c = (chan == 0) ? 127 : (chan - 1); if (rid.ridacq_map[c / 16] & (1 << (c % 16)) || chan == 0) { fcp->isp_loopstate = LOOP_HAVE_LINK; isp_async(isp, ISPASYNC_CHANGE_NOTIFY, chan, ISPASYNC_CHANGE_OTHER); } else { fcp->isp_loopstate = LOOP_NIL; isp_async(isp, ISPASYNC_LOOP_DOWN, chan); } } } else { fcparam *fcp = FCPARAM(isp, rid.ridacq_vp_index); if (rid.ridacq_vp_status == RIDACQ_STS_COMPLETE || rid.ridacq_vp_status == RIDACQ_STS_CHANGED) { fcp->isp_loopstate = LOOP_HAVE_LINK; isp_async(isp, ISPASYNC_CHANGE_NOTIFY, rid.ridacq_vp_index, ISPASYNC_CHANGE_OTHER); } else { fcp->isp_loopstate = LOOP_NIL; isp_async(isp, ISPASYNC_LOOP_DOWN, rid.ridacq_vp_index); } } return (1); case RQSTYPE_ATIO: case RQSTYPE_CTIO: case RQSTYPE_ENABLE_LUN: case RQSTYPE_MODIFY_LUN: case RQSTYPE_NOTIFY: case RQSTYPE_NOTIFY_ACK: case RQSTYPE_CTIO1: case RQSTYPE_ATIO2: case RQSTYPE_CTIO2: case RQSTYPE_CTIO3: case RQSTYPE_CTIO7: case RQSTYPE_ABTS_RCVD: case RQSTYPE_ABTS_RSP: isp->isp_rsltccmplt++; /* count as a response completion */ #ifdef ISP_TARGET_MODE if (isp_target_notify(isp, (ispstatusreq_t *) hp, optrp)) { return (1); } #endif /* FALLTHROUGH */ case RQSTYPE_REQUEST: default: ISP_DELAY(100); if (type != isp_get_response_type(isp, hp)) { /* * This is questionable- we're just papering over * something we've seen on SMP linux in target * mode- we don't really know what's happening * here that causes us to think we've gotten * an entry, but that either the entry isn't * filled out yet or our CPU read data is stale. */ isp_prt(isp, ISP_LOGINFO, "unstable type in response queue"); return (-1); } isp_prt(isp, ISP_LOGWARN, "Unhandled Response Type 0x%x", isp_get_response_type(isp, hp)); return (0); } } static void isp_parse_status(ispsoftc_t *isp, ispstatusreq_t *sp, XS_T *xs, long *rp) { switch (sp->req_completion_status & 0xff) { case RQCS_COMPLETE: if (XS_NOERR(xs)) { XS_SETERR(xs, HBA_NOERROR); } return; case RQCS_INCOMPLETE: if ((sp->req_state_flags & RQSF_GOT_TARGET) == 0) { isp_xs_prt(isp, xs, ISP_LOG_WARN1, "Selection Timeout @ %s:%d", __func__, __LINE__); if (XS_NOERR(xs)) { XS_SETERR(xs, HBA_SELTIMEOUT); *rp = XS_XFRLEN(xs); } return; } isp_xs_prt(isp, xs, ISP_LOGERR, "Command Incomplete, state 0x%x", sp->req_state_flags); break; case RQCS_DMA_ERROR: isp_xs_prt(isp, xs, ISP_LOGERR, "DMA Error"); *rp = XS_XFRLEN(xs); break; case RQCS_TRANSPORT_ERROR: { char buf[172]; ISP_SNPRINTF(buf, sizeof (buf), "states=>"); if (sp->req_state_flags & RQSF_GOT_BUS) { ISP_SNPRINTF(buf, sizeof (buf), "%s GOT_BUS", buf); } if (sp->req_state_flags & RQSF_GOT_TARGET) { ISP_SNPRINTF(buf, sizeof (buf), "%s GOT_TGT", buf); } if (sp->req_state_flags & RQSF_SENT_CDB) { ISP_SNPRINTF(buf, sizeof (buf), "%s SENT_CDB", buf); } if (sp->req_state_flags & RQSF_XFRD_DATA) { ISP_SNPRINTF(buf, sizeof (buf), "%s XFRD_DATA", buf); } if (sp->req_state_flags & RQSF_GOT_STATUS) { ISP_SNPRINTF(buf, sizeof (buf), "%s GOT_STS", buf); } if (sp->req_state_flags & RQSF_GOT_SENSE) { ISP_SNPRINTF(buf, sizeof (buf), "%s GOT_SNS", buf); } if (sp->req_state_flags & RQSF_XFER_COMPLETE) { ISP_SNPRINTF(buf, sizeof (buf), "%s XFR_CMPLT", buf); } ISP_SNPRINTF(buf, sizeof (buf), "%s\nstatus=>", buf); if (sp->req_status_flags & RQSTF_DISCONNECT) { ISP_SNPRINTF(buf, sizeof (buf), "%s Disconnect", buf); } if (sp->req_status_flags & RQSTF_SYNCHRONOUS) { ISP_SNPRINTF(buf, sizeof (buf), "%s Sync_xfr", buf); } if (sp->req_status_flags & RQSTF_PARITY_ERROR) { ISP_SNPRINTF(buf, sizeof (buf), "%s Parity", buf); } if (sp->req_status_flags & RQSTF_BUS_RESET) { ISP_SNPRINTF(buf, sizeof (buf), "%s Bus_Reset", buf); } if (sp->req_status_flags & RQSTF_DEVICE_RESET) { ISP_SNPRINTF(buf, sizeof (buf), "%s Device_Reset", buf); } if (sp->req_status_flags & RQSTF_ABORTED) { ISP_SNPRINTF(buf, sizeof (buf), "%s Aborted", buf); } if (sp->req_status_flags & RQSTF_TIMEOUT) { ISP_SNPRINTF(buf, sizeof (buf), "%s Timeout", buf); } if (sp->req_status_flags & RQSTF_NEGOTIATION) { ISP_SNPRINTF(buf, sizeof (buf), "%s Negotiation", buf); } isp_xs_prt(isp, xs, ISP_LOGERR, "Transport Error: %s", buf); *rp = XS_XFRLEN(xs); break; } case RQCS_RESET_OCCURRED: { int chan; isp_xs_prt(isp, xs, ISP_LOGWARN, "Bus Reset destroyed command"); for (chan = 0; chan < isp->isp_nchan; chan++) { FCPARAM(isp, chan)->sendmarker = 1; } if (XS_NOERR(xs)) { XS_SETERR(xs, HBA_BUSRESET); } *rp = XS_XFRLEN(xs); return; } case RQCS_ABORTED: isp_xs_prt(isp, xs, ISP_LOGERR, "Command Aborted"); ISP_SET_SENDMARKER(isp, XS_CHANNEL(xs), 1); if (XS_NOERR(xs)) { XS_SETERR(xs, HBA_ABORTED); } return; case RQCS_TIMEOUT: isp_xs_prt(isp, xs, ISP_LOGWARN, "Command timed out"); /* * XXX: Check to see if we logged out of the device. */ if (XS_NOERR(xs)) { XS_SETERR(xs, HBA_CMDTIMEOUT); } return; case RQCS_DATA_OVERRUN: XS_SET_RESID(xs, sp->req_resid); isp_xs_prt(isp, xs, ISP_LOGERR, "data overrun (%ld)", (long) XS_GET_RESID(xs)); if (XS_NOERR(xs)) { XS_SETERR(xs, HBA_DATAOVR); } return; case RQCS_COMMAND_OVERRUN: isp_xs_prt(isp, xs, ISP_LOGERR, "command overrun"); break; case RQCS_STATUS_OVERRUN: isp_xs_prt(isp, xs, ISP_LOGERR, "status overrun"); break; case RQCS_BAD_MESSAGE: isp_xs_prt(isp, xs, ISP_LOGERR, "msg not COMMAND COMPLETE after status"); break; case RQCS_NO_MESSAGE_OUT: isp_xs_prt(isp, xs, ISP_LOGERR, "No MESSAGE OUT phase after selection"); break; case RQCS_EXT_ID_FAILED: isp_xs_prt(isp, xs, ISP_LOGERR, "EXTENDED IDENTIFY failed"); break; case RQCS_IDE_MSG_FAILED: isp_xs_prt(isp, xs, ISP_LOGERR, "INITIATOR DETECTED ERROR rejected"); break; case RQCS_ABORT_MSG_FAILED: isp_xs_prt(isp, xs, ISP_LOGERR, "ABORT OPERATION rejected"); break; case RQCS_REJECT_MSG_FAILED: isp_xs_prt(isp, xs, ISP_LOGERR, "MESSAGE REJECT rejected"); break; case RQCS_NOP_MSG_FAILED: isp_xs_prt(isp, xs, ISP_LOGERR, "NOP rejected"); break; case RQCS_PARITY_ERROR_MSG_FAILED: isp_xs_prt(isp, xs, ISP_LOGERR, "MESSAGE PARITY ERROR rejected"); break; case RQCS_DEVICE_RESET_MSG_FAILED: isp_xs_prt(isp, xs, ISP_LOGWARN, "BUS DEVICE RESET rejected"); break; case RQCS_ID_MSG_FAILED: isp_xs_prt(isp, xs, ISP_LOGERR, "IDENTIFY rejected"); break; case RQCS_UNEXP_BUS_FREE: isp_xs_prt(isp, xs, ISP_LOGERR, "Unexpected Bus Free"); break; case RQCS_DATA_UNDERRUN: { if (IS_FC(isp)) { int ru_marked = (sp->req_scsi_status & RQCS_RU) != 0; if (!ru_marked || sp->req_resid > XS_XFRLEN(xs)) { isp_xs_prt(isp, xs, ISP_LOGWARN, bun, XS_XFRLEN(xs), sp->req_resid, (ru_marked)? "marked" : "not marked"); if (XS_NOERR(xs)) { XS_SETERR(xs, HBA_BOTCH); } return; } } XS_SET_RESID(xs, sp->req_resid); if (XS_NOERR(xs)) { XS_SETERR(xs, HBA_NOERROR); } return; } case RQCS_XACT_ERR1: isp_xs_prt(isp, xs, ISP_LOGERR, "HBA attempted queued transaction with disconnect not set"); break; case RQCS_XACT_ERR2: isp_xs_prt(isp, xs, ISP_LOGERR, "HBA attempted queued transaction to target routine %jx", (uintmax_t)XS_LUN(xs)); break; case RQCS_XACT_ERR3: isp_xs_prt(isp, xs, ISP_LOGERR, "HBA attempted queued cmd when queueing disabled"); break; case RQCS_BAD_ENTRY: isp_prt(isp, ISP_LOGERR, "Invalid IOCB entry type detected"); break; case RQCS_QUEUE_FULL: isp_xs_prt(isp, xs, ISP_LOG_WARN1, "internal queues full status 0x%x", *XS_STSP(xs)); /* * If QFULL or some other status byte is set, then this * isn't an error, per se. * * Unfortunately, some QLogic f/w writers have, in * some cases, ommitted to *set* status to QFULL. */ #if 0 if (*XS_STSP(xs) != SCSI_GOOD && XS_NOERR(xs)) { XS_SETERR(xs, HBA_NOERROR); return; } #endif *XS_STSP(xs) = SCSI_QFULL; XS_SETERR(xs, HBA_NOERROR); return; case RQCS_PHASE_SKIPPED: isp_xs_prt(isp, xs, ISP_LOGERR, "SCSI phase skipped"); break; case RQCS_ARQS_FAILED: isp_xs_prt(isp, xs, ISP_LOGERR, "Auto Request Sense Failed"); if (XS_NOERR(xs)) { XS_SETERR(xs, HBA_ARQFAIL); } return; case RQCS_WIDE_FAILED: isp_xs_prt(isp, xs, ISP_LOGERR, "Wide Negotiation Failed"); if (IS_SCSI(isp)) { sdparam *sdp = SDPARAM(isp, XS_CHANNEL(xs)); sdp->isp_devparam[XS_TGT(xs)].goal_flags &= ~DPARM_WIDE; sdp->isp_devparam[XS_TGT(xs)].dev_update = 1; sdp->update = 1; } if (XS_NOERR(xs)) { XS_SETERR(xs, HBA_NOERROR); } return; case RQCS_SYNCXFER_FAILED: isp_xs_prt(isp, xs, ISP_LOGERR, "SDTR Message Failed"); if (IS_SCSI(isp)) { sdparam *sdp = SDPARAM(isp, XS_CHANNEL(xs)); sdp += XS_CHANNEL(xs); sdp->isp_devparam[XS_TGT(xs)].goal_flags &= ~DPARM_SYNC; sdp->isp_devparam[XS_TGT(xs)].dev_update = 1; sdp->update = 1; } break; case RQCS_LVD_BUSERR: isp_xs_prt(isp, xs, ISP_LOGERR, "Bad LVD condition"); break; case RQCS_PORT_UNAVAILABLE: /* * No such port on the loop. Moral equivalent of SELTIMEO */ case RQCS_PORT_LOGGED_OUT: { const char *reason; uint8_t sts = sp->req_completion_status & 0xff; /* * It was there (maybe)- treat as a selection timeout. */ if (sts == RQCS_PORT_UNAVAILABLE) { reason = "unavailable"; } else { reason = "logout"; } isp_prt(isp, ISP_LOGINFO, "port %s for target %d", reason, XS_TGT(xs)); /* * If we're on a local loop, force a LIP (which is overkill) * to force a re-login of this unit. If we're on fabric, * then we'll have to log in again as a matter of course. */ if (FCPARAM(isp, 0)->isp_topo == TOPO_NL_PORT || FCPARAM(isp, 0)->isp_topo == TOPO_FL_PORT) { mbreg_t mbs; MBSINIT(&mbs, MBOX_INIT_LIP, MBLOGALL, 0); if (ISP_CAP_2KLOGIN(isp)) { mbs.ibits = (1 << 10); } isp_mboxcmd_qnw(isp, &mbs, 1); } if (XS_NOERR(xs)) { XS_SETERR(xs, HBA_SELTIMEOUT); } return; } case RQCS_PORT_CHANGED: isp_prt(isp, ISP_LOGWARN, "port changed for target %d", XS_TGT(xs)); if (XS_NOERR(xs)) { XS_SETERR(xs, HBA_SELTIMEOUT); } return; case RQCS_PORT_BUSY: isp_prt(isp, ISP_LOGWARN, "port busy for target %d", XS_TGT(xs)); if (XS_NOERR(xs)) { XS_SETERR(xs, HBA_TGTBSY); } return; default: isp_prt(isp, ISP_LOGERR, "Unknown Completion Status 0x%x", sp->req_completion_status); break; } if (XS_NOERR(xs)) { XS_SETERR(xs, HBA_BOTCH); } } static void isp_parse_status_24xx(ispsoftc_t *isp, isp24xx_statusreq_t *sp, XS_T *xs, long *rp) { int ru_marked, sv_marked; int chan = XS_CHANNEL(xs); switch (sp->req_completion_status) { case RQCS_COMPLETE: if (XS_NOERR(xs)) { XS_SETERR(xs, HBA_NOERROR); } return; case RQCS_DMA_ERROR: isp_xs_prt(isp, xs, ISP_LOGERR, "DMA error"); break; case RQCS_TRANSPORT_ERROR: isp_xs_prt(isp, xs, ISP_LOGERR, "Transport Error"); break; case RQCS_RESET_OCCURRED: isp_xs_prt(isp, xs, ISP_LOGWARN, "reset destroyed command"); FCPARAM(isp, chan)->sendmarker = 1; if (XS_NOERR(xs)) { XS_SETERR(xs, HBA_BUSRESET); } return; case RQCS_ABORTED: isp_xs_prt(isp, xs, ISP_LOGERR, "Command Aborted"); FCPARAM(isp, chan)->sendmarker = 1; if (XS_NOERR(xs)) { XS_SETERR(xs, HBA_ABORTED); } return; case RQCS_TIMEOUT: isp_xs_prt(isp, xs, ISP_LOGWARN, "Command Timed Out"); if (XS_NOERR(xs)) { XS_SETERR(xs, HBA_CMDTIMEOUT); } return; case RQCS_DATA_OVERRUN: XS_SET_RESID(xs, sp->req_resid); isp_xs_prt(isp, xs, ISP_LOGERR, "Data Overrun"); if (XS_NOERR(xs)) { XS_SETERR(xs, HBA_DATAOVR); } return; case RQCS_24XX_DRE: /* data reassembly error */ isp_prt(isp, ISP_LOGERR, "Chan %d data reassembly error for target %d", chan, XS_TGT(xs)); if (XS_NOERR(xs)) { XS_SETERR(xs, HBA_ABORTED); } *rp = XS_XFRLEN(xs); return; case RQCS_24XX_TABORT: /* aborted by target */ isp_prt(isp, ISP_LOGERR, "Chan %d target %d sent ABTS", chan, XS_TGT(xs)); if (XS_NOERR(xs)) { XS_SETERR(xs, HBA_ABORTED); } return; case RQCS_DATA_UNDERRUN: ru_marked = (sp->req_scsi_status & RQCS_RU) != 0; /* * We can get an underrun w/o things being marked * if we got a non-zero status. */ sv_marked = (sp->req_scsi_status & (RQCS_SV|RQCS_RV)) != 0; if ((ru_marked == 0 && sv_marked == 0) || (sp->req_resid > XS_XFRLEN(xs))) { isp_xs_prt(isp, xs, ISP_LOGWARN, bun, XS_XFRLEN(xs), sp->req_resid, (ru_marked)? "marked" : "not marked"); if (XS_NOERR(xs)) { XS_SETERR(xs, HBA_BOTCH); } return; } XS_SET_RESID(xs, sp->req_resid); isp_xs_prt(isp, xs, ISP_LOG_WARN1, "Data Underrun (%d) for command 0x%x", sp->req_resid, XS_CDBP(xs)[0] & 0xff); if (XS_NOERR(xs)) { XS_SETERR(xs, HBA_NOERROR); } return; case RQCS_PORT_UNAVAILABLE: /* * No such port on the loop. Moral equivalent of SELTIMEO */ case RQCS_PORT_LOGGED_OUT: { const char *reason; uint8_t sts = sp->req_completion_status & 0xff; /* * It was there (maybe)- treat as a selection timeout. */ if (sts == RQCS_PORT_UNAVAILABLE) { reason = "unavailable"; } else { reason = "logout"; } isp_prt(isp, ISP_LOGINFO, "Chan %d port %s for target %d", chan, reason, XS_TGT(xs)); /* * There is no MBOX_INIT_LIP for the 24XX. */ if (XS_NOERR(xs)) { XS_SETERR(xs, HBA_SELTIMEOUT); } return; } case RQCS_PORT_CHANGED: isp_prt(isp, ISP_LOGWARN, "port changed for target %d chan %d", XS_TGT(xs), chan); if (XS_NOERR(xs)) { XS_SETERR(xs, HBA_SELTIMEOUT); } return; case RQCS_24XX_ENOMEM: /* f/w resource unavailable */ isp_prt(isp, ISP_LOGWARN, "f/w resource unavailable for target %d chan %d", XS_TGT(xs), chan); if (XS_NOERR(xs)) { *XS_STSP(xs) = SCSI_BUSY; XS_SETERR(xs, HBA_TGTBSY); } return; case RQCS_24XX_TMO: /* task management overrun */ isp_prt(isp, ISP_LOGWARN, "command for target %d overlapped task management for chan %d", XS_TGT(xs), chan); if (XS_NOERR(xs)) { *XS_STSP(xs) = SCSI_BUSY; XS_SETERR(xs, HBA_TGTBSY); } return; default: isp_prt(isp, ISP_LOGERR, "Unknown Completion Status 0x%x on chan %d", sp->req_completion_status, chan); break; } if (XS_NOERR(xs)) { XS_SETERR(xs, HBA_BOTCH); } } static void isp_fastpost_complete(ispsoftc_t *isp, uint32_t fph) { XS_T *xs; if (fph == 0) { return; } xs = isp_find_xs(isp, fph); if (xs == NULL) { isp_prt(isp, ISP_LOGWARN, "Command for fast post handle 0x%x not found", fph); return; } isp_destroy_handle(isp, fph); /* * Since we don't have a result queue entry item, * we must believe that SCSI status is zero and * that all data transferred. */ XS_SET_RESID(xs, 0); *XS_STSP(xs) = SCSI_GOOD; if (XS_XFRLEN(xs)) { ISP_DMAFREE(isp, xs, fph); } if (isp->isp_nactive) { isp->isp_nactive--; } isp->isp_fphccmplt++; isp_done(xs); } static int isp_mbox_continue(ispsoftc_t *isp) { mbreg_t mbs; uint16_t *ptr; uint32_t offset; switch (isp->isp_lastmbxcmd) { case MBOX_WRITE_RAM_WORD: case MBOX_READ_RAM_WORD: case MBOX_WRITE_RAM_WORD_EXTENDED: case MBOX_READ_RAM_WORD_EXTENDED: break; default: return (1); } if (isp->isp_mboxtmp[0] != MBOX_COMMAND_COMPLETE) { isp->isp_mbxwrk0 = 0; return (-1); } /* * Clear the previous interrupt. */ if (IS_24XX(isp)) { ISP_WRITE(isp, BIU2400_HCCR, HCCR_2400_CMD_CLEAR_RISC_INT); } else { ISP_WRITE(isp, HCCR, HCCR_CMD_CLEAR_RISC_INT); ISP_WRITE(isp, BIU_SEMA, 0); } /* * Continue with next word. */ ISP_MEMZERO(&mbs, sizeof (mbs)); ptr = isp->isp_mbxworkp; switch (isp->isp_lastmbxcmd) { case MBOX_WRITE_RAM_WORD: mbs.param[1] = isp->isp_mbxwrk1++; mbs.param[2] = *ptr++; break; case MBOX_READ_RAM_WORD: *ptr++ = isp->isp_mboxtmp[2]; mbs.param[1] = isp->isp_mbxwrk1++; break; case MBOX_WRITE_RAM_WORD_EXTENDED: if (IS_24XX(isp)) { uint32_t *lptr = (uint32_t *)ptr; mbs.param[2] = lptr[0]; mbs.param[3] = lptr[0] >> 16; lptr++; ptr = (uint16_t *)lptr; } else { mbs.param[2] = *ptr++; } offset = isp->isp_mbxwrk1; offset |= isp->isp_mbxwrk8 << 16; mbs.param[1] = offset; mbs.param[8] = offset >> 16; offset++; isp->isp_mbxwrk1 = offset; isp->isp_mbxwrk8 = offset >> 16; break; case MBOX_READ_RAM_WORD_EXTENDED: if (IS_24XX(isp)) { uint32_t *lptr = (uint32_t *)ptr; uint32_t val = isp->isp_mboxtmp[2]; val |= (isp->isp_mboxtmp[3]) << 16; *lptr++ = val; ptr = (uint16_t *)lptr; } else { *ptr++ = isp->isp_mboxtmp[2]; } offset = isp->isp_mbxwrk1; offset |= isp->isp_mbxwrk8 << 16; mbs.param[1] = offset; mbs.param[8] = offset >> 16; offset++; isp->isp_mbxwrk1 = offset; isp->isp_mbxwrk8 = offset >> 16; break; } isp->isp_mbxworkp = ptr; isp->isp_mbxwrk0--; mbs.param[0] = isp->isp_lastmbxcmd; mbs.logval = MBLOGALL; isp_mboxcmd_qnw(isp, &mbs, 0); return (0); } #define ISP_SCSI_IBITS(op) (mbpscsi[((op)<<1)]) #define ISP_SCSI_OBITS(op) (mbpscsi[((op)<<1) + 1]) #define ISP_SCSI_OPMAP(in, out) in, out static const uint8_t mbpscsi[] = { ISP_SCSI_OPMAP(0x01, 0x01), /* 0x00: MBOX_NO_OP */ ISP_SCSI_OPMAP(0x1f, 0x01), /* 0x01: MBOX_LOAD_RAM */ ISP_SCSI_OPMAP(0x03, 0x01), /* 0x02: MBOX_EXEC_FIRMWARE */ ISP_SCSI_OPMAP(0x1f, 0x01), /* 0x03: MBOX_DUMP_RAM */ ISP_SCSI_OPMAP(0x07, 0x07), /* 0x04: MBOX_WRITE_RAM_WORD */ ISP_SCSI_OPMAP(0x03, 0x07), /* 0x05: MBOX_READ_RAM_WORD */ ISP_SCSI_OPMAP(0x3f, 0x3f), /* 0x06: MBOX_MAILBOX_REG_TEST */ ISP_SCSI_OPMAP(0x07, 0x07), /* 0x07: MBOX_VERIFY_CHECKSUM */ ISP_SCSI_OPMAP(0x01, 0x0f), /* 0x08: MBOX_ABOUT_FIRMWARE */ ISP_SCSI_OPMAP(0x00, 0x00), /* 0x09: */ ISP_SCSI_OPMAP(0x00, 0x00), /* 0x0a: */ ISP_SCSI_OPMAP(0x00, 0x00), /* 0x0b: */ ISP_SCSI_OPMAP(0x00, 0x00), /* 0x0c: */ ISP_SCSI_OPMAP(0x00, 0x00), /* 0x0d: */ ISP_SCSI_OPMAP(0x01, 0x05), /* 0x0e: MBOX_CHECK_FIRMWARE */ ISP_SCSI_OPMAP(0x00, 0x00), /* 0x0f: */ ISP_SCSI_OPMAP(0x1f, 0x1f), /* 0x10: MBOX_INIT_REQ_QUEUE */ ISP_SCSI_OPMAP(0x3f, 0x3f), /* 0x11: MBOX_INIT_RES_QUEUE */ ISP_SCSI_OPMAP(0x0f, 0x0f), /* 0x12: MBOX_EXECUTE_IOCB */ ISP_SCSI_OPMAP(0x03, 0x03), /* 0x13: MBOX_WAKE_UP */ ISP_SCSI_OPMAP(0x01, 0x3f), /* 0x14: MBOX_STOP_FIRMWARE */ ISP_SCSI_OPMAP(0x0f, 0x0f), /* 0x15: MBOX_ABORT */ ISP_SCSI_OPMAP(0x03, 0x03), /* 0x16: MBOX_ABORT_DEVICE */ ISP_SCSI_OPMAP(0x07, 0x07), /* 0x17: MBOX_ABORT_TARGET */ ISP_SCSI_OPMAP(0x07, 0x07), /* 0x18: MBOX_BUS_RESET */ ISP_SCSI_OPMAP(0x03, 0x07), /* 0x19: MBOX_STOP_QUEUE */ ISP_SCSI_OPMAP(0x03, 0x07), /* 0x1a: MBOX_START_QUEUE */ ISP_SCSI_OPMAP(0x03, 0x07), /* 0x1b: MBOX_SINGLE_STEP_QUEUE */ ISP_SCSI_OPMAP(0x03, 0x07), /* 0x1c: MBOX_ABORT_QUEUE */ ISP_SCSI_OPMAP(0x03, 0x4f), /* 0x1d: MBOX_GET_DEV_QUEUE_STATUS */ ISP_SCSI_OPMAP(0x00, 0x00), /* 0x1e: */ ISP_SCSI_OPMAP(0x01, 0x07), /* 0x1f: MBOX_GET_FIRMWARE_STATUS */ ISP_SCSI_OPMAP(0x01, 0x07), /* 0x20: MBOX_GET_INIT_SCSI_ID */ ISP_SCSI_OPMAP(0x01, 0x07), /* 0x21: MBOX_GET_SELECT_TIMEOUT */ ISP_SCSI_OPMAP(0x01, 0xc7), /* 0x22: MBOX_GET_RETRY_COUNT */ ISP_SCSI_OPMAP(0x01, 0x07), /* 0x23: MBOX_GET_TAG_AGE_LIMIT */ ISP_SCSI_OPMAP(0x01, 0x03), /* 0x24: MBOX_GET_CLOCK_RATE */ ISP_SCSI_OPMAP(0x01, 0x07), /* 0x25: MBOX_GET_ACT_NEG_STATE */ ISP_SCSI_OPMAP(0x01, 0x07), /* 0x26: MBOX_GET_ASYNC_DATA_SETUP_TIME */ ISP_SCSI_OPMAP(0x01, 0x07), /* 0x27: MBOX_GET_PCI_PARAMS */ ISP_SCSI_OPMAP(0x03, 0x4f), /* 0x28: MBOX_GET_TARGET_PARAMS */ ISP_SCSI_OPMAP(0x03, 0x0f), /* 0x29: MBOX_GET_DEV_QUEUE_PARAMS */ ISP_SCSI_OPMAP(0x01, 0x07), /* 0x2a: MBOX_GET_RESET_DELAY_PARAMS */ ISP_SCSI_OPMAP(0x00, 0x00), /* 0x2b: */ ISP_SCSI_OPMAP(0x00, 0x00), /* 0x2c: */ ISP_SCSI_OPMAP(0x00, 0x00), /* 0x2d: */ ISP_SCSI_OPMAP(0x00, 0x00), /* 0x2e: */ ISP_SCSI_OPMAP(0x00, 0x00), /* 0x2f: */ ISP_SCSI_OPMAP(0x03, 0x03), /* 0x30: MBOX_SET_INIT_SCSI_ID */ ISP_SCSI_OPMAP(0x07, 0x07), /* 0x31: MBOX_SET_SELECT_TIMEOUT */ ISP_SCSI_OPMAP(0xc7, 0xc7), /* 0x32: MBOX_SET_RETRY_COUNT */ ISP_SCSI_OPMAP(0x07, 0x07), /* 0x33: MBOX_SET_TAG_AGE_LIMIT */ ISP_SCSI_OPMAP(0x03, 0x03), /* 0x34: MBOX_SET_CLOCK_RATE */ ISP_SCSI_OPMAP(0x07, 0x07), /* 0x35: MBOX_SET_ACT_NEG_STATE */ ISP_SCSI_OPMAP(0x07, 0x07), /* 0x36: MBOX_SET_ASYNC_DATA_SETUP_TIME */ ISP_SCSI_OPMAP(0x07, 0x07), /* 0x37: MBOX_SET_PCI_CONTROL_PARAMS */ ISP_SCSI_OPMAP(0x4f, 0x4f), /* 0x38: MBOX_SET_TARGET_PARAMS */ ISP_SCSI_OPMAP(0x0f, 0x0f), /* 0x39: MBOX_SET_DEV_QUEUE_PARAMS */ ISP_SCSI_OPMAP(0x07, 0x07), /* 0x3a: MBOX_SET_RESET_DELAY_PARAMS */ ISP_SCSI_OPMAP(0x00, 0x00), /* 0x3b: */ ISP_SCSI_OPMAP(0x00, 0x00), /* 0x3c: */ ISP_SCSI_OPMAP(0x00, 0x00), /* 0x3d: */ ISP_SCSI_OPMAP(0x00, 0x00), /* 0x3e: */ ISP_SCSI_OPMAP(0x00, 0x00), /* 0x3f: */ ISP_SCSI_OPMAP(0x01, 0x03), /* 0x40: MBOX_RETURN_BIOS_BLOCK_ADDR */ ISP_SCSI_OPMAP(0x3f, 0x01), /* 0x41: MBOX_WRITE_FOUR_RAM_WORDS */ ISP_SCSI_OPMAP(0x03, 0x07), /* 0x42: MBOX_EXEC_BIOS_IOCB */ ISP_SCSI_OPMAP(0x00, 0x00), /* 0x43: */ ISP_SCSI_OPMAP(0x00, 0x00), /* 0x44: */ ISP_SCSI_OPMAP(0x03, 0x03), /* 0x45: SET SYSTEM PARAMETER */ ISP_SCSI_OPMAP(0x01, 0x03), /* 0x46: GET SYSTEM PARAMETER */ ISP_SCSI_OPMAP(0x00, 0x00), /* 0x47: */ ISP_SCSI_OPMAP(0x01, 0xcf), /* 0x48: GET SCAM CONFIGURATION */ ISP_SCSI_OPMAP(0xcf, 0xcf), /* 0x49: SET SCAM CONFIGURATION */ ISP_SCSI_OPMAP(0x03, 0x03), /* 0x4a: MBOX_SET_FIRMWARE_FEATURES */ ISP_SCSI_OPMAP(0x01, 0x03), /* 0x4b: MBOX_GET_FIRMWARE_FEATURES */ ISP_SCSI_OPMAP(0x00, 0x00), /* 0x4c: */ ISP_SCSI_OPMAP(0x00, 0x00), /* 0x4d: */ ISP_SCSI_OPMAP(0x00, 0x00), /* 0x4e: */ ISP_SCSI_OPMAP(0x00, 0x00), /* 0x4f: */ ISP_SCSI_OPMAP(0xdf, 0xdf), /* 0x50: LOAD RAM A64 */ ISP_SCSI_OPMAP(0xdf, 0xdf), /* 0x51: DUMP RAM A64 */ ISP_SCSI_OPMAP(0xdf, 0xff), /* 0x52: INITIALIZE REQUEST QUEUE A64 */ ISP_SCSI_OPMAP(0xef, 0xff), /* 0x53: INITIALIZE RESPONSE QUEUE A64 */ ISP_SCSI_OPMAP(0xcf, 0x01), /* 0x54: EXECUCUTE COMMAND IOCB A64 */ ISP_SCSI_OPMAP(0x07, 0x01), /* 0x55: ENABLE TARGET MODE */ ISP_SCSI_OPMAP(0x03, 0x0f), /* 0x56: GET TARGET STATUS */ ISP_SCSI_OPMAP(0x00, 0x00), /* 0x57: */ ISP_SCSI_OPMAP(0x00, 0x00), /* 0x58: */ ISP_SCSI_OPMAP(0x00, 0x00), /* 0x59: */ ISP_SCSI_OPMAP(0x03, 0x03), /* 0x5a: SET DATA OVERRUN RECOVERY MODE */ ISP_SCSI_OPMAP(0x01, 0x03), /* 0x5b: GET DATA OVERRUN RECOVERY MODE */ ISP_SCSI_OPMAP(0x0f, 0x0f), /* 0x5c: SET HOST DATA */ ISP_SCSI_OPMAP(0x01, 0x01) /* 0x5d: GET NOST DATA */ }; #define MAX_SCSI_OPCODE 0x5d static const char *scsi_mbcmd_names[] = { "NO-OP", "LOAD RAM", "EXEC FIRMWARE", "DUMP RAM", "WRITE RAM WORD", "READ RAM WORD", "MAILBOX REG TEST", "VERIFY CHECKSUM", "ABOUT FIRMWARE", NULL, NULL, NULL, NULL, NULL, "CHECK FIRMWARE", NULL, "INIT REQUEST QUEUE", "INIT RESULT QUEUE", "EXECUTE IOCB", "WAKE UP", "STOP FIRMWARE", "ABORT", "ABORT DEVICE", "ABORT TARGET", "BUS RESET", "STOP QUEUE", "START QUEUE", "SINGLE STEP QUEUE", "ABORT QUEUE", "GET DEV QUEUE STATUS", NULL, "GET FIRMWARE STATUS", "GET INIT SCSI ID", "GET SELECT TIMEOUT", "GET RETRY COUNT", "GET TAG AGE LIMIT", "GET CLOCK RATE", "GET ACT NEG STATE", "GET ASYNC DATA SETUP TIME", "GET PCI PARAMS", "GET TARGET PARAMS", "GET DEV QUEUE PARAMS", "GET RESET DELAY PARAMS", NULL, NULL, NULL, NULL, NULL, "SET INIT SCSI ID", "SET SELECT TIMEOUT", "SET RETRY COUNT", "SET TAG AGE LIMIT", "SET CLOCK RATE", "SET ACT NEG STATE", "SET ASYNC DATA SETUP TIME", "SET PCI CONTROL PARAMS", "SET TARGET PARAMS", "SET DEV QUEUE PARAMS", "SET RESET DELAY PARAMS", NULL, NULL, NULL, NULL, NULL, "RETURN BIOS BLOCK ADDR", "WRITE FOUR RAM WORDS", "EXEC BIOS IOCB", NULL, NULL, "SET SYSTEM PARAMETER", "GET SYSTEM PARAMETER", NULL, "GET SCAM CONFIGURATION", "SET SCAM CONFIGURATION", "SET FIRMWARE FEATURES", "GET FIRMWARE FEATURES", NULL, NULL, NULL, NULL, "LOAD RAM A64", "DUMP RAM A64", "INITIALIZE REQUEST QUEUE A64", "INITIALIZE RESPONSE QUEUE A64", "EXECUTE IOCB A64", "ENABLE TARGET MODE", "GET TARGET MODE STATE", NULL, NULL, NULL, "SET DATA OVERRUN RECOVERY MODE", "GET DATA OVERRUN RECOVERY MODE", "SET HOST DATA", "GET NOST DATA", }; #define ISP_FC_IBITS(op) ((mbpfc[((op)<<3) + 0] << 24) | (mbpfc[((op)<<3) + 1] << 16) | (mbpfc[((op)<<3) + 2] << 8) | (mbpfc[((op)<<3) + 3])) #define ISP_FC_OBITS(op) ((mbpfc[((op)<<3) + 4] << 24) | (mbpfc[((op)<<3) + 5] << 16) | (mbpfc[((op)<<3) + 6] << 8) | (mbpfc[((op)<<3) + 7])) #define ISP_FC_OPMAP(in0, out0) 0, 0, 0, in0, 0, 0, 0, out0 #define ISP_FC_OPMAP_HALF(in1, in0, out1, out0) 0, 0, in1, in0, 0, 0, out1, out0 #define ISP_FC_OPMAP_FULL(in3, in2, in1, in0, out3, out2, out1, out0) in3, in2, in1, in0, out3, out2, out1, out0 static const uint32_t mbpfc[] = { ISP_FC_OPMAP(0x01, 0x01), /* 0x00: MBOX_NO_OP */ ISP_FC_OPMAP(0x1f, 0x01), /* 0x01: MBOX_LOAD_RAM */ ISP_FC_OPMAP_HALF(0x07, 0xff, 0x00, 0x03), /* 0x02: MBOX_EXEC_FIRMWARE */ ISP_FC_OPMAP(0xdf, 0x01), /* 0x03: MBOX_DUMP_RAM */ ISP_FC_OPMAP(0x07, 0x07), /* 0x04: MBOX_WRITE_RAM_WORD */ ISP_FC_OPMAP(0x03, 0x07), /* 0x05: MBOX_READ_RAM_WORD */ ISP_FC_OPMAP_FULL(0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff), /* 0x06: MBOX_MAILBOX_REG_TEST */ ISP_FC_OPMAP(0x07, 0x07), /* 0x07: MBOX_VERIFY_CHECKSUM */ ISP_FC_OPMAP_FULL(0x0, 0x0, 0x0, 0x01, 0x0, 0x3, 0x80, 0x7f), /* 0x08: MBOX_ABOUT_FIRMWARE */ ISP_FC_OPMAP(0xdf, 0x01), /* 0x09: MBOX_LOAD_RISC_RAM_2100 */ ISP_FC_OPMAP(0xdf, 0x01), /* 0x0a: DUMP RAM */ ISP_FC_OPMAP_HALF(0x1, 0xff, 0x0, 0x01), /* 0x0b: MBOX_LOAD_RISC_RAM */ ISP_FC_OPMAP(0x00, 0x00), /* 0x0c: */ ISP_FC_OPMAP_HALF(0x1, 0x0f, 0x0, 0x01), /* 0x0d: MBOX_WRITE_RAM_WORD_EXTENDED */ ISP_FC_OPMAP(0x01, 0x05), /* 0x0e: MBOX_CHECK_FIRMWARE */ ISP_FC_OPMAP_HALF(0x1, 0x03, 0x0, 0x0d), /* 0x0f: MBOX_READ_RAM_WORD_EXTENDED */ ISP_FC_OPMAP(0x1f, 0x11), /* 0x10: MBOX_INIT_REQ_QUEUE */ ISP_FC_OPMAP(0x2f, 0x21), /* 0x11: MBOX_INIT_RES_QUEUE */ ISP_FC_OPMAP(0x0f, 0x01), /* 0x12: MBOX_EXECUTE_IOCB */ ISP_FC_OPMAP(0x03, 0x03), /* 0x13: MBOX_WAKE_UP */ ISP_FC_OPMAP_HALF(0x1, 0xff, 0x0, 0x03), /* 0x14: MBOX_STOP_FIRMWARE */ ISP_FC_OPMAP(0x4f, 0x01), /* 0x15: MBOX_ABORT */ ISP_FC_OPMAP(0x07, 0x01), /* 0x16: MBOX_ABORT_DEVICE */ ISP_FC_OPMAP(0x07, 0x01), /* 0x17: MBOX_ABORT_TARGET */ ISP_FC_OPMAP(0x03, 0x03), /* 0x18: MBOX_BUS_RESET */ ISP_FC_OPMAP(0x07, 0x05), /* 0x19: MBOX_STOP_QUEUE */ ISP_FC_OPMAP(0x07, 0x05), /* 0x1a: MBOX_START_QUEUE */ ISP_FC_OPMAP(0x07, 0x05), /* 0x1b: MBOX_SINGLE_STEP_QUEUE */ ISP_FC_OPMAP(0x07, 0x05), /* 0x1c: MBOX_ABORT_QUEUE */ ISP_FC_OPMAP(0x07, 0x03), /* 0x1d: MBOX_GET_DEV_QUEUE_STATUS */ ISP_FC_OPMAP(0x00, 0x00), /* 0x1e: */ ISP_FC_OPMAP(0x01, 0x07), /* 0x1f: MBOX_GET_FIRMWARE_STATUS */ ISP_FC_OPMAP_HALF(0x2, 0x01, 0x7e, 0xcf), /* 0x20: MBOX_GET_LOOP_ID */ ISP_FC_OPMAP(0x00, 0x00), /* 0x21: */ ISP_FC_OPMAP(0x01, 0x07), /* 0x22: MBOX_GET_RETRY_COUNT */ ISP_FC_OPMAP(0x00, 0x00), /* 0x23: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x24: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x25: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x26: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x27: */ ISP_FC_OPMAP(0x01, 0x03), /* 0x28: MBOX_GET_FIRMWARE_OPTIONS */ ISP_FC_OPMAP(0x03, 0x07), /* 0x29: MBOX_GET_PORT_QUEUE_PARAMS */ ISP_FC_OPMAP(0x00, 0x00), /* 0x2a: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x2b: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x2c: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x2d: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x2e: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x2f: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x30: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x31: */ ISP_FC_OPMAP(0x07, 0x07), /* 0x32: MBOX_SET_RETRY_COUNT */ ISP_FC_OPMAP(0x00, 0x00), /* 0x33: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x34: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x35: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x36: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x37: */ ISP_FC_OPMAP(0x0f, 0x01), /* 0x38: MBOX_SET_FIRMWARE_OPTIONS */ ISP_FC_OPMAP(0x0f, 0x07), /* 0x39: MBOX_SET_PORT_QUEUE_PARAMS */ ISP_FC_OPMAP(0x00, 0x00), /* 0x3a: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x3b: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x3c: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x3d: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x3e: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x3f: */ ISP_FC_OPMAP(0x03, 0x01), /* 0x40: MBOX_LOOP_PORT_BYPASS */ ISP_FC_OPMAP(0x03, 0x01), /* 0x41: MBOX_LOOP_PORT_ENABLE */ ISP_FC_OPMAP_HALF(0x0, 0x01, 0x3, 0xcf), /* 0x42: MBOX_GET_RESOURCE_COUNT */ ISP_FC_OPMAP(0x01, 0x01), /* 0x43: MBOX_REQUEST_OFFLINE_MODE */ ISP_FC_OPMAP(0x00, 0x00), /* 0x44: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x45: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x46: */ ISP_FC_OPMAP(0xcf, 0x03), /* 0x47: GET PORT_DATABASE ENHANCED */ ISP_FC_OPMAP(0xcf, 0x0f), /* 0x48: MBOX_INIT_FIRMWARE_MULTI_ID */ ISP_FC_OPMAP(0xcd, 0x01), /* 0x49: MBOX_GET_VP_DATABASE */ ISP_FC_OPMAP_HALF(0x2, 0xcd, 0x0, 0x01), /* 0x4a: MBOX_GET_VP_DATABASE_ENTRY */ ISP_FC_OPMAP(0x00, 0x00), /* 0x4b: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x4c: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x4d: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x4e: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x4f: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x50: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x51: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x52: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x53: */ ISP_FC_OPMAP(0xcf, 0x01), /* 0x54: EXECUTE IOCB A64 */ ISP_FC_OPMAP(0x00, 0x00), /* 0x55: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x56: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x57: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x58: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x59: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x5a: */ ISP_FC_OPMAP(0x03, 0x01), /* 0x5b: MBOX_DRIVER_HEARTBEAT */ ISP_FC_OPMAP(0xcf, 0x01), /* 0x5c: MBOX_FW_HEARTBEAT */ ISP_FC_OPMAP(0x07, 0x1f), /* 0x5d: MBOX_GET_SET_DATA_RATE */ ISP_FC_OPMAP(0x00, 0x00), /* 0x5e: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x5f: */ ISP_FC_OPMAP(0xcf, 0x0f), /* 0x60: MBOX_INIT_FIRMWARE */ ISP_FC_OPMAP(0x00, 0x00), /* 0x61: */ ISP_FC_OPMAP(0x01, 0x01), /* 0x62: MBOX_INIT_LIP */ ISP_FC_OPMAP(0xcd, 0x03), /* 0x63: MBOX_GET_FC_AL_POSITION_MAP */ ISP_FC_OPMAP(0xcf, 0x01), /* 0x64: MBOX_GET_PORT_DB */ ISP_FC_OPMAP(0x07, 0x01), /* 0x65: MBOX_CLEAR_ACA */ ISP_FC_OPMAP(0x07, 0x01), /* 0x66: MBOX_TARGET_RESET */ ISP_FC_OPMAP(0x07, 0x01), /* 0x67: MBOX_CLEAR_TASK_SET */ ISP_FC_OPMAP(0x07, 0x01), /* 0x68: MBOX_ABORT_TASK_SET */ ISP_FC_OPMAP(0x01, 0x07), /* 0x69: MBOX_GET_FW_STATE */ ISP_FC_OPMAP_HALF(0x6, 0x03, 0x0, 0xcf), /* 0x6a: MBOX_GET_PORT_NAME */ ISP_FC_OPMAP(0xcf, 0x01), /* 0x6b: MBOX_GET_LINK_STATUS */ ISP_FC_OPMAP(0x0f, 0x01), /* 0x6c: MBOX_INIT_LIP_RESET */ ISP_FC_OPMAP(0x00, 0x00), /* 0x6d: */ ISP_FC_OPMAP(0xcf, 0x03), /* 0x6e: MBOX_SEND_SNS */ ISP_FC_OPMAP(0x0f, 0x07), /* 0x6f: MBOX_FABRIC_LOGIN */ ISP_FC_OPMAP_HALF(0x02, 0x03, 0x00, 0x03), /* 0x70: MBOX_SEND_CHANGE_REQUEST */ ISP_FC_OPMAP(0x03, 0x03), /* 0x71: MBOX_FABRIC_LOGOUT */ ISP_FC_OPMAP(0x0f, 0x0f), /* 0x72: MBOX_INIT_LIP_LOGIN */ ISP_FC_OPMAP(0x00, 0x00), /* 0x73: */ ISP_FC_OPMAP(0x07, 0x01), /* 0x74: LOGIN LOOP PORT */ ISP_FC_OPMAP_HALF(0x03, 0xcf, 0x00, 0x07), /* 0x75: GET PORT/NODE NAME LIST */ ISP_FC_OPMAP(0x4f, 0x01), /* 0x76: SET VENDOR ID */ ISP_FC_OPMAP(0xcd, 0x01), /* 0x77: INITIALIZE IP MAILBOX */ ISP_FC_OPMAP(0x00, 0x00), /* 0x78: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x79: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x7a: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x7b: */ ISP_FC_OPMAP_HALF(0x03, 0x4f, 0x00, 0x07), /* 0x7c: Get ID List */ ISP_FC_OPMAP(0xcf, 0x01), /* 0x7d: SEND LFA */ ISP_FC_OPMAP(0x0f, 0x01) /* 0x7e: LUN RESET */ }; #define MAX_FC_OPCODE 0x7e /* * Footnotes * * (1): this sets bits 21..16 in mailbox register #8, which we nominally * do not access at this time in the core driver. The caller is * responsible for setting this register first (Gross!). The assumption * is that we won't overflow. */ static const char *fc_mbcmd_names[] = { "NO-OP", /* 00h */ "LOAD RAM", "EXEC FIRMWARE", "DUMP RAM", "WRITE RAM WORD", "READ RAM WORD", "MAILBOX REG TEST", "VERIFY CHECKSUM", "ABOUT FIRMWARE", "LOAD RAM (2100)", "DUMP RAM", "LOAD RISC RAM", "DUMP RISC RAM", "WRITE RAM WORD EXTENDED", "CHECK FIRMWARE", "READ RAM WORD EXTENDED", "INIT REQUEST QUEUE", /* 10h */ "INIT RESULT QUEUE", "EXECUTE IOCB", "WAKE UP", "STOP FIRMWARE", "ABORT", "ABORT DEVICE", "ABORT TARGET", "BUS RESET", "STOP QUEUE", "START QUEUE", "SINGLE STEP QUEUE", "ABORT QUEUE", "GET DEV QUEUE STATUS", NULL, "GET FIRMWARE STATUS", "GET LOOP ID", /* 20h */ NULL, "GET TIMEOUT PARAMS", NULL, NULL, NULL, NULL, NULL, "GET FIRMWARE OPTIONS", "GET PORT QUEUE PARAMS", "GENERATE SYSTEM ERROR", NULL, NULL, NULL, NULL, NULL, "WRITE SFP", /* 30h */ "READ SFP", "SET TIMEOUT PARAMS", NULL, NULL, NULL, NULL, NULL, "SET FIRMWARE OPTIONS", "SET PORT QUEUE PARAMS", NULL, "SET FC LED CONF", NULL, "RESTART NIC FIRMWARE", "ACCESS CONTROL", NULL, "LOOP PORT BYPASS", /* 40h */ "LOOP PORT ENABLE", "GET RESOURCE COUNT", "REQUEST NON PARTICIPATING MODE", "DIAGNOSTIC ECHO TEST", "DIAGNOSTIC LOOPBACK", NULL, "GET PORT DATABASE ENHANCED", "INIT FIRMWARE MULTI ID", "GET VP DATABASE", "GET VP DATABASE ENTRY", NULL, NULL, NULL, NULL, NULL, "GET FCF LIST", /* 50h */ "GET DCBX PARAMETERS", NULL, "HOST MEMORY COPY", "EXECUTE IOCB A64", NULL, NULL, "SEND RNID", NULL, "SET PARAMETERS", "GET PARAMETERS", "DRIVER HEARTBEAT", "FIRMWARE HEARTBEAT", "GET/SET DATA RATE", "SEND RNFT", NULL, "INIT FIRMWARE", /* 60h */ "GET INIT CONTROL BLOCK", "INIT LIP", "GET FC-AL POSITION MAP", "GET PORT DATABASE", "CLEAR ACA", "TARGET RESET", "CLEAR TASK SET", "ABORT TASK SET", "GET FW STATE", "GET PORT NAME", "GET LINK STATUS", "INIT LIP RESET", "GET LINK STATS & PRIVATE DATA CNTS", "SEND SNS", "FABRIC LOGIN", "SEND CHANGE REQUEST", /* 70h */ "FABRIC LOGOUT", "INIT LIP LOGIN", NULL, "LOGIN LOOP PORT", "GET PORT/NODE NAME LIST", "SET VENDOR ID", "INITIALIZE IP MAILBOX", NULL, NULL, "GET XGMAC STATS", NULL, "GET ID LIST", "SEND LFA", "LUN RESET" }; static void isp_mboxcmd_qnw(ispsoftc_t *isp, mbreg_t *mbp, int nodelay) { unsigned int ibits, obits, box, opcode; opcode = mbp->param[0]; if (IS_FC(isp)) { ibits = ISP_FC_IBITS(opcode); obits = ISP_FC_OBITS(opcode); } else { ibits = ISP_SCSI_IBITS(opcode); obits = ISP_SCSI_OBITS(opcode); } ibits |= mbp->ibits; obits |= mbp->obits; for (box = 0; box < ISP_NMBOX(isp); box++) { if (ibits & (1 << box)) { ISP_WRITE(isp, MBOX_OFF(box), mbp->param[box]); } if (nodelay == 0) { isp->isp_mboxtmp[box] = mbp->param[box] = 0; } } if (nodelay == 0) { isp->isp_lastmbxcmd = opcode; isp->isp_obits = obits; isp->isp_mboxbsy = 1; } if (IS_24XX(isp)) { ISP_WRITE(isp, BIU2400_HCCR, HCCR_2400_CMD_SET_HOST_INT); } else { ISP_WRITE(isp, HCCR, HCCR_CMD_SET_HOST_INT); } /* * Oddly enough, if we're not delaying for an answer, * delay a bit to give the f/w a chance to pick up the * command. */ if (nodelay) { ISP_DELAY(1000); } } static void isp_mboxcmd(ispsoftc_t *isp, mbreg_t *mbp) { const char *cname, *xname, *sname; char tname[16], mname[16]; unsigned int ibits, obits, box, opcode; opcode = mbp->param[0]; if (IS_FC(isp)) { if (opcode > MAX_FC_OPCODE) { mbp->param[0] = MBOX_INVALID_COMMAND; isp_prt(isp, ISP_LOGERR, "Unknown Command 0x%x", opcode); return; } cname = fc_mbcmd_names[opcode]; ibits = ISP_FC_IBITS(opcode); obits = ISP_FC_OBITS(opcode); } else { if (opcode > MAX_SCSI_OPCODE) { mbp->param[0] = MBOX_INVALID_COMMAND; isp_prt(isp, ISP_LOGERR, "Unknown Command 0x%x", opcode); return; } cname = scsi_mbcmd_names[opcode]; ibits = ISP_SCSI_IBITS(opcode); obits = ISP_SCSI_OBITS(opcode); } if (cname == NULL) { cname = tname; ISP_SNPRINTF(tname, sizeof tname, "opcode %x", opcode); } isp_prt(isp, ISP_LOGDEBUG3, "Mailbox Command '%s'", cname); /* * Pick up any additional bits that the caller might have set. */ ibits |= mbp->ibits; obits |= mbp->obits; /* * Mask any bits that the caller wants us to mask */ ibits &= mbp->ibitm; obits &= mbp->obitm; if (ibits == 0 && obits == 0) { mbp->param[0] = MBOX_COMMAND_PARAM_ERROR; isp_prt(isp, ISP_LOGERR, "no parameters for 0x%x", opcode); return; } /* * Get exclusive usage of mailbox registers. */ if (MBOX_ACQUIRE(isp)) { mbp->param[0] = MBOX_REGS_BUSY; goto out; } for (box = 0; box < ISP_NMBOX(isp); box++) { if (ibits & (1 << box)) { isp_prt(isp, ISP_LOGDEBUG3, "IN mbox %d = 0x%04x", box, mbp->param[box]); ISP_WRITE(isp, MBOX_OFF(box), mbp->param[box]); } isp->isp_mboxtmp[box] = mbp->param[box] = 0; } isp->isp_lastmbxcmd = opcode; /* * We assume that we can't overwrite a previous command. */ isp->isp_obits = obits; isp->isp_mboxbsy = 1; /* * Set Host Interrupt condition so that RISC will pick up mailbox regs. */ if (IS_24XX(isp)) { ISP_WRITE(isp, BIU2400_HCCR, HCCR_2400_CMD_SET_HOST_INT); } else { ISP_WRITE(isp, HCCR, HCCR_CMD_SET_HOST_INT); } /* * While we haven't finished the command, spin our wheels here. */ MBOX_WAIT_COMPLETE(isp, mbp); /* * Did the command time out? */ if (mbp->param[0] == MBOX_TIMEOUT) { isp->isp_mboxbsy = 0; MBOX_RELEASE(isp); goto out; } /* * Copy back output registers. */ for (box = 0; box < ISP_NMBOX(isp); box++) { if (obits & (1 << box)) { mbp->param[box] = isp->isp_mboxtmp[box]; isp_prt(isp, ISP_LOGDEBUG3, "OUT mbox %d = 0x%04x", box, mbp->param[box]); } } isp->isp_mboxbsy = 0; MBOX_RELEASE(isp); out: if (mbp->logval == 0 || mbp->param[0] == MBOX_COMMAND_COMPLETE) return; if ((mbp->param[0] & 0xbfe0) == 0 && (mbp->logval & MBLOGMASK(mbp->param[0])) == 0) return; xname = NULL; sname = ""; switch (mbp->param[0]) { case MBOX_INVALID_COMMAND: xname = "INVALID COMMAND"; break; case MBOX_HOST_INTERFACE_ERROR: xname = "HOST INTERFACE ERROR"; break; case MBOX_TEST_FAILED: xname = "TEST FAILED"; break; case MBOX_COMMAND_ERROR: xname = "COMMAND ERROR"; ISP_SNPRINTF(mname, sizeof(mname), " subcode 0x%x", mbp->param[1]); sname = mname; break; case MBOX_COMMAND_PARAM_ERROR: xname = "COMMAND PARAMETER ERROR"; break; case MBOX_PORT_ID_USED: xname = "PORT ID ALREADY IN USE"; break; case MBOX_LOOP_ID_USED: xname = "LOOP ID ALREADY IN USE"; break; case MBOX_ALL_IDS_USED: xname = "ALL LOOP IDS IN USE"; break; case MBOX_NOT_LOGGED_IN: xname = "NOT LOGGED IN"; break; case MBOX_LINK_DOWN_ERROR: xname = "LINK DOWN ERROR"; break; case MBOX_LOOPBACK_ERROR: xname = "LOOPBACK ERROR"; break; case MBOX_CHECKSUM_ERROR: xname = "CHECKSUM ERROR"; break; case MBOX_INVALID_PRODUCT_KEY: xname = "INVALID PRODUCT KEY"; break; case MBOX_REGS_BUSY: xname = "REGISTERS BUSY"; break; case MBOX_TIMEOUT: xname = "TIMEOUT"; break; default: ISP_SNPRINTF(mname, sizeof mname, "error 0x%x", mbp->param[0]); xname = mname; break; } if (xname) { isp_prt(isp, ISP_LOGALL, "Mailbox Command '%s' failed (%s%s)", cname, xname, sname); } } static int isp_fw_state(ispsoftc_t *isp, int chan) { if (IS_FC(isp)) { mbreg_t mbs; MBSINIT(&mbs, MBOX_GET_FW_STATE, MBLOGALL, 0); isp_mboxcmd(isp, &mbs); if (mbs.param[0] == MBOX_COMMAND_COMPLETE) { return (mbs.param[1]); } } return (FW_ERROR); } static void isp_spi_update(ispsoftc_t *isp, int chan) { int tgt; mbreg_t mbs; sdparam *sdp; if (IS_FC(isp)) { /* * There are no 'per-bus' settings for Fibre Channel. */ return; } sdp = SDPARAM(isp, chan); sdp->update = 0; for (tgt = 0; tgt < MAX_TARGETS; tgt++) { uint16_t flags, period, offset; int get; if (sdp->isp_devparam[tgt].dev_enable == 0) { sdp->isp_devparam[tgt].dev_update = 0; sdp->isp_devparam[tgt].dev_refresh = 0; isp_prt(isp, ISP_LOGDEBUG0, "skipping target %d bus %d update", tgt, chan); continue; } /* * If the goal is to update the status of the device, * take what's in goal_flags and try and set the device * toward that. Otherwise, if we're just refreshing the * current device state, get the current parameters. */ MBSINIT(&mbs, 0, MBLOGALL, 0); /* * Refresh overrides set */ if (sdp->isp_devparam[tgt].dev_refresh) { mbs.param[0] = MBOX_GET_TARGET_PARAMS; get = 1; } else if (sdp->isp_devparam[tgt].dev_update) { mbs.param[0] = MBOX_SET_TARGET_PARAMS; /* * Make sure goal_flags has "Renegotiate on Error" * on and "Freeze Queue on Error" off. */ sdp->isp_devparam[tgt].goal_flags |= DPARM_RENEG; sdp->isp_devparam[tgt].goal_flags &= ~DPARM_QFRZ; mbs.param[2] = sdp->isp_devparam[tgt].goal_flags; /* * Insist that PARITY must be enabled * if SYNC or WIDE is enabled. */ if ((mbs.param[2] & (DPARM_SYNC|DPARM_WIDE)) != 0) { mbs.param[2] |= DPARM_PARITY; } if (mbs.param[2] & DPARM_SYNC) { mbs.param[3] = (sdp->isp_devparam[tgt].goal_offset << 8) | (sdp->isp_devparam[tgt].goal_period); } /* * A command completion later that has * RQSTF_NEGOTIATION set can cause * the dev_refresh/announce cycle also. * * Note: It is really important to update our current * flags with at least the state of TAG capabilities- * otherwise we might try and send a tagged command * when we have it all turned off. So change it here * to say that current already matches goal. */ sdp->isp_devparam[tgt].actv_flags &= ~DPARM_TQING; sdp->isp_devparam[tgt].actv_flags |= (sdp->isp_devparam[tgt].goal_flags & DPARM_TQING); isp_prt(isp, ISP_LOGDEBUG0, "bus %d set tgt %d flags 0x%x off 0x%x period 0x%x", chan, tgt, mbs.param[2], mbs.param[3] >> 8, mbs.param[3] & 0xff); get = 0; } else { continue; } mbs.param[1] = (chan << 15) | (tgt << 8); isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { continue; } if (get == 0) { sdp->sendmarker = 1; sdp->isp_devparam[tgt].dev_update = 0; sdp->isp_devparam[tgt].dev_refresh = 1; } else { sdp->isp_devparam[tgt].dev_refresh = 0; flags = mbs.param[2]; period = mbs.param[3] & 0xff; offset = mbs.param[3] >> 8; sdp->isp_devparam[tgt].actv_flags = flags; sdp->isp_devparam[tgt].actv_period = period; sdp->isp_devparam[tgt].actv_offset = offset; isp_async(isp, ISPASYNC_NEW_TGT_PARAMS, chan, tgt); } } for (tgt = 0; tgt < MAX_TARGETS; tgt++) { if (sdp->isp_devparam[tgt].dev_update || sdp->isp_devparam[tgt].dev_refresh) { sdp->update = 1; break; } } } static void isp_setdfltsdparm(ispsoftc_t *isp) { int tgt; sdparam *sdp, *sdp1; sdp = SDPARAM(isp, 0); if (IS_DUALBUS(isp)) sdp1 = sdp + 1; else sdp1 = NULL; /* * Establish some default parameters. */ sdp->isp_cmd_dma_burst_enable = 0; sdp->isp_data_dma_burst_enabl = 1; sdp->isp_fifo_threshold = 0; sdp->isp_initiator_id = DEFAULT_IID(isp, 0); if (isp->isp_type >= ISP_HA_SCSI_1040) { sdp->isp_async_data_setup = 9; } else { sdp->isp_async_data_setup = 6; } sdp->isp_selection_timeout = 250; sdp->isp_max_queue_depth = MAXISPREQUEST(isp); sdp->isp_tag_aging = 8; sdp->isp_bus_reset_delay = 5; /* * Don't retry selection, busy or queue full automatically- reflect * these back to us. */ sdp->isp_retry_count = 0; sdp->isp_retry_delay = 0; for (tgt = 0; tgt < MAX_TARGETS; tgt++) { sdp->isp_devparam[tgt].exc_throttle = ISP_EXEC_THROTTLE; sdp->isp_devparam[tgt].dev_enable = 1; } /* * The trick here is to establish a default for the default (honk!) * state (goal_flags). Then try and get the current status from * the card to fill in the current state. We don't, in fact, set * the default to the SAFE default state- that's not the goal state. */ for (tgt = 0; tgt < MAX_TARGETS; tgt++) { uint8_t off, per; sdp->isp_devparam[tgt].actv_offset = 0; sdp->isp_devparam[tgt].actv_period = 0; sdp->isp_devparam[tgt].actv_flags = 0; sdp->isp_devparam[tgt].goal_flags = sdp->isp_devparam[tgt].nvrm_flags = DPARM_DEFAULT; /* * We default to Wide/Fast for versions less than a 1040 * (unless it's SBus). */ if (IS_ULTRA3(isp)) { off = ISP_80M_SYNCPARMS >> 8; per = ISP_80M_SYNCPARMS & 0xff; } else if (IS_ULTRA2(isp)) { off = ISP_40M_SYNCPARMS >> 8; per = ISP_40M_SYNCPARMS & 0xff; } else if (IS_1240(isp)) { off = ISP_20M_SYNCPARMS >> 8; per = ISP_20M_SYNCPARMS & 0xff; } else if ((isp->isp_bustype == ISP_BT_SBUS && isp->isp_type < ISP_HA_SCSI_1020A) || (isp->isp_bustype == ISP_BT_PCI && isp->isp_type < ISP_HA_SCSI_1040) || (isp->isp_clock && isp->isp_clock < 60) || (sdp->isp_ultramode == 0)) { off = ISP_10M_SYNCPARMS >> 8; per = ISP_10M_SYNCPARMS & 0xff; } else { off = ISP_20M_SYNCPARMS_1040 >> 8; per = ISP_20M_SYNCPARMS_1040 & 0xff; } sdp->isp_devparam[tgt].goal_offset = sdp->isp_devparam[tgt].nvrm_offset = off; sdp->isp_devparam[tgt].goal_period = sdp->isp_devparam[tgt].nvrm_period = per; } /* * If we're a dual bus card, just copy the data over */ if (sdp1) { *sdp1 = *sdp; sdp1->isp_initiator_id = DEFAULT_IID(isp, 1); } /* * If we've not been told to avoid reading NVRAM, try and read it. * If we're successful reading it, we can then return because NVRAM * will tell us what the desired settings are. Otherwise, we establish * some reasonable 'fake' nvram and goal defaults. */ if ((isp->isp_confopts & ISP_CFG_NONVRAM) == 0) { mbreg_t mbs; if (isp_read_nvram(isp, 0) == 0) { if (IS_DUALBUS(isp)) { if (isp_read_nvram(isp, 1) == 0) { return; } } } MBSINIT(&mbs, MBOX_GET_ACT_NEG_STATE, MBLOGNONE, 0); isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { sdp->isp_req_ack_active_neg = 1; sdp->isp_data_line_active_neg = 1; if (sdp1) { sdp1->isp_req_ack_active_neg = 1; sdp1->isp_data_line_active_neg = 1; } } else { sdp->isp_req_ack_active_neg = (mbs.param[1] >> 4) & 0x1; sdp->isp_data_line_active_neg = (mbs.param[1] >> 5) & 0x1; if (sdp1) { sdp1->isp_req_ack_active_neg = (mbs.param[2] >> 4) & 0x1; sdp1->isp_data_line_active_neg = (mbs.param[2] >> 5) & 0x1; } } } } static void isp_setdfltfcparm(ispsoftc_t *isp, int chan) { fcparam *fcp = FCPARAM(isp, chan); /* * Establish some default parameters. */ fcp->role = DEFAULT_ROLE(isp, chan); fcp->isp_maxalloc = ICB_DFLT_ALLOC; fcp->isp_retry_delay = ICB_DFLT_RDELAY; fcp->isp_retry_count = ICB_DFLT_RCOUNT; fcp->isp_loopid = DEFAULT_LOOPID(isp, chan); fcp->isp_wwnn_nvram = DEFAULT_NODEWWN(isp, chan); fcp->isp_wwpn_nvram = DEFAULT_PORTWWN(isp, chan); fcp->isp_fwoptions = 0; fcp->isp_xfwoptions = 0; fcp->isp_zfwoptions = 0; fcp->isp_lasthdl = NIL_HANDLE; if (IS_24XX(isp)) { fcp->isp_fwoptions |= ICB2400_OPT1_FAIRNESS; fcp->isp_fwoptions |= ICB2400_OPT1_HARD_ADDRESS; if (isp->isp_confopts & ISP_CFG_FULL_DUPLEX) { fcp->isp_fwoptions |= ICB2400_OPT1_FULL_DUPLEX; } fcp->isp_fwoptions |= ICB2400_OPT1_BOTH_WWNS; } else { fcp->isp_fwoptions |= ICBOPT_FAIRNESS; fcp->isp_fwoptions |= ICBOPT_PDBCHANGE_AE; fcp->isp_fwoptions |= ICBOPT_HARD_ADDRESS; if (isp->isp_confopts & ISP_CFG_FULL_DUPLEX) { fcp->isp_fwoptions |= ICBOPT_FULL_DUPLEX; } /* * Make sure this is turned off now until we get * extended options from NVRAM */ fcp->isp_fwoptions &= ~ICBOPT_EXTENDED; } /* * Now try and read NVRAM unless told to not do so. * This will set fcparam's isp_wwnn_nvram && isp_wwpn_nvram. */ if ((isp->isp_confopts & ISP_CFG_NONVRAM) == 0) { int i, j = 0; /* * Give a couple of tries at reading NVRAM. */ for (i = 0; i < 2; i++) { j = isp_read_nvram(isp, chan); if (j == 0) { break; } } if (j) { isp->isp_confopts |= ISP_CFG_NONVRAM; } } fcp->isp_wwnn = ACTIVE_NODEWWN(isp, chan); fcp->isp_wwpn = ACTIVE_PORTWWN(isp, chan); isp_prt(isp, ISP_LOGCONFIG, "Chan %d 0x%08x%08x/0x%08x%08x Role %s", chan, (uint32_t) (fcp->isp_wwnn >> 32), (uint32_t) (fcp->isp_wwnn), (uint32_t) (fcp->isp_wwpn >> 32), (uint32_t) (fcp->isp_wwpn), isp_class3_roles[fcp->role]); } /* * Re-initialize the ISP and complete all orphaned commands * with a 'botched' notice. The reset/init routines should * not disturb an already active list of commands. */ int isp_reinit(ispsoftc_t *isp, int do_load_defaults) { int i, res = 0; if (isp->isp_state == ISP_RUNSTATE) isp_deinit(isp); if (isp->isp_state != ISP_RESETSTATE) isp_reset(isp, do_load_defaults); if (isp->isp_state != ISP_RESETSTATE) { res = EIO; isp_prt(isp, ISP_LOGERR, "%s: cannot reset card", __func__); ISP_DISABLE_INTS(isp); goto cleanup; } isp_init(isp); if (isp->isp_state > ISP_RESETSTATE && isp->isp_state != ISP_RUNSTATE) { res = EIO; isp_prt(isp, ISP_LOGERR, "%s: cannot init card", __func__); ISP_DISABLE_INTS(isp); if (IS_FC(isp)) { /* * If we're in ISP_ROLE_NONE, turn off the lasers. */ if (!IS_24XX(isp)) { ISP_WRITE(isp, BIU2100_CSR, BIU2100_FPM0_REGS); ISP_WRITE(isp, FPM_DIAG_CONFIG, FPM_SOFT_RESET); ISP_WRITE(isp, BIU2100_CSR, BIU2100_FB_REGS); ISP_WRITE(isp, FBM_CMD, FBMCMD_FIFO_RESET_ALL); ISP_WRITE(isp, BIU2100_CSR, BIU2100_RISC_REGS); } } } cleanup: isp->isp_nactive = 0; isp_clear_commands(isp); if (IS_FC(isp)) { for (i = 0; i < isp->isp_nchan; i++) isp_clear_portdb(isp, i); } return (res); } /* * NVRAM Routines */ static int isp_read_nvram(ispsoftc_t *isp, int bus) { int i, amt, retval; uint8_t csum, minversion; union { uint8_t _x[ISP2400_NVRAM_SIZE]; uint16_t _s[ISP2400_NVRAM_SIZE>>1]; } _n; #define nvram_data _n._x #define nvram_words _n._s if (IS_24XX(isp)) { return (isp_read_nvram_2400(isp, nvram_data)); } else if (IS_FC(isp)) { amt = ISP2100_NVRAM_SIZE; minversion = 1; } else if (IS_ULTRA2(isp)) { amt = ISP1080_NVRAM_SIZE; minversion = 0; } else { amt = ISP_NVRAM_SIZE; minversion = 2; } for (i = 0; i < amt>>1; i++) { isp_rdnvram_word(isp, i, &nvram_words[i]); } if (nvram_data[0] != 'I' || nvram_data[1] != 'S' || nvram_data[2] != 'P') { if (isp->isp_bustype != ISP_BT_SBUS) { isp_prt(isp, ISP_LOGWARN, "invalid NVRAM header"); isp_prt(isp, ISP_LOGDEBUG0, "%x %x %x", nvram_data[0], nvram_data[1], nvram_data[2]); } retval = -1; goto out; } for (csum = 0, i = 0; i < amt; i++) { csum += nvram_data[i]; } if (csum != 0) { isp_prt(isp, ISP_LOGWARN, "invalid NVRAM checksum"); retval = -1; goto out; } if (ISP_NVRAM_VERSION(nvram_data) < minversion) { isp_prt(isp, ISP_LOGWARN, "version %d NVRAM not understood", ISP_NVRAM_VERSION(nvram_data)); retval = -1; goto out; } if (IS_ULTRA3(isp)) { isp_parse_nvram_12160(isp, bus, nvram_data); } else if (IS_1080(isp)) { isp_parse_nvram_1080(isp, bus, nvram_data); } else if (IS_1280(isp) || IS_1240(isp)) { isp_parse_nvram_1080(isp, bus, nvram_data); } else if (IS_SCSI(isp)) { isp_parse_nvram_1020(isp, nvram_data); } else { isp_parse_nvram_2100(isp, nvram_data); } retval = 0; out: return (retval); #undef nvram_data #undef nvram_words } static int isp_read_nvram_2400(ispsoftc_t *isp, uint8_t *nvram_data) { int retval = 0; uint32_t addr, csum, lwrds, *dptr; if (isp->isp_port) { addr = ISP2400_NVRAM_PORT1_ADDR; } else { addr = ISP2400_NVRAM_PORT0_ADDR; } dptr = (uint32_t *) nvram_data; for (lwrds = 0; lwrds < ISP2400_NVRAM_SIZE >> 2; lwrds++) { isp_rd_2400_nvram(isp, addr++, dptr++); } if (nvram_data[0] != 'I' || nvram_data[1] != 'S' || nvram_data[2] != 'P') { isp_prt(isp, ISP_LOGWARN, "invalid NVRAM header (%x %x %x)", nvram_data[0], nvram_data[1], nvram_data[2]); retval = -1; goto out; } dptr = (uint32_t *) nvram_data; for (csum = 0, lwrds = 0; lwrds < ISP2400_NVRAM_SIZE >> 2; lwrds++) { uint32_t tmp; ISP_IOXGET_32(isp, &dptr[lwrds], tmp); csum += tmp; } if (csum != 0) { isp_prt(isp, ISP_LOGWARN, "invalid NVRAM checksum"); retval = -1; goto out; } isp_parse_nvram_2400(isp, nvram_data); out: return (retval); } static void isp_rdnvram_word(ispsoftc_t *isp, int wo, uint16_t *rp) { int i, cbits; uint16_t bit, rqst, junk; ISP_WRITE(isp, BIU_NVRAM, BIU_NVRAM_SELECT); ISP_DELAY(10); ISP_WRITE(isp, BIU_NVRAM, BIU_NVRAM_SELECT|BIU_NVRAM_CLOCK); ISP_DELAY(10); if (IS_FC(isp)) { wo &= ((ISP2100_NVRAM_SIZE >> 1) - 1); if (IS_2312(isp) && isp->isp_port) { wo += 128; } rqst = (ISP_NVRAM_READ << 8) | wo; cbits = 10; } else if (IS_ULTRA2(isp)) { wo &= ((ISP1080_NVRAM_SIZE >> 1) - 1); rqst = (ISP_NVRAM_READ << 8) | wo; cbits = 10; } else { wo &= ((ISP_NVRAM_SIZE >> 1) - 1); rqst = (ISP_NVRAM_READ << 6) | wo; cbits = 8; } /* * Clock the word select request out... */ for (i = cbits; i >= 0; i--) { if ((rqst >> i) & 1) { bit = BIU_NVRAM_SELECT | BIU_NVRAM_DATAOUT; } else { bit = BIU_NVRAM_SELECT; } ISP_WRITE(isp, BIU_NVRAM, bit); ISP_DELAY(10); junk = ISP_READ(isp, BIU_NVRAM); /* force PCI flush */ ISP_WRITE(isp, BIU_NVRAM, bit | BIU_NVRAM_CLOCK); ISP_DELAY(10); junk = ISP_READ(isp, BIU_NVRAM); /* force PCI flush */ ISP_WRITE(isp, BIU_NVRAM, bit); ISP_DELAY(10); junk = ISP_READ(isp, BIU_NVRAM); /* force PCI flush */ } /* * Now read the result back in (bits come back in MSB format). */ *rp = 0; for (i = 0; i < 16; i++) { uint16_t rv; *rp <<= 1; ISP_WRITE(isp, BIU_NVRAM, BIU_NVRAM_SELECT|BIU_NVRAM_CLOCK); ISP_DELAY(10); rv = ISP_READ(isp, BIU_NVRAM); if (rv & BIU_NVRAM_DATAIN) { *rp |= 1; } ISP_DELAY(10); ISP_WRITE(isp, BIU_NVRAM, BIU_NVRAM_SELECT); ISP_DELAY(10); junk = ISP_READ(isp, BIU_NVRAM); /* force PCI flush */ } ISP_WRITE(isp, BIU_NVRAM, 0); ISP_DELAY(10); junk = ISP_READ(isp, BIU_NVRAM); /* force PCI flush */ ISP_SWIZZLE_NVRAM_WORD(isp, rp); } static void isp_rd_2400_nvram(ispsoftc_t *isp, uint32_t addr, uint32_t *rp) { int loops = 0; uint32_t base = 0x7ffe0000; uint32_t tmp = 0; - if (IS_25XX(isp)) { + if (IS_26XX(isp)) { + base = 0x7fe7c000; /* XXX: Observation, may be wrong. */ + } else if (IS_25XX(isp)) { base = 0x7ff00000 | 0x48000; } ISP_WRITE(isp, BIU2400_FLASH_ADDR, base | addr); for (loops = 0; loops < 5000; loops++) { ISP_DELAY(10); tmp = ISP_READ(isp, BIU2400_FLASH_ADDR); if ((tmp & (1U << 31)) != 0) { break; } } if (tmp & (1U << 31)) { *rp = ISP_READ(isp, BIU2400_FLASH_DATA); ISP_SWIZZLE_NVRAM_LONG(isp, rp); } else { *rp = 0xffffffff; } } static void isp_parse_nvram_1020(ispsoftc_t *isp, uint8_t *nvram_data) { sdparam *sdp = SDPARAM(isp, 0); int tgt; sdp->isp_fifo_threshold = ISP_NVRAM_FIFO_THRESHOLD(nvram_data) | (ISP_NVRAM_FIFO_THRESHOLD_128(nvram_data) << 2); if ((isp->isp_confopts & ISP_CFG_OWNLOOPID) == 0) sdp->isp_initiator_id = ISP_NVRAM_INITIATOR_ID(nvram_data); sdp->isp_bus_reset_delay = ISP_NVRAM_BUS_RESET_DELAY(nvram_data); sdp->isp_retry_count = ISP_NVRAM_BUS_RETRY_COUNT(nvram_data); sdp->isp_retry_delay = ISP_NVRAM_BUS_RETRY_DELAY(nvram_data); sdp->isp_async_data_setup = ISP_NVRAM_ASYNC_DATA_SETUP_TIME(nvram_data); if (isp->isp_type >= ISP_HA_SCSI_1040) { if (sdp->isp_async_data_setup < 9) { sdp->isp_async_data_setup = 9; } } else { if (sdp->isp_async_data_setup != 6) { sdp->isp_async_data_setup = 6; } } sdp->isp_req_ack_active_neg = ISP_NVRAM_REQ_ACK_ACTIVE_NEGATION(nvram_data); sdp->isp_data_line_active_neg = ISP_NVRAM_DATA_LINE_ACTIVE_NEGATION(nvram_data); sdp->isp_data_dma_burst_enabl = ISP_NVRAM_DATA_DMA_BURST_ENABLE(nvram_data); sdp->isp_cmd_dma_burst_enable = ISP_NVRAM_CMD_DMA_BURST_ENABLE(nvram_data); sdp->isp_tag_aging = ISP_NVRAM_TAG_AGE_LIMIT(nvram_data); sdp->isp_selection_timeout = ISP_NVRAM_SELECTION_TIMEOUT(nvram_data); sdp->isp_max_queue_depth = ISP_NVRAM_MAX_QUEUE_DEPTH(nvram_data); sdp->isp_fast_mttr = ISP_NVRAM_FAST_MTTR_ENABLE(nvram_data); for (tgt = 0; tgt < MAX_TARGETS; tgt++) { sdp->isp_devparam[tgt].dev_enable = ISP_NVRAM_TGT_DEVICE_ENABLE(nvram_data, tgt); sdp->isp_devparam[tgt].exc_throttle = ISP_NVRAM_TGT_EXEC_THROTTLE(nvram_data, tgt); sdp->isp_devparam[tgt].nvrm_offset = ISP_NVRAM_TGT_SYNC_OFFSET(nvram_data, tgt); sdp->isp_devparam[tgt].nvrm_period = ISP_NVRAM_TGT_SYNC_PERIOD(nvram_data, tgt); /* * We probably shouldn't lie about this, but it * it makes it much safer if we limit NVRAM values * to sanity. */ if (isp->isp_type < ISP_HA_SCSI_1040) { /* * If we're not ultra, we can't possibly * be a shorter period than this. */ if (sdp->isp_devparam[tgt].nvrm_period < 0x19) { sdp->isp_devparam[tgt].nvrm_period = 0x19; } if (sdp->isp_devparam[tgt].nvrm_offset > 0xc) { sdp->isp_devparam[tgt].nvrm_offset = 0x0c; } } else { if (sdp->isp_devparam[tgt].nvrm_offset > 0x8) { sdp->isp_devparam[tgt].nvrm_offset = 0x8; } } sdp->isp_devparam[tgt].nvrm_flags = 0; if (ISP_NVRAM_TGT_RENEG(nvram_data, tgt)) sdp->isp_devparam[tgt].nvrm_flags |= DPARM_RENEG; sdp->isp_devparam[tgt].nvrm_flags |= DPARM_ARQ; if (ISP_NVRAM_TGT_TQING(nvram_data, tgt)) sdp->isp_devparam[tgt].nvrm_flags |= DPARM_TQING; if (ISP_NVRAM_TGT_SYNC(nvram_data, tgt)) sdp->isp_devparam[tgt].nvrm_flags |= DPARM_SYNC; if (ISP_NVRAM_TGT_WIDE(nvram_data, tgt)) sdp->isp_devparam[tgt].nvrm_flags |= DPARM_WIDE; if (ISP_NVRAM_TGT_PARITY(nvram_data, tgt)) sdp->isp_devparam[tgt].nvrm_flags |= DPARM_PARITY; if (ISP_NVRAM_TGT_DISC(nvram_data, tgt)) sdp->isp_devparam[tgt].nvrm_flags |= DPARM_DISC; sdp->isp_devparam[tgt].actv_flags = 0; /* we don't know */ sdp->isp_devparam[tgt].goal_offset = sdp->isp_devparam[tgt].nvrm_offset; sdp->isp_devparam[tgt].goal_period = sdp->isp_devparam[tgt].nvrm_period; sdp->isp_devparam[tgt].goal_flags = sdp->isp_devparam[tgt].nvrm_flags; } } static void isp_parse_nvram_1080(ispsoftc_t *isp, int bus, uint8_t *nvram_data) { sdparam *sdp = SDPARAM(isp, bus); int tgt; sdp->isp_fifo_threshold = ISP1080_NVRAM_FIFO_THRESHOLD(nvram_data); if ((isp->isp_confopts & ISP_CFG_OWNLOOPID) == 0) sdp->isp_initiator_id = ISP1080_NVRAM_INITIATOR_ID(nvram_data, bus); sdp->isp_bus_reset_delay = ISP1080_NVRAM_BUS_RESET_DELAY(nvram_data, bus); sdp->isp_retry_count = ISP1080_NVRAM_BUS_RETRY_COUNT(nvram_data, bus); sdp->isp_retry_delay = ISP1080_NVRAM_BUS_RETRY_DELAY(nvram_data, bus); sdp->isp_async_data_setup = ISP1080_NVRAM_ASYNC_DATA_SETUP_TIME(nvram_data, bus); sdp->isp_req_ack_active_neg = ISP1080_NVRAM_REQ_ACK_ACTIVE_NEGATION(nvram_data, bus); sdp->isp_data_line_active_neg = ISP1080_NVRAM_DATA_LINE_ACTIVE_NEGATION(nvram_data, bus); sdp->isp_data_dma_burst_enabl = ISP1080_NVRAM_BURST_ENABLE(nvram_data); sdp->isp_cmd_dma_burst_enable = ISP1080_NVRAM_BURST_ENABLE(nvram_data); sdp->isp_selection_timeout = ISP1080_NVRAM_SELECTION_TIMEOUT(nvram_data, bus); sdp->isp_max_queue_depth = ISP1080_NVRAM_MAX_QUEUE_DEPTH(nvram_data, bus); for (tgt = 0; tgt < MAX_TARGETS; tgt++) { sdp->isp_devparam[tgt].dev_enable = ISP1080_NVRAM_TGT_DEVICE_ENABLE(nvram_data, tgt, bus); sdp->isp_devparam[tgt].exc_throttle = ISP1080_NVRAM_TGT_EXEC_THROTTLE(nvram_data, tgt, bus); sdp->isp_devparam[tgt].nvrm_offset = ISP1080_NVRAM_TGT_SYNC_OFFSET(nvram_data, tgt, bus); sdp->isp_devparam[tgt].nvrm_period = ISP1080_NVRAM_TGT_SYNC_PERIOD(nvram_data, tgt, bus); sdp->isp_devparam[tgt].nvrm_flags = 0; if (ISP1080_NVRAM_TGT_RENEG(nvram_data, tgt, bus)) sdp->isp_devparam[tgt].nvrm_flags |= DPARM_RENEG; sdp->isp_devparam[tgt].nvrm_flags |= DPARM_ARQ; if (ISP1080_NVRAM_TGT_TQING(nvram_data, tgt, bus)) sdp->isp_devparam[tgt].nvrm_flags |= DPARM_TQING; if (ISP1080_NVRAM_TGT_SYNC(nvram_data, tgt, bus)) sdp->isp_devparam[tgt].nvrm_flags |= DPARM_SYNC; if (ISP1080_NVRAM_TGT_WIDE(nvram_data, tgt, bus)) sdp->isp_devparam[tgt].nvrm_flags |= DPARM_WIDE; if (ISP1080_NVRAM_TGT_PARITY(nvram_data, tgt, bus)) sdp->isp_devparam[tgt].nvrm_flags |= DPARM_PARITY; if (ISP1080_NVRAM_TGT_DISC(nvram_data, tgt, bus)) sdp->isp_devparam[tgt].nvrm_flags |= DPARM_DISC; sdp->isp_devparam[tgt].actv_flags = 0; sdp->isp_devparam[tgt].goal_offset = sdp->isp_devparam[tgt].nvrm_offset; sdp->isp_devparam[tgt].goal_period = sdp->isp_devparam[tgt].nvrm_period; sdp->isp_devparam[tgt].goal_flags = sdp->isp_devparam[tgt].nvrm_flags; } } static void isp_parse_nvram_12160(ispsoftc_t *isp, int bus, uint8_t *nvram_data) { sdparam *sdp = SDPARAM(isp, bus); int tgt; sdp->isp_fifo_threshold = ISP12160_NVRAM_FIFO_THRESHOLD(nvram_data); if ((isp->isp_confopts & ISP_CFG_OWNLOOPID) == 0) sdp->isp_initiator_id = ISP12160_NVRAM_INITIATOR_ID(nvram_data, bus); sdp->isp_bus_reset_delay = ISP12160_NVRAM_BUS_RESET_DELAY(nvram_data, bus); sdp->isp_retry_count = ISP12160_NVRAM_BUS_RETRY_COUNT(nvram_data, bus); sdp->isp_retry_delay = ISP12160_NVRAM_BUS_RETRY_DELAY(nvram_data, bus); sdp->isp_async_data_setup = ISP12160_NVRAM_ASYNC_DATA_SETUP_TIME(nvram_data, bus); sdp->isp_req_ack_active_neg = ISP12160_NVRAM_REQ_ACK_ACTIVE_NEGATION(nvram_data, bus); sdp->isp_data_line_active_neg = ISP12160_NVRAM_DATA_LINE_ACTIVE_NEGATION(nvram_data, bus); sdp->isp_data_dma_burst_enabl = ISP12160_NVRAM_BURST_ENABLE(nvram_data); sdp->isp_cmd_dma_burst_enable = ISP12160_NVRAM_BURST_ENABLE(nvram_data); sdp->isp_selection_timeout = ISP12160_NVRAM_SELECTION_TIMEOUT(nvram_data, bus); sdp->isp_max_queue_depth = ISP12160_NVRAM_MAX_QUEUE_DEPTH(nvram_data, bus); for (tgt = 0; tgt < MAX_TARGETS; tgt++) { sdp->isp_devparam[tgt].dev_enable = ISP12160_NVRAM_TGT_DEVICE_ENABLE(nvram_data, tgt, bus); sdp->isp_devparam[tgt].exc_throttle = ISP12160_NVRAM_TGT_EXEC_THROTTLE(nvram_data, tgt, bus); sdp->isp_devparam[tgt].nvrm_offset = ISP12160_NVRAM_TGT_SYNC_OFFSET(nvram_data, tgt, bus); sdp->isp_devparam[tgt].nvrm_period = ISP12160_NVRAM_TGT_SYNC_PERIOD(nvram_data, tgt, bus); sdp->isp_devparam[tgt].nvrm_flags = 0; if (ISP12160_NVRAM_TGT_RENEG(nvram_data, tgt, bus)) sdp->isp_devparam[tgt].nvrm_flags |= DPARM_RENEG; sdp->isp_devparam[tgt].nvrm_flags |= DPARM_ARQ; if (ISP12160_NVRAM_TGT_TQING(nvram_data, tgt, bus)) sdp->isp_devparam[tgt].nvrm_flags |= DPARM_TQING; if (ISP12160_NVRAM_TGT_SYNC(nvram_data, tgt, bus)) sdp->isp_devparam[tgt].nvrm_flags |= DPARM_SYNC; if (ISP12160_NVRAM_TGT_WIDE(nvram_data, tgt, bus)) sdp->isp_devparam[tgt].nvrm_flags |= DPARM_WIDE; if (ISP12160_NVRAM_TGT_PARITY(nvram_data, tgt, bus)) sdp->isp_devparam[tgt].nvrm_flags |= DPARM_PARITY; if (ISP12160_NVRAM_TGT_DISC(nvram_data, tgt, bus)) sdp->isp_devparam[tgt].nvrm_flags |= DPARM_DISC; sdp->isp_devparam[tgt].actv_flags = 0; sdp->isp_devparam[tgt].goal_offset = sdp->isp_devparam[tgt].nvrm_offset; sdp->isp_devparam[tgt].goal_period = sdp->isp_devparam[tgt].nvrm_period; sdp->isp_devparam[tgt].goal_flags = sdp->isp_devparam[tgt].nvrm_flags; } } static void isp_parse_nvram_2100(ispsoftc_t *isp, uint8_t *nvram_data) { fcparam *fcp = FCPARAM(isp, 0); uint64_t wwn; /* * There is NVRAM storage for both Port and Node entities- * but the Node entity appears to be unused on all the cards * I can find. However, we should account for this being set * at some point in the future. * * Qlogic WWNs have an NAA of 2, but usually nothing shows up in * bits 48..60. In the case of the 2202, it appears that they do * use bit 48 to distinguish between the two instances on the card. * The 2204, which I've never seen, *probably* extends this method. */ wwn = ISP2100_NVRAM_PORT_NAME(nvram_data); if (wwn) { isp_prt(isp, ISP_LOGCONFIG, "NVRAM Port WWN 0x%08x%08x", (uint32_t) (wwn >> 32), (uint32_t) (wwn)); if ((wwn >> 60) == 0) { wwn |= (((uint64_t) 2)<< 60); } } fcp->isp_wwpn_nvram = wwn; if (IS_2200(isp) || IS_23XX(isp)) { wwn = ISP2100_NVRAM_NODE_NAME(nvram_data); if (wwn) { isp_prt(isp, ISP_LOGCONFIG, "NVRAM Node WWN 0x%08x%08x", (uint32_t) (wwn >> 32), (uint32_t) (wwn)); if ((wwn >> 60) == 0) { wwn |= (((uint64_t) 2)<< 60); } } else { wwn = fcp->isp_wwpn_nvram & ~((uint64_t) 0xfff << 48); } } else { wwn &= ~((uint64_t) 0xfff << 48); } fcp->isp_wwnn_nvram = wwn; fcp->isp_maxalloc = ISP2100_NVRAM_MAXIOCBALLOCATION(nvram_data); if ((isp->isp_confopts & ISP_CFG_OWNFSZ) == 0) { DEFAULT_FRAMESIZE(isp) = ISP2100_NVRAM_MAXFRAMELENGTH(nvram_data); } fcp->isp_retry_delay = ISP2100_NVRAM_RETRY_DELAY(nvram_data); fcp->isp_retry_count = ISP2100_NVRAM_RETRY_COUNT(nvram_data); if ((isp->isp_confopts & ISP_CFG_OWNLOOPID) == 0) { fcp->isp_loopid = ISP2100_NVRAM_HARDLOOPID(nvram_data); } if ((isp->isp_confopts & ISP_CFG_OWNEXCTHROTTLE) == 0) { DEFAULT_EXEC_THROTTLE(isp) = ISP2100_NVRAM_EXECUTION_THROTTLE(nvram_data); } fcp->isp_fwoptions = ISP2100_NVRAM_OPTIONS(nvram_data); isp_prt(isp, ISP_LOGDEBUG0, "NVRAM 0x%08x%08x 0x%08x%08x maxalloc %d maxframelen %d", (uint32_t) (fcp->isp_wwnn_nvram >> 32), (uint32_t) fcp->isp_wwnn_nvram, (uint32_t) (fcp->isp_wwpn_nvram >> 32), (uint32_t) fcp->isp_wwpn_nvram, ISP2100_NVRAM_MAXIOCBALLOCATION(nvram_data), ISP2100_NVRAM_MAXFRAMELENGTH(nvram_data)); isp_prt(isp, ISP_LOGDEBUG0, "execthrottle %d fwoptions 0x%x hardloop %d tov %d", ISP2100_NVRAM_EXECUTION_THROTTLE(nvram_data), ISP2100_NVRAM_OPTIONS(nvram_data), ISP2100_NVRAM_HARDLOOPID(nvram_data), ISP2100_NVRAM_TOV(nvram_data)); fcp->isp_xfwoptions = ISP2100_XFW_OPTIONS(nvram_data); fcp->isp_zfwoptions = ISP2100_ZFW_OPTIONS(nvram_data); isp_prt(isp, ISP_LOGDEBUG0, "xfwoptions 0x%x zfw options 0x%x", ISP2100_XFW_OPTIONS(nvram_data), ISP2100_ZFW_OPTIONS(nvram_data)); } static void isp_parse_nvram_2400(ispsoftc_t *isp, uint8_t *nvram_data) { fcparam *fcp = FCPARAM(isp, 0); uint64_t wwn; isp_prt(isp, ISP_LOGDEBUG0, "NVRAM 0x%08x%08x 0x%08x%08x exchg_cnt %d maxframelen %d", (uint32_t) (ISP2400_NVRAM_NODE_NAME(nvram_data) >> 32), (uint32_t) (ISP2400_NVRAM_NODE_NAME(nvram_data)), (uint32_t) (ISP2400_NVRAM_PORT_NAME(nvram_data) >> 32), (uint32_t) (ISP2400_NVRAM_PORT_NAME(nvram_data)), ISP2400_NVRAM_EXCHANGE_COUNT(nvram_data), ISP2400_NVRAM_MAXFRAMELENGTH(nvram_data)); isp_prt(isp, ISP_LOGDEBUG0, "NVRAM execthr %d loopid %d fwopt1 0x%x fwopt2 0x%x fwopt3 0x%x", ISP2400_NVRAM_EXECUTION_THROTTLE(nvram_data), ISP2400_NVRAM_HARDLOOPID(nvram_data), ISP2400_NVRAM_FIRMWARE_OPTIONS1(nvram_data), ISP2400_NVRAM_FIRMWARE_OPTIONS2(nvram_data), ISP2400_NVRAM_FIRMWARE_OPTIONS3(nvram_data)); wwn = ISP2400_NVRAM_PORT_NAME(nvram_data); fcp->isp_wwpn_nvram = wwn; wwn = ISP2400_NVRAM_NODE_NAME(nvram_data); if (wwn) { if ((wwn >> 60) != 2 && (wwn >> 60) != 5) { wwn = 0; } } if (wwn == 0 && (fcp->isp_wwpn_nvram >> 60) == 2) { wwn = fcp->isp_wwpn_nvram; wwn &= ~((uint64_t) 0xfff << 48); } fcp->isp_wwnn_nvram = wwn; if (ISP2400_NVRAM_EXCHANGE_COUNT(nvram_data)) { fcp->isp_maxalloc = ISP2400_NVRAM_EXCHANGE_COUNT(nvram_data); } if ((isp->isp_confopts & ISP_CFG_OWNFSZ) == 0) { DEFAULT_FRAMESIZE(isp) = ISP2400_NVRAM_MAXFRAMELENGTH(nvram_data); } if ((isp->isp_confopts & ISP_CFG_OWNLOOPID) == 0) { fcp->isp_loopid = ISP2400_NVRAM_HARDLOOPID(nvram_data); } if ((isp->isp_confopts & ISP_CFG_OWNEXCTHROTTLE) == 0) { DEFAULT_EXEC_THROTTLE(isp) = ISP2400_NVRAM_EXECUTION_THROTTLE(nvram_data); } fcp->isp_fwoptions = ISP2400_NVRAM_FIRMWARE_OPTIONS1(nvram_data); fcp->isp_xfwoptions = ISP2400_NVRAM_FIRMWARE_OPTIONS2(nvram_data); fcp->isp_zfwoptions = ISP2400_NVRAM_FIRMWARE_OPTIONS3(nvram_data); } Index: user/ngie/more-tests2/sys/dev/isp/isp_pci.c =================================================================== --- user/ngie/more-tests2/sys/dev/isp/isp_pci.c (revision 292053) +++ user/ngie/more-tests2/sys/dev/isp/isp_pci.c (revision 292054) @@ -1,2084 +1,2095 @@ /*- * Copyright (c) 1997-2008 by Matthew Jacob * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice immediately at the beginning of the file, without modification, * this list of conditions, and the following disclaimer. * 2. The name of the author may not be used to endorse or promote products * derived from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * PCI specific probe and attach routines for Qlogic ISP SCSI adapters. * FreeBSD Version. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef __sparc64__ #include #include #endif #include static uint32_t isp_pci_rd_reg(ispsoftc_t *, int); static void isp_pci_wr_reg(ispsoftc_t *, int, uint32_t); static uint32_t isp_pci_rd_reg_1080(ispsoftc_t *, int); static void isp_pci_wr_reg_1080(ispsoftc_t *, int, uint32_t); static uint32_t isp_pci_rd_reg_2400(ispsoftc_t *, int); static void isp_pci_wr_reg_2400(ispsoftc_t *, int, uint32_t); static uint32_t isp_pci_rd_reg_2600(ispsoftc_t *, int); static void isp_pci_wr_reg_2600(ispsoftc_t *, int, uint32_t); static int isp_pci_rd_isr(ispsoftc_t *, uint16_t *, uint16_t *, uint16_t *); static int isp_pci_rd_isr_2300(ispsoftc_t *, uint16_t *, uint16_t *, uint16_t *); static int isp_pci_rd_isr_2400(ispsoftc_t *, uint16_t *, uint16_t *, uint16_t *); static int isp_pci_mbxdma(ispsoftc_t *); static int isp_pci_dmasetup(ispsoftc_t *, XS_T *, void *); static void isp_pci_reset0(ispsoftc_t *); static void isp_pci_reset1(ispsoftc_t *); static void isp_pci_dumpregs(ispsoftc_t *, const char *); static struct ispmdvec mdvec = { isp_pci_rd_isr, isp_pci_rd_reg, isp_pci_wr_reg, isp_pci_mbxdma, isp_pci_dmasetup, isp_common_dmateardown, isp_pci_reset0, isp_pci_reset1, isp_pci_dumpregs, NULL, BIU_BURST_ENABLE|BIU_PCI_CONF1_FIFO_64 }; static struct ispmdvec mdvec_1080 = { isp_pci_rd_isr, isp_pci_rd_reg_1080, isp_pci_wr_reg_1080, isp_pci_mbxdma, isp_pci_dmasetup, isp_common_dmateardown, isp_pci_reset0, isp_pci_reset1, isp_pci_dumpregs, NULL, BIU_BURST_ENABLE|BIU_PCI_CONF1_FIFO_64 }; static struct ispmdvec mdvec_12160 = { isp_pci_rd_isr, isp_pci_rd_reg_1080, isp_pci_wr_reg_1080, isp_pci_mbxdma, isp_pci_dmasetup, isp_common_dmateardown, isp_pci_reset0, isp_pci_reset1, isp_pci_dumpregs, NULL, BIU_BURST_ENABLE|BIU_PCI_CONF1_FIFO_64 }; static struct ispmdvec mdvec_2100 = { isp_pci_rd_isr, isp_pci_rd_reg, isp_pci_wr_reg, isp_pci_mbxdma, isp_pci_dmasetup, isp_common_dmateardown, isp_pci_reset0, isp_pci_reset1, isp_pci_dumpregs }; static struct ispmdvec mdvec_2200 = { isp_pci_rd_isr, isp_pci_rd_reg, isp_pci_wr_reg, isp_pci_mbxdma, isp_pci_dmasetup, isp_common_dmateardown, isp_pci_reset0, isp_pci_reset1, isp_pci_dumpregs }; static struct ispmdvec mdvec_2300 = { isp_pci_rd_isr_2300, isp_pci_rd_reg, isp_pci_wr_reg, isp_pci_mbxdma, isp_pci_dmasetup, isp_common_dmateardown, isp_pci_reset0, isp_pci_reset1, isp_pci_dumpregs }; static struct ispmdvec mdvec_2400 = { isp_pci_rd_isr_2400, isp_pci_rd_reg_2400, isp_pci_wr_reg_2400, isp_pci_mbxdma, isp_pci_dmasetup, isp_common_dmateardown, isp_pci_reset0, isp_pci_reset1, NULL }; static struct ispmdvec mdvec_2500 = { isp_pci_rd_isr_2400, isp_pci_rd_reg_2400, isp_pci_wr_reg_2400, isp_pci_mbxdma, isp_pci_dmasetup, isp_common_dmateardown, isp_pci_reset0, isp_pci_reset1, NULL }; static struct ispmdvec mdvec_2600 = { isp_pci_rd_isr_2400, isp_pci_rd_reg_2600, isp_pci_wr_reg_2600, isp_pci_mbxdma, isp_pci_dmasetup, isp_common_dmateardown, isp_pci_reset0, isp_pci_reset1, NULL }; #ifndef PCIM_CMD_INVEN #define PCIM_CMD_INVEN 0x10 #endif #ifndef PCIM_CMD_BUSMASTEREN #define PCIM_CMD_BUSMASTEREN 0x0004 #endif #ifndef PCIM_CMD_PERRESPEN #define PCIM_CMD_PERRESPEN 0x0040 #endif #ifndef PCIM_CMD_SEREN #define PCIM_CMD_SEREN 0x0100 #endif #ifndef PCIM_CMD_INTX_DISABLE #define PCIM_CMD_INTX_DISABLE 0x0400 #endif #ifndef PCIR_COMMAND #define PCIR_COMMAND 0x04 #endif #ifndef PCIR_CACHELNSZ #define PCIR_CACHELNSZ 0x0c #endif #ifndef PCIR_LATTIMER #define PCIR_LATTIMER 0x0d #endif #ifndef PCIR_ROMADDR #define PCIR_ROMADDR 0x30 #endif #ifndef PCI_VENDOR_QLOGIC #define PCI_VENDOR_QLOGIC 0x1077 #endif #ifndef PCI_PRODUCT_QLOGIC_ISP1020 #define PCI_PRODUCT_QLOGIC_ISP1020 0x1020 #endif #ifndef PCI_PRODUCT_QLOGIC_ISP1080 #define PCI_PRODUCT_QLOGIC_ISP1080 0x1080 #endif #ifndef PCI_PRODUCT_QLOGIC_ISP10160 #define PCI_PRODUCT_QLOGIC_ISP10160 0x1016 #endif #ifndef PCI_PRODUCT_QLOGIC_ISP12160 #define PCI_PRODUCT_QLOGIC_ISP12160 0x1216 #endif #ifndef PCI_PRODUCT_QLOGIC_ISP1240 #define PCI_PRODUCT_QLOGIC_ISP1240 0x1240 #endif #ifndef PCI_PRODUCT_QLOGIC_ISP1280 #define PCI_PRODUCT_QLOGIC_ISP1280 0x1280 #endif #ifndef PCI_PRODUCT_QLOGIC_ISP2100 #define PCI_PRODUCT_QLOGIC_ISP2100 0x2100 #endif #ifndef PCI_PRODUCT_QLOGIC_ISP2200 #define PCI_PRODUCT_QLOGIC_ISP2200 0x2200 #endif #ifndef PCI_PRODUCT_QLOGIC_ISP2300 #define PCI_PRODUCT_QLOGIC_ISP2300 0x2300 #endif #ifndef PCI_PRODUCT_QLOGIC_ISP2312 #define PCI_PRODUCT_QLOGIC_ISP2312 0x2312 #endif #ifndef PCI_PRODUCT_QLOGIC_ISP2322 #define PCI_PRODUCT_QLOGIC_ISP2322 0x2322 #endif #ifndef PCI_PRODUCT_QLOGIC_ISP2422 #define PCI_PRODUCT_QLOGIC_ISP2422 0x2422 #endif #ifndef PCI_PRODUCT_QLOGIC_ISP2432 #define PCI_PRODUCT_QLOGIC_ISP2432 0x2432 #endif #ifndef PCI_PRODUCT_QLOGIC_ISP2532 #define PCI_PRODUCT_QLOGIC_ISP2532 0x2532 #endif #ifndef PCI_PRODUCT_QLOGIC_ISP6312 #define PCI_PRODUCT_QLOGIC_ISP6312 0x6312 #endif #ifndef PCI_PRODUCT_QLOGIC_ISP6322 #define PCI_PRODUCT_QLOGIC_ISP6322 0x6322 #endif #ifndef PCI_PRODUCT_QLOGIC_ISP5432 #define PCI_PRODUCT_QLOGIC_ISP5432 0x5432 #endif #ifndef PCI_PRODUCT_QLOGIC_ISP2031 #define PCI_PRODUCT_QLOGIC_ISP2031 0x2031 #endif +#ifndef PCI_PRODUCT_QLOGIC_ISP8031 +#define PCI_PRODUCT_QLOGIC_ISP8031 0x8031 +#endif + #define PCI_QLOGIC_ISP5432 \ ((PCI_PRODUCT_QLOGIC_ISP5432 << 16) | PCI_VENDOR_QLOGIC) #define PCI_QLOGIC_ISP1020 \ ((PCI_PRODUCT_QLOGIC_ISP1020 << 16) | PCI_VENDOR_QLOGIC) #define PCI_QLOGIC_ISP1080 \ ((PCI_PRODUCT_QLOGIC_ISP1080 << 16) | PCI_VENDOR_QLOGIC) #define PCI_QLOGIC_ISP10160 \ ((PCI_PRODUCT_QLOGIC_ISP10160 << 16) | PCI_VENDOR_QLOGIC) #define PCI_QLOGIC_ISP12160 \ ((PCI_PRODUCT_QLOGIC_ISP12160 << 16) | PCI_VENDOR_QLOGIC) #define PCI_QLOGIC_ISP1240 \ ((PCI_PRODUCT_QLOGIC_ISP1240 << 16) | PCI_VENDOR_QLOGIC) #define PCI_QLOGIC_ISP1280 \ ((PCI_PRODUCT_QLOGIC_ISP1280 << 16) | PCI_VENDOR_QLOGIC) #define PCI_QLOGIC_ISP2100 \ ((PCI_PRODUCT_QLOGIC_ISP2100 << 16) | PCI_VENDOR_QLOGIC) #define PCI_QLOGIC_ISP2200 \ ((PCI_PRODUCT_QLOGIC_ISP2200 << 16) | PCI_VENDOR_QLOGIC) #define PCI_QLOGIC_ISP2300 \ ((PCI_PRODUCT_QLOGIC_ISP2300 << 16) | PCI_VENDOR_QLOGIC) #define PCI_QLOGIC_ISP2312 \ ((PCI_PRODUCT_QLOGIC_ISP2312 << 16) | PCI_VENDOR_QLOGIC) #define PCI_QLOGIC_ISP2322 \ ((PCI_PRODUCT_QLOGIC_ISP2322 << 16) | PCI_VENDOR_QLOGIC) #define PCI_QLOGIC_ISP2422 \ ((PCI_PRODUCT_QLOGIC_ISP2422 << 16) | PCI_VENDOR_QLOGIC) #define PCI_QLOGIC_ISP2432 \ ((PCI_PRODUCT_QLOGIC_ISP2432 << 16) | PCI_VENDOR_QLOGIC) #define PCI_QLOGIC_ISP2532 \ ((PCI_PRODUCT_QLOGIC_ISP2532 << 16) | PCI_VENDOR_QLOGIC) #define PCI_QLOGIC_ISP6312 \ ((PCI_PRODUCT_QLOGIC_ISP6312 << 16) | PCI_VENDOR_QLOGIC) #define PCI_QLOGIC_ISP6322 \ ((PCI_PRODUCT_QLOGIC_ISP6322 << 16) | PCI_VENDOR_QLOGIC) #define PCI_QLOGIC_ISP2031 \ ((PCI_PRODUCT_QLOGIC_ISP2031 << 16) | PCI_VENDOR_QLOGIC) +#define PCI_QLOGIC_ISP8031 \ + ((PCI_PRODUCT_QLOGIC_ISP8031 << 16) | PCI_VENDOR_QLOGIC) + /* * Odd case for some AMI raid cards... We need to *not* attach to this. */ #define AMI_RAID_SUBVENDOR_ID 0x101e #define PCI_DFLT_LTNCY 0x40 #define PCI_DFLT_LNSZ 0x10 static int isp_pci_probe (device_t); static int isp_pci_attach (device_t); static int isp_pci_detach (device_t); #define ISP_PCD(isp) ((struct isp_pcisoftc *)isp)->pci_dev struct isp_pcisoftc { ispsoftc_t pci_isp; device_t pci_dev; struct resource * regs; struct resource * regs2; void * irq; int iqd; int rtp; int rgd; int rtp2; int rgd2; void * ih; int16_t pci_poff[_NREG_BLKS]; bus_dma_tag_t dmat; int msicount; }; static device_method_t isp_pci_methods[] = { /* Device interface */ DEVMETHOD(device_probe, isp_pci_probe), DEVMETHOD(device_attach, isp_pci_attach), DEVMETHOD(device_detach, isp_pci_detach), { 0, 0 } }; static driver_t isp_pci_driver = { "isp", isp_pci_methods, sizeof (struct isp_pcisoftc) }; static devclass_t isp_devclass; DRIVER_MODULE(isp, pci, isp_pci_driver, isp_devclass, 0, 0); MODULE_DEPEND(isp, cam, 1, 1, 1); MODULE_DEPEND(isp, firmware, 1, 1, 1); static int isp_nvports = 0; static int isp_pci_probe(device_t dev) { switch ((pci_get_device(dev) << 16) | (pci_get_vendor(dev))) { case PCI_QLOGIC_ISP1020: device_set_desc(dev, "Qlogic ISP 1020/1040 PCI SCSI Adapter"); break; case PCI_QLOGIC_ISP1080: device_set_desc(dev, "Qlogic ISP 1080 PCI SCSI Adapter"); break; case PCI_QLOGIC_ISP1240: device_set_desc(dev, "Qlogic ISP 1240 PCI SCSI Adapter"); break; case PCI_QLOGIC_ISP1280: device_set_desc(dev, "Qlogic ISP 1280 PCI SCSI Adapter"); break; case PCI_QLOGIC_ISP10160: device_set_desc(dev, "Qlogic ISP 10160 PCI SCSI Adapter"); break; case PCI_QLOGIC_ISP12160: if (pci_get_subvendor(dev) == AMI_RAID_SUBVENDOR_ID) { return (ENXIO); } device_set_desc(dev, "Qlogic ISP 12160 PCI SCSI Adapter"); break; case PCI_QLOGIC_ISP2100: device_set_desc(dev, "Qlogic ISP 2100 PCI FC-AL Adapter"); break; case PCI_QLOGIC_ISP2200: device_set_desc(dev, "Qlogic ISP 2200 PCI FC-AL Adapter"); break; case PCI_QLOGIC_ISP2300: device_set_desc(dev, "Qlogic ISP 2300 PCI FC-AL Adapter"); break; case PCI_QLOGIC_ISP2312: device_set_desc(dev, "Qlogic ISP 2312 PCI FC-AL Adapter"); break; case PCI_QLOGIC_ISP2322: device_set_desc(dev, "Qlogic ISP 2322 PCI FC-AL Adapter"); break; case PCI_QLOGIC_ISP2422: device_set_desc(dev, "Qlogic ISP 2422 PCI FC-AL Adapter"); break; case PCI_QLOGIC_ISP2432: device_set_desc(dev, "Qlogic ISP 2432 PCI FC-AL Adapter"); break; case PCI_QLOGIC_ISP2532: device_set_desc(dev, "Qlogic ISP 2532 PCI FC-AL Adapter"); break; case PCI_QLOGIC_ISP5432: device_set_desc(dev, "Qlogic ISP 5432 PCI FC-AL Adapter"); break; case PCI_QLOGIC_ISP6312: device_set_desc(dev, "Qlogic ISP 6312 PCI FC-AL Adapter"); break; case PCI_QLOGIC_ISP6322: device_set_desc(dev, "Qlogic ISP 6322 PCI FC-AL Adapter"); break; case PCI_QLOGIC_ISP2031: device_set_desc(dev, "Qlogic ISP 2031 PCI FC-AL Adapter"); break; + case PCI_QLOGIC_ISP8031: + device_set_desc(dev, "Qlogic ISP 8031 PCI FCoE Adapter"); + break; default: return (ENXIO); } if (isp_announced == 0 && bootverbose) { printf("Qlogic ISP Driver, FreeBSD Version %d.%d, " "Core Version %d.%d\n", ISP_PLATFORM_VERSION_MAJOR, ISP_PLATFORM_VERSION_MINOR, ISP_CORE_VERSION_MAJOR, ISP_CORE_VERSION_MINOR); isp_announced++; } /* * XXXX: Here is where we might load the f/w module * XXXX: (or increase a reference count to it). */ return (BUS_PROBE_DEFAULT); } static void isp_get_generic_options(device_t dev, ispsoftc_t *isp) { int tval; tval = 0; if (resource_int_value(device_get_name(dev), device_get_unit(dev), "fwload_disable", &tval) == 0 && tval != 0) { isp->isp_confopts |= ISP_CFG_NORELOAD; } tval = 0; if (resource_int_value(device_get_name(dev), device_get_unit(dev), "ignore_nvram", &tval) == 0 && tval != 0) { isp->isp_confopts |= ISP_CFG_NONVRAM; } tval = 0; (void) resource_int_value(device_get_name(dev), device_get_unit(dev), "debug", &tval); if (tval) { isp->isp_dblev = tval; } else { isp->isp_dblev = ISP_LOGWARN|ISP_LOGERR; } if (bootverbose) { isp->isp_dblev |= ISP_LOGCONFIG|ISP_LOGINFO; } tval = -1; (void) resource_int_value(device_get_name(dev), device_get_unit(dev), "vports", &tval); if (tval > 0 && tval <= 254) { isp_nvports = tval; } tval = 7; (void) resource_int_value(device_get_name(dev), device_get_unit(dev), "quickboot_time", &tval); isp_quickboot_time = tval; } static void isp_get_specific_options(device_t dev, int chan, ispsoftc_t *isp) { const char *sptr; int tval = 0; char prefix[12], name[16]; if (chan == 0) prefix[0] = 0; else snprintf(prefix, sizeof(prefix), "chan%d.", chan); snprintf(name, sizeof(name), "%siid", prefix); if (resource_int_value(device_get_name(dev), device_get_unit(dev), name, &tval)) { if (IS_FC(isp)) { ISP_FC_PC(isp, chan)->default_id = 109 - chan; } else { #ifdef __sparc64__ ISP_SPI_PC(isp, chan)->iid = OF_getscsinitid(dev); #else ISP_SPI_PC(isp, chan)->iid = 7; #endif } } else { if (IS_FC(isp)) { ISP_FC_PC(isp, chan)->default_id = tval - chan; } else { ISP_SPI_PC(isp, chan)->iid = tval; } isp->isp_confopts |= ISP_CFG_OWNLOOPID; } if (IS_SCSI(isp)) return; tval = -1; snprintf(name, sizeof(name), "%srole", prefix); if (resource_int_value(device_get_name(dev), device_get_unit(dev), name, &tval) == 0) { switch (tval) { case ISP_ROLE_NONE: case ISP_ROLE_INITIATOR: case ISP_ROLE_TARGET: case ISP_ROLE_BOTH: device_printf(dev, "Chan %d setting role to 0x%x\n", chan, tval); break; default: tval = -1; break; } } if (tval == -1) { tval = ISP_DEFAULT_ROLES; } ISP_FC_PC(isp, chan)->def_role = tval; tval = 0; snprintf(name, sizeof(name), "%sfullduplex", prefix); if (resource_int_value(device_get_name(dev), device_get_unit(dev), name, &tval) == 0 && tval != 0) { isp->isp_confopts |= ISP_CFG_FULL_DUPLEX; } sptr = 0; snprintf(name, sizeof(name), "%stopology", prefix); if (resource_string_value(device_get_name(dev), device_get_unit(dev), name, (const char **) &sptr) == 0 && sptr != 0) { if (strcmp(sptr, "lport") == 0) { isp->isp_confopts |= ISP_CFG_LPORT; } else if (strcmp(sptr, "nport") == 0) { isp->isp_confopts |= ISP_CFG_NPORT; } else if (strcmp(sptr, "lport-only") == 0) { isp->isp_confopts |= ISP_CFG_LPORT_ONLY; } else if (strcmp(sptr, "nport-only") == 0) { isp->isp_confopts |= ISP_CFG_NPORT_ONLY; } } tval = 0; snprintf(name, sizeof(name), "%snofctape", prefix); (void) resource_int_value(device_get_name(dev), device_get_unit(dev), name, &tval); if (tval) { isp->isp_confopts |= ISP_CFG_NOFCTAPE; } tval = 0; snprintf(name, sizeof(name), "%sfctape", prefix); (void) resource_int_value(device_get_name(dev), device_get_unit(dev), name, &tval); if (tval) { isp->isp_confopts &= ~ISP_CFG_NOFCTAPE; isp->isp_confopts |= ISP_CFG_FCTAPE; } /* * Because the resource_*_value functions can neither return * 64 bit integer values, nor can they be directly coerced * to interpret the right hand side of the assignment as * you want them to interpret it, we have to force WWN * hint replacement to specify WWN strings with a leading * 'w' (e..g w50000000aaaa0001). Sigh. */ sptr = 0; snprintf(name, sizeof(name), "%sportwwn", prefix); tval = resource_string_value(device_get_name(dev), device_get_unit(dev), name, (const char **) &sptr); if (tval == 0 && sptr != 0 && *sptr++ == 'w') { char *eptr = 0; ISP_FC_PC(isp, chan)->def_wwpn = strtouq(sptr, &eptr, 16); if (eptr < sptr + 16 || ISP_FC_PC(isp, chan)->def_wwpn == -1) { device_printf(dev, "mangled portwwn hint '%s'\n", sptr); ISP_FC_PC(isp, chan)->def_wwpn = 0; } } sptr = 0; snprintf(name, sizeof(name), "%snodewwn", prefix); tval = resource_string_value(device_get_name(dev), device_get_unit(dev), name, (const char **) &sptr); if (tval == 0 && sptr != 0 && *sptr++ == 'w') { char *eptr = 0; ISP_FC_PC(isp, chan)->def_wwnn = strtouq(sptr, &eptr, 16); if (eptr < sptr + 16 || ISP_FC_PC(isp, chan)->def_wwnn == 0) { device_printf(dev, "mangled nodewwn hint '%s'\n", sptr); ISP_FC_PC(isp, chan)->def_wwnn = 0; } } tval = -1; snprintf(name, sizeof(name), "%sloop_down_limit", prefix); (void) resource_int_value(device_get_name(dev), device_get_unit(dev), name, &tval); if (tval >= 0 && tval < 0xffff) { ISP_FC_PC(isp, chan)->loop_down_limit = tval; } else { ISP_FC_PC(isp, chan)->loop_down_limit = isp_loop_down_limit; } tval = -1; snprintf(name, sizeof(name), "%sgone_device_time", prefix); (void) resource_int_value(device_get_name(dev), device_get_unit(dev), name, &tval); if (tval >= 0 && tval < 0xffff) { ISP_FC_PC(isp, chan)->gone_device_time = tval; } else { ISP_FC_PC(isp, chan)->gone_device_time = isp_gone_device_time; } } static int isp_pci_attach(device_t dev) { int i, locksetup = 0; uint32_t data, cmd, linesz, did; struct isp_pcisoftc *pcs; ispsoftc_t *isp; size_t psize, xsize; char fwname[32]; pcs = device_get_softc(dev); if (pcs == NULL) { device_printf(dev, "cannot get softc\n"); return (ENOMEM); } memset(pcs, 0, sizeof (*pcs)); pcs->pci_dev = dev; isp = &pcs->pci_isp; isp->isp_dev = dev; isp->isp_nchan = 1; if (sizeof (bus_addr_t) > 4) isp->isp_osinfo.sixtyfourbit = 1; /* * Get Generic Options */ isp_nvports = 0; isp_get_generic_options(dev, isp); linesz = PCI_DFLT_LNSZ; pcs->irq = pcs->regs = pcs->regs2 = NULL; pcs->rgd = pcs->rtp = pcs->iqd = 0; pcs->pci_dev = dev; pcs->pci_poff[BIU_BLOCK >> _BLK_REG_SHFT] = BIU_REGS_OFF; pcs->pci_poff[MBOX_BLOCK >> _BLK_REG_SHFT] = PCI_MBOX_REGS_OFF; pcs->pci_poff[SXP_BLOCK >> _BLK_REG_SHFT] = PCI_SXP_REGS_OFF; pcs->pci_poff[RISC_BLOCK >> _BLK_REG_SHFT] = PCI_RISC_REGS_OFF; pcs->pci_poff[DMA_BLOCK >> _BLK_REG_SHFT] = DMA_REGS_OFF; switch (pci_get_devid(dev)) { case PCI_QLOGIC_ISP1020: did = 0x1040; isp->isp_mdvec = &mdvec; isp->isp_type = ISP_HA_SCSI_UNKNOWN; break; case PCI_QLOGIC_ISP1080: did = 0x1080; isp->isp_mdvec = &mdvec_1080; isp->isp_type = ISP_HA_SCSI_1080; pcs->pci_poff[DMA_BLOCK >> _BLK_REG_SHFT] = ISP1080_DMA_REGS_OFF; break; case PCI_QLOGIC_ISP1240: did = 0x1080; isp->isp_mdvec = &mdvec_1080; isp->isp_type = ISP_HA_SCSI_1240; isp->isp_nchan = 2; pcs->pci_poff[DMA_BLOCK >> _BLK_REG_SHFT] = ISP1080_DMA_REGS_OFF; break; case PCI_QLOGIC_ISP1280: did = 0x1080; isp->isp_mdvec = &mdvec_1080; isp->isp_type = ISP_HA_SCSI_1280; pcs->pci_poff[DMA_BLOCK >> _BLK_REG_SHFT] = ISP1080_DMA_REGS_OFF; break; case PCI_QLOGIC_ISP10160: did = 0x12160; isp->isp_mdvec = &mdvec_12160; isp->isp_type = ISP_HA_SCSI_10160; pcs->pci_poff[DMA_BLOCK >> _BLK_REG_SHFT] = ISP1080_DMA_REGS_OFF; break; case PCI_QLOGIC_ISP12160: did = 0x12160; isp->isp_nchan = 2; isp->isp_mdvec = &mdvec_12160; isp->isp_type = ISP_HA_SCSI_12160; pcs->pci_poff[DMA_BLOCK >> _BLK_REG_SHFT] = ISP1080_DMA_REGS_OFF; break; case PCI_QLOGIC_ISP2100: did = 0x2100; isp->isp_mdvec = &mdvec_2100; isp->isp_type = ISP_HA_FC_2100; pcs->pci_poff[MBOX_BLOCK >> _BLK_REG_SHFT] = PCI_MBOX_REGS2100_OFF; if (pci_get_revid(dev) < 3) { /* * XXX: Need to get the actual revision * XXX: number of the 2100 FB. At any rate, * XXX: lower cache line size for early revision * XXX; boards. */ linesz = 1; } break; case PCI_QLOGIC_ISP2200: did = 0x2200; isp->isp_mdvec = &mdvec_2200; isp->isp_type = ISP_HA_FC_2200; pcs->pci_poff[MBOX_BLOCK >> _BLK_REG_SHFT] = PCI_MBOX_REGS2100_OFF; break; case PCI_QLOGIC_ISP2300: did = 0x2300; isp->isp_mdvec = &mdvec_2300; isp->isp_type = ISP_HA_FC_2300; pcs->pci_poff[MBOX_BLOCK >> _BLK_REG_SHFT] = PCI_MBOX_REGS2300_OFF; break; case PCI_QLOGIC_ISP2312: case PCI_QLOGIC_ISP6312: did = 0x2300; isp->isp_mdvec = &mdvec_2300; isp->isp_type = ISP_HA_FC_2312; pcs->pci_poff[MBOX_BLOCK >> _BLK_REG_SHFT] = PCI_MBOX_REGS2300_OFF; break; case PCI_QLOGIC_ISP2322: case PCI_QLOGIC_ISP6322: did = 0x2322; isp->isp_mdvec = &mdvec_2300; isp->isp_type = ISP_HA_FC_2322; pcs->pci_poff[MBOX_BLOCK >> _BLK_REG_SHFT] = PCI_MBOX_REGS2300_OFF; break; case PCI_QLOGIC_ISP2422: case PCI_QLOGIC_ISP2432: did = 0x2400; isp->isp_nchan += isp_nvports; isp->isp_mdvec = &mdvec_2400; isp->isp_type = ISP_HA_FC_2400; pcs->pci_poff[MBOX_BLOCK >> _BLK_REG_SHFT] = PCI_MBOX_REGS2400_OFF; break; case PCI_QLOGIC_ISP2532: did = 0x2500; isp->isp_nchan += isp_nvports; isp->isp_mdvec = &mdvec_2500; isp->isp_type = ISP_HA_FC_2500; pcs->pci_poff[MBOX_BLOCK >> _BLK_REG_SHFT] = PCI_MBOX_REGS2400_OFF; break; case PCI_QLOGIC_ISP5432: did = 0x2500; isp->isp_mdvec = &mdvec_2500; isp->isp_type = ISP_HA_FC_2500; pcs->pci_poff[MBOX_BLOCK >> _BLK_REG_SHFT] = PCI_MBOX_REGS2400_OFF; break; case PCI_QLOGIC_ISP2031: + case PCI_QLOGIC_ISP8031: did = 0x2600; isp->isp_nchan += isp_nvports; isp->isp_mdvec = &mdvec_2600; isp->isp_type = ISP_HA_FC_2600; pcs->pci_poff[MBOX_BLOCK >> _BLK_REG_SHFT] = PCI_MBOX_REGS2400_OFF; break; default: device_printf(dev, "unknown device type\n"); goto bad; break; } isp->isp_revision = pci_get_revid(dev); if (IS_26XX(isp)) { pcs->rtp = SYS_RES_MEMORY; pcs->rgd = PCIR_BAR(0); pcs->regs = bus_alloc_resource_any(dev, pcs->rtp, &pcs->rgd, RF_ACTIVE); pcs->rtp2 = SYS_RES_MEMORY; pcs->rgd2 = PCIR_BAR(4); pcs->regs2 = bus_alloc_resource_any(dev, pcs->rtp2, &pcs->rgd2, RF_ACTIVE); } else { pcs->rtp = SYS_RES_MEMORY; pcs->rgd = PCIR_BAR(1); pcs->regs = bus_alloc_resource_any(dev, pcs->rtp, &pcs->rgd, RF_ACTIVE); if (pcs->regs == NULL) { pcs->rtp = SYS_RES_IOPORT; pcs->rgd = PCIR_BAR(0); pcs->regs = bus_alloc_resource_any(dev, pcs->rtp, &pcs->rgd, RF_ACTIVE); } } if (pcs->regs == NULL) { device_printf(dev, "Unable to map any ports\n"); goto bad; } if (bootverbose) { device_printf(dev, "Using %s space register mapping\n", (pcs->rtp == SYS_RES_IOPORT)? "I/O" : "Memory"); } isp->isp_regs = pcs->regs; isp->isp_regs2 = pcs->regs2; if (IS_FC(isp)) { psize = sizeof (fcparam); xsize = sizeof (struct isp_fc); } else { psize = sizeof (sdparam); xsize = sizeof (struct isp_spi); } psize *= isp->isp_nchan; xsize *= isp->isp_nchan; isp->isp_param = malloc(psize, M_DEVBUF, M_NOWAIT | M_ZERO); if (isp->isp_param == NULL) { device_printf(dev, "cannot allocate parameter data\n"); goto bad; } isp->isp_osinfo.pc.ptr = malloc(xsize, M_DEVBUF, M_NOWAIT | M_ZERO); if (isp->isp_osinfo.pc.ptr == NULL) { device_printf(dev, "cannot allocate parameter data\n"); goto bad; } /* * Now that we know who we are (roughly) get/set specific options */ for (i = 0; i < isp->isp_nchan; i++) { isp_get_specific_options(dev, i, isp); } isp->isp_osinfo.fw = NULL; if (isp->isp_osinfo.fw == NULL) { snprintf(fwname, sizeof (fwname), "isp_%04x", did); isp->isp_osinfo.fw = firmware_get(fwname); } if (isp->isp_osinfo.fw != NULL) { isp_prt(isp, ISP_LOGCONFIG, "loaded firmware %s", fwname); isp->isp_mdvec->dv_ispfw = isp->isp_osinfo.fw->data; } /* * Make sure that SERR, PERR, WRITE INVALIDATE and BUSMASTER are set. */ cmd = pci_read_config(dev, PCIR_COMMAND, 2); cmd |= PCIM_CMD_SEREN | PCIM_CMD_PERRESPEN | PCIM_CMD_BUSMASTEREN | PCIM_CMD_INVEN; if (IS_2300(isp)) { /* per QLogic errata */ cmd &= ~PCIM_CMD_INVEN; } if (IS_2322(isp) || pci_get_devid(dev) == PCI_QLOGIC_ISP6312) { cmd &= ~PCIM_CMD_INTX_DISABLE; } if (IS_24XX(isp)) { cmd &= ~PCIM_CMD_INTX_DISABLE; } pci_write_config(dev, PCIR_COMMAND, cmd, 2); /* * Make sure the Cache Line Size register is set sensibly. */ data = pci_read_config(dev, PCIR_CACHELNSZ, 1); if (data == 0 || (linesz != PCI_DFLT_LNSZ && data != linesz)) { isp_prt(isp, ISP_LOGDEBUG0, "set PCI line size to %d from %d", linesz, data); data = linesz; pci_write_config(dev, PCIR_CACHELNSZ, data, 1); } /* * Make sure the Latency Timer is sane. */ data = pci_read_config(dev, PCIR_LATTIMER, 1); if (data < PCI_DFLT_LTNCY) { data = PCI_DFLT_LTNCY; isp_prt(isp, ISP_LOGDEBUG0, "set PCI latency to %d", data); pci_write_config(dev, PCIR_LATTIMER, data, 1); } /* * Make sure we've disabled the ROM. */ data = pci_read_config(dev, PCIR_ROMADDR, 4); data &= ~1; pci_write_config(dev, PCIR_ROMADDR, data, 4); /* * Do MSI * * NB: MSI-X needs to be disabled for the 2432 (PCI-Express) */ if (IS_24XX(isp) || IS_2322(isp)) { pcs->msicount = pci_msi_count(dev); if (pcs->msicount > 1) { pcs->msicount = 1; } if (pci_alloc_msi(dev, &pcs->msicount) == 0) { pcs->iqd = 1; } else { pcs->iqd = 0; } } pcs->irq = bus_alloc_resource_any(dev, SYS_RES_IRQ, &pcs->iqd, RF_ACTIVE | RF_SHAREABLE); if (pcs->irq == NULL) { device_printf(dev, "could not allocate interrupt\n"); goto bad; } /* Make sure the lock is set up. */ mtx_init(&isp->isp_osinfo.lock, "isp", NULL, MTX_DEF); locksetup++; if (isp_setup_intr(dev, pcs->irq, ISP_IFLAGS, NULL, isp_platform_intr, isp, &pcs->ih)) { device_printf(dev, "could not setup interrupt\n"); goto bad; } /* * Last minute checks... */ if (IS_23XX(isp) || IS_24XX(isp)) { isp->isp_port = pci_get_function(dev); } /* * Make sure we're in reset state. */ ISP_LOCK(isp); if (isp_reinit(isp, 1) != 0) { ISP_UNLOCK(isp); goto bad; } ISP_UNLOCK(isp); if (isp_attach(isp)) { ISP_LOCK(isp); isp_uninit(isp); ISP_UNLOCK(isp); goto bad; } return (0); bad: if (pcs->ih) { (void) bus_teardown_intr(dev, pcs->irq, pcs->ih); } if (locksetup) { mtx_destroy(&isp->isp_osinfo.lock); } if (pcs->irq) { (void) bus_release_resource(dev, SYS_RES_IRQ, pcs->iqd, pcs->irq); } if (pcs->msicount) { pci_release_msi(dev); } if (pcs->regs) (void) bus_release_resource(dev, pcs->rtp, pcs->rgd, pcs->regs); if (pcs->regs2) (void) bus_release_resource(dev, pcs->rtp2, pcs->rgd2, pcs->regs2); if (pcs->pci_isp.isp_param) { free(pcs->pci_isp.isp_param, M_DEVBUF); pcs->pci_isp.isp_param = NULL; } if (pcs->pci_isp.isp_osinfo.pc.ptr) { free(pcs->pci_isp.isp_osinfo.pc.ptr, M_DEVBUF); pcs->pci_isp.isp_osinfo.pc.ptr = NULL; } return (ENXIO); } static int isp_pci_detach(device_t dev) { struct isp_pcisoftc *pcs; ispsoftc_t *isp; int status; pcs = device_get_softc(dev); if (pcs == NULL) { return (ENXIO); } isp = (ispsoftc_t *) pcs; status = isp_detach(isp); if (status) return (status); ISP_LOCK(isp); isp_uninit(isp); if (pcs->ih) { (void) bus_teardown_intr(dev, pcs->irq, pcs->ih); } ISP_UNLOCK(isp); mtx_destroy(&isp->isp_osinfo.lock); (void) bus_release_resource(dev, SYS_RES_IRQ, pcs->iqd, pcs->irq); if (pcs->msicount) { pci_release_msi(dev); } (void) bus_release_resource(dev, pcs->rtp, pcs->rgd, pcs->regs); if (pcs->regs2) (void) bus_release_resource(dev, pcs->rtp2, pcs->rgd2, pcs->regs2); /* * XXX: THERE IS A LOT OF LEAKAGE HERE */ if (pcs->pci_isp.isp_param) { free(pcs->pci_isp.isp_param, M_DEVBUF); pcs->pci_isp.isp_param = NULL; } if (pcs->pci_isp.isp_osinfo.pc.ptr) { free(pcs->pci_isp.isp_osinfo.pc.ptr, M_DEVBUF); pcs->pci_isp.isp_osinfo.pc.ptr = NULL; } return (0); } #define IspVirt2Off(a, x) \ (((struct isp_pcisoftc *)a)->pci_poff[((x) & _BLK_REG_MASK) >> \ _BLK_REG_SHFT] + ((x) & 0xfff)) #define BXR2(isp, off) bus_read_2((isp)->isp_regs, (off)) #define BXW2(isp, off, v) bus_write_2((isp)->isp_regs, (off), (v)) #define BXR4(isp, off) bus_read_4((isp)->isp_regs, (off)) #define BXW4(isp, off, v) bus_write_4((isp)->isp_regs, (off), (v)) #define B2R4(isp, off) bus_read_4((isp)->isp_regs2, (off)) #define B2W4(isp, off, v) bus_write_4((isp)->isp_regs2, (off), (v)) static ISP_INLINE int isp_pci_rd_debounced(ispsoftc_t *isp, int off, uint16_t *rp) { uint32_t val0, val1; int i = 0; do { val0 = BXR2(isp, IspVirt2Off(isp, off)); val1 = BXR2(isp, IspVirt2Off(isp, off)); } while (val0 != val1 && ++i < 1000); if (val0 != val1) { return (1); } *rp = val0; return (0); } static int isp_pci_rd_isr(ispsoftc_t *isp, uint16_t *isrp, uint16_t *semap, uint16_t *info) { uint16_t isr, sema; if (IS_2100(isp)) { if (isp_pci_rd_debounced(isp, BIU_ISR, &isr)) { return (0); } if (isp_pci_rd_debounced(isp, BIU_SEMA, &sema)) { return (0); } } else { isr = BXR2(isp, IspVirt2Off(isp, BIU_ISR)); sema = BXR2(isp, IspVirt2Off(isp, BIU_SEMA)); } isp_prt(isp, ISP_LOGDEBUG3, "ISR 0x%x SEMA 0x%x", isr, sema); isr &= INT_PENDING_MASK(isp); sema &= BIU_SEMA_LOCK; if (isr == 0 && sema == 0) { return (0); } *isrp = isr; if ((*semap = sema) != 0) { if (IS_2100(isp)) { if (isp_pci_rd_debounced(isp, OUTMAILBOX0, info)) { return (0); } } else { *info = BXR2(isp, IspVirt2Off(isp, OUTMAILBOX0)); } } return (1); } static int isp_pci_rd_isr_2300(ispsoftc_t *isp, uint16_t *isrp, uint16_t *semap, uint16_t *info) { uint32_t hccr, r2hisr; if (!(BXR2(isp, IspVirt2Off(isp, BIU_ISR) & BIU2100_ISR_RISC_INT))) { *isrp = 0; return (0); } r2hisr = BXR4(isp, IspVirt2Off(isp, BIU_R2HSTSLO)); isp_prt(isp, ISP_LOGDEBUG3, "RISC2HOST ISR 0x%x", r2hisr); if ((r2hisr & BIU_R2HST_INTR) == 0) { *isrp = 0; return (0); } switch ((*isrp = r2hisr & BIU_R2HST_ISTAT_MASK)) { case ISPR2HST_ROM_MBX_OK: case ISPR2HST_ROM_MBX_FAIL: case ISPR2HST_MBX_OK: case ISPR2HST_MBX_FAIL: case ISPR2HST_ASYNC_EVENT: *semap = 1; break; case ISPR2HST_RIO_16: *info = ASYNC_RIO16_1; *semap = 1; return (1); case ISPR2HST_FPOST: *info = ASYNC_CMD_CMPLT; *semap = 1; return (1); case ISPR2HST_FPOST_CTIO: *info = ASYNC_CTIO_DONE; *semap = 1; return (1); case ISPR2HST_RSPQ_UPDATE: *semap = 0; break; default: hccr = ISP_READ(isp, HCCR); if (hccr & HCCR_PAUSE) { ISP_WRITE(isp, HCCR, HCCR_RESET); isp_prt(isp, ISP_LOGERR, "RISC paused at interrupt (%x->%x)", hccr, ISP_READ(isp, HCCR)); ISP_WRITE(isp, BIU_ICR, 0); } else { isp_prt(isp, ISP_LOGERR, "unknown interrupt 0x%x\n", r2hisr); } return (0); } *info = (r2hisr >> 16); return (1); } static int isp_pci_rd_isr_2400(ispsoftc_t *isp, uint16_t *isrp, uint16_t *semap, uint16_t *info) { uint32_t r2hisr; r2hisr = BXR4(isp, IspVirt2Off(isp, BIU2400_R2HSTSLO)); isp_prt(isp, ISP_LOGDEBUG3, "RISC2HOST ISR 0x%x", r2hisr); if ((r2hisr & BIU_R2HST_INTR) == 0) { *isrp = 0; return (0); } switch ((*isrp = r2hisr & BIU_R2HST_ISTAT_MASK)) { case ISPR2HST_ROM_MBX_OK: case ISPR2HST_ROM_MBX_FAIL: case ISPR2HST_MBX_OK: case ISPR2HST_MBX_FAIL: case ISPR2HST_ASYNC_EVENT: *semap = 1; break; case ISPR2HST_RSPQ_UPDATE: case ISPR2HST_RSPQ_UPDATE2: case ISPR2HST_ATIO_UPDATE: case ISPR2HST_ATIO_RSPQ_UPDATE: case ISPR2HST_ATIO_UPDATE2: *semap = 0; break; default: ISP_WRITE(isp, BIU2400_HCCR, HCCR_2400_CMD_CLEAR_RISC_INT); isp_prt(isp, ISP_LOGERR, "unknown interrupt 0x%x\n", r2hisr); return (0); } *info = (r2hisr >> 16); return (1); } static uint32_t isp_pci_rd_reg(ispsoftc_t *isp, int regoff) { uint16_t rv; int oldconf = 0; if ((regoff & _BLK_REG_MASK) == SXP_BLOCK) { /* * We will assume that someone has paused the RISC processor. */ oldconf = BXR2(isp, IspVirt2Off(isp, BIU_CONF1)); BXW2(isp, IspVirt2Off(isp, BIU_CONF1), oldconf | BIU_PCI_CONF1_SXP); MEMORYBARRIER(isp, SYNC_REG, IspVirt2Off(isp, BIU_CONF1), 2, -1); } rv = BXR2(isp, IspVirt2Off(isp, regoff)); if ((regoff & _BLK_REG_MASK) == SXP_BLOCK) { BXW2(isp, IspVirt2Off(isp, BIU_CONF1), oldconf); MEMORYBARRIER(isp, SYNC_REG, IspVirt2Off(isp, BIU_CONF1), 2, -1); } return (rv); } static void isp_pci_wr_reg(ispsoftc_t *isp, int regoff, uint32_t val) { int oldconf = 0; if ((regoff & _BLK_REG_MASK) == SXP_BLOCK) { /* * We will assume that someone has paused the RISC processor. */ oldconf = BXR2(isp, IspVirt2Off(isp, BIU_CONF1)); BXW2(isp, IspVirt2Off(isp, BIU_CONF1), oldconf | BIU_PCI_CONF1_SXP); MEMORYBARRIER(isp, SYNC_REG, IspVirt2Off(isp, BIU_CONF1), 2, -1); } BXW2(isp, IspVirt2Off(isp, regoff), val); MEMORYBARRIER(isp, SYNC_REG, IspVirt2Off(isp, regoff), 2, -1); if ((regoff & _BLK_REG_MASK) == SXP_BLOCK) { BXW2(isp, IspVirt2Off(isp, BIU_CONF1), oldconf); MEMORYBARRIER(isp, SYNC_REG, IspVirt2Off(isp, BIU_CONF1), 2, -1); } } static uint32_t isp_pci_rd_reg_1080(ispsoftc_t *isp, int regoff) { uint32_t rv, oc = 0; if ((regoff & _BLK_REG_MASK) == SXP_BLOCK) { uint32_t tc; /* * We will assume that someone has paused the RISC processor. */ oc = BXR2(isp, IspVirt2Off(isp, BIU_CONF1)); tc = oc & ~BIU_PCI1080_CONF1_DMA; if (regoff & SXP_BANK1_SELECT) tc |= BIU_PCI1080_CONF1_SXP1; else tc |= BIU_PCI1080_CONF1_SXP0; BXW2(isp, IspVirt2Off(isp, BIU_CONF1), tc); MEMORYBARRIER(isp, SYNC_REG, IspVirt2Off(isp, BIU_CONF1), 2, -1); } else if ((regoff & _BLK_REG_MASK) == DMA_BLOCK) { oc = BXR2(isp, IspVirt2Off(isp, BIU_CONF1)); BXW2(isp, IspVirt2Off(isp, BIU_CONF1), oc | BIU_PCI1080_CONF1_DMA); MEMORYBARRIER(isp, SYNC_REG, IspVirt2Off(isp, BIU_CONF1), 2, -1); } rv = BXR2(isp, IspVirt2Off(isp, regoff)); if (oc) { BXW2(isp, IspVirt2Off(isp, BIU_CONF1), oc); MEMORYBARRIER(isp, SYNC_REG, IspVirt2Off(isp, BIU_CONF1), 2, -1); } return (rv); } static void isp_pci_wr_reg_1080(ispsoftc_t *isp, int regoff, uint32_t val) { int oc = 0; if ((regoff & _BLK_REG_MASK) == SXP_BLOCK) { uint32_t tc; /* * We will assume that someone has paused the RISC processor. */ oc = BXR2(isp, IspVirt2Off(isp, BIU_CONF1)); tc = oc & ~BIU_PCI1080_CONF1_DMA; if (regoff & SXP_BANK1_SELECT) tc |= BIU_PCI1080_CONF1_SXP1; else tc |= BIU_PCI1080_CONF1_SXP0; BXW2(isp, IspVirt2Off(isp, BIU_CONF1), tc); MEMORYBARRIER(isp, SYNC_REG, IspVirt2Off(isp, BIU_CONF1), 2, -1); } else if ((regoff & _BLK_REG_MASK) == DMA_BLOCK) { oc = BXR2(isp, IspVirt2Off(isp, BIU_CONF1)); BXW2(isp, IspVirt2Off(isp, BIU_CONF1), oc | BIU_PCI1080_CONF1_DMA); MEMORYBARRIER(isp, SYNC_REG, IspVirt2Off(isp, BIU_CONF1), 2, -1); } BXW2(isp, IspVirt2Off(isp, regoff), val); MEMORYBARRIER(isp, SYNC_REG, IspVirt2Off(isp, regoff), 2, -1); if (oc) { BXW2(isp, IspVirt2Off(isp, BIU_CONF1), oc); MEMORYBARRIER(isp, SYNC_REG, IspVirt2Off(isp, BIU_CONF1), 2, -1); } } static uint32_t isp_pci_rd_reg_2400(ispsoftc_t *isp, int regoff) { uint32_t rv; int block = regoff & _BLK_REG_MASK; switch (block) { case BIU_BLOCK: break; case MBOX_BLOCK: return (BXR2(isp, IspVirt2Off(isp, regoff))); case SXP_BLOCK: isp_prt(isp, ISP_LOGERR, "SXP_BLOCK read at 0x%x", regoff); return (0xffffffff); case RISC_BLOCK: isp_prt(isp, ISP_LOGERR, "RISC_BLOCK read at 0x%x", regoff); return (0xffffffff); case DMA_BLOCK: isp_prt(isp, ISP_LOGERR, "DMA_BLOCK read at 0x%x", regoff); return (0xffffffff); default: isp_prt(isp, ISP_LOGERR, "unknown block read at 0x%x", regoff); return (0xffffffff); } switch (regoff) { case BIU2400_FLASH_ADDR: case BIU2400_FLASH_DATA: case BIU2400_ICR: case BIU2400_ISR: case BIU2400_CSR: case BIU2400_REQINP: case BIU2400_REQOUTP: case BIU2400_RSPINP: case BIU2400_RSPOUTP: case BIU2400_PRI_REQINP: case BIU2400_PRI_REQOUTP: case BIU2400_ATIO_RSPINP: case BIU2400_ATIO_RSPOUTP: case BIU2400_HCCR: case BIU2400_GPIOD: case BIU2400_GPIOE: case BIU2400_HSEMA: rv = BXR4(isp, IspVirt2Off(isp, regoff)); break; case BIU2400_R2HSTSLO: rv = BXR4(isp, IspVirt2Off(isp, regoff)); break; case BIU2400_R2HSTSHI: rv = BXR4(isp, IspVirt2Off(isp, regoff)) >> 16; break; default: isp_prt(isp, ISP_LOGERR, "unknown register read at 0x%x", regoff); rv = 0xffffffff; break; } return (rv); } static void isp_pci_wr_reg_2400(ispsoftc_t *isp, int regoff, uint32_t val) { int block = regoff & _BLK_REG_MASK; switch (block) { case BIU_BLOCK: break; case MBOX_BLOCK: BXW2(isp, IspVirt2Off(isp, regoff), val); MEMORYBARRIER(isp, SYNC_REG, IspVirt2Off(isp, regoff), 2, -1); return; case SXP_BLOCK: isp_prt(isp, ISP_LOGERR, "SXP_BLOCK write at 0x%x", regoff); return; case RISC_BLOCK: isp_prt(isp, ISP_LOGERR, "RISC_BLOCK write at 0x%x", regoff); return; case DMA_BLOCK: isp_prt(isp, ISP_LOGERR, "DMA_BLOCK write at 0x%x", regoff); return; default: isp_prt(isp, ISP_LOGERR, "unknown block write at 0x%x", regoff); break; } switch (regoff) { case BIU2400_FLASH_ADDR: case BIU2400_FLASH_DATA: case BIU2400_ICR: case BIU2400_ISR: case BIU2400_CSR: case BIU2400_REQINP: case BIU2400_REQOUTP: case BIU2400_RSPINP: case BIU2400_RSPOUTP: case BIU2400_PRI_REQINP: case BIU2400_PRI_REQOUTP: case BIU2400_ATIO_RSPINP: case BIU2400_ATIO_RSPOUTP: case BIU2400_HCCR: case BIU2400_GPIOD: case BIU2400_GPIOE: case BIU2400_HSEMA: BXW4(isp, IspVirt2Off(isp, regoff), val); #ifdef MEMORYBARRIERW if (regoff == BIU2400_REQINP || regoff == BIU2400_RSPOUTP || regoff == BIU2400_PRI_REQINP || regoff == BIU2400_ATIO_RSPOUTP) MEMORYBARRIERW(isp, SYNC_REG, IspVirt2Off(isp, regoff), 4, -1) else #endif MEMORYBARRIER(isp, SYNC_REG, IspVirt2Off(isp, regoff), 4, -1); break; default: isp_prt(isp, ISP_LOGERR, "unknown register write at 0x%x", regoff); break; } } static uint32_t isp_pci_rd_reg_2600(ispsoftc_t *isp, int regoff) { uint32_t rv; switch (regoff) { case BIU2400_PRI_REQINP: case BIU2400_PRI_REQOUTP: isp_prt(isp, ISP_LOGERR, "unknown register read at 0x%x", regoff); rv = 0xffffffff; break; case BIU2400_REQINP: rv = B2R4(isp, 0x00); break; case BIU2400_REQOUTP: rv = B2R4(isp, 0x04); break; case BIU2400_RSPINP: rv = B2R4(isp, 0x08); break; case BIU2400_RSPOUTP: rv = B2R4(isp, 0x0c); break; case BIU2400_ATIO_RSPINP: rv = B2R4(isp, 0x10); break; case BIU2400_ATIO_RSPOUTP: rv = B2R4(isp, 0x14); break; default: rv = isp_pci_rd_reg_2400(isp, regoff); break; } return (rv); } static void isp_pci_wr_reg_2600(ispsoftc_t *isp, int regoff, uint32_t val) { int off; switch (regoff) { case BIU2400_PRI_REQINP: case BIU2400_PRI_REQOUTP: isp_prt(isp, ISP_LOGERR, "unknown register write at 0x%x", regoff); return; case BIU2400_REQINP: off = 0x00; break; case BIU2400_REQOUTP: off = 0x04; break; case BIU2400_RSPINP: off = 0x08; break; case BIU2400_RSPOUTP: off = 0x0c; break; case BIU2400_ATIO_RSPINP: off = 0x10; break; case BIU2400_ATIO_RSPOUTP: off = 0x14; break; default: isp_pci_wr_reg_2400(isp, regoff, val); return; } B2W4(isp, off, val); } struct imush { ispsoftc_t *isp; caddr_t vbase; int chan; int error; }; static void imc(void *, bus_dma_segment_t *, int, int); static void imc1(void *, bus_dma_segment_t *, int, int); static void imc(void *arg, bus_dma_segment_t *segs, int nseg, int error) { struct imush *imushp = (struct imush *) arg; isp_ecmd_t *ecmd; if (error) { imushp->error = error; return; } if (nseg != 1) { imushp->error = EINVAL; return; } isp_prt(imushp->isp, ISP_LOGDEBUG0, "request/result area @ 0x%jx/0x%jx", (uintmax_t) segs->ds_addr, (uintmax_t) segs->ds_len); imushp->isp->isp_rquest = imushp->vbase; imushp->isp->isp_rquest_dma = segs->ds_addr; segs->ds_addr += ISP_QUEUE_SIZE(RQUEST_QUEUE_LEN(imushp->isp)); imushp->vbase += ISP_QUEUE_SIZE(RQUEST_QUEUE_LEN(imushp->isp)); imushp->isp->isp_result_dma = segs->ds_addr; imushp->isp->isp_result = imushp->vbase; segs->ds_addr += ISP_QUEUE_SIZE(RESULT_QUEUE_LEN(imushp->isp)); imushp->vbase += ISP_QUEUE_SIZE(RESULT_QUEUE_LEN(imushp->isp)); if (imushp->isp->isp_type >= ISP_HA_FC_2200) { imushp->isp->isp_osinfo.ecmd_dma = segs->ds_addr; imushp->isp->isp_osinfo.ecmd_free = (isp_ecmd_t *)imushp->vbase; imushp->isp->isp_osinfo.ecmd_base = imushp->isp->isp_osinfo.ecmd_free; for (ecmd = imushp->isp->isp_osinfo.ecmd_free; ecmd < &imushp->isp->isp_osinfo.ecmd_free[N_XCMDS]; ecmd++) { if (ecmd == &imushp->isp->isp_osinfo.ecmd_free[N_XCMDS - 1]) { ecmd->next = NULL; } else { ecmd->next = ecmd + 1; } } } #ifdef ISP_TARGET_MODE segs->ds_addr += (N_XCMDS * XCMD_SIZE); imushp->vbase += (N_XCMDS * XCMD_SIZE); if (IS_24XX(imushp->isp)) { imushp->isp->isp_atioq_dma = segs->ds_addr; imushp->isp->isp_atioq = imushp->vbase; } #endif } static void imc1(void *arg, bus_dma_segment_t *segs, int nseg, int error) { struct imush *imushp = (struct imush *) arg; if (error) { imushp->error = error; return; } if (nseg != 1) { imushp->error = EINVAL; return; } isp_prt(imushp->isp, ISP_LOGDEBUG0, "scdma @ 0x%jx/0x%jx", (uintmax_t) segs->ds_addr, (uintmax_t) segs->ds_len); FCPARAM(imushp->isp, imushp->chan)->isp_scdma = segs->ds_addr; FCPARAM(imushp->isp, imushp->chan)->isp_scratch = imushp->vbase; } static int isp_pci_mbxdma(ispsoftc_t *isp) { caddr_t base; uint32_t len, nsegs; int i, error, cmap = 0; bus_size_t slim; /* segment size */ bus_addr_t llim; /* low limit of unavailable dma */ bus_addr_t hlim; /* high limit of unavailable dma */ struct imush im; /* * Already been here? If so, leave... */ if (isp->isp_rquest) { return (0); } ISP_UNLOCK(isp); if (isp->isp_maxcmds == 0) { isp_prt(isp, ISP_LOGERR, "maxcmds not set"); ISP_LOCK(isp); return (1); } hlim = BUS_SPACE_MAXADDR; if (IS_ULTRA2(isp) || IS_FC(isp) || IS_1240(isp)) { if (sizeof (bus_size_t) > 4) { slim = (bus_size_t) (1ULL << 32); } else { slim = (bus_size_t) (1UL << 31); } llim = BUS_SPACE_MAXADDR; } else { llim = BUS_SPACE_MAXADDR_32BIT; slim = (1UL << 24); } len = isp->isp_maxcmds * sizeof (struct isp_pcmd); isp->isp_osinfo.pcmd_pool = (struct isp_pcmd *) malloc(len, M_DEVBUF, M_WAITOK | M_ZERO); if (isp->isp_osinfo.pcmd_pool == NULL) { isp_prt(isp, ISP_LOGERR, "cannot allocate pcmds"); ISP_LOCK(isp); return (1); } if (isp->isp_osinfo.sixtyfourbit) { nsegs = ISP_NSEG64_MAX; } else { nsegs = ISP_NSEG_MAX; } if (isp_dma_tag_create(BUS_DMA_ROOTARG(ISP_PCD(isp)), 1, slim, llim, hlim, NULL, NULL, BUS_SPACE_MAXSIZE, nsegs, slim, 0, &isp->isp_osinfo.dmat)) { free(isp->isp_osinfo.pcmd_pool, M_DEVBUF); ISP_LOCK(isp); isp_prt(isp, ISP_LOGERR, "could not create master dma tag"); return (1); } len = sizeof (isp_hdl_t) * isp->isp_maxcmds; isp->isp_xflist = (isp_hdl_t *) malloc(len, M_DEVBUF, M_WAITOK | M_ZERO); if (isp->isp_xflist == NULL) { free(isp->isp_osinfo.pcmd_pool, M_DEVBUF); ISP_LOCK(isp); isp_prt(isp, ISP_LOGERR, "cannot alloc xflist array"); return (1); } for (len = 0; len < isp->isp_maxcmds - 1; len++) { isp->isp_xflist[len].cmd = &isp->isp_xflist[len+1]; } isp->isp_xffree = isp->isp_xflist; #ifdef ISP_TARGET_MODE len = sizeof (isp_hdl_t) * isp->isp_maxcmds; isp->isp_tgtlist = (isp_hdl_t *) malloc(len, M_DEVBUF, M_WAITOK | M_ZERO); if (isp->isp_tgtlist == NULL) { free(isp->isp_osinfo.pcmd_pool, M_DEVBUF); free(isp->isp_xflist, M_DEVBUF); ISP_LOCK(isp); isp_prt(isp, ISP_LOGERR, "cannot alloc tgtlist array"); return (1); } for (len = 0; len < isp->isp_maxcmds - 1; len++) { isp->isp_tgtlist[len].cmd = &isp->isp_tgtlist[len+1]; } isp->isp_tgtfree = isp->isp_tgtlist; #endif /* * Allocate and map the request and result queues (and ATIO queue * if we're a 2400 supporting target mode), and a region for * external dma addressable command/status structures (23XX and * later). */ len = ISP_QUEUE_SIZE(RQUEST_QUEUE_LEN(isp)); len += ISP_QUEUE_SIZE(RESULT_QUEUE_LEN(isp)); #ifdef ISP_TARGET_MODE if (IS_24XX(isp)) { len += ISP_QUEUE_SIZE(RESULT_QUEUE_LEN(isp)); } #endif if (isp->isp_type >= ISP_HA_FC_2200) { len += (N_XCMDS * XCMD_SIZE); } /* * Create a tag for the control spaces. We don't always need this * to be 32 bits, but we do this for simplicity and speed's sake. */ if (isp_dma_tag_create(isp->isp_osinfo.dmat, QENTRY_LEN, slim, BUS_SPACE_MAXADDR_32BIT, BUS_SPACE_MAXADDR, NULL, NULL, len, 1, slim, 0, &isp->isp_osinfo.cdmat)) { isp_prt(isp, ISP_LOGERR, "cannot create a dma tag for control spaces"); free(isp->isp_osinfo.pcmd_pool, M_DEVBUF); free(isp->isp_xflist, M_DEVBUF); #ifdef ISP_TARGET_MODE free(isp->isp_tgtlist, M_DEVBUF); #endif ISP_LOCK(isp); return (1); } if (bus_dmamem_alloc(isp->isp_osinfo.cdmat, (void **)&base, BUS_DMA_NOWAIT | BUS_DMA_COHERENT, &isp->isp_osinfo.cdmap) != 0) { isp_prt(isp, ISP_LOGERR, "cannot allocate %d bytes of CCB memory", len); bus_dma_tag_destroy(isp->isp_osinfo.cdmat); free(isp->isp_osinfo.pcmd_pool, M_DEVBUF); free(isp->isp_xflist, M_DEVBUF); #ifdef ISP_TARGET_MODE free(isp->isp_tgtlist, M_DEVBUF); #endif ISP_LOCK(isp); return (1); } im.isp = isp; im.chan = 0; im.vbase = base; im.error = 0; bus_dmamap_load(isp->isp_osinfo.cdmat, isp->isp_osinfo.cdmap, base, len, imc, &im, 0); if (im.error) { isp_prt(isp, ISP_LOGERR, "error %d loading dma map for control areas", im.error); goto bad; } if (IS_FC(isp)) { for (cmap = 0; cmap < isp->isp_nchan; cmap++) { struct isp_fc *fc = ISP_FC_PC(isp, cmap); if (isp_dma_tag_create(isp->isp_osinfo.dmat, 64, slim, BUS_SPACE_MAXADDR_32BIT, BUS_SPACE_MAXADDR, NULL, NULL, ISP_FC_SCRLEN, 1, slim, 0, &fc->tdmat)) { goto bad; } if (bus_dmamem_alloc(fc->tdmat, (void **)&base, BUS_DMA_NOWAIT | BUS_DMA_COHERENT, &fc->tdmap) != 0) { bus_dma_tag_destroy(fc->tdmat); goto bad; } im.isp = isp; im.chan = cmap; im.vbase = base; im.error = 0; bus_dmamap_load(fc->tdmat, fc->tdmap, base, ISP_FC_SCRLEN, imc1, &im, 0); if (im.error) { bus_dmamem_free(fc->tdmat, base, fc->tdmap); bus_dma_tag_destroy(fc->tdmat); goto bad; } if (!IS_2100(isp)) { for (i = 0; i < INITIAL_NEXUS_COUNT; i++) { struct isp_nexus *n = malloc(sizeof (struct isp_nexus), M_DEVBUF, M_NOWAIT | M_ZERO); if (n == NULL) { while (fc->nexus_free_list) { n = fc->nexus_free_list; fc->nexus_free_list = n->next; free(n, M_DEVBUF); } goto bad; } n->next = fc->nexus_free_list; fc->nexus_free_list = n; } } } } for (i = 0; i < isp->isp_maxcmds; i++) { struct isp_pcmd *pcmd = &isp->isp_osinfo.pcmd_pool[i]; error = bus_dmamap_create(isp->isp_osinfo.dmat, 0, &pcmd->dmap); if (error) { isp_prt(isp, ISP_LOGERR, "error %d creating per-cmd DMA maps", error); while (--i >= 0) { bus_dmamap_destroy(isp->isp_osinfo.dmat, isp->isp_osinfo.pcmd_pool[i].dmap); } goto bad; } callout_init_mtx(&pcmd->wdog, &isp->isp_osinfo.lock, 0); if (i == isp->isp_maxcmds-1) { pcmd->next = NULL; } else { pcmd->next = &isp->isp_osinfo.pcmd_pool[i+1]; } } isp->isp_osinfo.pcmd_free = &isp->isp_osinfo.pcmd_pool[0]; ISP_LOCK(isp); return (0); bad: while (--cmap >= 0) { struct isp_fc *fc = ISP_FC_PC(isp, cmap); bus_dmamap_unload(fc->tdmat, fc->tdmap); bus_dmamem_free(fc->tdmat, base, fc->tdmap); bus_dma_tag_destroy(fc->tdmat); while (fc->nexus_free_list) { struct isp_nexus *n = fc->nexus_free_list; fc->nexus_free_list = n->next; free(n, M_DEVBUF); } } if (isp->isp_rquest_dma != 0) bus_dmamap_unload(isp->isp_osinfo.cdmat, isp->isp_osinfo.cdmap); bus_dmamem_free(isp->isp_osinfo.cdmat, base, isp->isp_osinfo.cdmap); bus_dma_tag_destroy(isp->isp_osinfo.cdmat); free(isp->isp_xflist, M_DEVBUF); #ifdef ISP_TARGET_MODE free(isp->isp_tgtlist, M_DEVBUF); #endif free(isp->isp_osinfo.pcmd_pool, M_DEVBUF); isp->isp_rquest = NULL; ISP_LOCK(isp); return (1); } typedef struct { ispsoftc_t *isp; void *cmd_token; void *rq; /* original request */ int error; bus_size_t mapsize; } mush_t; #define MUSHERR_NOQENTRIES -2 #ifdef ISP_TARGET_MODE static void tdma2_2(void *, bus_dma_segment_t *, int, bus_size_t, int); static void tdma2(void *, bus_dma_segment_t *, int, int); static void tdma2_2(void *arg, bus_dma_segment_t *dm_segs, int nseg, bus_size_t mapsize, int error) { mush_t *mp; mp = (mush_t *)arg; mp->mapsize = mapsize; tdma2(arg, dm_segs, nseg, error); } static void tdma2(void *arg, bus_dma_segment_t *dm_segs, int nseg, int error) { mush_t *mp; ispsoftc_t *isp; struct ccb_scsiio *csio; isp_ddir_t ddir; ispreq_t *rq; mp = (mush_t *) arg; if (error) { mp->error = error; return; } csio = mp->cmd_token; isp = mp->isp; rq = mp->rq; if (nseg) { if (isp->isp_osinfo.sixtyfourbit) { if (nseg >= ISP_NSEG64_MAX) { isp_prt(isp, ISP_LOGERR, "number of segments (%d) exceed maximum we can support (%d)", nseg, ISP_NSEG64_MAX); mp->error = EFAULT; return; } if (rq->req_header.rqs_entry_type == RQSTYPE_CTIO2) { rq->req_header.rqs_entry_type = RQSTYPE_CTIO3; } } else { if (nseg >= ISP_NSEG_MAX) { isp_prt(isp, ISP_LOGERR, "number of segments (%d) exceed maximum we can support (%d)", nseg, ISP_NSEG_MAX); mp->error = EFAULT; return; } } if ((csio->ccb_h.flags & CAM_DIR_MASK) == CAM_DIR_IN) { bus_dmamap_sync(isp->isp_osinfo.dmat, PISP_PCMD(csio)->dmap, BUS_DMASYNC_PREWRITE); ddir = ISP_TO_DEVICE; } else if ((csio->ccb_h.flags & CAM_DIR_MASK) == CAM_DIR_OUT) { bus_dmamap_sync(isp->isp_osinfo.dmat, PISP_PCMD(csio)->dmap, BUS_DMASYNC_PREREAD); ddir = ISP_FROM_DEVICE; } else { dm_segs = NULL; nseg = 0; ddir = ISP_NOXFR; } } else { dm_segs = NULL; nseg = 0; ddir = ISP_NOXFR; } error = isp_send_tgt_cmd(isp, rq, dm_segs, nseg, XS_XFRLEN(csio), ddir, &csio->sense_data, csio->sense_len); switch (error) { case CMD_EAGAIN: mp->error = MUSHERR_NOQENTRIES; case CMD_QUEUED: break; default: mp->error = EIO; } } #endif static void dma2_2(void *, bus_dma_segment_t *, int, bus_size_t, int); static void dma2(void *, bus_dma_segment_t *, int, int); static void dma2_2(void *arg, bus_dma_segment_t *dm_segs, int nseg, bus_size_t mapsize, int error) { mush_t *mp; mp = (mush_t *)arg; mp->mapsize = mapsize; dma2(arg, dm_segs, nseg, error); } static void dma2(void *arg, bus_dma_segment_t *dm_segs, int nseg, int error) { mush_t *mp; ispsoftc_t *isp; struct ccb_scsiio *csio; isp_ddir_t ddir; ispreq_t *rq; mp = (mush_t *) arg; if (error) { mp->error = error; return; } csio = mp->cmd_token; isp = mp->isp; rq = mp->rq; if (nseg) { if (isp->isp_osinfo.sixtyfourbit) { if (nseg >= ISP_NSEG64_MAX) { isp_prt(isp, ISP_LOGERR, "number of segments (%d) exceed maximum we can support (%d)", nseg, ISP_NSEG64_MAX); mp->error = EFAULT; return; } if (rq->req_header.rqs_entry_type == RQSTYPE_T2RQS) { rq->req_header.rqs_entry_type = RQSTYPE_T3RQS; } else if (rq->req_header.rqs_entry_type == RQSTYPE_REQUEST) { rq->req_header.rqs_entry_type = RQSTYPE_A64; } } else { if (nseg >= ISP_NSEG_MAX) { isp_prt(isp, ISP_LOGERR, "number of segments (%d) exceed maximum we can support (%d)", nseg, ISP_NSEG_MAX); mp->error = EFAULT; return; } } if ((csio->ccb_h.flags & CAM_DIR_MASK) == CAM_DIR_IN) { bus_dmamap_sync(isp->isp_osinfo.dmat, PISP_PCMD(csio)->dmap, BUS_DMASYNC_PREREAD); ddir = ISP_FROM_DEVICE; } else if ((csio->ccb_h.flags & CAM_DIR_MASK) == CAM_DIR_OUT) { bus_dmamap_sync(isp->isp_osinfo.dmat, PISP_PCMD(csio)->dmap, BUS_DMASYNC_PREWRITE); ddir = ISP_TO_DEVICE; } else { ddir = ISP_NOXFR; } } else { dm_segs = NULL; nseg = 0; ddir = ISP_NOXFR; } error = isp_send_cmd(isp, rq, dm_segs, nseg, XS_XFRLEN(csio), ddir, (ispds64_t *)csio->req_map); switch (error) { case CMD_EAGAIN: mp->error = MUSHERR_NOQENTRIES; break; case CMD_QUEUED: break; default: mp->error = EIO; break; } } static int isp_pci_dmasetup(ispsoftc_t *isp, struct ccb_scsiio *csio, void *ff) { mush_t mush, *mp; void (*eptr)(void *, bus_dma_segment_t *, int, int); void (*eptr2)(void *, bus_dma_segment_t *, int, bus_size_t, int); int error; mp = &mush; mp->isp = isp; mp->cmd_token = csio; mp->rq = ff; mp->error = 0; mp->mapsize = 0; #ifdef ISP_TARGET_MODE if (csio->ccb_h.func_code == XPT_CONT_TARGET_IO) { eptr = tdma2; eptr2 = tdma2_2; } else #endif { eptr = dma2; eptr2 = dma2_2; } error = bus_dmamap_load_ccb(isp->isp_osinfo.dmat, PISP_PCMD(csio)->dmap, (union ccb *)csio, eptr, mp, 0); if (error == EINPROGRESS) { bus_dmamap_unload(isp->isp_osinfo.dmat, PISP_PCMD(csio)->dmap); mp->error = EINVAL; isp_prt(isp, ISP_LOGERR, "deferred dma allocation not supported"); } else if (error && mp->error == 0) { #ifdef DIAGNOSTIC isp_prt(isp, ISP_LOGERR, "error %d in dma mapping code", error); #endif mp->error = error; } if (mp->error) { int retval = CMD_COMPLETE; if (mp->error == MUSHERR_NOQENTRIES) { retval = CMD_EAGAIN; } else if (mp->error == EFBIG) { csio->ccb_h.status = CAM_REQ_TOO_BIG; } else if (mp->error == EINVAL) { csio->ccb_h.status = CAM_REQ_INVALID; } else { csio->ccb_h.status = CAM_UNREC_HBA_ERROR; } return (retval); } return (CMD_QUEUED); } static void isp_pci_reset0(ispsoftc_t *isp) { ISP_DISABLE_INTS(isp); } static void isp_pci_reset1(ispsoftc_t *isp) { if (!IS_24XX(isp)) { /* Make sure the BIOS is disabled */ isp_pci_wr_reg(isp, HCCR, PCI_HCCR_CMD_BIOS); } /* and enable interrupts */ ISP_ENABLE_INTS(isp); } static void isp_pci_dumpregs(ispsoftc_t *isp, const char *msg) { struct isp_pcisoftc *pcs = (struct isp_pcisoftc *)isp; if (msg) printf("%s: %s\n", device_get_nameunit(isp->isp_dev), msg); else printf("%s:\n", device_get_nameunit(isp->isp_dev)); if (IS_SCSI(isp)) printf(" biu_conf1=%x", ISP_READ(isp, BIU_CONF1)); else printf(" biu_csr=%x", ISP_READ(isp, BIU2100_CSR)); printf(" biu_icr=%x biu_isr=%x biu_sema=%x ", ISP_READ(isp, BIU_ICR), ISP_READ(isp, BIU_ISR), ISP_READ(isp, BIU_SEMA)); printf("risc_hccr=%x\n", ISP_READ(isp, HCCR)); if (IS_SCSI(isp)) { ISP_WRITE(isp, HCCR, HCCR_CMD_PAUSE); printf(" cdma_conf=%x cdma_sts=%x cdma_fifostat=%x\n", ISP_READ(isp, CDMA_CONF), ISP_READ(isp, CDMA_STATUS), ISP_READ(isp, CDMA_FIFO_STS)); printf(" ddma_conf=%x ddma_sts=%x ddma_fifostat=%x\n", ISP_READ(isp, DDMA_CONF), ISP_READ(isp, DDMA_STATUS), ISP_READ(isp, DDMA_FIFO_STS)); printf(" sxp_int=%x sxp_gross=%x sxp(scsi_ctrl)=%x\n", ISP_READ(isp, SXP_INTERRUPT), ISP_READ(isp, SXP_GROSS_ERR), ISP_READ(isp, SXP_PINS_CTRL)); ISP_WRITE(isp, HCCR, HCCR_CMD_RELEASE); } printf(" mbox regs: %x %x %x %x %x\n", ISP_READ(isp, OUTMAILBOX0), ISP_READ(isp, OUTMAILBOX1), ISP_READ(isp, OUTMAILBOX2), ISP_READ(isp, OUTMAILBOX3), ISP_READ(isp, OUTMAILBOX4)); printf(" PCI Status Command/Status=%x\n", pci_read_config(pcs->pci_dev, PCIR_COMMAND, 1)); } Index: user/ngie/more-tests2/sys/dev/sfxge/common/efx.h =================================================================== --- user/ngie/more-tests2/sys/dev/sfxge/common/efx.h (revision 292053) +++ user/ngie/more-tests2/sys/dev/sfxge/common/efx.h (revision 292054) @@ -1,2327 +1,2330 @@ /*- * Copyright (c) 2006-2015 Solarflare Communications Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are met: * * 1. Redistributions of source code must retain the above copyright notice, * this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright notice, * this list of conditions and the following disclaimer in the documentation * and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR * OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, * EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * The views and conclusions contained in the software and documentation are * those of the authors and should not be interpreted as representing official * policies, either expressed or implied, of the FreeBSD Project. * * $FreeBSD$ */ #ifndef _SYS_EFX_H #define _SYS_EFX_H #include "efsys.h" #include "efx_phy_ids.h" #ifdef __cplusplus extern "C" { #endif #define EFX_STATIC_ASSERT(_cond) \ ((void)sizeof(char[(_cond) ? 1 : -1])) #define EFX_ARRAY_SIZE(_array) \ (sizeof(_array) / sizeof((_array)[0])) #define EFX_FIELD_OFFSET(_type, _field) \ ((size_t) &(((_type *)0)->_field)) /* Return codes */ typedef __success(return == 0) int efx_rc_t; /* Chip families */ typedef enum efx_family_e { EFX_FAMILY_INVALID, EFX_FAMILY_FALCON, EFX_FAMILY_SIENA, EFX_FAMILY_HUNTINGTON, EFX_FAMILY_NTYPES } efx_family_t; extern __checkReturn efx_rc_t efx_family( __in uint16_t venid, __in uint16_t devid, __out efx_family_t *efp); extern __checkReturn efx_rc_t efx_infer_family( __in efsys_bar_t *esbp, __out efx_family_t *efp); #define EFX_PCI_VENID_SFC 0x1924 #define EFX_PCI_DEVID_FALCON 0x0710 /* SFC4000 */ #define EFX_PCI_DEVID_BETHPAGE 0x0803 /* SFC9020 */ #define EFX_PCI_DEVID_SIENA 0x0813 /* SFL9021 */ #define EFX_PCI_DEVID_SIENA_F1_UNINIT 0x0810 #define EFX_PCI_DEVID_HUNTINGTON_PF_UNINIT 0x0901 #define EFX_PCI_DEVID_FARMINGDALE 0x0903 /* SFC9120 PF */ #define EFX_PCI_DEVID_GREENPORT 0x0923 /* SFC9140 PF */ #define EFX_PCI_DEVID_FARMINGDALE_VF 0x1903 /* SFC9120 VF */ #define EFX_PCI_DEVID_GREENPORT_VF 0x1923 /* SFC9140 VF */ #define EFX_MEM_BAR 2 /* Error codes */ enum { EFX_ERR_INVALID, EFX_ERR_SRAM_OOB, EFX_ERR_BUFID_DC_OOB, EFX_ERR_MEM_PERR, EFX_ERR_RBUF_OWN, EFX_ERR_TBUF_OWN, EFX_ERR_RDESQ_OWN, EFX_ERR_TDESQ_OWN, EFX_ERR_EVQ_OWN, EFX_ERR_EVFF_OFLO, EFX_ERR_ILL_ADDR, EFX_ERR_SRAM_PERR, EFX_ERR_NCODES }; /* Calculate the IEEE 802.3 CRC32 of a MAC addr */ extern __checkReturn uint32_t efx_crc32_calculate( __in uint32_t crc_init, __in_ecount(length) uint8_t const *input, __in int length); /* Type prototypes */ typedef struct efx_rxq_s efx_rxq_t; /* NIC */ typedef struct efx_nic_s efx_nic_t; #define EFX_NIC_FUNC_PRIMARY 0x00000001 #define EFX_NIC_FUNC_LINKCTRL 0x00000002 #define EFX_NIC_FUNC_TRUSTED 0x00000004 extern __checkReturn efx_rc_t efx_nic_create( __in efx_family_t family, __in efsys_identifier_t *esip, __in efsys_bar_t *esbp, __in efsys_lock_t *eslp, __deref_out efx_nic_t **enpp); extern __checkReturn efx_rc_t efx_nic_probe( __in efx_nic_t *enp); #if EFSYS_OPT_PCIE_TUNE extern __checkReturn efx_rc_t efx_nic_pcie_tune( __in efx_nic_t *enp, unsigned int nlanes); extern __checkReturn efx_rc_t efx_nic_pcie_extended_sync( __in efx_nic_t *enp); #endif /* EFSYS_OPT_PCIE_TUNE */ extern __checkReturn efx_rc_t efx_nic_init( __in efx_nic_t *enp); extern __checkReturn efx_rc_t efx_nic_reset( __in efx_nic_t *enp); #if EFSYS_OPT_DIAG extern __checkReturn efx_rc_t efx_nic_register_test( __in efx_nic_t *enp); #endif /* EFSYS_OPT_DIAG */ extern void efx_nic_fini( __in efx_nic_t *enp); extern void efx_nic_unprobe( __in efx_nic_t *enp); extern void efx_nic_destroy( __in efx_nic_t *enp); #if EFSYS_OPT_MCDI #if EFSYS_OPT_HUNTINGTON /* Huntington requires MCDIv2 commands */ #define WITH_MCDI_V2 1 #endif typedef struct efx_mcdi_req_s efx_mcdi_req_t; typedef enum efx_mcdi_exception_e { EFX_MCDI_EXCEPTION_MC_REBOOT, EFX_MCDI_EXCEPTION_MC_BADASSERT, } efx_mcdi_exception_t; #if EFSYS_OPT_MCDI_LOGGING typedef enum efx_log_msg_e { EFX_LOG_INVALID, EFX_LOG_MCDI_REQUEST, EFX_LOG_MCDI_RESPONSE, } efx_log_msg_t; #endif /* EFSYS_OPT_MCDI_LOGGING */ typedef struct efx_mcdi_transport_s { void *emt_context; efsys_mem_t *emt_dma_mem; void (*emt_execute)(void *, efx_mcdi_req_t *); void (*emt_ev_cpl)(void *); void (*emt_exception)(void *, efx_mcdi_exception_t); #if EFSYS_OPT_MCDI_LOGGING void (*emt_logger)(void *, efx_log_msg_t, void *, size_t, void *, size_t); #endif /* EFSYS_OPT_MCDI_LOGGING */ +#if EFSYS_OPT_MCDI_PROXY_AUTH + void (*emt_ev_proxy_response)(void *, uint32_t, efx_rc_t); +#endif /* EFSYS_OPT_MCDI_PROXY_AUTH */ } efx_mcdi_transport_t; extern __checkReturn efx_rc_t efx_mcdi_init( __in efx_nic_t *enp, __in const efx_mcdi_transport_t *mtp); extern __checkReturn efx_rc_t efx_mcdi_reboot( __in efx_nic_t *enp); void efx_mcdi_new_epoch( __in efx_nic_t *enp); extern void efx_mcdi_request_start( __in efx_nic_t *enp, __in efx_mcdi_req_t *emrp, __in boolean_t ev_cpl); extern __checkReturn boolean_t efx_mcdi_request_poll( __in efx_nic_t *enp); extern __checkReturn boolean_t efx_mcdi_request_abort( __in efx_nic_t *enp); extern void efx_mcdi_fini( __in efx_nic_t *enp); #endif /* EFSYS_OPT_MCDI */ /* INTR */ #define EFX_NINTR_FALCON 64 #define EFX_NINTR_SIENA 1024 typedef enum efx_intr_type_e { EFX_INTR_INVALID = 0, EFX_INTR_LINE, EFX_INTR_MESSAGE, EFX_INTR_NTYPES } efx_intr_type_t; #define EFX_INTR_SIZE (sizeof (efx_oword_t)) extern __checkReturn efx_rc_t efx_intr_init( __in efx_nic_t *enp, __in efx_intr_type_t type, __in efsys_mem_t *esmp); extern void efx_intr_enable( __in efx_nic_t *enp); extern void efx_intr_disable( __in efx_nic_t *enp); extern void efx_intr_disable_unlocked( __in efx_nic_t *enp); #define EFX_INTR_NEVQS 32 extern __checkReturn efx_rc_t efx_intr_trigger( __in efx_nic_t *enp, __in unsigned int level); extern void efx_intr_status_line( __in efx_nic_t *enp, __out boolean_t *fatalp, __out uint32_t *maskp); extern void efx_intr_status_message( __in efx_nic_t *enp, __in unsigned int message, __out boolean_t *fatalp); extern void efx_intr_fatal( __in efx_nic_t *enp); extern void efx_intr_fini( __in efx_nic_t *enp); /* MAC */ #if EFSYS_OPT_MAC_STATS /* START MKCONFIG GENERATED EfxHeaderMacBlock e323546097fd7c65 */ typedef enum efx_mac_stat_e { EFX_MAC_RX_OCTETS, EFX_MAC_RX_PKTS, EFX_MAC_RX_UNICST_PKTS, EFX_MAC_RX_MULTICST_PKTS, EFX_MAC_RX_BRDCST_PKTS, EFX_MAC_RX_PAUSE_PKTS, EFX_MAC_RX_LE_64_PKTS, EFX_MAC_RX_65_TO_127_PKTS, EFX_MAC_RX_128_TO_255_PKTS, EFX_MAC_RX_256_TO_511_PKTS, EFX_MAC_RX_512_TO_1023_PKTS, EFX_MAC_RX_1024_TO_15XX_PKTS, EFX_MAC_RX_GE_15XX_PKTS, EFX_MAC_RX_ERRORS, EFX_MAC_RX_FCS_ERRORS, EFX_MAC_RX_DROP_EVENTS, EFX_MAC_RX_FALSE_CARRIER_ERRORS, EFX_MAC_RX_SYMBOL_ERRORS, EFX_MAC_RX_ALIGN_ERRORS, EFX_MAC_RX_INTERNAL_ERRORS, EFX_MAC_RX_JABBER_PKTS, EFX_MAC_RX_LANE0_CHAR_ERR, EFX_MAC_RX_LANE1_CHAR_ERR, EFX_MAC_RX_LANE2_CHAR_ERR, EFX_MAC_RX_LANE3_CHAR_ERR, EFX_MAC_RX_LANE0_DISP_ERR, EFX_MAC_RX_LANE1_DISP_ERR, EFX_MAC_RX_LANE2_DISP_ERR, EFX_MAC_RX_LANE3_DISP_ERR, EFX_MAC_RX_MATCH_FAULT, EFX_MAC_RX_NODESC_DROP_CNT, EFX_MAC_TX_OCTETS, EFX_MAC_TX_PKTS, EFX_MAC_TX_UNICST_PKTS, EFX_MAC_TX_MULTICST_PKTS, EFX_MAC_TX_BRDCST_PKTS, EFX_MAC_TX_PAUSE_PKTS, EFX_MAC_TX_LE_64_PKTS, EFX_MAC_TX_65_TO_127_PKTS, EFX_MAC_TX_128_TO_255_PKTS, EFX_MAC_TX_256_TO_511_PKTS, EFX_MAC_TX_512_TO_1023_PKTS, EFX_MAC_TX_1024_TO_15XX_PKTS, EFX_MAC_TX_GE_15XX_PKTS, EFX_MAC_TX_ERRORS, EFX_MAC_TX_SGL_COL_PKTS, EFX_MAC_TX_MULT_COL_PKTS, EFX_MAC_TX_EX_COL_PKTS, EFX_MAC_TX_LATE_COL_PKTS, EFX_MAC_TX_DEF_PKTS, EFX_MAC_TX_EX_DEF_PKTS, EFX_MAC_PM_TRUNC_BB_OVERFLOW, EFX_MAC_PM_DISCARD_BB_OVERFLOW, EFX_MAC_PM_TRUNC_VFIFO_FULL, EFX_MAC_PM_DISCARD_VFIFO_FULL, EFX_MAC_PM_TRUNC_QBB, EFX_MAC_PM_DISCARD_QBB, EFX_MAC_PM_DISCARD_MAPPING, EFX_MAC_RXDP_Q_DISABLED_PKTS, EFX_MAC_RXDP_DI_DROPPED_PKTS, EFX_MAC_RXDP_STREAMING_PKTS, EFX_MAC_RXDP_HLB_FETCH, EFX_MAC_RXDP_HLB_WAIT, EFX_MAC_VADAPTER_RX_UNICAST_PACKETS, EFX_MAC_VADAPTER_RX_UNICAST_BYTES, EFX_MAC_VADAPTER_RX_MULTICAST_PACKETS, EFX_MAC_VADAPTER_RX_MULTICAST_BYTES, EFX_MAC_VADAPTER_RX_BROADCAST_PACKETS, EFX_MAC_VADAPTER_RX_BROADCAST_BYTES, EFX_MAC_VADAPTER_RX_BAD_PACKETS, EFX_MAC_VADAPTER_RX_BAD_BYTES, EFX_MAC_VADAPTER_RX_OVERFLOW, EFX_MAC_VADAPTER_TX_UNICAST_PACKETS, EFX_MAC_VADAPTER_TX_UNICAST_BYTES, EFX_MAC_VADAPTER_TX_MULTICAST_PACKETS, EFX_MAC_VADAPTER_TX_MULTICAST_BYTES, EFX_MAC_VADAPTER_TX_BROADCAST_PACKETS, EFX_MAC_VADAPTER_TX_BROADCAST_BYTES, EFX_MAC_VADAPTER_TX_BAD_PACKETS, EFX_MAC_VADAPTER_TX_BAD_BYTES, EFX_MAC_VADAPTER_TX_OVERFLOW, EFX_MAC_NSTATS } efx_mac_stat_t; /* END MKCONFIG GENERATED EfxHeaderMacBlock */ #endif /* EFSYS_OPT_MAC_STATS */ typedef enum efx_link_mode_e { EFX_LINK_UNKNOWN = 0, EFX_LINK_DOWN, EFX_LINK_10HDX, EFX_LINK_10FDX, EFX_LINK_100HDX, EFX_LINK_100FDX, EFX_LINK_1000HDX, EFX_LINK_1000FDX, EFX_LINK_10000FDX, EFX_LINK_40000FDX, EFX_LINK_NMODES } efx_link_mode_t; #define EFX_MAC_ADDR_LEN 6 #define EFX_MAC_ADDR_IS_MULTICAST(_address) (((uint8_t*)_address)[0] & 0x01) #define EFX_MAC_MULTICAST_LIST_MAX 256 #define EFX_MAC_SDU_MAX 9202 #define EFX_MAC_PDU(_sdu) \ P2ROUNDUP(((_sdu) \ + /* EtherII */ 14 \ + /* VLAN */ 4 \ + /* CRC */ 4 \ + /* bug16011 */ 16), \ (1 << 3)) #define EFX_MAC_PDU_MIN 60 #define EFX_MAC_PDU_MAX EFX_MAC_PDU(EFX_MAC_SDU_MAX) extern __checkReturn efx_rc_t efx_mac_pdu_set( __in efx_nic_t *enp, __in size_t pdu); extern __checkReturn efx_rc_t efx_mac_addr_set( __in efx_nic_t *enp, __in uint8_t *addr); extern __checkReturn efx_rc_t efx_mac_filter_set( __in efx_nic_t *enp, __in boolean_t all_unicst, __in boolean_t mulcst, __in boolean_t all_mulcst, __in boolean_t brdcst); extern __checkReturn efx_rc_t efx_mac_multicast_list_set( __in efx_nic_t *enp, __in_ecount(6*count) uint8_t const *addrs, __in int count); extern __checkReturn efx_rc_t efx_mac_filter_default_rxq_set( __in efx_nic_t *enp, __in efx_rxq_t *erp, __in boolean_t using_rss); extern void efx_mac_filter_default_rxq_clear( __in efx_nic_t *enp); extern __checkReturn efx_rc_t efx_mac_drain( __in efx_nic_t *enp, __in boolean_t enabled); extern __checkReturn efx_rc_t efx_mac_up( __in efx_nic_t *enp, __out boolean_t *mac_upp); #define EFX_FCNTL_RESPOND 0x00000001 #define EFX_FCNTL_GENERATE 0x00000002 extern __checkReturn efx_rc_t efx_mac_fcntl_set( __in efx_nic_t *enp, __in unsigned int fcntl, __in boolean_t autoneg); extern void efx_mac_fcntl_get( __in efx_nic_t *enp, __out unsigned int *fcntl_wantedp, __out unsigned int *fcntl_linkp); #define EFX_MAC_HASH_BITS (1 << 8) extern __checkReturn efx_rc_t efx_pktfilter_init( __in efx_nic_t *enp); extern void efx_pktfilter_fini( __in efx_nic_t *enp); extern __checkReturn efx_rc_t efx_pktfilter_set( __in efx_nic_t *enp, __in boolean_t unicst, __in boolean_t brdcst); extern __checkReturn efx_rc_t efx_mac_hash_set( __in efx_nic_t *enp, __in_ecount(EFX_MAC_HASH_BITS) unsigned int const *bucket); #if EFSYS_OPT_MCAST_FILTER_LIST extern __checkReturn efx_rc_t efx_pktfilter_mcast_list_set( __in efx_nic_t *enp, __in uint8_t const *addrs, __in int count); #endif /* EFSYS_OPT_MCAST_FILTER_LIST */ extern __checkReturn efx_rc_t efx_pktfilter_mcast_all( __in efx_nic_t *enp); #if EFSYS_OPT_MAC_STATS #if EFSYS_OPT_NAMES extern __checkReturn const char * efx_mac_stat_name( __in efx_nic_t *enp, __in unsigned int id); #endif /* EFSYS_OPT_NAMES */ #define EFX_MAC_STATS_SIZE 0x400 /* * Upload mac statistics supported by the hardware into the given buffer. * * The reference buffer must be at least %EFX_MAC_STATS_SIZE bytes, * and page aligned. * * The hardware will only DMA statistics that it understands (of course). * Drivers should not make any assumptions about which statistics are * supported, especially when the statistics are generated by firmware. * * Thus, drivers should zero this buffer before use, so that not-understood * statistics read back as zero. */ extern __checkReturn efx_rc_t efx_mac_stats_upload( __in efx_nic_t *enp, __in efsys_mem_t *esmp); extern __checkReturn efx_rc_t efx_mac_stats_periodic( __in efx_nic_t *enp, __in efsys_mem_t *esmp, __in uint16_t period_ms, __in boolean_t events); extern __checkReturn efx_rc_t efx_mac_stats_update( __in efx_nic_t *enp, __in efsys_mem_t *esmp, __inout_ecount(EFX_MAC_NSTATS) efsys_stat_t *stat, __inout_opt uint32_t *generationp); #endif /* EFSYS_OPT_MAC_STATS */ /* MON */ typedef enum efx_mon_type_e { EFX_MON_INVALID = 0, EFX_MON_NULL, EFX_MON_LM87, EFX_MON_MAX6647, EFX_MON_SFC90X0, EFX_MON_SFC91X0, EFX_MON_NTYPES } efx_mon_type_t; #if EFSYS_OPT_NAMES extern const char * efx_mon_name( __in efx_nic_t *enp); #endif /* EFSYS_OPT_NAMES */ extern __checkReturn efx_rc_t efx_mon_init( __in efx_nic_t *enp); #if EFSYS_OPT_MON_STATS #define EFX_MON_STATS_PAGE_SIZE 0x100 #define EFX_MON_MASK_ELEMENT_SIZE 32 /* START MKCONFIG GENERATED MonitorHeaderStatsBlock c79c86b62a144846 */ typedef enum efx_mon_stat_e { EFX_MON_STAT_2_5V, EFX_MON_STAT_VCCP1, EFX_MON_STAT_VCC, EFX_MON_STAT_5V, EFX_MON_STAT_12V, EFX_MON_STAT_VCCP2, EFX_MON_STAT_EXT_TEMP, EFX_MON_STAT_INT_TEMP, EFX_MON_STAT_AIN1, EFX_MON_STAT_AIN2, EFX_MON_STAT_INT_COOLING, EFX_MON_STAT_EXT_COOLING, EFX_MON_STAT_1V, EFX_MON_STAT_1_2V, EFX_MON_STAT_1_8V, EFX_MON_STAT_3_3V, EFX_MON_STAT_1_2VA, EFX_MON_STAT_VREF, EFX_MON_STAT_VAOE, EFX_MON_STAT_AOE_TEMP, EFX_MON_STAT_PSU_AOE_TEMP, EFX_MON_STAT_PSU_TEMP, EFX_MON_STAT_FAN0, EFX_MON_STAT_FAN1, EFX_MON_STAT_FAN2, EFX_MON_STAT_FAN3, EFX_MON_STAT_FAN4, EFX_MON_STAT_VAOE_IN, EFX_MON_STAT_IAOE, EFX_MON_STAT_IAOE_IN, EFX_MON_STAT_NIC_POWER, EFX_MON_STAT_0_9V, EFX_MON_STAT_I0_9V, EFX_MON_STAT_I1_2V, EFX_MON_STAT_0_9V_ADC, EFX_MON_STAT_INT_TEMP2, EFX_MON_STAT_VREG_TEMP, EFX_MON_STAT_VREG_0_9V_TEMP, EFX_MON_STAT_VREG_1_2V_TEMP, EFX_MON_STAT_INT_VPTAT, EFX_MON_STAT_INT_ADC_TEMP, EFX_MON_STAT_EXT_VPTAT, EFX_MON_STAT_EXT_ADC_TEMP, EFX_MON_STAT_AMBIENT_TEMP, EFX_MON_STAT_AIRFLOW, EFX_MON_STAT_VDD08D_VSS08D_CSR, EFX_MON_STAT_VDD08D_VSS08D_CSR_EXTADC, EFX_MON_STAT_HOTPOINT_TEMP, EFX_MON_STAT_PHY_POWER_SWITCH_PORT0, EFX_MON_STAT_PHY_POWER_SWITCH_PORT1, EFX_MON_STAT_MUM_VCC, EFX_MON_STAT_0V9_A, EFX_MON_STAT_I0V9_A, EFX_MON_STAT_0V9_A_TEMP, EFX_MON_STAT_0V9_B, EFX_MON_STAT_I0V9_B, EFX_MON_STAT_0V9_B_TEMP, EFX_MON_STAT_CCOM_AVREG_1V2_SUPPLY, EFX_MON_STAT_CCOM_AVREG_1V2_SUPPLY_EXT_ADC, EFX_MON_STAT_CCOM_AVREG_1V8_SUPPLY, EFX_MON_STAT_CCOM_AVREG_1V8_SUPPLY_EXT_ADC, EFX_MON_STAT_CONTROLLER_MASTER_VPTAT, EFX_MON_STAT_CONTROLLER_MASTER_INTERNAL_TEMP, EFX_MON_STAT_CONTROLLER_MASTER_VPTAT_EXT_ADC, EFX_MON_STAT_CONTROLLER_MASTER_INTERNAL_TEMP_EXT_ADC, EFX_MON_STAT_CONTROLLER_SLAVE_VPTAT, EFX_MON_STAT_CONTROLLER_SLAVE_INTERNAL_TEMP, EFX_MON_STAT_CONTROLLER_SLAVE_VPTAT_EXT_ADC, EFX_MON_STAT_CONTROLLER_SLAVE_INTERNAL_TEMP_EXT_ADC, EFX_MON_NSTATS } efx_mon_stat_t; /* END MKCONFIG GENERATED MonitorHeaderStatsBlock */ typedef enum efx_mon_stat_state_e { EFX_MON_STAT_STATE_OK = 0, EFX_MON_STAT_STATE_WARNING = 1, EFX_MON_STAT_STATE_FATAL = 2, EFX_MON_STAT_STATE_BROKEN = 3, EFX_MON_STAT_STATE_NO_READING = 4, } efx_mon_stat_state_t; typedef struct efx_mon_stat_value_s { uint16_t emsv_value; uint16_t emsv_state; } efx_mon_stat_value_t; #if EFSYS_OPT_NAMES extern const char * efx_mon_stat_name( __in efx_nic_t *enp, __in efx_mon_stat_t id); #endif /* EFSYS_OPT_NAMES */ extern __checkReturn efx_rc_t efx_mon_stats_update( __in efx_nic_t *enp, __in efsys_mem_t *esmp, __inout_ecount(EFX_MON_NSTATS) efx_mon_stat_value_t *values); #endif /* EFSYS_OPT_MON_STATS */ extern void efx_mon_fini( __in efx_nic_t *enp); /* PHY */ #define PMA_PMD_MMD 1 #define PCS_MMD 3 #define PHY_XS_MMD 4 #define DTE_XS_MMD 5 #define AN_MMD 7 #define CL22EXT_MMD 29 #define MAXMMD ((1 << 5) - 1) extern __checkReturn efx_rc_t efx_phy_verify( __in efx_nic_t *enp); #if EFSYS_OPT_PHY_LED_CONTROL typedef enum efx_phy_led_mode_e { EFX_PHY_LED_DEFAULT = 0, EFX_PHY_LED_OFF, EFX_PHY_LED_ON, EFX_PHY_LED_FLASH, EFX_PHY_LED_NMODES } efx_phy_led_mode_t; extern __checkReturn efx_rc_t efx_phy_led_set( __in efx_nic_t *enp, __in efx_phy_led_mode_t mode); #endif /* EFSYS_OPT_PHY_LED_CONTROL */ extern __checkReturn efx_rc_t efx_port_init( __in efx_nic_t *enp); #if EFSYS_OPT_LOOPBACK typedef enum efx_loopback_type_e { EFX_LOOPBACK_OFF = 0, EFX_LOOPBACK_DATA = 1, EFX_LOOPBACK_GMAC = 2, EFX_LOOPBACK_XGMII = 3, EFX_LOOPBACK_XGXS = 4, EFX_LOOPBACK_XAUI = 5, EFX_LOOPBACK_GMII = 6, EFX_LOOPBACK_SGMII = 7, EFX_LOOPBACK_XGBR = 8, EFX_LOOPBACK_XFI = 9, EFX_LOOPBACK_XAUI_FAR = 10, EFX_LOOPBACK_GMII_FAR = 11, EFX_LOOPBACK_SGMII_FAR = 12, EFX_LOOPBACK_XFI_FAR = 13, EFX_LOOPBACK_GPHY = 14, EFX_LOOPBACK_PHY_XS = 15, EFX_LOOPBACK_PCS = 16, EFX_LOOPBACK_PMA_PMD = 17, EFX_LOOPBACK_XPORT = 18, EFX_LOOPBACK_XGMII_WS = 19, EFX_LOOPBACK_XAUI_WS = 20, EFX_LOOPBACK_XAUI_WS_FAR = 21, EFX_LOOPBACK_XAUI_WS_NEAR = 22, EFX_LOOPBACK_GMII_WS = 23, EFX_LOOPBACK_XFI_WS = 24, EFX_LOOPBACK_XFI_WS_FAR = 25, EFX_LOOPBACK_PHYXS_WS = 26, EFX_LOOPBACK_PMA_INT = 27, EFX_LOOPBACK_SD_NEAR = 28, EFX_LOOPBACK_SD_FAR = 29, EFX_LOOPBACK_PMA_INT_WS = 30, EFX_LOOPBACK_SD_FEP2_WS = 31, EFX_LOOPBACK_SD_FEP1_5_WS = 32, EFX_LOOPBACK_SD_FEP_WS = 33, EFX_LOOPBACK_SD_FES_WS = 34, EFX_LOOPBACK_NTYPES } efx_loopback_type_t; typedef enum efx_loopback_kind_e { EFX_LOOPBACK_KIND_OFF = 0, EFX_LOOPBACK_KIND_ALL, EFX_LOOPBACK_KIND_MAC, EFX_LOOPBACK_KIND_PHY, EFX_LOOPBACK_NKINDS } efx_loopback_kind_t; extern void efx_loopback_mask( __in efx_loopback_kind_t loopback_kind, __out efx_qword_t *maskp); extern __checkReturn efx_rc_t efx_port_loopback_set( __in efx_nic_t *enp, __in efx_link_mode_t link_mode, __in efx_loopback_type_t type); #if EFSYS_OPT_NAMES extern __checkReturn const char * efx_loopback_type_name( __in efx_nic_t *enp, __in efx_loopback_type_t type); #endif /* EFSYS_OPT_NAMES */ #endif /* EFSYS_OPT_LOOPBACK */ extern __checkReturn efx_rc_t efx_port_poll( __in efx_nic_t *enp, __out_opt efx_link_mode_t *link_modep); extern void efx_port_fini( __in efx_nic_t *enp); typedef enum efx_phy_cap_type_e { EFX_PHY_CAP_INVALID = 0, EFX_PHY_CAP_10HDX, EFX_PHY_CAP_10FDX, EFX_PHY_CAP_100HDX, EFX_PHY_CAP_100FDX, EFX_PHY_CAP_1000HDX, EFX_PHY_CAP_1000FDX, EFX_PHY_CAP_10000FDX, EFX_PHY_CAP_PAUSE, EFX_PHY_CAP_ASYM, EFX_PHY_CAP_AN, EFX_PHY_CAP_40000FDX, EFX_PHY_CAP_NTYPES } efx_phy_cap_type_t; #define EFX_PHY_CAP_CURRENT 0x00000000 #define EFX_PHY_CAP_DEFAULT 0x00000001 #define EFX_PHY_CAP_PERM 0x00000002 extern void efx_phy_adv_cap_get( __in efx_nic_t *enp, __in uint32_t flag, __out uint32_t *maskp); extern __checkReturn efx_rc_t efx_phy_adv_cap_set( __in efx_nic_t *enp, __in uint32_t mask); extern void efx_phy_lp_cap_get( __in efx_nic_t *enp, __out uint32_t *maskp); extern __checkReturn efx_rc_t efx_phy_oui_get( __in efx_nic_t *enp, __out uint32_t *ouip); typedef enum efx_phy_media_type_e { EFX_PHY_MEDIA_INVALID = 0, EFX_PHY_MEDIA_XAUI, EFX_PHY_MEDIA_CX4, EFX_PHY_MEDIA_KX4, EFX_PHY_MEDIA_XFP, EFX_PHY_MEDIA_SFP_PLUS, EFX_PHY_MEDIA_BASE_T, EFX_PHY_MEDIA_QSFP_PLUS, EFX_PHY_MEDIA_NTYPES } efx_phy_media_type_t; /* Get the type of medium currently used. If the board has ports for * modules, a module is present, and we recognise the media type of * the module, then this will be the media type of the module. * Otherwise it will be the media type of the port. */ extern void efx_phy_media_type_get( __in efx_nic_t *enp, __out efx_phy_media_type_t *typep); #if EFSYS_OPT_PHY_STATS /* START MKCONFIG GENERATED PhyHeaderStatsBlock 30ed56ad501f8e36 */ typedef enum efx_phy_stat_e { EFX_PHY_STAT_OUI, EFX_PHY_STAT_PMA_PMD_LINK_UP, EFX_PHY_STAT_PMA_PMD_RX_FAULT, EFX_PHY_STAT_PMA_PMD_TX_FAULT, EFX_PHY_STAT_PMA_PMD_REV_A, EFX_PHY_STAT_PMA_PMD_REV_B, EFX_PHY_STAT_PMA_PMD_REV_C, EFX_PHY_STAT_PMA_PMD_REV_D, EFX_PHY_STAT_PCS_LINK_UP, EFX_PHY_STAT_PCS_RX_FAULT, EFX_PHY_STAT_PCS_TX_FAULT, EFX_PHY_STAT_PCS_BER, EFX_PHY_STAT_PCS_BLOCK_ERRORS, EFX_PHY_STAT_PHY_XS_LINK_UP, EFX_PHY_STAT_PHY_XS_RX_FAULT, EFX_PHY_STAT_PHY_XS_TX_FAULT, EFX_PHY_STAT_PHY_XS_ALIGN, EFX_PHY_STAT_PHY_XS_SYNC_A, EFX_PHY_STAT_PHY_XS_SYNC_B, EFX_PHY_STAT_PHY_XS_SYNC_C, EFX_PHY_STAT_PHY_XS_SYNC_D, EFX_PHY_STAT_AN_LINK_UP, EFX_PHY_STAT_AN_MASTER, EFX_PHY_STAT_AN_LOCAL_RX_OK, EFX_PHY_STAT_AN_REMOTE_RX_OK, EFX_PHY_STAT_CL22EXT_LINK_UP, EFX_PHY_STAT_SNR_A, EFX_PHY_STAT_SNR_B, EFX_PHY_STAT_SNR_C, EFX_PHY_STAT_SNR_D, EFX_PHY_STAT_PMA_PMD_SIGNAL_A, EFX_PHY_STAT_PMA_PMD_SIGNAL_B, EFX_PHY_STAT_PMA_PMD_SIGNAL_C, EFX_PHY_STAT_PMA_PMD_SIGNAL_D, EFX_PHY_STAT_AN_COMPLETE, EFX_PHY_STAT_PMA_PMD_REV_MAJOR, EFX_PHY_STAT_PMA_PMD_REV_MINOR, EFX_PHY_STAT_PMA_PMD_REV_MICRO, EFX_PHY_STAT_PCS_FW_VERSION_0, EFX_PHY_STAT_PCS_FW_VERSION_1, EFX_PHY_STAT_PCS_FW_VERSION_2, EFX_PHY_STAT_PCS_FW_VERSION_3, EFX_PHY_STAT_PCS_FW_BUILD_YY, EFX_PHY_STAT_PCS_FW_BUILD_MM, EFX_PHY_STAT_PCS_FW_BUILD_DD, EFX_PHY_STAT_PCS_OP_MODE, EFX_PHY_NSTATS } efx_phy_stat_t; /* END MKCONFIG GENERATED PhyHeaderStatsBlock */ #if EFSYS_OPT_NAMES extern const char * efx_phy_stat_name( __in efx_nic_t *enp, __in efx_phy_stat_t stat); #endif /* EFSYS_OPT_NAMES */ #define EFX_PHY_STATS_SIZE 0x100 extern __checkReturn efx_rc_t efx_phy_stats_update( __in efx_nic_t *enp, __in efsys_mem_t *esmp, __inout_ecount(EFX_PHY_NSTATS) uint32_t *stat); #endif /* EFSYS_OPT_PHY_STATS */ #if EFSYS_OPT_PHY_PROPS #if EFSYS_OPT_NAMES extern const char * efx_phy_prop_name( __in efx_nic_t *enp, __in unsigned int id); #endif /* EFSYS_OPT_NAMES */ #define EFX_PHY_PROP_DEFAULT 0x00000001 extern __checkReturn efx_rc_t efx_phy_prop_get( __in efx_nic_t *enp, __in unsigned int id, __in uint32_t flags, __out uint32_t *valp); extern __checkReturn efx_rc_t efx_phy_prop_set( __in efx_nic_t *enp, __in unsigned int id, __in uint32_t val); #endif /* EFSYS_OPT_PHY_PROPS */ #if EFSYS_OPT_BIST typedef enum efx_bist_type_e { EFX_BIST_TYPE_UNKNOWN, EFX_BIST_TYPE_PHY_NORMAL, EFX_BIST_TYPE_PHY_CABLE_SHORT, EFX_BIST_TYPE_PHY_CABLE_LONG, EFX_BIST_TYPE_MC_MEM, /* Test the MC DMEM and IMEM */ EFX_BIST_TYPE_SAT_MEM, /* Test the DMEM and IMEM of satellite cpus*/ EFX_BIST_TYPE_REG, /* Test the register memories */ EFX_BIST_TYPE_NTYPES, } efx_bist_type_t; typedef enum efx_bist_result_e { EFX_BIST_RESULT_UNKNOWN, EFX_BIST_RESULT_RUNNING, EFX_BIST_RESULT_PASSED, EFX_BIST_RESULT_FAILED, } efx_bist_result_t; typedef enum efx_phy_cable_status_e { EFX_PHY_CABLE_STATUS_OK, EFX_PHY_CABLE_STATUS_INVALID, EFX_PHY_CABLE_STATUS_OPEN, EFX_PHY_CABLE_STATUS_INTRAPAIRSHORT, EFX_PHY_CABLE_STATUS_INTERPAIRSHORT, EFX_PHY_CABLE_STATUS_BUSY, } efx_phy_cable_status_t; typedef enum efx_bist_value_e { EFX_BIST_PHY_CABLE_LENGTH_A, EFX_BIST_PHY_CABLE_LENGTH_B, EFX_BIST_PHY_CABLE_LENGTH_C, EFX_BIST_PHY_CABLE_LENGTH_D, EFX_BIST_PHY_CABLE_STATUS_A, EFX_BIST_PHY_CABLE_STATUS_B, EFX_BIST_PHY_CABLE_STATUS_C, EFX_BIST_PHY_CABLE_STATUS_D, EFX_BIST_FAULT_CODE, /* Memory BIST specific values. These match to the MC_CMD_BIST_POLL * response. */ EFX_BIST_MEM_TEST, EFX_BIST_MEM_ADDR, EFX_BIST_MEM_BUS, EFX_BIST_MEM_EXPECT, EFX_BIST_MEM_ACTUAL, EFX_BIST_MEM_ECC, EFX_BIST_MEM_ECC_PARITY, EFX_BIST_MEM_ECC_FATAL, EFX_BIST_NVALUES, } efx_bist_value_t; extern __checkReturn efx_rc_t efx_bist_enable_offline( __in efx_nic_t *enp); extern __checkReturn efx_rc_t efx_bist_start( __in efx_nic_t *enp, __in efx_bist_type_t type); extern __checkReturn efx_rc_t efx_bist_poll( __in efx_nic_t *enp, __in efx_bist_type_t type, __out efx_bist_result_t *resultp, __out_opt uint32_t *value_maskp, __out_ecount_opt(count) unsigned long *valuesp, __in size_t count); extern void efx_bist_stop( __in efx_nic_t *enp, __in efx_bist_type_t type); #endif /* EFSYS_OPT_BIST */ #define EFX_FEATURE_IPV6 0x00000001 #define EFX_FEATURE_LFSR_HASH_INSERT 0x00000002 #define EFX_FEATURE_LINK_EVENTS 0x00000004 #define EFX_FEATURE_PERIODIC_MAC_STATS 0x00000008 #define EFX_FEATURE_WOL 0x00000010 #define EFX_FEATURE_MCDI 0x00000020 #define EFX_FEATURE_LOOKAHEAD_SPLIT 0x00000040 #define EFX_FEATURE_MAC_HEADER_FILTERS 0x00000080 #define EFX_FEATURE_TURBO 0x00000100 #define EFX_FEATURE_MCDI_DMA 0x00000200 #define EFX_FEATURE_TX_SRC_FILTERS 0x00000400 #define EFX_FEATURE_PIO_BUFFERS 0x00000800 #define EFX_FEATURE_FW_ASSISTED_TSO 0x00001000 typedef struct efx_nic_cfg_s { uint32_t enc_board_type; uint32_t enc_phy_type; #if EFSYS_OPT_NAMES char enc_phy_name[21]; #endif char enc_phy_revision[21]; efx_mon_type_t enc_mon_type; #if EFSYS_OPT_MON_STATS uint32_t enc_mon_stat_dma_buf_size; uint32_t enc_mon_stat_mask[(EFX_MON_NSTATS + 31) / 32]; #endif unsigned int enc_features; uint8_t enc_mac_addr[6]; uint8_t enc_port; /* PHY port number */ uint32_t enc_func_flags; uint32_t enc_intr_vec_base; uint32_t enc_intr_limit; uint32_t enc_evq_limit; uint32_t enc_txq_limit; uint32_t enc_rxq_limit; uint32_t enc_buftbl_limit; uint32_t enc_piobuf_limit; uint32_t enc_piobuf_size; uint32_t enc_evq_timer_quantum_ns; uint32_t enc_evq_timer_max_us; uint32_t enc_clk_mult; uint32_t enc_rx_prefix_size; uint32_t enc_rx_buf_align_start; uint32_t enc_rx_buf_align_end; #if EFSYS_OPT_LOOPBACK efx_qword_t enc_loopback_types[EFX_LINK_NMODES]; #endif /* EFSYS_OPT_LOOPBACK */ #if EFSYS_OPT_PHY_FLAGS uint32_t enc_phy_flags_mask; #endif /* EFSYS_OPT_PHY_FLAGS */ #if EFSYS_OPT_PHY_LED_CONTROL uint32_t enc_led_mask; #endif /* EFSYS_OPT_PHY_LED_CONTROL */ #if EFSYS_OPT_PHY_STATS uint64_t enc_phy_stat_mask; #endif /* EFSYS_OPT_PHY_STATS */ #if EFSYS_OPT_PHY_PROPS unsigned int enc_phy_nprops; #endif /* EFSYS_OPT_PHY_PROPS */ #if EFSYS_OPT_SIENA uint8_t enc_mcdi_mdio_channel; #if EFSYS_OPT_PHY_STATS uint32_t enc_mcdi_phy_stat_mask; #endif /* EFSYS_OPT_PHY_STATS */ #endif /* EFSYS_OPT_SIENA */ #if (EFSYS_OPT_SIENA || EFSYS_OPT_HUNTINGTON) #if EFSYS_OPT_MON_STATS uint32_t *enc_mcdi_sensor_maskp; uint32_t enc_mcdi_sensor_mask_size; #endif /* EFSYS_OPT_MON_STATS */ #endif /* (EFSYS_OPT_SIENA | EFSYS_OPT_HUNTINGTON) */ #if EFSYS_OPT_BIST uint32_t enc_bist_mask; #endif /* EFSYS_OPT_BIST */ #if EFSYS_OPT_HUNTINGTON uint32_t enc_pf; uint32_t enc_vf; uint32_t enc_privilege_mask; #endif /* EFSYS_OPT_HUNTINGTON */ boolean_t enc_bug26807_workaround; boolean_t enc_bug35388_workaround; boolean_t enc_bug41750_workaround; boolean_t enc_rx_batching_enabled; /* Maximum number of descriptors completed in an rx event. */ uint32_t enc_rx_batch_max; /* Number of rx descriptors the hardware requires for a push. */ uint32_t enc_rx_push_align; /* * Maximum number of bytes into the packet the TCP header can start for * the hardware to apply TSO packet edits. */ uint32_t enc_tx_tso_tcp_header_offset_limit; boolean_t enc_fw_assisted_tso_enabled; boolean_t enc_hw_tx_insert_vlan_enabled; /* Datapath firmware vadapter/vport/vswitch support */ boolean_t enc_datapath_cap_evb; boolean_t enc_rx_disable_scatter_supported; boolean_t enc_allow_set_mac_with_installed_filters; /* External port identifier */ uint8_t enc_external_port; uint32_t enc_mcdi_max_payload_length; } efx_nic_cfg_t; #define EFX_PCI_FUNCTION_IS_PF(_encp) ((_encp)->enc_vf == 0xffff) #define EFX_PCI_FUNCTION_IS_VF(_encp) ((_encp)->enc_vf != 0xffff) #define EFX_PCI_FUNCTION(_encp) \ (EFX_PCI_FUNCTION_IS_PF(_encp) ? (_encp)->enc_pf : (_encp)->enc_vf) #define EFX_PCI_VF_PARENT(_encp) ((_encp)->enc_pf) extern const efx_nic_cfg_t * efx_nic_cfg_get( __in efx_nic_t *enp); /* Driver resource limits (minimum required/maximum usable). */ typedef struct efx_drv_limits_s { uint32_t edl_min_evq_count; uint32_t edl_max_evq_count; uint32_t edl_min_rxq_count; uint32_t edl_max_rxq_count; uint32_t edl_min_txq_count; uint32_t edl_max_txq_count; /* PIO blocks (sub-allocated from piobuf) */ uint32_t edl_min_pio_alloc_size; uint32_t edl_max_pio_alloc_count; } efx_drv_limits_t; extern __checkReturn efx_rc_t efx_nic_set_drv_limits( __inout efx_nic_t *enp, __in efx_drv_limits_t *edlp); typedef enum efx_nic_region_e { EFX_REGION_VI, /* Memory BAR UC mapping */ EFX_REGION_PIO_WRITE_VI, /* Memory BAR WC mapping */ } efx_nic_region_t; extern __checkReturn efx_rc_t efx_nic_get_bar_region( __in efx_nic_t *enp, __in efx_nic_region_t region, __out uint32_t *offsetp, __out size_t *sizep); extern __checkReturn efx_rc_t efx_nic_get_vi_pool( __in efx_nic_t *enp, __out uint32_t *evq_countp, __out uint32_t *rxq_countp, __out uint32_t *txq_countp); #if EFSYS_OPT_VPD typedef enum efx_vpd_tag_e { EFX_VPD_ID = 0x02, EFX_VPD_END = 0x0f, EFX_VPD_RO = 0x10, EFX_VPD_RW = 0x11, } efx_vpd_tag_t; typedef uint16_t efx_vpd_keyword_t; typedef struct efx_vpd_value_s { efx_vpd_tag_t evv_tag; efx_vpd_keyword_t evv_keyword; uint8_t evv_length; uint8_t evv_value[0x100]; } efx_vpd_value_t; #define EFX_VPD_KEYWORD(x, y) ((x) | ((y) << 8)) extern __checkReturn efx_rc_t efx_vpd_init( __in efx_nic_t *enp); extern __checkReturn efx_rc_t efx_vpd_size( __in efx_nic_t *enp, __out size_t *sizep); extern __checkReturn efx_rc_t efx_vpd_read( __in efx_nic_t *enp, __out_bcount(size) caddr_t data, __in size_t size); extern __checkReturn efx_rc_t efx_vpd_verify( __in efx_nic_t *enp, __in_bcount(size) caddr_t data, __in size_t size); extern __checkReturn efx_rc_t efx_vpd_reinit( __in efx_nic_t *enp, __in_bcount(size) caddr_t data, __in size_t size); extern __checkReturn efx_rc_t efx_vpd_get( __in efx_nic_t *enp, __in_bcount(size) caddr_t data, __in size_t size, __inout efx_vpd_value_t *evvp); extern __checkReturn efx_rc_t efx_vpd_set( __in efx_nic_t *enp, __inout_bcount(size) caddr_t data, __in size_t size, __in efx_vpd_value_t *evvp); extern __checkReturn efx_rc_t efx_vpd_next( __in efx_nic_t *enp, __inout_bcount(size) caddr_t data, __in size_t size, __out efx_vpd_value_t *evvp, __inout unsigned int *contp); extern __checkReturn efx_rc_t efx_vpd_write( __in efx_nic_t *enp, __in_bcount(size) caddr_t data, __in size_t size); extern void efx_vpd_fini( __in efx_nic_t *enp); #endif /* EFSYS_OPT_VPD */ /* NVRAM */ #if EFSYS_OPT_NVRAM typedef enum efx_nvram_type_e { EFX_NVRAM_INVALID = 0, EFX_NVRAM_BOOTROM, EFX_NVRAM_BOOTROM_CFG, EFX_NVRAM_MC_FIRMWARE, EFX_NVRAM_MC_GOLDEN, EFX_NVRAM_PHY, EFX_NVRAM_NULLPHY, EFX_NVRAM_FPGA, EFX_NVRAM_FCFW, EFX_NVRAM_CPLD, EFX_NVRAM_FPGA_BACKUP, EFX_NVRAM_DYNAMIC_CFG, EFX_NVRAM_NTYPES, } efx_nvram_type_t; extern __checkReturn efx_rc_t efx_nvram_init( __in efx_nic_t *enp); #if EFSYS_OPT_DIAG extern __checkReturn efx_rc_t efx_nvram_test( __in efx_nic_t *enp); #endif /* EFSYS_OPT_DIAG */ extern __checkReturn efx_rc_t efx_nvram_size( __in efx_nic_t *enp, __in efx_nvram_type_t type, __out size_t *sizep); extern __checkReturn efx_rc_t efx_nvram_rw_start( __in efx_nic_t *enp, __in efx_nvram_type_t type, __out_opt size_t *pref_chunkp); extern void efx_nvram_rw_finish( __in efx_nic_t *enp, __in efx_nvram_type_t type); extern __checkReturn efx_rc_t efx_nvram_get_version( __in efx_nic_t *enp, __in efx_nvram_type_t type, __out uint32_t *subtypep, __out_ecount(4) uint16_t version[4]); extern __checkReturn efx_rc_t efx_nvram_read_chunk( __in efx_nic_t *enp, __in efx_nvram_type_t type, __in unsigned int offset, __out_bcount(size) caddr_t data, __in size_t size); extern __checkReturn efx_rc_t efx_nvram_set_version( __in efx_nic_t *enp, __in efx_nvram_type_t type, __in_ecount(4) uint16_t version[4]); /* Validate contents of TLV formatted partition */ extern __checkReturn efx_rc_t efx_nvram_tlv_validate( __in efx_nic_t *enp, __in uint32_t partn, __in_bcount(partn_size) caddr_t partn_data, __in size_t partn_size); extern __checkReturn efx_rc_t efx_nvram_erase( __in efx_nic_t *enp, __in efx_nvram_type_t type); extern __checkReturn efx_rc_t efx_nvram_write_chunk( __in efx_nic_t *enp, __in efx_nvram_type_t type, __in unsigned int offset, __in_bcount(size) caddr_t data, __in size_t size); extern void efx_nvram_fini( __in efx_nic_t *enp); #endif /* EFSYS_OPT_NVRAM */ #if EFSYS_OPT_BOOTCFG extern efx_rc_t efx_bootcfg_read( __in efx_nic_t *enp, __out_bcount(size) caddr_t data, __in size_t size); extern efx_rc_t efx_bootcfg_write( __in efx_nic_t *enp, __in_bcount(size) caddr_t data, __in size_t size); #endif /* EFSYS_OPT_BOOTCFG */ #if EFSYS_OPT_WOL typedef enum efx_wol_type_e { EFX_WOL_TYPE_INVALID, EFX_WOL_TYPE_MAGIC, EFX_WOL_TYPE_BITMAP, EFX_WOL_TYPE_LINK, EFX_WOL_NTYPES, } efx_wol_type_t; typedef enum efx_lightsout_offload_type_e { EFX_LIGHTSOUT_OFFLOAD_TYPE_INVALID, EFX_LIGHTSOUT_OFFLOAD_TYPE_ARP, EFX_LIGHTSOUT_OFFLOAD_TYPE_NS, } efx_lightsout_offload_type_t; #define EFX_WOL_BITMAP_MASK_SIZE (48) #define EFX_WOL_BITMAP_VALUE_SIZE (128) typedef union efx_wol_param_u { struct { uint8_t mac_addr[6]; } ewp_magic; struct { uint8_t mask[EFX_WOL_BITMAP_MASK_SIZE]; /* 1 bit per byte */ uint8_t value[EFX_WOL_BITMAP_VALUE_SIZE]; /* value to match */ uint8_t value_len; } ewp_bitmap; } efx_wol_param_t; typedef union efx_lightsout_offload_param_u { struct { uint8_t mac_addr[6]; uint32_t ip; } elop_arp; struct { uint8_t mac_addr[6]; uint32_t solicited_node[4]; uint32_t ip[4]; } elop_ns; } efx_lightsout_offload_param_t; extern __checkReturn efx_rc_t efx_wol_init( __in efx_nic_t *enp); extern __checkReturn efx_rc_t efx_wol_filter_clear( __in efx_nic_t *enp); extern __checkReturn efx_rc_t efx_wol_filter_add( __in efx_nic_t *enp, __in efx_wol_type_t type, __in efx_wol_param_t *paramp, __out uint32_t *filter_idp); extern __checkReturn efx_rc_t efx_wol_filter_remove( __in efx_nic_t *enp, __in uint32_t filter_id); extern __checkReturn efx_rc_t efx_lightsout_offload_add( __in efx_nic_t *enp, __in efx_lightsout_offload_type_t type, __in efx_lightsout_offload_param_t *paramp, __out uint32_t *filter_idp); extern __checkReturn efx_rc_t efx_lightsout_offload_remove( __in efx_nic_t *enp, __in efx_lightsout_offload_type_t type, __in uint32_t filter_id); extern void efx_wol_fini( __in efx_nic_t *enp); #endif /* EFSYS_OPT_WOL */ #if EFSYS_OPT_DIAG typedef enum efx_pattern_type_t { EFX_PATTERN_BYTE_INCREMENT = 0, EFX_PATTERN_ALL_THE_SAME, EFX_PATTERN_BIT_ALTERNATE, EFX_PATTERN_BYTE_ALTERNATE, EFX_PATTERN_BYTE_CHANGING, EFX_PATTERN_BIT_SWEEP, EFX_PATTERN_NTYPES } efx_pattern_type_t; typedef void (*efx_sram_pattern_fn_t)( __in size_t row, __in boolean_t negate, __out efx_qword_t *eqp); extern __checkReturn efx_rc_t efx_sram_test( __in efx_nic_t *enp, __in efx_pattern_type_t type); #endif /* EFSYS_OPT_DIAG */ extern __checkReturn efx_rc_t efx_sram_buf_tbl_set( __in efx_nic_t *enp, __in uint32_t id, __in efsys_mem_t *esmp, __in size_t n); extern void efx_sram_buf_tbl_clear( __in efx_nic_t *enp, __in uint32_t id, __in size_t n); #define EFX_BUF_TBL_SIZE 0x20000 #define EFX_BUF_SIZE 4096 /* EV */ typedef struct efx_evq_s efx_evq_t; #if EFSYS_OPT_QSTATS /* START MKCONFIG GENERATED EfxHeaderEventQueueBlock 6f3843f5fe7cc843 */ typedef enum efx_ev_qstat_e { EV_ALL, EV_RX, EV_RX_OK, EV_RX_FRM_TRUNC, EV_RX_TOBE_DISC, EV_RX_PAUSE_FRM_ERR, EV_RX_BUF_OWNER_ID_ERR, EV_RX_IPV4_HDR_CHKSUM_ERR, EV_RX_TCP_UDP_CHKSUM_ERR, EV_RX_ETH_CRC_ERR, EV_RX_IP_FRAG_ERR, EV_RX_MCAST_PKT, EV_RX_MCAST_HASH_MATCH, EV_RX_TCP_IPV4, EV_RX_TCP_IPV6, EV_RX_UDP_IPV4, EV_RX_UDP_IPV6, EV_RX_OTHER_IPV4, EV_RX_OTHER_IPV6, EV_RX_NON_IP, EV_RX_BATCH, EV_TX, EV_TX_WQ_FF_FULL, EV_TX_PKT_ERR, EV_TX_PKT_TOO_BIG, EV_TX_UNEXPECTED, EV_GLOBAL, EV_GLOBAL_MNT, EV_DRIVER, EV_DRIVER_SRM_UPD_DONE, EV_DRIVER_TX_DESCQ_FLS_DONE, EV_DRIVER_RX_DESCQ_FLS_DONE, EV_DRIVER_RX_DESCQ_FLS_FAILED, EV_DRIVER_RX_DSC_ERROR, EV_DRIVER_TX_DSC_ERROR, EV_DRV_GEN, EV_MCDI_RESPONSE, EV_NQSTATS } efx_ev_qstat_t; /* END MKCONFIG GENERATED EfxHeaderEventQueueBlock */ #endif /* EFSYS_OPT_QSTATS */ extern __checkReturn efx_rc_t efx_ev_init( __in efx_nic_t *enp); extern void efx_ev_fini( __in efx_nic_t *enp); #define EFX_EVQ_MAXNEVS 32768 #define EFX_EVQ_MINNEVS 512 #define EFX_EVQ_SIZE(_nevs) ((_nevs) * sizeof (efx_qword_t)) #define EFX_EVQ_NBUFS(_nevs) (EFX_EVQ_SIZE(_nevs) / EFX_BUF_SIZE) extern __checkReturn efx_rc_t efx_ev_qcreate( __in efx_nic_t *enp, __in unsigned int index, __in efsys_mem_t *esmp, __in size_t n, __in uint32_t id, __deref_out efx_evq_t **eepp); extern void efx_ev_qpost( __in efx_evq_t *eep, __in uint16_t data); typedef __checkReturn boolean_t (*efx_initialized_ev_t)( __in_opt void *arg); #define EFX_PKT_UNICAST 0x0004 #define EFX_PKT_START 0x0008 #define EFX_PKT_VLAN_TAGGED 0x0010 #define EFX_CKSUM_TCPUDP 0x0020 #define EFX_CKSUM_IPV4 0x0040 #define EFX_PKT_CONT 0x0080 #define EFX_CHECK_VLAN 0x0100 #define EFX_PKT_TCP 0x0200 #define EFX_PKT_UDP 0x0400 #define EFX_PKT_IPV4 0x0800 #define EFX_PKT_IPV6 0x1000 #define EFX_PKT_PREFIX_LEN 0x2000 #define EFX_ADDR_MISMATCH 0x4000 #define EFX_DISCARD 0x8000 #define EFX_EV_RX_NLABELS 32 #define EFX_EV_TX_NLABELS 32 typedef __checkReturn boolean_t (*efx_rx_ev_t)( __in_opt void *arg, __in uint32_t label, __in uint32_t id, __in uint32_t size, __in uint16_t flags); typedef __checkReturn boolean_t (*efx_tx_ev_t)( __in_opt void *arg, __in uint32_t label, __in uint32_t id); #define EFX_EXCEPTION_RX_RECOVERY 0x00000001 #define EFX_EXCEPTION_RX_DSC_ERROR 0x00000002 #define EFX_EXCEPTION_TX_DSC_ERROR 0x00000003 #define EFX_EXCEPTION_UNKNOWN_SENSOREVT 0x00000004 #define EFX_EXCEPTION_FWALERT_SRAM 0x00000005 #define EFX_EXCEPTION_UNKNOWN_FWALERT 0x00000006 #define EFX_EXCEPTION_RX_ERROR 0x00000007 #define EFX_EXCEPTION_TX_ERROR 0x00000008 #define EFX_EXCEPTION_EV_ERROR 0x00000009 typedef __checkReturn boolean_t (*efx_exception_ev_t)( __in_opt void *arg, __in uint32_t label, __in uint32_t data); typedef __checkReturn boolean_t (*efx_rxq_flush_done_ev_t)( __in_opt void *arg, __in uint32_t rxq_index); typedef __checkReturn boolean_t (*efx_rxq_flush_failed_ev_t)( __in_opt void *arg, __in uint32_t rxq_index); typedef __checkReturn boolean_t (*efx_txq_flush_done_ev_t)( __in_opt void *arg, __in uint32_t txq_index); typedef __checkReturn boolean_t (*efx_software_ev_t)( __in_opt void *arg, __in uint16_t magic); typedef __checkReturn boolean_t (*efx_sram_ev_t)( __in_opt void *arg, __in uint32_t code); #define EFX_SRAM_CLEAR 0 #define EFX_SRAM_UPDATE 1 #define EFX_SRAM_ILLEGAL_CLEAR 2 typedef __checkReturn boolean_t (*efx_wake_up_ev_t)( __in_opt void *arg, __in uint32_t label); typedef __checkReturn boolean_t (*efx_timer_ev_t)( __in_opt void *arg, __in uint32_t label); typedef __checkReturn boolean_t (*efx_link_change_ev_t)( __in_opt void *arg, __in efx_link_mode_t link_mode); #if EFSYS_OPT_MON_STATS typedef __checkReturn boolean_t (*efx_monitor_ev_t)( __in_opt void *arg, __in efx_mon_stat_t id, __in efx_mon_stat_value_t value); #endif /* EFSYS_OPT_MON_STATS */ #if EFSYS_OPT_MAC_STATS typedef __checkReturn boolean_t (*efx_mac_stats_ev_t)( __in_opt void *arg, __in uint32_t generation ); #endif /* EFSYS_OPT_MAC_STATS */ typedef struct efx_ev_callbacks_s { efx_initialized_ev_t eec_initialized; efx_rx_ev_t eec_rx; efx_tx_ev_t eec_tx; efx_exception_ev_t eec_exception; efx_rxq_flush_done_ev_t eec_rxq_flush_done; efx_rxq_flush_failed_ev_t eec_rxq_flush_failed; efx_txq_flush_done_ev_t eec_txq_flush_done; efx_software_ev_t eec_software; efx_sram_ev_t eec_sram; efx_wake_up_ev_t eec_wake_up; efx_timer_ev_t eec_timer; efx_link_change_ev_t eec_link_change; #if EFSYS_OPT_MON_STATS efx_monitor_ev_t eec_monitor; #endif /* EFSYS_OPT_MON_STATS */ #if EFSYS_OPT_MAC_STATS efx_mac_stats_ev_t eec_mac_stats; #endif /* EFSYS_OPT_MAC_STATS */ } efx_ev_callbacks_t; extern __checkReturn boolean_t efx_ev_qpending( __in efx_evq_t *eep, __in unsigned int count); #if EFSYS_OPT_EV_PREFETCH extern void efx_ev_qprefetch( __in efx_evq_t *eep, __in unsigned int count); #endif /* EFSYS_OPT_EV_PREFETCH */ extern void efx_ev_qpoll( __in efx_evq_t *eep, __inout unsigned int *countp, __in const efx_ev_callbacks_t *eecp, __in_opt void *arg); extern __checkReturn efx_rc_t efx_ev_qmoderate( __in efx_evq_t *eep, __in unsigned int us); extern __checkReturn efx_rc_t efx_ev_qprime( __in efx_evq_t *eep, __in unsigned int count); #if EFSYS_OPT_QSTATS #if EFSYS_OPT_NAMES extern const char * efx_ev_qstat_name( __in efx_nic_t *enp, __in unsigned int id); #endif /* EFSYS_OPT_NAMES */ extern void efx_ev_qstats_update( __in efx_evq_t *eep, __inout_ecount(EV_NQSTATS) efsys_stat_t *stat); #endif /* EFSYS_OPT_QSTATS */ extern void efx_ev_qdestroy( __in efx_evq_t *eep); /* RX */ extern __checkReturn efx_rc_t efx_rx_init( __inout efx_nic_t *enp); extern void efx_rx_fini( __in efx_nic_t *enp); #if EFSYS_OPT_RX_HDR_SPLIT __checkReturn efx_rc_t efx_rx_hdr_split_enable( __in efx_nic_t *enp, __in unsigned int hdr_buf_size, __in unsigned int pld_buf_size); #endif /* EFSYS_OPT_RX_HDR_SPLIT */ #if EFSYS_OPT_RX_SCATTER __checkReturn efx_rc_t efx_rx_scatter_enable( __in efx_nic_t *enp, __in unsigned int buf_size); #endif /* EFSYS_OPT_RX_SCATTER */ #if EFSYS_OPT_RX_SCALE typedef enum efx_rx_hash_alg_e { EFX_RX_HASHALG_LFSR = 0, EFX_RX_HASHALG_TOEPLITZ } efx_rx_hash_alg_t; typedef enum efx_rx_hash_type_e { EFX_RX_HASH_IPV4 = 0, EFX_RX_HASH_TCPIPV4, EFX_RX_HASH_IPV6, EFX_RX_HASH_TCPIPV6, } efx_rx_hash_type_t; typedef enum efx_rx_hash_support_e { EFX_RX_HASH_UNAVAILABLE = 0, /* Hardware hash not inserted */ EFX_RX_HASH_AVAILABLE /* Insert hash with/without RSS */ } efx_rx_hash_support_t; #define EFX_RSS_TBL_SIZE 128 /* Rows in RX indirection table */ #define EFX_MAXRSS 64 /* RX indirection entry range */ #define EFX_MAXRSS_LEGACY 16 /* See bug16611 and bug17213 */ typedef enum efx_rx_scale_support_e { EFX_RX_SCALE_UNAVAILABLE = 0, /* Not supported */ EFX_RX_SCALE_EXCLUSIVE, /* Writable key/indirection table */ EFX_RX_SCALE_SHARED /* Read-only key/indirection table */ } efx_rx_scale_support_t; extern __checkReturn efx_rc_t efx_rx_hash_support_get( __in efx_nic_t *enp, __out efx_rx_hash_support_t *supportp); extern __checkReturn efx_rc_t efx_rx_scale_support_get( __in efx_nic_t *enp, __out efx_rx_scale_support_t *supportp); extern __checkReturn efx_rc_t efx_rx_scale_mode_set( __in efx_nic_t *enp, __in efx_rx_hash_alg_t alg, __in efx_rx_hash_type_t type, __in boolean_t insert); extern __checkReturn efx_rc_t efx_rx_scale_tbl_set( __in efx_nic_t *enp, __in_ecount(n) unsigned int *table, __in size_t n); extern __checkReturn efx_rc_t efx_rx_scale_key_set( __in efx_nic_t *enp, __in_ecount(n) uint8_t *key, __in size_t n); extern uint32_t efx_psuedo_hdr_hash_get( __in efx_nic_t *enp, __in efx_rx_hash_alg_t func, __in uint8_t *buffer); #endif /* EFSYS_OPT_RX_SCALE */ extern __checkReturn efx_rc_t efx_psuedo_hdr_pkt_length_get( __in efx_nic_t *enp, __in uint8_t *buffer, __out uint16_t *pkt_lengthp); #define EFX_RXQ_MAXNDESCS 4096 #define EFX_RXQ_MINNDESCS 512 #define EFX_RXQ_SIZE(_ndescs) ((_ndescs) * sizeof (efx_qword_t)) #define EFX_RXQ_NBUFS(_ndescs) (EFX_RXQ_SIZE(_ndescs) / EFX_BUF_SIZE) #define EFX_RXQ_LIMIT(_ndescs) ((_ndescs) - 16) #define EFX_RXQ_DC_NDESCS(_dcsize) (8 << _dcsize) typedef enum efx_rxq_type_e { EFX_RXQ_TYPE_DEFAULT, EFX_RXQ_TYPE_SPLIT_HEADER, EFX_RXQ_TYPE_SPLIT_PAYLOAD, EFX_RXQ_TYPE_SCATTER, EFX_RXQ_NTYPES } efx_rxq_type_t; extern __checkReturn efx_rc_t efx_rx_qcreate( __in efx_nic_t *enp, __in unsigned int index, __in unsigned int label, __in efx_rxq_type_t type, __in efsys_mem_t *esmp, __in size_t n, __in uint32_t id, __in efx_evq_t *eep, __deref_out efx_rxq_t **erpp); typedef struct efx_buffer_s { efsys_dma_addr_t eb_addr; size_t eb_size; boolean_t eb_eop; } efx_buffer_t; typedef struct efx_desc_s { efx_qword_t ed_eq; } efx_desc_t; extern void efx_rx_qpost( __in efx_rxq_t *erp, __in_ecount(n) efsys_dma_addr_t *addrp, __in size_t size, __in unsigned int n, __in unsigned int completed, __in unsigned int added); extern void efx_rx_qpush( __in efx_rxq_t *erp, __in unsigned int added, __inout unsigned int *pushedp); extern __checkReturn efx_rc_t efx_rx_qflush( __in efx_rxq_t *erp); extern void efx_rx_qenable( __in efx_rxq_t *erp); extern void efx_rx_qdestroy( __in efx_rxq_t *erp); /* TX */ typedef struct efx_txq_s efx_txq_t; #if EFSYS_OPT_QSTATS /* START MKCONFIG GENERATED EfxHeaderTransmitQueueBlock 12dff8778598b2db */ typedef enum efx_tx_qstat_e { TX_POST, TX_POST_PIO, TX_NQSTATS } efx_tx_qstat_t; /* END MKCONFIG GENERATED EfxHeaderTransmitQueueBlock */ #endif /* EFSYS_OPT_QSTATS */ extern __checkReturn efx_rc_t efx_tx_init( __in efx_nic_t *enp); extern void efx_tx_fini( __in efx_nic_t *enp); #define EFX_BUG35388_WORKAROUND(_encp) \ (((_encp) == NULL) ? 1 : ((_encp)->enc_bug35388_workaround != 0)) #define EFX_TXQ_MAXNDESCS(_encp) \ ((EFX_BUG35388_WORKAROUND(_encp)) ? 2048 : 4096) #define EFX_TXQ_MINNDESCS 512 #define EFX_TXQ_SIZE(_ndescs) ((_ndescs) * sizeof (efx_qword_t)) #define EFX_TXQ_NBUFS(_ndescs) (EFX_TXQ_SIZE(_ndescs) / EFX_BUF_SIZE) #define EFX_TXQ_LIMIT(_ndescs) ((_ndescs) - 16) #define EFX_TXQ_DC_NDESCS(_dcsize) (8 << _dcsize) #define EFX_TXQ_MAX_BUFS 8 /* Maximum independent of EFX_BUG35388_WORKAROUND. */ #define EFX_TXQ_CKSUM_IPV4 0x0001 #define EFX_TXQ_CKSUM_TCPUDP 0x0002 extern __checkReturn efx_rc_t efx_tx_qcreate( __in efx_nic_t *enp, __in unsigned int index, __in unsigned int label, __in efsys_mem_t *esmp, __in size_t n, __in uint32_t id, __in uint16_t flags, __in efx_evq_t *eep, __deref_out efx_txq_t **etpp, __out unsigned int *addedp); extern __checkReturn efx_rc_t efx_tx_qpost( __in efx_txq_t *etp, __in_ecount(n) efx_buffer_t *eb, __in unsigned int n, __in unsigned int completed, __inout unsigned int *addedp); extern __checkReturn efx_rc_t efx_tx_qpace( __in efx_txq_t *etp, __in unsigned int ns); extern void efx_tx_qpush( __in efx_txq_t *etp, __in unsigned int added, __in unsigned int pushed); extern __checkReturn efx_rc_t efx_tx_qflush( __in efx_txq_t *etp); extern void efx_tx_qenable( __in efx_txq_t *etp); extern __checkReturn efx_rc_t efx_tx_qpio_enable( __in efx_txq_t *etp); extern void efx_tx_qpio_disable( __in efx_txq_t *etp); extern __checkReturn efx_rc_t efx_tx_qpio_write( __in efx_txq_t *etp, __in_ecount(buf_length) uint8_t *buffer, __in size_t buf_length, __in size_t pio_buf_offset); extern __checkReturn efx_rc_t efx_tx_qpio_post( __in efx_txq_t *etp, __in size_t pkt_length, __in unsigned int completed, __inout unsigned int *addedp); extern __checkReturn efx_rc_t efx_tx_qdesc_post( __in efx_txq_t *etp, __in_ecount(n) efx_desc_t *ed, __in unsigned int n, __in unsigned int completed, __inout unsigned int *addedp); extern void efx_tx_qdesc_dma_create( __in efx_txq_t *etp, __in efsys_dma_addr_t addr, __in size_t size, __in boolean_t eop, __out efx_desc_t *edp); extern void efx_tx_qdesc_tso_create( __in efx_txq_t *etp, __in uint16_t ipv4_id, __in uint32_t tcp_seq, __in uint8_t tcp_flags, __out efx_desc_t *edp); extern void efx_tx_qdesc_vlantci_create( __in efx_txq_t *etp, __in uint16_t tci, __out efx_desc_t *edp); #if EFSYS_OPT_QSTATS #if EFSYS_OPT_NAMES extern const char * efx_tx_qstat_name( __in efx_nic_t *etp, __in unsigned int id); #endif /* EFSYS_OPT_NAMES */ extern void efx_tx_qstats_update( __in efx_txq_t *etp, __inout_ecount(TX_NQSTATS) efsys_stat_t *stat); #endif /* EFSYS_OPT_QSTATS */ extern void efx_tx_qdestroy( __in efx_txq_t *etp); /* FILTER */ #if EFSYS_OPT_FILTER #define EFX_ETHER_TYPE_IPV4 0x0800 #define EFX_ETHER_TYPE_IPV6 0x86DD #define EFX_IPPROTO_TCP 6 #define EFX_IPPROTO_UDP 17 typedef enum efx_filter_flag_e { EFX_FILTER_FLAG_RX_RSS = 0x01, /* use RSS to spread across * multiple queues */ EFX_FILTER_FLAG_RX_SCATTER = 0x02, /* enable RX scatter */ EFX_FILTER_FLAG_RX_OVER_AUTO = 0x04, /* Override an automatic filter * (priority EFX_FILTER_PRI_AUTO). * May only be set by the filter * implementation for each type. * A removal request will * restore the automatic filter * in its place. */ EFX_FILTER_FLAG_RX = 0x08, /* Filter is for RX */ EFX_FILTER_FLAG_TX = 0x10, /* Filter is for TX */ } efx_filter_flag_t; typedef enum efx_filter_match_flags_e { EFX_FILTER_MATCH_REM_HOST = 0x0001, /* Match by remote IP host * address */ EFX_FILTER_MATCH_LOC_HOST = 0x0002, /* Match by local IP host * address */ EFX_FILTER_MATCH_REM_MAC = 0x0004, /* Match by remote MAC address */ EFX_FILTER_MATCH_REM_PORT = 0x0008, /* Match by remote TCP/UDP port */ EFX_FILTER_MATCH_LOC_MAC = 0x0010, /* Match by remote TCP/UDP port */ EFX_FILTER_MATCH_LOC_PORT = 0x0020, /* Match by local TCP/UDP port */ EFX_FILTER_MATCH_ETHER_TYPE = 0x0040, /* Match by Ether-type */ EFX_FILTER_MATCH_INNER_VID = 0x0080, /* Match by inner VLAN ID */ EFX_FILTER_MATCH_OUTER_VID = 0x0100, /* Match by outer VLAN ID */ EFX_FILTER_MATCH_IP_PROTO = 0x0200, /* Match by IP transport * protocol */ EFX_FILTER_MATCH_LOC_MAC_IG = 0x0400, /* Match by local MAC address * I/G bit. Used for RX default * unicast and multicast/ * broadcast filters. */ } efx_filter_match_flags_t; typedef enum efx_filter_priority_s { EFX_FILTER_PRI_HINT = 0, /* Performance hint */ EFX_FILTER_PRI_AUTO, /* Automatic filter based on device * address list or hardware * requirements. This may only be used * by the filter implementation for * each NIC type. */ EFX_FILTER_PRI_MANUAL, /* Manually configured filter */ EFX_FILTER_PRI_REQUIRED, /* Required for correct behaviour of the * client (e.g. SR-IOV, HyperV VMQ etc.) */ } efx_filter_priority_t; /* * FIXME: All these fields are assumed to be in little-endian byte order. * It may be better for some to be big-endian. See bug42804. */ typedef struct efx_filter_spec_s { uint32_t efs_match_flags:12; uint32_t efs_priority:2; uint32_t efs_flags:6; uint32_t efs_dmaq_id:12; uint32_t efs_rss_context; uint16_t efs_outer_vid; uint16_t efs_inner_vid; uint8_t efs_loc_mac[EFX_MAC_ADDR_LEN]; uint8_t efs_rem_mac[EFX_MAC_ADDR_LEN]; uint16_t efs_ether_type; uint8_t efs_ip_proto; uint16_t efs_loc_port; uint16_t efs_rem_port; efx_oword_t efs_rem_host; efx_oword_t efs_loc_host; } efx_filter_spec_t; /* Default values for use in filter specifications */ #define EFX_FILTER_SPEC_RSS_CONTEXT_DEFAULT 0xffffffff #define EFX_FILTER_SPEC_RX_DMAQ_ID_DROP 0xfff #define EFX_FILTER_SPEC_VID_UNSPEC 0xffff extern __checkReturn efx_rc_t efx_filter_init( __in efx_nic_t *enp); extern void efx_filter_fini( __in efx_nic_t *enp); extern __checkReturn efx_rc_t efx_filter_insert( __in efx_nic_t *enp, __inout efx_filter_spec_t *spec); extern __checkReturn efx_rc_t efx_filter_remove( __in efx_nic_t *enp, __inout efx_filter_spec_t *spec); extern __checkReturn efx_rc_t efx_filter_restore( __in efx_nic_t *enp); extern __checkReturn efx_rc_t efx_filter_supported_filters( __in efx_nic_t *enp, __out uint32_t *list, __out size_t *length); extern void efx_filter_spec_init_rx( __inout efx_filter_spec_t *spec, __in efx_filter_priority_t priority, __in efx_filter_flag_t flags, __in efx_rxq_t *erp); extern void efx_filter_spec_init_tx( __inout efx_filter_spec_t *spec, __in efx_txq_t *etp); extern __checkReturn efx_rc_t efx_filter_spec_set_ipv4_local( __inout efx_filter_spec_t *spec, __in uint8_t proto, __in uint32_t host, __in uint16_t port); extern __checkReturn efx_rc_t efx_filter_spec_set_ipv4_full( __inout efx_filter_spec_t *spec, __in uint8_t proto, __in uint32_t lhost, __in uint16_t lport, __in uint32_t rhost, __in uint16_t rport); extern __checkReturn efx_rc_t efx_filter_spec_set_eth_local( __inout efx_filter_spec_t *spec, __in uint16_t vid, __in const uint8_t *addr); extern __checkReturn efx_rc_t efx_filter_spec_set_uc_def( __inout efx_filter_spec_t *spec); extern __checkReturn efx_rc_t efx_filter_spec_set_mc_def( __inout efx_filter_spec_t *spec); #endif /* EFSYS_OPT_FILTER */ /* HASH */ extern __checkReturn uint32_t efx_hash_dwords( __in_ecount(count) uint32_t const *input, __in size_t count, __in uint32_t init); extern __checkReturn uint32_t efx_hash_bytes( __in_ecount(length) uint8_t const *input, __in size_t length, __in uint32_t init); #ifdef __cplusplus } #endif #endif /* _SYS_EFX_H */ Index: user/ngie/more-tests2/sys/dev/sfxge/common/efx_impl.h =================================================================== --- user/ngie/more-tests2/sys/dev/sfxge/common/efx_impl.h (revision 292053) +++ user/ngie/more-tests2/sys/dev/sfxge/common/efx_impl.h (revision 292054) @@ -1,1173 +1,1174 @@ /*- * Copyright (c) 2007-2015 Solarflare Communications Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are met: * * 1. Redistributions of source code must retain the above copyright notice, * this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright notice, * this list of conditions and the following disclaimer in the documentation * and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR * OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, * EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * The views and conclusions contained in the software and documentation are * those of the authors and should not be interpreted as representing official * policies, either expressed or implied, of the FreeBSD Project. * * $FreeBSD$ */ #ifndef _SYS_EFX_IMPL_H #define _SYS_EFX_IMPL_H #include "efsys.h" #include "efx.h" #include "efx_regs.h" #include "efx_regs_ef10.h" /* FIXME: Add definition for driver generated software events */ #ifndef ESE_DZ_EV_CODE_DRV_GEN_EV #define ESE_DZ_EV_CODE_DRV_GEN_EV FSE_AZ_EV_CODE_DRV_GEN_EV #endif #include "efx_check.h" #if EFSYS_OPT_FALCON #include "falcon_impl.h" #endif /* EFSYS_OPT_FALCON */ #if EFSYS_OPT_SIENA #include "siena_impl.h" #endif /* EFSYS_OPT_SIENA */ #if EFSYS_OPT_HUNTINGTON #include "hunt_impl.h" #endif /* EFSYS_OPT_HUNTINGTON */ #ifdef __cplusplus extern "C" { #endif #define EFX_MOD_MCDI 0x00000001 #define EFX_MOD_PROBE 0x00000002 #define EFX_MOD_NVRAM 0x00000004 #define EFX_MOD_VPD 0x00000008 #define EFX_MOD_NIC 0x00000010 #define EFX_MOD_INTR 0x00000020 #define EFX_MOD_EV 0x00000040 #define EFX_MOD_RX 0x00000080 #define EFX_MOD_TX 0x00000100 #define EFX_MOD_PORT 0x00000200 #define EFX_MOD_MON 0x00000400 #define EFX_MOD_WOL 0x00000800 #define EFX_MOD_FILTER 0x00001000 #define EFX_MOD_PKTFILTER 0x00002000 #define EFX_RESET_MAC 0x00000001 #define EFX_RESET_PHY 0x00000002 #define EFX_RESET_RXQ_ERR 0x00000004 #define EFX_RESET_TXQ_ERR 0x00000008 typedef enum efx_mac_type_e { EFX_MAC_INVALID = 0, EFX_MAC_FALCON_GMAC, EFX_MAC_FALCON_XMAC, EFX_MAC_SIENA, EFX_MAC_HUNTINGTON, EFX_MAC_NTYPES } efx_mac_type_t; typedef struct efx_ev_ops_s { efx_rc_t (*eevo_init)(efx_nic_t *); void (*eevo_fini)(efx_nic_t *); efx_rc_t (*eevo_qcreate)(efx_nic_t *, unsigned int, efsys_mem_t *, size_t, uint32_t, efx_evq_t *); void (*eevo_qdestroy)(efx_evq_t *); efx_rc_t (*eevo_qprime)(efx_evq_t *, unsigned int); void (*eevo_qpost)(efx_evq_t *, uint16_t); efx_rc_t (*eevo_qmoderate)(efx_evq_t *, unsigned int); #if EFSYS_OPT_QSTATS void (*eevo_qstats_update)(efx_evq_t *, efsys_stat_t *); #endif } efx_ev_ops_t; typedef struct efx_tx_ops_s { efx_rc_t (*etxo_init)(efx_nic_t *); void (*etxo_fini)(efx_nic_t *); efx_rc_t (*etxo_qcreate)(efx_nic_t *, unsigned int, unsigned int, efsys_mem_t *, size_t, uint32_t, uint16_t, efx_evq_t *, efx_txq_t *, unsigned int *); void (*etxo_qdestroy)(efx_txq_t *); efx_rc_t (*etxo_qpost)(efx_txq_t *, efx_buffer_t *, unsigned int, unsigned int, unsigned int *); void (*etxo_qpush)(efx_txq_t *, unsigned int, unsigned int); efx_rc_t (*etxo_qpace)(efx_txq_t *, unsigned int); efx_rc_t (*etxo_qflush)(efx_txq_t *); void (*etxo_qenable)(efx_txq_t *); efx_rc_t (*etxo_qpio_enable)(efx_txq_t *); void (*etxo_qpio_disable)(efx_txq_t *); efx_rc_t (*etxo_qpio_write)(efx_txq_t *,uint8_t *, size_t, size_t); efx_rc_t (*etxo_qpio_post)(efx_txq_t *, size_t, unsigned int, unsigned int *); efx_rc_t (*etxo_qdesc_post)(efx_txq_t *, efx_desc_t *, unsigned int, unsigned int, unsigned int *); void (*etxo_qdesc_dma_create)(efx_txq_t *, efsys_dma_addr_t, size_t, boolean_t, efx_desc_t *); void (*etxo_qdesc_tso_create)(efx_txq_t *, uint16_t, uint32_t, uint8_t, efx_desc_t *); void (*etxo_qdesc_vlantci_create)(efx_txq_t *, uint16_t, efx_desc_t *); #if EFSYS_OPT_QSTATS void (*etxo_qstats_update)(efx_txq_t *, efsys_stat_t *); #endif } efx_tx_ops_t; typedef struct efx_rx_ops_s { efx_rc_t (*erxo_init)(efx_nic_t *); void (*erxo_fini)(efx_nic_t *); #if EFSYS_OPT_RX_HDR_SPLIT efx_rc_t (*erxo_hdr_split_enable)(efx_nic_t *, unsigned int, unsigned int); #endif #if EFSYS_OPT_RX_SCATTER efx_rc_t (*erxo_scatter_enable)(efx_nic_t *, unsigned int); #endif #if EFSYS_OPT_RX_SCALE efx_rc_t (*erxo_scale_mode_set)(efx_nic_t *, efx_rx_hash_alg_t, efx_rx_hash_type_t, boolean_t); efx_rc_t (*erxo_scale_key_set)(efx_nic_t *, uint8_t *, size_t); efx_rc_t (*erxo_scale_tbl_set)(efx_nic_t *, unsigned int *, size_t); #endif void (*erxo_qpost)(efx_rxq_t *, efsys_dma_addr_t *, size_t, unsigned int, unsigned int, unsigned int); void (*erxo_qpush)(efx_rxq_t *, unsigned int, unsigned int *); efx_rc_t (*erxo_qflush)(efx_rxq_t *); void (*erxo_qenable)(efx_rxq_t *); efx_rc_t (*erxo_qcreate)(efx_nic_t *enp, unsigned int, unsigned int, efx_rxq_type_t, efsys_mem_t *, size_t, uint32_t, efx_evq_t *, efx_rxq_t *); void (*erxo_qdestroy)(efx_rxq_t *); } efx_rx_ops_t; typedef struct efx_mac_ops_s { efx_rc_t (*emo_reset)(efx_nic_t *); /* optional */ efx_rc_t (*emo_poll)(efx_nic_t *, efx_link_mode_t *); efx_rc_t (*emo_up)(efx_nic_t *, boolean_t *); efx_rc_t (*emo_addr_set)(efx_nic_t *); efx_rc_t (*emo_reconfigure)(efx_nic_t *); efx_rc_t (*emo_multicast_list_set)(efx_nic_t *); efx_rc_t (*emo_filter_default_rxq_set)(efx_nic_t *, efx_rxq_t *, boolean_t); void (*emo_filter_default_rxq_clear)(efx_nic_t *); #if EFSYS_OPT_LOOPBACK efx_rc_t (*emo_loopback_set)(efx_nic_t *, efx_link_mode_t, efx_loopback_type_t); #endif /* EFSYS_OPT_LOOPBACK */ #if EFSYS_OPT_MAC_STATS efx_rc_t (*emo_stats_upload)(efx_nic_t *, efsys_mem_t *); efx_rc_t (*emo_stats_periodic)(efx_nic_t *, efsys_mem_t *, uint16_t, boolean_t); efx_rc_t (*emo_stats_update)(efx_nic_t *, efsys_mem_t *, efsys_stat_t *, uint32_t *); #endif /* EFSYS_OPT_MAC_STATS */ } efx_mac_ops_t; typedef struct efx_phy_ops_s { efx_rc_t (*epo_power)(efx_nic_t *, boolean_t); /* optional */ efx_rc_t (*epo_reset)(efx_nic_t *); efx_rc_t (*epo_reconfigure)(efx_nic_t *); efx_rc_t (*epo_verify)(efx_nic_t *); efx_rc_t (*epo_uplink_check)(efx_nic_t *, boolean_t *); /* optional */ efx_rc_t (*epo_downlink_check)(efx_nic_t *, efx_link_mode_t *, unsigned int *, uint32_t *); efx_rc_t (*epo_oui_get)(efx_nic_t *, uint32_t *); #if EFSYS_OPT_PHY_STATS efx_rc_t (*epo_stats_update)(efx_nic_t *, efsys_mem_t *, uint32_t *); #endif /* EFSYS_OPT_PHY_STATS */ #if EFSYS_OPT_PHY_PROPS #if EFSYS_OPT_NAMES const char *(*epo_prop_name)(efx_nic_t *, unsigned int); #endif /* EFSYS_OPT_PHY_PROPS */ efx_rc_t (*epo_prop_get)(efx_nic_t *, unsigned int, uint32_t, uint32_t *); efx_rc_t (*epo_prop_set)(efx_nic_t *, unsigned int, uint32_t); #endif /* EFSYS_OPT_PHY_PROPS */ #if EFSYS_OPT_BIST efx_rc_t (*epo_bist_enable_offline)(efx_nic_t *); efx_rc_t (*epo_bist_start)(efx_nic_t *, efx_bist_type_t); efx_rc_t (*epo_bist_poll)(efx_nic_t *, efx_bist_type_t, efx_bist_result_t *, uint32_t *, unsigned long *, size_t); void (*epo_bist_stop)(efx_nic_t *, efx_bist_type_t); #endif /* EFSYS_OPT_BIST */ } efx_phy_ops_t; #if EFSYS_OPT_FILTER typedef struct efx_filter_ops_s { efx_rc_t (*efo_init)(efx_nic_t *); void (*efo_fini)(efx_nic_t *); efx_rc_t (*efo_restore)(efx_nic_t *); efx_rc_t (*efo_add)(efx_nic_t *, efx_filter_spec_t *, boolean_t may_replace); efx_rc_t (*efo_delete)(efx_nic_t *, efx_filter_spec_t *); efx_rc_t (*efo_supported_filters)(efx_nic_t *, uint32_t *, size_t *); efx_rc_t (*efo_reconfigure)(efx_nic_t *, uint8_t const *, boolean_t, boolean_t, boolean_t, boolean_t, uint8_t const *, int); } efx_filter_ops_t; extern __checkReturn efx_rc_t efx_filter_reconfigure( __in efx_nic_t *enp, __in_ecount(6) uint8_t const *mac_addr, __in boolean_t all_unicst, __in boolean_t mulcst, __in boolean_t all_mulcst, __in boolean_t brdcst, __in_ecount(6*count) uint8_t const *addrs, __in int count); #endif /* EFSYS_OPT_FILTER */ typedef struct efx_pktfilter_ops_s { efx_rc_t (*epfo_set)(efx_nic_t *, boolean_t unicst, boolean_t brdcast); #if EFSYS_OPT_MCAST_FILTER_LIST efx_rc_t (*epfo_mcast_list_set)(efx_nic_t *, uint8_t const *addrs, int count); #endif /* EFSYS_OPT_MCAST_FILTER_LIST */ efx_rc_t (*epfo_mcast_all)(efx_nic_t *); } efx_pktfilter_ops_t; typedef struct efx_port_s { efx_mac_type_t ep_mac_type; uint32_t ep_phy_type; uint8_t ep_port; uint32_t ep_mac_pdu; uint8_t ep_mac_addr[6]; efx_link_mode_t ep_link_mode; boolean_t ep_all_unicst; boolean_t ep_mulcst; boolean_t ep_all_mulcst; boolean_t ep_brdcst; unsigned int ep_fcntl; boolean_t ep_fcntl_autoneg; efx_oword_t ep_multicst_hash[2]; uint8_t ep_mulcst_addr_list[EFX_MAC_ADDR_LEN * EFX_MAC_MULTICAST_LIST_MAX]; uint32_t ep_mulcst_addr_count; #if EFSYS_OPT_LOOPBACK efx_loopback_type_t ep_loopback_type; efx_link_mode_t ep_loopback_link_mode; #endif /* EFSYS_OPT_LOOPBACK */ #if EFSYS_OPT_PHY_FLAGS uint32_t ep_phy_flags; #endif /* EFSYS_OPT_PHY_FLAGS */ #if EFSYS_OPT_PHY_LED_CONTROL efx_phy_led_mode_t ep_phy_led_mode; #endif /* EFSYS_OPT_PHY_LED_CONTROL */ efx_phy_media_type_t ep_fixed_port_type; efx_phy_media_type_t ep_module_type; uint32_t ep_adv_cap_mask; uint32_t ep_lp_cap_mask; uint32_t ep_default_adv_cap_mask; uint32_t ep_phy_cap_mask; #if EFSYS_OPT_PHY_TXC43128 || EFSYS_OPT_PHY_QT2025C union { struct { unsigned int bug10934_count; } ep_txc43128; struct { unsigned int bug17190_count; } ep_qt2025c; }; #endif boolean_t ep_mac_poll_needed; /* falcon only */ boolean_t ep_mac_up; /* falcon only */ uint32_t ep_fwver; /* falcon only */ boolean_t ep_mac_drain; boolean_t ep_mac_stats_pending; #if EFSYS_OPT_BIST efx_bist_type_t ep_current_bist; #endif efx_mac_ops_t *ep_emop; efx_phy_ops_t *ep_epop; } efx_port_t; typedef struct efx_mon_ops_s { efx_rc_t (*emo_reset)(efx_nic_t *); efx_rc_t (*emo_reconfigure)(efx_nic_t *); #if EFSYS_OPT_MON_STATS efx_rc_t (*emo_stats_update)(efx_nic_t *, efsys_mem_t *, efx_mon_stat_value_t *); #endif /* EFSYS_OPT_MON_STATS */ } efx_mon_ops_t; typedef struct efx_mon_s { efx_mon_type_t em_type; efx_mon_ops_t *em_emop; } efx_mon_t; typedef struct efx_intr_ops_s { efx_rc_t (*eio_init)(efx_nic_t *, efx_intr_type_t, efsys_mem_t *); void (*eio_enable)(efx_nic_t *); void (*eio_disable)(efx_nic_t *); void (*eio_disable_unlocked)(efx_nic_t *); efx_rc_t (*eio_trigger)(efx_nic_t *, unsigned int); void (*eio_fini)(efx_nic_t *); } efx_intr_ops_t; typedef struct efx_intr_s { efx_intr_ops_t *ei_eiop; efsys_mem_t *ei_esmp; efx_intr_type_t ei_type; unsigned int ei_level; } efx_intr_t; typedef struct efx_nic_ops_s { efx_rc_t (*eno_probe)(efx_nic_t *); efx_rc_t (*eno_set_drv_limits)(efx_nic_t *, efx_drv_limits_t*); efx_rc_t (*eno_reset)(efx_nic_t *); efx_rc_t (*eno_init)(efx_nic_t *); efx_rc_t (*eno_get_vi_pool)(efx_nic_t *, uint32_t *); efx_rc_t (*eno_get_bar_region)(efx_nic_t *, efx_nic_region_t, uint32_t *, size_t *); #if EFSYS_OPT_DIAG efx_rc_t (*eno_sram_test)(efx_nic_t *, efx_sram_pattern_fn_t); efx_rc_t (*eno_register_test)(efx_nic_t *); #endif /* EFSYS_OPT_DIAG */ void (*eno_fini)(efx_nic_t *); void (*eno_unprobe)(efx_nic_t *); } efx_nic_ops_t; #ifndef EFX_TXQ_LIMIT_TARGET #define EFX_TXQ_LIMIT_TARGET 259 #endif #ifndef EFX_RXQ_LIMIT_TARGET #define EFX_RXQ_LIMIT_TARGET 512 #endif #ifndef EFX_TXQ_DC_SIZE #define EFX_TXQ_DC_SIZE 1 /* 16 descriptors */ #endif #ifndef EFX_RXQ_DC_SIZE #define EFX_RXQ_DC_SIZE 3 /* 64 descriptors */ #endif #if EFSYS_OPT_FILTER typedef struct falconsiena_filter_spec_s { uint8_t fsfs_type; uint32_t fsfs_flags; uint32_t fsfs_dmaq_id; uint32_t fsfs_dword[3]; } falconsiena_filter_spec_t; typedef enum falconsiena_filter_type_e { EFX_FS_FILTER_RX_TCP_FULL, /* TCP/IPv4 4-tuple {dIP,dTCP,sIP,sTCP} */ EFX_FS_FILTER_RX_TCP_WILD, /* TCP/IPv4 dest {dIP,dTCP, -, -} */ EFX_FS_FILTER_RX_UDP_FULL, /* UDP/IPv4 4-tuple {dIP,dUDP,sIP,sUDP} */ EFX_FS_FILTER_RX_UDP_WILD, /* UDP/IPv4 dest {dIP,dUDP, -, -} */ #if EFSYS_OPT_SIENA EFX_FS_FILTER_RX_MAC_FULL, /* Ethernet {dMAC,VLAN} */ EFX_FS_FILTER_RX_MAC_WILD, /* Ethernet {dMAC, -} */ EFX_FS_FILTER_TX_TCP_FULL, /* TCP/IPv4 {dIP,dTCP,sIP,sTCP} */ EFX_FS_FILTER_TX_TCP_WILD, /* TCP/IPv4 { -, -,sIP,sTCP} */ EFX_FS_FILTER_TX_UDP_FULL, /* UDP/IPv4 {dIP,dTCP,sIP,sTCP} */ EFX_FS_FILTER_TX_UDP_WILD, /* UDP/IPv4 source (host, port) */ EFX_FS_FILTER_TX_MAC_FULL, /* Ethernet source (MAC address, VLAN ID) */ EFX_FS_FILTER_TX_MAC_WILD, /* Ethernet source (MAC address) */ #endif /* EFSYS_OPT_SIENA */ EFX_FS_FILTER_NTYPES } falconsiena_filter_type_t; typedef enum falconsiena_filter_tbl_id_e { EFX_FS_FILTER_TBL_RX_IP = 0, EFX_FS_FILTER_TBL_RX_MAC, EFX_FS_FILTER_TBL_TX_IP, EFX_FS_FILTER_TBL_TX_MAC, EFX_FS_FILTER_NTBLS } falconsiena_filter_tbl_id_t; typedef struct falconsiena_filter_tbl_s { int fsft_size; /* number of entries */ int fsft_used; /* active count */ uint32_t *fsft_bitmap; /* active bitmap */ falconsiena_filter_spec_t *fsft_spec; /* array of saved specs */ } falconsiena_filter_tbl_t; typedef struct falconsiena_filter_s { falconsiena_filter_tbl_t fsf_tbl[EFX_FS_FILTER_NTBLS]; unsigned int fsf_depth[EFX_FS_FILTER_NTYPES]; } falconsiena_filter_t; typedef struct efx_filter_s { #if EFSYS_OPT_FALCON || EFSYS_OPT_SIENA falconsiena_filter_t *ef_falconsiena_filter; #endif /* EFSYS_OPT_FALCON || EFSYS_OPT_SIENA */ #if EFSYS_OPT_HUNTINGTON hunt_filter_table_t *ef_hunt_filter_table; #endif /* EFSYS_OPT_HUNTINGTON */ } efx_filter_t; extern void falconsiena_filter_tbl_clear( __in efx_nic_t *enp, __in falconsiena_filter_tbl_id_t tbl); #endif /* EFSYS_OPT_FILTER */ #if EFSYS_OPT_MCDI typedef struct efx_mcdi_ops_s { efx_rc_t (*emco_init)(efx_nic_t *, const efx_mcdi_transport_t *); void (*emco_request_copyin)(efx_nic_t *, efx_mcdi_req_t *, unsigned int, boolean_t, boolean_t); boolean_t (*emco_request_poll)(efx_nic_t *); void (*emco_request_copyout)(efx_nic_t *, efx_mcdi_req_t *); efx_rc_t (*emco_poll_reboot)(efx_nic_t *); void (*emco_fini)(efx_nic_t *); efx_rc_t (*emco_fw_update_supported)(efx_nic_t *, boolean_t *); efx_rc_t (*emco_macaddr_change_supported)(efx_nic_t *, boolean_t *); efx_rc_t (*emco_link_control_supported)(efx_nic_t *, boolean_t *); + efx_rc_t (*emco_mac_spoofing_supported)(efx_nic_t *, boolean_t *); void (*emco_read_response)(efx_nic_t *, void *, size_t, size_t); } efx_mcdi_ops_t; typedef struct efx_mcdi_s { efx_mcdi_ops_t *em_emcop; const efx_mcdi_transport_t *em_emtp; efx_mcdi_iface_t em_emip; } efx_mcdi_t; #endif /* EFSYS_OPT_MCDI */ #if EFSYS_OPT_NVRAM typedef struct efx_nvram_ops_s { #if EFSYS_OPT_DIAG efx_rc_t (*envo_test)(efx_nic_t *); #endif /* EFSYS_OPT_DIAG */ efx_rc_t (*envo_size)(efx_nic_t *, efx_nvram_type_t, size_t *); efx_rc_t (*envo_get_version)(efx_nic_t *, efx_nvram_type_t, uint32_t *, uint16_t *); efx_rc_t (*envo_rw_start)(efx_nic_t *, efx_nvram_type_t, size_t *); efx_rc_t (*envo_read_chunk)(efx_nic_t *, efx_nvram_type_t, unsigned int, caddr_t, size_t); efx_rc_t (*envo_erase)(efx_nic_t *, efx_nvram_type_t); efx_rc_t (*envo_write_chunk)(efx_nic_t *, efx_nvram_type_t, unsigned int, caddr_t, size_t); void (*envo_rw_finish)(efx_nic_t *, efx_nvram_type_t); efx_rc_t (*envo_set_version)(efx_nic_t *, efx_nvram_type_t, uint16_t *); } efx_nvram_ops_t; #endif /* EFSYS_OPT_NVRAM */ #if EFSYS_OPT_VPD typedef struct efx_vpd_ops_s { efx_rc_t (*evpdo_init)(efx_nic_t *); efx_rc_t (*evpdo_size)(efx_nic_t *, size_t *); efx_rc_t (*evpdo_read)(efx_nic_t *, caddr_t, size_t); efx_rc_t (*evpdo_verify)(efx_nic_t *, caddr_t, size_t); efx_rc_t (*evpdo_reinit)(efx_nic_t *, caddr_t, size_t); efx_rc_t (*evpdo_get)(efx_nic_t *, caddr_t, size_t, efx_vpd_value_t *); efx_rc_t (*evpdo_set)(efx_nic_t *, caddr_t, size_t, efx_vpd_value_t *); efx_rc_t (*evpdo_next)(efx_nic_t *, caddr_t, size_t, efx_vpd_value_t *, unsigned int *); efx_rc_t (*evpdo_write)(efx_nic_t *, caddr_t, size_t); void (*evpdo_fini)(efx_nic_t *); } efx_vpd_ops_t; #endif /* EFSYS_OPT_VPD */ #if EFSYS_OPT_VPD || EFSYS_OPT_NVRAM __checkReturn efx_rc_t efx_mcdi_nvram_partitions( __in efx_nic_t *enp, __out_bcount(size) caddr_t data, __in size_t size, __out unsigned int *npartnp); __checkReturn efx_rc_t efx_mcdi_nvram_metadata( __in efx_nic_t *enp, __in uint32_t partn, __out uint32_t *subtypep, __out_ecount(4) uint16_t version[4], __out_bcount_opt(size) char *descp, __in size_t size); __checkReturn efx_rc_t efx_mcdi_nvram_info( __in efx_nic_t *enp, __in uint32_t partn, __out_opt size_t *sizep, __out_opt uint32_t *addressp, __out_opt uint32_t *erase_sizep, __out_opt uint32_t *write_sizep); __checkReturn efx_rc_t efx_mcdi_nvram_update_start( __in efx_nic_t *enp, __in uint32_t partn); __checkReturn efx_rc_t efx_mcdi_nvram_read( __in efx_nic_t *enp, __in uint32_t partn, __in uint32_t offset, __out_bcount(size) caddr_t data, __in size_t size); __checkReturn efx_rc_t efx_mcdi_nvram_erase( __in efx_nic_t *enp, __in uint32_t partn, __in uint32_t offset, __in size_t size); __checkReturn efx_rc_t efx_mcdi_nvram_write( __in efx_nic_t *enp, __in uint32_t partn, __in uint32_t offset, __out_bcount(size) caddr_t data, __in size_t size); __checkReturn efx_rc_t efx_mcdi_nvram_update_finish( __in efx_nic_t *enp, __in uint32_t partn, __in boolean_t reboot); #if EFSYS_OPT_DIAG __checkReturn efx_rc_t efx_mcdi_nvram_test( __in efx_nic_t *enp, __in uint32_t partn); #endif /* EFSYS_OPT_DIAG */ #endif /* EFSYS_OPT_VPD || EFSYS_OPT_NVRAM */ typedef struct efx_drv_cfg_s { uint32_t edc_min_vi_count; uint32_t edc_max_vi_count; uint32_t edc_max_piobuf_count; uint32_t edc_pio_alloc_size; } efx_drv_cfg_t; struct efx_nic_s { uint32_t en_magic; efx_family_t en_family; uint32_t en_features; efsys_identifier_t *en_esip; efsys_lock_t *en_eslp; efsys_bar_t *en_esbp; unsigned int en_mod_flags; unsigned int en_reset_flags; efx_nic_cfg_t en_nic_cfg; efx_drv_cfg_t en_drv_cfg; efx_port_t en_port; efx_mon_t en_mon; efx_intr_t en_intr; uint32_t en_ev_qcount; uint32_t en_rx_qcount; uint32_t en_tx_qcount; efx_nic_ops_t *en_enop; efx_ev_ops_t *en_eevop; efx_tx_ops_t *en_etxop; efx_rx_ops_t *en_erxop; #if EFSYS_OPT_FILTER efx_filter_t en_filter; efx_filter_ops_t *en_efop; #endif /* EFSYS_OPT_FILTER */ efx_pktfilter_ops_t *en_epfop; #if EFSYS_OPT_MCDI efx_mcdi_t en_mcdi; #endif /* EFSYS_OPT_MCDI */ #if EFSYS_OPT_NVRAM efx_nvram_type_t en_nvram_locked; efx_nvram_ops_t *en_envop; #endif /* EFSYS_OPT_NVRAM */ #if EFSYS_OPT_VPD efx_vpd_ops_t *en_evpdop; #endif /* EFSYS_OPT_VPD */ #if EFSYS_OPT_RX_SCALE efx_rx_hash_support_t en_hash_support; efx_rx_scale_support_t en_rss_support; uint32_t en_rss_context; #endif /* EFSYS_OPT_RX_SCALE */ uint32_t en_vport_id; union { #if EFSYS_OPT_FALCON struct { falcon_spi_dev_t enu_fsd[FALCON_SPI_NTYPES]; falcon_i2c_t enu_fip; boolean_t enu_i2c_locked; #if EFSYS_OPT_FALCON_NIC_CFG_OVERRIDE const uint8_t *enu_forced_cfg; #endif /* EFSYS_OPT_FALCON_NIC_CFG_OVERRIDE */ uint8_t enu_mon_devid; #if EFSYS_OPT_PCIE_TUNE unsigned int enu_nlanes; #endif /* EFSYS_OPT_PCIE_TUNE */ uint16_t enu_board_rev; boolean_t enu_internal_sram; uint8_t enu_sram_num_bank; uint8_t enu_sram_bank_size; } falcon; #endif /* EFSYS_OPT_FALCON */ #if EFSYS_OPT_SIENA struct { #if EFSYS_OPT_NVRAM || EFSYS_OPT_VPD unsigned int enu_partn_mask; #endif /* EFSYS_OPT_NVRAM || EFSYS_OPT_VPD */ #if EFSYS_OPT_VPD caddr_t enu_svpd; size_t enu_svpd_length; #endif /* EFSYS_OPT_VPD */ int enu_unused; } siena; #endif /* EFSYS_OPT_SIENA */ #if EFSYS_OPT_HUNTINGTON struct { int enu_vi_base; int enu_vi_count; #if EFSYS_OPT_VPD caddr_t enu_svpd; size_t enu_svpd_length; #endif /* EFSYS_OPT_VPD */ efx_piobuf_handle_t enu_piobuf_handle[HUNT_PIOBUF_NBUFS]; uint32_t enu_piobuf_count; uint32_t enu_pio_alloc_map[HUNT_PIOBUF_NBUFS]; uint32_t enu_pio_write_vi_base; /* Memory BAR mapping regions */ uint32_t enu_uc_mem_map_offset; size_t enu_uc_mem_map_size; uint32_t enu_wc_mem_map_offset; size_t enu_wc_mem_map_size; } hunt; #endif /* EFSYS_OPT_HUNTINGTON */ } en_u; }; #define EFX_NIC_MAGIC 0x02121996 typedef boolean_t (*efx_ev_handler_t)(efx_evq_t *, efx_qword_t *, const efx_ev_callbacks_t *, void *); typedef struct efx_evq_rxq_state_s { unsigned int eers_rx_read_ptr; unsigned int eers_rx_mask; } efx_evq_rxq_state_t; struct efx_evq_s { uint32_t ee_magic; efx_nic_t *ee_enp; unsigned int ee_index; unsigned int ee_mask; efsys_mem_t *ee_esmp; #if EFSYS_OPT_QSTATS uint32_t ee_stat[EV_NQSTATS]; #endif /* EFSYS_OPT_QSTATS */ efx_ev_handler_t ee_rx; efx_ev_handler_t ee_tx; efx_ev_handler_t ee_driver; efx_ev_handler_t ee_global; efx_ev_handler_t ee_drv_gen; #if EFSYS_OPT_MCDI efx_ev_handler_t ee_mcdi; #endif /* EFSYS_OPT_MCDI */ efx_evq_rxq_state_t ee_rxq_state[EFX_EV_RX_NLABELS]; }; #define EFX_EVQ_MAGIC 0x08081997 #define EFX_EVQ_FALCON_TIMER_QUANTUM_NS 4968 /* 621 cycles */ #define EFX_EVQ_SIENA_TIMER_QUANTUM_NS 6144 /* 768 cycles */ struct efx_rxq_s { uint32_t er_magic; efx_nic_t *er_enp; efx_evq_t *er_eep; unsigned int er_index; unsigned int er_label; unsigned int er_mask; efsys_mem_t *er_esmp; }; #define EFX_RXQ_MAGIC 0x15022005 struct efx_txq_s { uint32_t et_magic; efx_nic_t *et_enp; unsigned int et_index; unsigned int et_mask; efsys_mem_t *et_esmp; #if EFSYS_OPT_HUNTINGTON uint32_t et_pio_bufnum; uint32_t et_pio_blknum; uint32_t et_pio_write_offset; uint32_t et_pio_offset; size_t et_pio_size; #endif #if EFSYS_OPT_QSTATS uint32_t et_stat[TX_NQSTATS]; #endif /* EFSYS_OPT_QSTATS */ }; #define EFX_TXQ_MAGIC 0x05092005 #define EFX_MAC_ADDR_COPY(_dst, _src) \ do { \ (_dst)[0] = (_src)[0]; \ (_dst)[1] = (_src)[1]; \ (_dst)[2] = (_src)[2]; \ (_dst)[3] = (_src)[3]; \ (_dst)[4] = (_src)[4]; \ (_dst)[5] = (_src)[5]; \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_MAC_BROADCAST_ADDR_SET(_dst) \ do { \ uint16_t *_d = (uint16_t *)(_dst); \ _d[0] = 0xffff; \ _d[1] = 0xffff; \ _d[2] = 0xffff; \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #if EFSYS_OPT_CHECK_REG #define EFX_CHECK_REG(_enp, _reg) \ do { \ const char *name = #_reg; \ char min = name[4]; \ char max = name[5]; \ char rev; \ \ switch ((_enp)->en_family) { \ case EFX_FAMILY_FALCON: \ rev = 'B'; \ break; \ \ case EFX_FAMILY_SIENA: \ rev = 'C'; \ break; \ \ case EFX_FAMILY_HUNTINGTON: \ rev = 'D'; \ break; \ \ default: \ rev = '?'; \ break; \ } \ \ EFSYS_ASSERT3S(rev, >=, min); \ EFSYS_ASSERT3S(rev, <=, max); \ \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #else #define EFX_CHECK_REG(_enp, _reg) do { \ _NOTE(CONSTANTCONDITION) \ } while(B_FALSE) #endif #define EFX_BAR_READD(_enp, _reg, _edp, _lock) \ do { \ EFX_CHECK_REG((_enp), (_reg)); \ EFSYS_BAR_READD((_enp)->en_esbp, _reg ## _OFST, \ (_edp), (_lock)); \ EFSYS_PROBE3(efx_bar_readd, const char *, #_reg, \ uint32_t, _reg ## _OFST, \ uint32_t, (_edp)->ed_u32[0]); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_BAR_WRITED(_enp, _reg, _edp, _lock) \ do { \ EFX_CHECK_REG((_enp), (_reg)); \ EFSYS_PROBE3(efx_bar_writed, const char *, #_reg, \ uint32_t, _reg ## _OFST, \ uint32_t, (_edp)->ed_u32[0]); \ EFSYS_BAR_WRITED((_enp)->en_esbp, _reg ## _OFST, \ (_edp), (_lock)); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_BAR_READQ(_enp, _reg, _eqp) \ do { \ EFX_CHECK_REG((_enp), (_reg)); \ EFSYS_BAR_READQ((_enp)->en_esbp, _reg ## _OFST, \ (_eqp)); \ EFSYS_PROBE4(efx_bar_readq, const char *, #_reg, \ uint32_t, _reg ## _OFST, \ uint32_t, (_eqp)->eq_u32[1], \ uint32_t, (_eqp)->eq_u32[0]); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_BAR_WRITEQ(_enp, _reg, _eqp) \ do { \ EFX_CHECK_REG((_enp), (_reg)); \ EFSYS_PROBE4(efx_bar_writeq, const char *, #_reg, \ uint32_t, _reg ## _OFST, \ uint32_t, (_eqp)->eq_u32[1], \ uint32_t, (_eqp)->eq_u32[0]); \ EFSYS_BAR_WRITEQ((_enp)->en_esbp, _reg ## _OFST, \ (_eqp)); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_BAR_READO(_enp, _reg, _eop) \ do { \ EFX_CHECK_REG((_enp), (_reg)); \ EFSYS_BAR_READO((_enp)->en_esbp, _reg ## _OFST, \ (_eop), B_TRUE); \ EFSYS_PROBE6(efx_bar_reado, const char *, #_reg, \ uint32_t, _reg ## _OFST, \ uint32_t, (_eop)->eo_u32[3], \ uint32_t, (_eop)->eo_u32[2], \ uint32_t, (_eop)->eo_u32[1], \ uint32_t, (_eop)->eo_u32[0]); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_BAR_WRITEO(_enp, _reg, _eop) \ do { \ EFX_CHECK_REG((_enp), (_reg)); \ EFSYS_PROBE6(efx_bar_writeo, const char *, #_reg, \ uint32_t, _reg ## _OFST, \ uint32_t, (_eop)->eo_u32[3], \ uint32_t, (_eop)->eo_u32[2], \ uint32_t, (_eop)->eo_u32[1], \ uint32_t, (_eop)->eo_u32[0]); \ EFSYS_BAR_WRITEO((_enp)->en_esbp, _reg ## _OFST, \ (_eop), B_TRUE); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_BAR_TBL_READD(_enp, _reg, _index, _edp, _lock) \ do { \ EFX_CHECK_REG((_enp), (_reg)); \ EFSYS_BAR_READD((_enp)->en_esbp, \ (_reg ## _OFST + ((_index) * _reg ## _STEP)), \ (_edp), (_lock)); \ EFSYS_PROBE4(efx_bar_tbl_readd, const char *, #_reg, \ uint32_t, (_index), \ uint32_t, _reg ## _OFST, \ uint32_t, (_edp)->ed_u32[0]); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_BAR_TBL_WRITED(_enp, _reg, _index, _edp, _lock) \ do { \ EFX_CHECK_REG((_enp), (_reg)); \ EFSYS_PROBE4(efx_bar_tbl_writed, const char *, #_reg, \ uint32_t, (_index), \ uint32_t, _reg ## _OFST, \ uint32_t, (_edp)->ed_u32[0]); \ EFSYS_BAR_WRITED((_enp)->en_esbp, \ (_reg ## _OFST + ((_index) * _reg ## _STEP)), \ (_edp), (_lock)); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_BAR_TBL_WRITED2(_enp, _reg, _index, _edp, _lock) \ do { \ EFX_CHECK_REG((_enp), (_reg)); \ EFSYS_PROBE4(efx_bar_tbl_writed, const char *, #_reg, \ uint32_t, (_index), \ uint32_t, _reg ## _OFST, \ uint32_t, (_edp)->ed_u32[0]); \ EFSYS_BAR_WRITED((_enp)->en_esbp, \ (_reg ## _OFST + \ (2 * sizeof (efx_dword_t)) + \ ((_index) * _reg ## _STEP)), \ (_edp), (_lock)); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_BAR_TBL_WRITED3(_enp, _reg, _index, _edp, _lock) \ do { \ EFX_CHECK_REG((_enp), (_reg)); \ EFSYS_PROBE4(efx_bar_tbl_writed, const char *, #_reg, \ uint32_t, (_index), \ uint32_t, _reg ## _OFST, \ uint32_t, (_edp)->ed_u32[0]); \ EFSYS_BAR_WRITED((_enp)->en_esbp, \ (_reg ## _OFST + \ (3 * sizeof (efx_dword_t)) + \ ((_index) * _reg ## _STEP)), \ (_edp), (_lock)); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_BAR_TBL_READQ(_enp, _reg, _index, _eqp) \ do { \ EFX_CHECK_REG((_enp), (_reg)); \ EFSYS_BAR_READQ((_enp)->en_esbp, \ (_reg ## _OFST + ((_index) * _reg ## _STEP)), \ (_eqp)); \ EFSYS_PROBE5(efx_bar_tbl_readq, const char *, #_reg, \ uint32_t, (_index), \ uint32_t, _reg ## _OFST, \ uint32_t, (_eqp)->eq_u32[1], \ uint32_t, (_eqp)->eq_u32[0]); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_BAR_TBL_WRITEQ(_enp, _reg, _index, _eqp) \ do { \ EFX_CHECK_REG((_enp), (_reg)); \ EFSYS_PROBE5(efx_bar_tbl_writeq, const char *, #_reg, \ uint32_t, (_index), \ uint32_t, _reg ## _OFST, \ uint32_t, (_eqp)->eq_u32[1], \ uint32_t, (_eqp)->eq_u32[0]); \ EFSYS_BAR_WRITEQ((_enp)->en_esbp, \ (_reg ## _OFST + ((_index) * _reg ## _STEP)), \ (_eqp)); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_BAR_TBL_READO(_enp, _reg, _index, _eop, _lock) \ do { \ EFX_CHECK_REG((_enp), (_reg)); \ EFSYS_BAR_READO((_enp)->en_esbp, \ (_reg ## _OFST + ((_index) * _reg ## _STEP)), \ (_eop), (_lock)); \ EFSYS_PROBE7(efx_bar_tbl_reado, const char *, #_reg, \ uint32_t, (_index), \ uint32_t, _reg ## _OFST, \ uint32_t, (_eop)->eo_u32[3], \ uint32_t, (_eop)->eo_u32[2], \ uint32_t, (_eop)->eo_u32[1], \ uint32_t, (_eop)->eo_u32[0]); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_BAR_TBL_WRITEO(_enp, _reg, _index, _eop, _lock) \ do { \ EFX_CHECK_REG((_enp), (_reg)); \ EFSYS_PROBE7(efx_bar_tbl_writeo, const char *, #_reg, \ uint32_t, (_index), \ uint32_t, _reg ## _OFST, \ uint32_t, (_eop)->eo_u32[3], \ uint32_t, (_eop)->eo_u32[2], \ uint32_t, (_eop)->eo_u32[1], \ uint32_t, (_eop)->eo_u32[0]); \ EFSYS_BAR_WRITEO((_enp)->en_esbp, \ (_reg ## _OFST + ((_index) * _reg ## _STEP)), \ (_eop), (_lock)); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) /* * Allow drivers to perform optimised 128-bit doorbell writes. * The DMA descriptor pointers (RX_DESC_UPD and TX_DESC_UPD) are * special-cased in the BIU on the Falcon/Siena and EF10 architectures to avoid * the need for locking in the host, and are the only ones known to be safe to * use 128-bites write with. */ #define EFX_BAR_TBL_DOORBELL_WRITEO(_enp, _reg, _index, _eop) \ do { \ EFX_CHECK_REG((_enp), (_reg)); \ EFSYS_PROBE7(efx_bar_tbl_doorbell_writeo, \ const char *, \ #_reg, \ uint32_t, (_index), \ uint32_t, _reg ## _OFST, \ uint32_t, (_eop)->eo_u32[3], \ uint32_t, (_eop)->eo_u32[2], \ uint32_t, (_eop)->eo_u32[1], \ uint32_t, (_eop)->eo_u32[0]); \ EFSYS_BAR_DOORBELL_WRITEO((_enp)->en_esbp, \ (_reg ## _OFST + ((_index) * _reg ## _STEP)), \ (_eop)); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_DMA_SYNC_QUEUE_FOR_DEVICE(_esmp, _entries, _wptr, _owptr) \ do { \ unsigned int _new = (_wptr); \ unsigned int _old = (_owptr); \ \ if ((_new) >= (_old)) \ EFSYS_DMA_SYNC_FOR_DEVICE((_esmp), \ (_old) * sizeof (efx_desc_t), \ ((_new) - (_old)) * sizeof (efx_desc_t)); \ else \ /* \ * It is cheaper to sync entire map than sync \ * two parts especially when offset/size are \ * ignored and entire map is synced in any case.\ */ \ EFSYS_DMA_SYNC_FOR_DEVICE((_esmp), \ 0, \ (_entries) * sizeof (efx_desc_t)); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) extern __checkReturn efx_rc_t efx_nic_biu_test( __in efx_nic_t *enp); extern __checkReturn efx_rc_t efx_mac_select( __in efx_nic_t *enp); extern void efx_mac_multicast_hash_compute( __in_ecount(6*count) uint8_t const *addrs, __in int count, __out efx_oword_t *hash_low, __out efx_oword_t *hash_high); extern __checkReturn efx_rc_t efx_phy_probe( __in efx_nic_t *enp); extern void efx_phy_unprobe( __in efx_nic_t *enp); #if EFSYS_OPT_VPD /* VPD utility functions */ extern __checkReturn efx_rc_t efx_vpd_hunk_length( __in_bcount(size) caddr_t data, __in size_t size, __out size_t *lengthp); extern __checkReturn efx_rc_t efx_vpd_hunk_verify( __in_bcount(size) caddr_t data, __in size_t size, __out_opt boolean_t *cksummedp); extern __checkReturn efx_rc_t efx_vpd_hunk_reinit( __in_bcount(size) caddr_t data, __in size_t size, __in boolean_t wantpid); extern __checkReturn efx_rc_t efx_vpd_hunk_get( __in_bcount(size) caddr_t data, __in size_t size, __in efx_vpd_tag_t tag, __in efx_vpd_keyword_t keyword, __out unsigned int *payloadp, __out uint8_t *paylenp); extern __checkReturn efx_rc_t efx_vpd_hunk_next( __in_bcount(size) caddr_t data, __in size_t size, __out efx_vpd_tag_t *tagp, __out efx_vpd_keyword_t *keyword, __out_bcount_opt(*paylenp) unsigned int *payloadp, __out_opt uint8_t *paylenp, __inout unsigned int *contp); extern __checkReturn efx_rc_t efx_vpd_hunk_set( __in_bcount(size) caddr_t data, __in size_t size, __in efx_vpd_value_t *evvp); #endif /* EFSYS_OPT_VPD */ #if EFSYS_OPT_DIAG extern efx_sram_pattern_fn_t __efx_sram_pattern_fns[]; typedef struct efx_register_set_s { unsigned int address; unsigned int step; unsigned int rows; efx_oword_t mask; } efx_register_set_t; extern __checkReturn efx_rc_t efx_nic_test_registers( __in efx_nic_t *enp, __in efx_register_set_t *rsp, __in size_t count); extern __checkReturn efx_rc_t efx_nic_test_tables( __in efx_nic_t *enp, __in efx_register_set_t *rsp, __in efx_pattern_type_t pattern, __in size_t count); #endif /* EFSYS_OPT_DIAG */ #if EFSYS_OPT_MCDI extern __checkReturn efx_rc_t efx_mcdi_set_workaround( __in efx_nic_t *enp, __in uint32_t type, __in boolean_t enabled, __out_opt uint32_t *flagsp); extern __checkReturn efx_rc_t efx_mcdi_get_workarounds( __in efx_nic_t *enp, __out_opt uint32_t *implementedp, __out_opt uint32_t *enabledp); #endif /* EFSYS_OPT_MCDI */ #ifdef __cplusplus } #endif #endif /* _SYS_EFX_IMPL_H */ Index: user/ngie/more-tests2/sys/dev/sfxge/common/efx_mcdi.c =================================================================== --- user/ngie/more-tests2/sys/dev/sfxge/common/efx_mcdi.c (revision 292053) +++ user/ngie/more-tests2/sys/dev/sfxge/common/efx_mcdi.c (revision 292054) @@ -1,1689 +1,1907 @@ /*- * Copyright (c) 2008-2015 Solarflare Communications Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are met: * * 1. Redistributions of source code must retain the above copyright notice, * this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright notice, * this list of conditions and the following disclaimer in the documentation * and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR * OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, * EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * The views and conclusions contained in the software and documentation are * those of the authors and should not be interpreted as representing official * policies, either expressed or implied, of the FreeBSD Project. */ #include __FBSDID("$FreeBSD$"); #include "efsys.h" #include "efx.h" #include "efx_types.h" #include "efx_regs.h" #include "efx_regs_mcdi.h" #include "efx_impl.h" #if EFSYS_OPT_MCDI #if EFSYS_OPT_SIENA static efx_mcdi_ops_t __efx_mcdi_siena_ops = { siena_mcdi_init, /* emco_init */ siena_mcdi_request_copyin, /* emco_request_copyin */ siena_mcdi_request_poll, /* emco_request_poll */ siena_mcdi_request_copyout, /* emco_request_copyout */ siena_mcdi_poll_reboot, /* emco_poll_reboot */ siena_mcdi_fini, /* emco_fini */ siena_mcdi_fw_update_supported, /* emco_fw_update_supported */ siena_mcdi_macaddr_change_supported, /* emco_macaddr_change_supported */ siena_mcdi_link_control_supported, /* emco_link_control_supported */ + NULL, /* emco_mac_spoofing_supported */ siena_mcdi_read_response, /* emco_read_response */ }; #endif /* EFSYS_OPT_SIENA */ #if EFSYS_OPT_HUNTINGTON static efx_mcdi_ops_t __efx_mcdi_hunt_ops = { hunt_mcdi_init, /* emco_init */ hunt_mcdi_request_copyin, /* emco_request_copyin */ hunt_mcdi_request_poll, /* emco_request_poll */ hunt_mcdi_request_copyout, /* emco_request_copyout */ hunt_mcdi_poll_reboot, /* emco_poll_reboot */ hunt_mcdi_fini, /* emco_fini */ hunt_mcdi_fw_update_supported, /* emco_fw_update_supported */ hunt_mcdi_macaddr_change_supported, /* emco_macaddr_change_supported */ hunt_mcdi_link_control_supported, /* emco_link_control_supported */ + hunt_mcdi_mac_spoofing_supported, + /* emco_mac_spoofing_supported */ hunt_mcdi_read_response, /* emco_read_response */ }; #endif /* EFSYS_OPT_HUNTINGTON */ __checkReturn efx_rc_t efx_mcdi_init( __in efx_nic_t *enp, __in const efx_mcdi_transport_t *emtp) { efx_mcdi_ops_t *emcop; efx_rc_t rc; EFSYS_ASSERT3U(enp->en_magic, ==, EFX_NIC_MAGIC); EFSYS_ASSERT3U(enp->en_mod_flags, ==, 0); switch (enp->en_family) { #if EFSYS_OPT_FALCON case EFX_FAMILY_FALCON: emcop = NULL; emtp = NULL; break; #endif /* EFSYS_OPT_FALCON */ #if EFSYS_OPT_SIENA case EFX_FAMILY_SIENA: emcop = (efx_mcdi_ops_t *)&__efx_mcdi_siena_ops; break; #endif /* EFSYS_OPT_SIENA */ #if EFSYS_OPT_HUNTINGTON case EFX_FAMILY_HUNTINGTON: emcop = (efx_mcdi_ops_t *)&__efx_mcdi_hunt_ops; break; #endif /* EFSYS_OPT_HUNTINGTON */ default: EFSYS_ASSERT(0); rc = ENOTSUP; goto fail1; } if (enp->en_features & EFX_FEATURE_MCDI_DMA) { /* MCDI requires a DMA buffer in host memory */ if ((emtp == NULL) || (emtp->emt_dma_mem) == NULL) { rc = EINVAL; goto fail2; } } enp->en_mcdi.em_emtp = emtp; if (emcop != NULL && emcop->emco_init != NULL) { if ((rc = emcop->emco_init(enp, emtp)) != 0) goto fail3; } enp->en_mcdi.em_emcop = emcop; enp->en_mod_flags |= EFX_MOD_MCDI; return (0); fail3: EFSYS_PROBE(fail3); fail2: EFSYS_PROBE(fail2); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); enp->en_mcdi.em_emcop = NULL; enp->en_mcdi.em_emtp = NULL; enp->en_mod_flags &= ~EFX_MOD_MCDI; return (rc); } void efx_mcdi_fini( __in efx_nic_t *enp) { efx_mcdi_iface_t *emip = &(enp->en_mcdi.em_emip); efx_mcdi_ops_t *emcop = enp->en_mcdi.em_emcop; EFSYS_ASSERT3U(enp->en_magic, ==, EFX_NIC_MAGIC); EFSYS_ASSERT3U(enp->en_mod_flags, ==, EFX_MOD_MCDI); if (emcop != NULL && emcop->emco_fini != NULL) emcop->emco_fini(enp); emip->emi_port = 0; emip->emi_aborted = 0; enp->en_mcdi.em_emcop = NULL; enp->en_mod_flags &= ~EFX_MOD_MCDI; } void efx_mcdi_new_epoch( __in efx_nic_t *enp) { efx_mcdi_iface_t *emip = &(enp->en_mcdi.em_emip); int state; /* Start a new epoch (allow fresh MCDI requests to succeed) */ EFSYS_LOCK(enp->en_eslp, state); emip->emi_new_epoch = B_TRUE; EFSYS_UNLOCK(enp->en_eslp, state); } void efx_mcdi_request_start( __in efx_nic_t *enp, __in efx_mcdi_req_t *emrp, __in boolean_t ev_cpl) { efx_mcdi_iface_t *emip = &(enp->en_mcdi.em_emip); efx_mcdi_ops_t *emcop = enp->en_mcdi.em_emcop; unsigned int seq; boolean_t new_epoch; int state; EFSYS_ASSERT3U(enp->en_magic, ==, EFX_NIC_MAGIC); EFSYS_ASSERT3U(enp->en_mod_flags, &, EFX_MOD_MCDI); EFSYS_ASSERT3U(enp->en_features, &, EFX_FEATURE_MCDI); if (emcop == NULL || emcop->emco_request_copyin == NULL) return; /* * efx_mcdi_request_start() is naturally serialised against both * efx_mcdi_request_poll() and efx_mcdi_ev_cpl()/efx_mcdi_ev_death(), * by virtue of there only being one outstanding MCDI request. * Unfortunately, upper layers may also call efx_mcdi_request_abort() * at any time, to timeout a pending mcdi request, That request may * then subsequently complete, meaning efx_mcdi_ev_cpl() or * efx_mcdi_ev_death() may end up running in parallel with * efx_mcdi_request_start(). This race is handled by ensuring that * %emi_pending_req, %emi_ev_cpl and %emi_seq are protected by the * en_eslp lock. */ EFSYS_LOCK(enp->en_eslp, state); EFSYS_ASSERT(emip->emi_pending_req == NULL); emip->emi_pending_req = emrp; emip->emi_ev_cpl = ev_cpl; emip->emi_poll_cnt = 0; seq = emip->emi_seq++ & EFX_MASK32(MCDI_HEADER_SEQ); new_epoch = emip->emi_new_epoch; EFSYS_UNLOCK(enp->en_eslp, state); emcop->emco_request_copyin(enp, emrp, seq, ev_cpl, new_epoch); } + + void +efx_mcdi_read_response_header( + __in efx_nic_t *enp, + __inout efx_mcdi_req_t *emrp) +{ +#if EFSYS_OPT_MCDI_LOGGING + const efx_mcdi_transport_t *emtp = enp->en_mcdi.em_emtp; +#endif /* EFSYS_OPT_MCDI_LOGGING */ + efx_mcdi_iface_t *emip = &(enp->en_mcdi.em_emip); + efx_mcdi_ops_t *emcop = enp->en_mcdi.em_emcop; + efx_dword_t hdr[2]; + unsigned int hdr_len; + unsigned int data_len; + unsigned int seq; + unsigned int cmd; + unsigned int error; + efx_rc_t rc; + + EFSYS_ASSERT(emrp != NULL); + + emcop->emco_read_response(enp, &hdr[0], 0, sizeof (hdr[0])); + hdr_len = sizeof (hdr[0]); + + cmd = EFX_DWORD_FIELD(hdr[0], MCDI_HEADER_CODE); + seq = EFX_DWORD_FIELD(hdr[0], MCDI_HEADER_SEQ); + error = EFX_DWORD_FIELD(hdr[0], MCDI_HEADER_ERROR); + + if (cmd != MC_CMD_V2_EXTN) { + data_len = EFX_DWORD_FIELD(hdr[0], MCDI_HEADER_DATALEN); + } else { + emcop->emco_read_response(enp, &hdr[1], hdr_len, + sizeof (hdr[1])); + hdr_len += sizeof (hdr[1]); + + cmd = EFX_DWORD_FIELD(hdr[1], MC_CMD_V2_EXTN_IN_EXTENDED_CMD); + data_len = + EFX_DWORD_FIELD(hdr[1], MC_CMD_V2_EXTN_IN_ACTUAL_LEN); + } + + if (error && (data_len == 0)) { + /* The MC has rebooted since the request was sent. */ + EFSYS_SPIN(EFX_MCDI_STATUS_SLEEP_US); + emcop->emco_poll_reboot(enp); + rc = EIO; + goto fail1; + } + if ((cmd != emrp->emr_cmd) || + (seq != ((emip->emi_seq - 1) & EFX_MASK32(MCDI_HEADER_SEQ)))) { + /* Response is for a different request */ + rc = EIO; + goto fail2; + } + if (error) { + efx_dword_t err[2]; + unsigned int err_len = MIN(data_len, sizeof (err)); + int err_code = MC_CMD_ERR_EPROTO; + int err_arg = 0; + + /* Read error code (and arg num for MCDI v2 commands) */ + emcop->emco_read_response(enp, &err, hdr_len, err_len); + + if (err_len >= (MC_CMD_ERR_CODE_OFST + sizeof (efx_dword_t))) + err_code = EFX_DWORD_FIELD(err[0], EFX_DWORD_0); +#ifdef WITH_MCDI_V2 + if (err_len >= (MC_CMD_ERR_ARG_OFST + sizeof (efx_dword_t))) + err_arg = EFX_DWORD_FIELD(err[1], EFX_DWORD_0); +#endif + emrp->emr_err_code = err_code; + emrp->emr_err_arg = err_arg; + +#if EFSYS_OPT_MCDI_PROXY_AUTH + if ((err_code == MC_CMD_ERR_PROXY_PENDING) && + (err_len == sizeof (err))) { + /* + * The MCDI request would normally fail with EPERM, but + * firmware has forwarded it to an authorization agent + * attached to a privileged PF. + * + * Save the authorization request handle. The client + * must wait for a PROXY_RESPONSE event, or timeout. + */ + emrp->emr_proxy_handle = err_arg; + } +#endif /* EFSYS_OPT_MCDI_PROXY_AUTH */ + +#if EFSYS_OPT_MCDI_LOGGING + if (emtp->emt_logger != NULL) { + emtp->emt_logger(emtp->emt_context, + EFX_LOG_MCDI_RESPONSE, + &hdr, hdr_len, + &err, err_len); + } +#endif /* EFSYS_OPT_MCDI_LOGGING */ + + if (!emrp->emr_quiet) { + EFSYS_PROBE3(mcdi_err_arg, int, emrp->emr_cmd, + int, err_code, int, err_arg); + } + + rc = efx_mcdi_request_errcode(err_code); + goto fail3; + } + + emrp->emr_rc = 0; + emrp->emr_out_length_used = data_len; +#if EFSYS_OPT_MCDI_PROXY_AUTH + emrp->emr_proxy_handle = 0; +#endif /* EFSYS_OPT_MCDI_PROXY_AUTH */ + return; + +fail3: + if (!emrp->emr_quiet) + EFSYS_PROBE(fail3); +fail2: + if (!emrp->emr_quiet) + EFSYS_PROBE(fail2); +fail1: + if (!emrp->emr_quiet) + EFSYS_PROBE1(fail1, efx_rc_t, rc); + + emrp->emr_rc = rc; + emrp->emr_out_length_used = 0; +} + + __checkReturn boolean_t efx_mcdi_request_poll( __in efx_nic_t *enp) { efx_mcdi_ops_t *emcop = enp->en_mcdi.em_emcop; boolean_t completed; EFSYS_ASSERT3U(enp->en_magic, ==, EFX_NIC_MAGIC); EFSYS_ASSERT3U(enp->en_mod_flags, &, EFX_MOD_MCDI); EFSYS_ASSERT3U(enp->en_features, &, EFX_FEATURE_MCDI); completed = B_FALSE; if (emcop != NULL && emcop->emco_request_poll != NULL) completed = emcop->emco_request_poll(enp); return (completed); } __checkReturn boolean_t efx_mcdi_request_abort( __in efx_nic_t *enp) { efx_mcdi_iface_t *emip = &(enp->en_mcdi.em_emip); efx_mcdi_req_t *emrp; boolean_t aborted; int state; EFSYS_ASSERT3U(enp->en_magic, ==, EFX_NIC_MAGIC); EFSYS_ASSERT3U(enp->en_mod_flags, &, EFX_MOD_MCDI); EFSYS_ASSERT3U(enp->en_features, &, EFX_FEATURE_MCDI); /* * efx_mcdi_ev_* may have already completed this event, and be * spinning/blocked on the upper layer lock. So it *is* legitimate * to for emi_pending_req to be NULL. If there is a pending event * completed request, then provide a "credit" to allow * efx_mcdi_ev_cpl() to accept a single spurious completion. */ EFSYS_LOCK(enp->en_eslp, state); emrp = emip->emi_pending_req; aborted = (emrp != NULL); if (aborted) { emip->emi_pending_req = NULL; /* Error the request */ emrp->emr_out_length_used = 0; emrp->emr_rc = ETIMEDOUT; /* Provide a credit for seqno/emr_pending_req mismatches */ if (emip->emi_ev_cpl) ++emip->emi_aborted; /* * The upper layer has called us, so we don't * need to complete the request. */ } EFSYS_UNLOCK(enp->en_eslp, state); return (aborted); } __checkReturn efx_rc_t efx_mcdi_request_errcode( __in unsigned int err) { switch (err) { /* MCDI v1 */ case MC_CMD_ERR_EPERM: return (EACCES); case MC_CMD_ERR_ENOENT: return (ENOENT); case MC_CMD_ERR_EINTR: return (EINTR); case MC_CMD_ERR_EACCES: return (EACCES); case MC_CMD_ERR_EBUSY: return (EBUSY); case MC_CMD_ERR_EINVAL: return (EINVAL); case MC_CMD_ERR_EDEADLK: return (EDEADLK); case MC_CMD_ERR_ENOSYS: return (ENOTSUP); case MC_CMD_ERR_ETIME: return (ETIMEDOUT); case MC_CMD_ERR_ENOTSUP: return (ENOTSUP); case MC_CMD_ERR_EALREADY: return (EALREADY); /* MCDI v2 */ #ifdef MC_CMD_ERR_EAGAIN case MC_CMD_ERR_EAGAIN: return (EAGAIN); #endif #ifdef MC_CMD_ERR_ENOSPC case MC_CMD_ERR_ENOSPC: return (ENOSPC); #endif case MC_CMD_ERR_ALLOC_FAIL: return (ENOMEM); case MC_CMD_ERR_NO_VADAPTOR: return (ENOENT); case MC_CMD_ERR_NO_EVB_PORT: return (ENOENT); case MC_CMD_ERR_NO_VSWITCH: return (ENODEV); case MC_CMD_ERR_VLAN_LIMIT: return (EINVAL); case MC_CMD_ERR_BAD_PCI_FUNC: return (ENODEV); case MC_CMD_ERR_BAD_VLAN_MODE: return (EINVAL); case MC_CMD_ERR_BAD_VSWITCH_TYPE: return (EINVAL); case MC_CMD_ERR_BAD_VPORT_TYPE: return (EINVAL); case MC_CMD_ERR_MAC_EXIST: return (EEXIST); + case MC_CMD_ERR_PROXY_PENDING: + return (EAGAIN); + default: EFSYS_PROBE1(mc_pcol_error, int, err); return (EIO); } } void efx_mcdi_raise_exception( __in efx_nic_t *enp, __in_opt efx_mcdi_req_t *emrp, __in int rc) { const efx_mcdi_transport_t *emtp = enp->en_mcdi.em_emtp; efx_mcdi_exception_t exception; /* Reboot or Assertion failure only */ EFSYS_ASSERT(rc == EIO || rc == EINTR); /* * If MC_CMD_REBOOT causes a reboot (dependent on parameters), * then the EIO is not worthy of an exception. */ if (emrp != NULL && emrp->emr_cmd == MC_CMD_REBOOT && rc == EIO) return; exception = (rc == EIO) ? EFX_MCDI_EXCEPTION_MC_REBOOT : EFX_MCDI_EXCEPTION_MC_BADASSERT; emtp->emt_exception(emtp->emt_context, exception); } static efx_rc_t efx_mcdi_poll_reboot( __in efx_nic_t *enp) { efx_mcdi_ops_t *emcop = enp->en_mcdi.em_emcop; return (emcop->emco_poll_reboot(enp)); } void efx_mcdi_execute( __in efx_nic_t *enp, __inout efx_mcdi_req_t *emrp) { const efx_mcdi_transport_t *emtp = enp->en_mcdi.em_emtp; EFSYS_ASSERT3U(enp->en_mod_flags, &, EFX_MOD_MCDI); EFSYS_ASSERT3U(enp->en_features, &, EFX_FEATURE_MCDI); emrp->emr_quiet = B_FALSE; emtp->emt_execute(emtp->emt_context, emrp); } void efx_mcdi_execute_quiet( __in efx_nic_t *enp, __inout efx_mcdi_req_t *emrp) { const efx_mcdi_transport_t *emtp = enp->en_mcdi.em_emtp; EFSYS_ASSERT3U(enp->en_mod_flags, &, EFX_MOD_MCDI); EFSYS_ASSERT3U(enp->en_features, &, EFX_FEATURE_MCDI); emrp->emr_quiet = B_TRUE; emtp->emt_execute(emtp->emt_context, emrp); } void efx_mcdi_ev_cpl( __in efx_nic_t *enp, __in unsigned int seq, __in unsigned int outlen, __in int errcode) { efx_mcdi_iface_t *emip = &(enp->en_mcdi.em_emip); const efx_mcdi_transport_t *emtp = enp->en_mcdi.em_emtp; efx_mcdi_ops_t *emcop = enp->en_mcdi.em_emcop; + efx_nic_cfg_t *encp = &enp->en_nic_cfg; efx_mcdi_req_t *emrp; int state; EFSYS_ASSERT3U(enp->en_mod_flags, &, EFX_MOD_MCDI); EFSYS_ASSERT3U(enp->en_features, &, EFX_FEATURE_MCDI); /* * Serialise against efx_mcdi_request_poll()/efx_mcdi_request_start() * when we're completing an aborted request. */ EFSYS_LOCK(enp->en_eslp, state); if (emip->emi_pending_req == NULL || !emip->emi_ev_cpl || (seq != ((emip->emi_seq - 1) & EFX_MASK32(MCDI_HEADER_SEQ)))) { EFSYS_ASSERT(emip->emi_aborted > 0); if (emip->emi_aborted > 0) --emip->emi_aborted; EFSYS_UNLOCK(enp->en_eslp, state); return; } emrp = emip->emi_pending_req; emip->emi_pending_req = NULL; EFSYS_UNLOCK(enp->en_eslp, state); + if (encp->enc_mcdi_max_payload_length > MCDI_CTL_SDU_LEN_MAX_V1) { + /* MCDIv2 response details do not fit into an event. */ + efx_mcdi_read_response_header(enp, emrp); + } else { + if (errcode != 0) { + if (!emrp->emr_quiet) { + EFSYS_PROBE2(mcdi_err, int, emrp->emr_cmd, + int, errcode); + } + emrp->emr_out_length_used = 0; + emrp->emr_rc = efx_mcdi_request_errcode(errcode); + } else { + emrp->emr_out_length_used = outlen; + emrp->emr_rc = 0; + } + } + if (errcode == 0) { + emcop->emco_request_copyout(enp, emrp); + } + + emtp->emt_ev_cpl(emtp->emt_context); +} + +#if EFSYS_OPT_MCDI_PROXY_AUTH + + __checkReturn efx_rc_t +efx_mcdi_get_proxy_handle( + __in efx_nic_t *enp, + __in efx_mcdi_req_t *emrp, + __out uint32_t *handlep) +{ + efx_mcdi_iface_t *emip = &(enp->en_mcdi.em_emip); + efx_rc_t rc; + /* - * Fill out the remaining hdr fields, and copyout the payload - * if the user supplied an output buffer. + * Return proxy handle from MCDI request that returned with error + * MC_MCD_ERR_PROXY_PENDING. This handle is used to wait for a matching + * PROXY_RESPONSE event. */ - if (errcode != 0) { - if (!emrp->emr_quiet) { - EFSYS_PROBE2(mcdi_err, int, emrp->emr_cmd, - int, errcode); - } - emrp->emr_out_length_used = 0; - emrp->emr_rc = efx_mcdi_request_errcode(errcode); + if ((emrp == NULL) || (handlep == NULL)) { + rc = EINVAL; + goto fail1; + } + if ((emrp->emr_rc != 0) && + (emrp->emr_err_code == MC_CMD_ERR_PROXY_PENDING)) { + *handlep = emrp->emr_proxy_handle; + rc = 0; } else { - emrp->emr_out_length_used = outlen; - emrp->emr_rc = 0; + *handlep = 0; + rc = ENOENT; } - emcop->emco_request_copyout(enp, emrp); + return (rc); - emtp->emt_ev_cpl(emtp->emt_context); +fail1: + EFSYS_PROBE1(fail1, efx_rc_t, rc); + return (rc); } void +efx_mcdi_ev_proxy_response( + __in efx_nic_t *enp, + __in unsigned int handle, + __in unsigned int status) +{ + const efx_mcdi_transport_t *emtp = enp->en_mcdi.em_emtp; + efx_rc_t rc; + + /* + * Handle results of an authorization request for a privileged MCDI + * command. If authorization was granted then we must re-issue the + * original MCDI request. If authorization failed or timed out, + * then the original MCDI request should be completed with the + * result code from this event. + */ + rc = (status == 0) ? 0 : efx_mcdi_request_errcode(status); + + emtp->emt_ev_proxy_response(emtp->emt_context, handle, rc); +} +#endif /* EFSYS_OPT_MCDI_PROXY_AUTH */ + + void efx_mcdi_ev_death( __in efx_nic_t *enp, __in int rc) { efx_mcdi_iface_t *emip = &(enp->en_mcdi.em_emip); const efx_mcdi_transport_t *emtp = enp->en_mcdi.em_emtp; efx_mcdi_req_t *emrp = NULL; boolean_t ev_cpl; int state; /* * The MCDI request (if there is one) has been terminated, either * by a BADASSERT or REBOOT event. * * If there is an outstanding event-completed MCDI operation, then we * will never receive the completion event (because both MCDI * completions and BADASSERT events are sent to the same evq). So * complete this MCDI op. * * This function might run in parallel with efx_mcdi_request_poll() * for poll completed mcdi requests, and also with * efx_mcdi_request_start() for post-watchdog completions. */ EFSYS_LOCK(enp->en_eslp, state); emrp = emip->emi_pending_req; ev_cpl = emip->emi_ev_cpl; if (emrp != NULL && emip->emi_ev_cpl) { emip->emi_pending_req = NULL; emrp->emr_out_length_used = 0; emrp->emr_rc = rc; ++emip->emi_aborted; } /* * Since we're running in parallel with a request, consume the * status word before dropping the lock. */ if (rc == EIO || rc == EINTR) { EFSYS_SPIN(EFX_MCDI_STATUS_SLEEP_US); (void) efx_mcdi_poll_reboot(enp); emip->emi_new_epoch = B_TRUE; } EFSYS_UNLOCK(enp->en_eslp, state); efx_mcdi_raise_exception(enp, emrp, rc); if (emrp != NULL && ev_cpl) emtp->emt_ev_cpl(emtp->emt_context); } __checkReturn efx_rc_t efx_mcdi_version( __in efx_nic_t *enp, __out_ecount_opt(4) uint16_t versionp[4], __out_opt uint32_t *buildp, __out_opt efx_mcdi_boot_t *statusp) { efx_mcdi_req_t req; uint8_t payload[MAX(MAX(MC_CMD_GET_VERSION_IN_LEN, MC_CMD_GET_VERSION_OUT_LEN), MAX(MC_CMD_GET_BOOT_STATUS_IN_LEN, MC_CMD_GET_BOOT_STATUS_OUT_LEN))]; efx_word_t *ver_words; uint16_t version[4]; uint32_t build; efx_mcdi_boot_t status; efx_rc_t rc; EFSYS_ASSERT3U(enp->en_features, &, EFX_FEATURE_MCDI); (void) memset(payload, 0, sizeof (payload)); req.emr_cmd = MC_CMD_GET_VERSION; req.emr_in_buf = payload; req.emr_in_length = MC_CMD_GET_VERSION_IN_LEN; req.emr_out_buf = payload; req.emr_out_length = MC_CMD_GET_VERSION_OUT_LEN; efx_mcdi_execute(enp, &req); if (req.emr_rc != 0) { rc = req.emr_rc; goto fail1; } /* bootrom support */ if (req.emr_out_length_used == MC_CMD_GET_VERSION_V0_OUT_LEN) { version[0] = version[1] = version[2] = version[3] = 0; build = MCDI_OUT_DWORD(req, GET_VERSION_OUT_FIRMWARE); goto version; } if (req.emr_out_length_used < MC_CMD_GET_VERSION_OUT_LEN) { rc = EMSGSIZE; goto fail2; } ver_words = MCDI_OUT2(req, efx_word_t, GET_VERSION_OUT_VERSION); version[0] = EFX_WORD_FIELD(ver_words[0], EFX_WORD_0); version[1] = EFX_WORD_FIELD(ver_words[1], EFX_WORD_0); version[2] = EFX_WORD_FIELD(ver_words[2], EFX_WORD_0); version[3] = EFX_WORD_FIELD(ver_words[3], EFX_WORD_0); build = MCDI_OUT_DWORD(req, GET_VERSION_OUT_FIRMWARE); version: /* The bootrom doesn't understand BOOT_STATUS */ if (MC_FW_VERSION_IS_BOOTLOADER(build)) { status = EFX_MCDI_BOOT_ROM; goto out; } (void) memset(payload, 0, sizeof (payload)); req.emr_cmd = MC_CMD_GET_BOOT_STATUS; req.emr_in_buf = payload; req.emr_in_length = MC_CMD_GET_BOOT_STATUS_IN_LEN; req.emr_out_buf = payload; req.emr_out_length = MC_CMD_GET_BOOT_STATUS_OUT_LEN; efx_mcdi_execute_quiet(enp, &req); if (req.emr_rc == EACCES) { /* Unprivileged functions cannot access BOOT_STATUS */ status = EFX_MCDI_BOOT_PRIMARY; version[0] = version[1] = version[2] = version[3] = 0; build = 0; goto out; } if (req.emr_rc != 0) { rc = req.emr_rc; goto fail3; } if (req.emr_out_length_used < MC_CMD_GET_BOOT_STATUS_OUT_LEN) { rc = EMSGSIZE; goto fail4; } if (MCDI_OUT_DWORD_FIELD(req, GET_BOOT_STATUS_OUT_FLAGS, GET_BOOT_STATUS_OUT_FLAGS_PRIMARY)) status = EFX_MCDI_BOOT_PRIMARY; else status = EFX_MCDI_BOOT_SECONDARY; out: if (versionp != NULL) memcpy(versionp, version, sizeof (version)); if (buildp != NULL) *buildp = build; if (statusp != NULL) *statusp = status; return (0); fail4: EFSYS_PROBE(fail4); fail3: EFSYS_PROBE(fail3); fail2: EFSYS_PROBE(fail2); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } static __checkReturn efx_rc_t efx_mcdi_do_reboot( __in efx_nic_t *enp, __in boolean_t after_assertion) { uint8_t payload[MAX(MC_CMD_REBOOT_IN_LEN, MC_CMD_REBOOT_OUT_LEN)]; efx_mcdi_req_t req; efx_rc_t rc; /* * We could require the caller to have caused en_mod_flags=0 to * call this function. This doesn't help the other port though, * who's about to get the MC ripped out from underneath them. * Since they have to cope with the subsequent fallout of MCDI * failures, we should as well. */ EFSYS_ASSERT3U(enp->en_magic, ==, EFX_NIC_MAGIC); (void) memset(payload, 0, sizeof (payload)); req.emr_cmd = MC_CMD_REBOOT; req.emr_in_buf = payload; req.emr_in_length = MC_CMD_REBOOT_IN_LEN; req.emr_out_buf = payload; req.emr_out_length = MC_CMD_REBOOT_OUT_LEN; MCDI_IN_SET_DWORD(req, REBOOT_IN_FLAGS, (after_assertion ? MC_CMD_REBOOT_FLAGS_AFTER_ASSERTION : 0)); efx_mcdi_execute_quiet(enp, &req); if (req.emr_rc == EACCES) { /* Unprivileged functions cannot reboot the MC. */ goto out; } /* A successful reboot request returns EIO. */ if (req.emr_rc != 0 && req.emr_rc != EIO) { rc = req.emr_rc; goto fail1; } out: return (0); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } __checkReturn efx_rc_t efx_mcdi_reboot( __in efx_nic_t *enp) { return (efx_mcdi_do_reboot(enp, B_FALSE)); } __checkReturn efx_rc_t efx_mcdi_exit_assertion_handler( __in efx_nic_t *enp) { return (efx_mcdi_do_reboot(enp, B_TRUE)); } __checkReturn efx_rc_t efx_mcdi_read_assertion( __in efx_nic_t *enp) { efx_mcdi_req_t req; uint8_t payload[MAX(MC_CMD_GET_ASSERTS_IN_LEN, MC_CMD_GET_ASSERTS_OUT_LEN)]; const char *reason; unsigned int flags; unsigned int index; unsigned int ofst; int retry; efx_rc_t rc; /* * Before we attempt to chat to the MC, we should verify that the MC * isn't in it's assertion handler, either due to a previous reboot, * or because we're reinitializing due to an eec_exception(). * * Use GET_ASSERTS to read any assertion state that may be present. * Retry this command twice. Once because a boot-time assertion failure * might cause the 1st MCDI request to fail. And once again because * we might race with efx_mcdi_exit_assertion_handler() running on * partner port(s) on the same NIC. */ retry = 2; do { (void) memset(payload, 0, sizeof (payload)); req.emr_cmd = MC_CMD_GET_ASSERTS; req.emr_in_buf = payload; req.emr_in_length = MC_CMD_GET_ASSERTS_IN_LEN; req.emr_out_buf = payload; req.emr_out_length = MC_CMD_GET_ASSERTS_OUT_LEN; MCDI_IN_SET_DWORD(req, GET_ASSERTS_IN_CLEAR, 1); efx_mcdi_execute_quiet(enp, &req); } while ((req.emr_rc == EINTR || req.emr_rc == EIO) && retry-- > 0); if (req.emr_rc != 0) { if (req.emr_rc == EACCES) { /* Unprivileged functions cannot clear assertions. */ goto out; } rc = req.emr_rc; goto fail1; } if (req.emr_out_length_used < MC_CMD_GET_ASSERTS_OUT_LEN) { rc = EMSGSIZE; goto fail2; } /* Print out any assertion state recorded */ flags = MCDI_OUT_DWORD(req, GET_ASSERTS_OUT_GLOBAL_FLAGS); if (flags == MC_CMD_GET_ASSERTS_FLAGS_NO_FAILS) return (0); reason = (flags == MC_CMD_GET_ASSERTS_FLAGS_SYS_FAIL) ? "system-level assertion" : (flags == MC_CMD_GET_ASSERTS_FLAGS_THR_FAIL) ? "thread-level assertion" : (flags == MC_CMD_GET_ASSERTS_FLAGS_WDOG_FIRED) ? "watchdog reset" : (flags == MC_CMD_GET_ASSERTS_FLAGS_ADDR_TRAP) ? "illegal address trap" : "unknown assertion"; EFSYS_PROBE3(mcpu_assertion, const char *, reason, unsigned int, MCDI_OUT_DWORD(req, GET_ASSERTS_OUT_SAVED_PC_OFFS), unsigned int, MCDI_OUT_DWORD(req, GET_ASSERTS_OUT_THREAD_OFFS)); /* Print out the registers (r1 ... r31) */ ofst = MC_CMD_GET_ASSERTS_OUT_GP_REGS_OFFS_OFST; for (index = 1; index < 1 + MC_CMD_GET_ASSERTS_OUT_GP_REGS_OFFS_NUM; index++) { EFSYS_PROBE2(mcpu_register, unsigned int, index, unsigned int, EFX_DWORD_FIELD(*MCDI_OUT(req, efx_dword_t, ofst), EFX_DWORD_0)); ofst += sizeof (efx_dword_t); } EFSYS_ASSERT(ofst <= MC_CMD_GET_ASSERTS_OUT_LEN); out: return (0); fail2: EFSYS_PROBE(fail2); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } /* * Internal routines for for specific MCDI requests. */ __checkReturn efx_rc_t efx_mcdi_drv_attach( __in efx_nic_t *enp, __in boolean_t attach) { efx_nic_cfg_t *encp = &(enp->en_nic_cfg); efx_mcdi_req_t req; uint8_t payload[MAX(MC_CMD_DRV_ATTACH_IN_LEN, MC_CMD_DRV_ATTACH_EXT_OUT_LEN)]; uint32_t flags; efx_rc_t rc; (void) memset(payload, 0, sizeof (payload)); req.emr_cmd = MC_CMD_DRV_ATTACH; req.emr_in_buf = payload; req.emr_in_length = MC_CMD_DRV_ATTACH_IN_LEN; req.emr_out_buf = payload; req.emr_out_length = MC_CMD_DRV_ATTACH_EXT_OUT_LEN; /* * Use DONT_CARE for the datapath firmware type to ensure that the * driver can attach to an unprivileged function. The datapath firmware * type to use is controlled by the 'sfboot' utility. */ MCDI_IN_SET_DWORD(req, DRV_ATTACH_IN_NEW_STATE, attach ? 1 : 0); MCDI_IN_SET_DWORD(req, DRV_ATTACH_IN_UPDATE, 1); MCDI_IN_SET_DWORD(req, DRV_ATTACH_IN_FIRMWARE_ID, MC_CMD_FW_DONT_CARE); efx_mcdi_execute(enp, &req); if (req.emr_rc != 0) { rc = req.emr_rc; goto fail1; } if (req.emr_out_length_used < MC_CMD_DRV_ATTACH_OUT_LEN) { rc = EMSGSIZE; goto fail2; } if (attach == B_FALSE) { flags = 0; } else if (enp->en_family == EFX_FAMILY_SIENA) { efx_mcdi_iface_t *emip = &(enp->en_mcdi.em_emip); /* Create synthetic privileges for Siena functions */ flags = EFX_NIC_FUNC_LINKCTRL | EFX_NIC_FUNC_TRUSTED; if (emip->emi_port == 1) flags |= EFX_NIC_FUNC_PRIMARY; } else { EFX_STATIC_ASSERT(EFX_NIC_FUNC_PRIMARY == (1u << MC_CMD_DRV_ATTACH_EXT_OUT_FLAG_PRIMARY)); EFX_STATIC_ASSERT(EFX_NIC_FUNC_LINKCTRL == (1u << MC_CMD_DRV_ATTACH_EXT_OUT_FLAG_LINKCTRL)); EFX_STATIC_ASSERT(EFX_NIC_FUNC_TRUSTED == (1u << MC_CMD_DRV_ATTACH_EXT_OUT_FLAG_TRUSTED)); /* Save function privilege flags (EF10 and later) */ if (req.emr_out_length_used < MC_CMD_DRV_ATTACH_EXT_OUT_LEN) { rc = EMSGSIZE; goto fail3; } flags = MCDI_OUT_DWORD(req, DRV_ATTACH_EXT_OUT_FUNC_FLAGS); } encp->enc_func_flags = flags; return (0); fail3: EFSYS_PROBE(fail3); fail2: EFSYS_PROBE(fail2); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } __checkReturn efx_rc_t efx_mcdi_get_board_cfg( __in efx_nic_t *enp, __out_opt uint32_t *board_typep, __out_opt efx_dword_t *capabilitiesp, __out_ecount_opt(6) uint8_t mac_addrp[6]) { efx_mcdi_iface_t *emip = &(enp->en_mcdi.em_emip); efx_mcdi_req_t req; uint8_t payload[MAX(MC_CMD_GET_BOARD_CFG_IN_LEN, MC_CMD_GET_BOARD_CFG_OUT_LENMIN)]; efx_rc_t rc; (void) memset(payload, 0, sizeof (payload)); req.emr_cmd = MC_CMD_GET_BOARD_CFG; req.emr_in_buf = payload; req.emr_in_length = MC_CMD_GET_BOARD_CFG_IN_LEN; req.emr_out_buf = payload; req.emr_out_length = MC_CMD_GET_BOARD_CFG_OUT_LENMIN; efx_mcdi_execute(enp, &req); if (req.emr_rc != 0) { rc = req.emr_rc; goto fail1; } if (req.emr_out_length_used < MC_CMD_GET_BOARD_CFG_OUT_LENMIN) { rc = EMSGSIZE; goto fail2; } if (mac_addrp != NULL) { uint8_t *addrp; if (emip->emi_port == 1) { addrp = MCDI_OUT2(req, uint8_t, GET_BOARD_CFG_OUT_MAC_ADDR_BASE_PORT0); } else if (emip->emi_port == 2) { addrp = MCDI_OUT2(req, uint8_t, GET_BOARD_CFG_OUT_MAC_ADDR_BASE_PORT1); } else { rc = EINVAL; goto fail3; } EFX_MAC_ADDR_COPY(mac_addrp, addrp); } if (capabilitiesp != NULL) { if (emip->emi_port == 1) { *capabilitiesp = *MCDI_OUT2(req, efx_dword_t, GET_BOARD_CFG_OUT_CAPABILITIES_PORT0); } else if (emip->emi_port == 2) { *capabilitiesp = *MCDI_OUT2(req, efx_dword_t, GET_BOARD_CFG_OUT_CAPABILITIES_PORT1); } else { rc = EINVAL; goto fail4; } } if (board_typep != NULL) { *board_typep = MCDI_OUT_DWORD(req, GET_BOARD_CFG_OUT_BOARD_TYPE); } return (0); fail4: EFSYS_PROBE(fail4); fail3: EFSYS_PROBE(fail3); fail2: EFSYS_PROBE(fail2); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } __checkReturn efx_rc_t efx_mcdi_get_resource_limits( __in efx_nic_t *enp, __out_opt uint32_t *nevqp, __out_opt uint32_t *nrxqp, __out_opt uint32_t *ntxqp) { efx_mcdi_req_t req; uint8_t payload[MAX(MC_CMD_GET_RESOURCE_LIMITS_IN_LEN, MC_CMD_GET_RESOURCE_LIMITS_OUT_LEN)]; efx_rc_t rc; (void) memset(payload, 0, sizeof (payload)); req.emr_cmd = MC_CMD_GET_RESOURCE_LIMITS; req.emr_in_buf = payload; req.emr_in_length = MC_CMD_GET_RESOURCE_LIMITS_IN_LEN; req.emr_out_buf = payload; req.emr_out_length = MC_CMD_GET_RESOURCE_LIMITS_OUT_LEN; efx_mcdi_execute(enp, &req); if (req.emr_rc != 0) { rc = req.emr_rc; goto fail1; } if (req.emr_out_length_used < MC_CMD_GET_RESOURCE_LIMITS_OUT_LEN) { rc = EMSGSIZE; goto fail2; } if (nevqp != NULL) *nevqp = MCDI_OUT_DWORD(req, GET_RESOURCE_LIMITS_OUT_EVQ); if (nrxqp != NULL) *nrxqp = MCDI_OUT_DWORD(req, GET_RESOURCE_LIMITS_OUT_RXQ); if (ntxqp != NULL) *ntxqp = MCDI_OUT_DWORD(req, GET_RESOURCE_LIMITS_OUT_TXQ); return (0); fail2: EFSYS_PROBE(fail2); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } __checkReturn efx_rc_t efx_mcdi_get_phy_cfg( __in efx_nic_t *enp) { efx_port_t *epp = &(enp->en_port); efx_nic_cfg_t *encp = &(enp->en_nic_cfg); efx_mcdi_req_t req; uint8_t payload[MAX(MC_CMD_GET_PHY_CFG_IN_LEN, MC_CMD_GET_PHY_CFG_OUT_LEN)]; efx_rc_t rc; (void) memset(payload, 0, sizeof (payload)); req.emr_cmd = MC_CMD_GET_PHY_CFG; req.emr_in_buf = payload; req.emr_in_length = MC_CMD_GET_PHY_CFG_IN_LEN; req.emr_out_buf = payload; req.emr_out_length = MC_CMD_GET_PHY_CFG_OUT_LEN; efx_mcdi_execute(enp, &req); if (req.emr_rc != 0) { rc = req.emr_rc; goto fail1; } if (req.emr_out_length_used < MC_CMD_GET_PHY_CFG_OUT_LEN) { rc = EMSGSIZE; goto fail2; } encp->enc_phy_type = MCDI_OUT_DWORD(req, GET_PHY_CFG_OUT_TYPE); #if EFSYS_OPT_NAMES (void) strncpy(encp->enc_phy_name, MCDI_OUT2(req, char, GET_PHY_CFG_OUT_NAME), MIN(sizeof (encp->enc_phy_name) - 1, MC_CMD_GET_PHY_CFG_OUT_NAME_LEN)); #endif /* EFSYS_OPT_NAMES */ (void) memset(encp->enc_phy_revision, 0, sizeof (encp->enc_phy_revision)); memcpy(encp->enc_phy_revision, MCDI_OUT2(req, char, GET_PHY_CFG_OUT_REVISION), MIN(sizeof (encp->enc_phy_revision) - 1, MC_CMD_GET_PHY_CFG_OUT_REVISION_LEN)); #if EFSYS_OPT_PHY_LED_CONTROL encp->enc_led_mask = ((1 << EFX_PHY_LED_DEFAULT) | (1 << EFX_PHY_LED_OFF) | (1 << EFX_PHY_LED_ON)); #endif /* EFSYS_OPT_PHY_LED_CONTROL */ #if EFSYS_OPT_PHY_PROPS encp->enc_phy_nprops = 0; #endif /* EFSYS_OPT_PHY_PROPS */ /* Get the media type of the fixed port, if recognised. */ EFX_STATIC_ASSERT(MC_CMD_MEDIA_XAUI == EFX_PHY_MEDIA_XAUI); EFX_STATIC_ASSERT(MC_CMD_MEDIA_CX4 == EFX_PHY_MEDIA_CX4); EFX_STATIC_ASSERT(MC_CMD_MEDIA_KX4 == EFX_PHY_MEDIA_KX4); EFX_STATIC_ASSERT(MC_CMD_MEDIA_XFP == EFX_PHY_MEDIA_XFP); EFX_STATIC_ASSERT(MC_CMD_MEDIA_SFP_PLUS == EFX_PHY_MEDIA_SFP_PLUS); EFX_STATIC_ASSERT(MC_CMD_MEDIA_BASE_T == EFX_PHY_MEDIA_BASE_T); EFX_STATIC_ASSERT(MC_CMD_MEDIA_QSFP_PLUS == EFX_PHY_MEDIA_QSFP_PLUS); epp->ep_fixed_port_type = MCDI_OUT_DWORD(req, GET_PHY_CFG_OUT_MEDIA_TYPE); if (epp->ep_fixed_port_type >= EFX_PHY_MEDIA_NTYPES) epp->ep_fixed_port_type = EFX_PHY_MEDIA_INVALID; epp->ep_phy_cap_mask = MCDI_OUT_DWORD(req, GET_PHY_CFG_OUT_SUPPORTED_CAP); #if EFSYS_OPT_PHY_FLAGS encp->enc_phy_flags_mask = MCDI_OUT_DWORD(req, GET_PHY_CFG_OUT_FLAGS); #endif /* EFSYS_OPT_PHY_FLAGS */ encp->enc_port = (uint8_t)MCDI_OUT_DWORD(req, GET_PHY_CFG_OUT_PRT); /* Populate internal state */ encp->enc_mcdi_mdio_channel = (uint8_t)MCDI_OUT_DWORD(req, GET_PHY_CFG_OUT_CHANNEL); #if EFSYS_OPT_PHY_STATS encp->enc_mcdi_phy_stat_mask = MCDI_OUT_DWORD(req, GET_PHY_CFG_OUT_STATS_MASK); #endif /* EFSYS_OPT_PHY_STATS */ #if EFSYS_OPT_BIST encp->enc_bist_mask = 0; if (MCDI_OUT_DWORD_FIELD(req, GET_PHY_CFG_OUT_FLAGS, GET_PHY_CFG_OUT_BIST_CABLE_SHORT)) encp->enc_bist_mask |= (1 << EFX_BIST_TYPE_PHY_CABLE_SHORT); if (MCDI_OUT_DWORD_FIELD(req, GET_PHY_CFG_OUT_FLAGS, GET_PHY_CFG_OUT_BIST_CABLE_LONG)) encp->enc_bist_mask |= (1 << EFX_BIST_TYPE_PHY_CABLE_LONG); if (MCDI_OUT_DWORD_FIELD(req, GET_PHY_CFG_OUT_FLAGS, GET_PHY_CFG_OUT_BIST)) encp->enc_bist_mask |= (1 << EFX_BIST_TYPE_PHY_NORMAL); #endif /* EFSYS_OPT_BIST */ return (0); fail2: EFSYS_PROBE(fail2); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } __checkReturn efx_rc_t efx_mcdi_firmware_update_supported( __in efx_nic_t *enp, __out boolean_t *supportedp) { efx_mcdi_ops_t *emcop = enp->en_mcdi.em_emcop; efx_rc_t rc; if (emcop != NULL && emcop->emco_fw_update_supported != NULL) { if ((rc = emcop->emco_fw_update_supported(enp, supportedp)) != 0) goto fail1; } else { /* Earlier devices always supported updates */ *supportedp = B_TRUE; } return (0); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } __checkReturn efx_rc_t efx_mcdi_macaddr_change_supported( __in efx_nic_t *enp, __out boolean_t *supportedp) { efx_mcdi_ops_t *emcop = enp->en_mcdi.em_emcop; efx_rc_t rc; if (emcop != NULL && emcop->emco_macaddr_change_supported != NULL) { if ((rc = emcop->emco_macaddr_change_supported(enp, supportedp)) != 0) goto fail1; } else { /* Earlier devices always supported MAC changes */ *supportedp = B_TRUE; } return (0); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } __checkReturn efx_rc_t efx_mcdi_link_control_supported( __in efx_nic_t *enp, __out boolean_t *supportedp) { efx_mcdi_ops_t *emcop = enp->en_mcdi.em_emcop; efx_rc_t rc; if (emcop != NULL && emcop->emco_link_control_supported != NULL) { if ((rc = emcop->emco_link_control_supported(enp, supportedp)) != 0) goto fail1; } else { /* Earlier devices always supported link control */ + *supportedp = B_TRUE; + } + + return (0); + +fail1: + EFSYS_PROBE1(fail1, efx_rc_t, rc); + + return (rc); +} + + __checkReturn efx_rc_t +efx_mcdi_mac_spoofing_supported( + __in efx_nic_t *enp, + __out boolean_t *supportedp) +{ + efx_mcdi_ops_t *emcop = enp->en_mcdi.em_emcop; + efx_rc_t rc; + + if (emcop != NULL && emcop->emco_mac_spoofing_supported != NULL) { + if ((rc = emcop->emco_mac_spoofing_supported(enp, supportedp)) + != 0) + goto fail1; + } else { + /* Earlier devices always supported MAC spoofing */ *supportedp = B_TRUE; } return (0); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } #if EFSYS_OPT_BIST #if EFSYS_OPT_HUNTINGTON /* * Enter bist offline mode. This is a fw mode which puts the NIC into a state * where memory BIST tests can be run and not much else can interfere or happen. * A reboot is required to exit this mode. */ __checkReturn efx_rc_t efx_mcdi_bist_enable_offline( __in efx_nic_t *enp) { efx_mcdi_req_t req; efx_rc_t rc; EFX_STATIC_ASSERT(MC_CMD_ENABLE_OFFLINE_BIST_IN_LEN == 0); EFX_STATIC_ASSERT(MC_CMD_ENABLE_OFFLINE_BIST_OUT_LEN == 0); req.emr_cmd = MC_CMD_ENABLE_OFFLINE_BIST; req.emr_in_buf = NULL; req.emr_in_length = 0; req.emr_out_buf = NULL; req.emr_out_length = 0; efx_mcdi_execute(enp, &req); if (req.emr_rc != 0) { rc = req.emr_rc; goto fail1; } return (0); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } #endif /* EFSYS_OPT_HUNTINGTON */ __checkReturn efx_rc_t efx_mcdi_bist_start( __in efx_nic_t *enp, __in efx_bist_type_t type) { efx_mcdi_req_t req; uint8_t payload[MAX(MC_CMD_START_BIST_IN_LEN, MC_CMD_START_BIST_OUT_LEN)]; efx_rc_t rc; (void) memset(payload, 0, sizeof (payload)); req.emr_cmd = MC_CMD_START_BIST; req.emr_in_buf = payload; req.emr_in_length = MC_CMD_START_BIST_IN_LEN; req.emr_out_buf = payload; req.emr_out_length = MC_CMD_START_BIST_OUT_LEN; switch (type) { case EFX_BIST_TYPE_PHY_NORMAL: MCDI_IN_SET_DWORD(req, START_BIST_IN_TYPE, MC_CMD_PHY_BIST); break; case EFX_BIST_TYPE_PHY_CABLE_SHORT: MCDI_IN_SET_DWORD(req, START_BIST_IN_TYPE, MC_CMD_PHY_BIST_CABLE_SHORT); break; case EFX_BIST_TYPE_PHY_CABLE_LONG: MCDI_IN_SET_DWORD(req, START_BIST_IN_TYPE, MC_CMD_PHY_BIST_CABLE_LONG); break; case EFX_BIST_TYPE_MC_MEM: MCDI_IN_SET_DWORD(req, START_BIST_IN_TYPE, MC_CMD_MC_MEM_BIST); break; case EFX_BIST_TYPE_SAT_MEM: MCDI_IN_SET_DWORD(req, START_BIST_IN_TYPE, MC_CMD_PORT_MEM_BIST); break; case EFX_BIST_TYPE_REG: MCDI_IN_SET_DWORD(req, START_BIST_IN_TYPE, MC_CMD_REG_BIST); break; default: EFSYS_ASSERT(0); } efx_mcdi_execute(enp, &req); if (req.emr_rc != 0) { rc = req.emr_rc; goto fail1; } return (0); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } #endif /* EFSYS_OPT_BIST */ /* Enable logging of some events (e.g. link state changes) */ __checkReturn efx_rc_t efx_mcdi_log_ctrl( __in efx_nic_t *enp) { efx_mcdi_req_t req; uint8_t payload[MAX(MC_CMD_LOG_CTRL_IN_LEN, MC_CMD_LOG_CTRL_OUT_LEN)]; efx_rc_t rc; (void) memset(payload, 0, sizeof (payload)); req.emr_cmd = MC_CMD_LOG_CTRL; req.emr_in_buf = payload; req.emr_in_length = MC_CMD_LOG_CTRL_IN_LEN; req.emr_out_buf = payload; req.emr_out_length = MC_CMD_LOG_CTRL_OUT_LEN; MCDI_IN_SET_DWORD(req, LOG_CTRL_IN_LOG_DEST, MC_CMD_LOG_CTRL_IN_LOG_DEST_EVQ); MCDI_IN_SET_DWORD(req, LOG_CTRL_IN_LOG_DEST_EVQ, 0); efx_mcdi_execute(enp, &req); if (req.emr_rc != 0) { rc = req.emr_rc; goto fail1; } return (0); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } #if EFSYS_OPT_MAC_STATS typedef enum efx_stats_action_e { EFX_STATS_CLEAR, EFX_STATS_UPLOAD, EFX_STATS_ENABLE_NOEVENTS, EFX_STATS_ENABLE_EVENTS, EFX_STATS_DISABLE, } efx_stats_action_t; static __checkReturn efx_rc_t efx_mcdi_mac_stats( __in efx_nic_t *enp, __in_opt efsys_mem_t *esmp, __in efx_stats_action_t action) { efx_mcdi_req_t req; uint8_t payload[MAX(MC_CMD_MAC_STATS_IN_LEN, MC_CMD_MAC_STATS_OUT_DMA_LEN)]; int clear = (action == EFX_STATS_CLEAR); int upload = (action == EFX_STATS_UPLOAD); int enable = (action == EFX_STATS_ENABLE_NOEVENTS); int events = (action == EFX_STATS_ENABLE_EVENTS); int disable = (action == EFX_STATS_DISABLE); efx_rc_t rc; (void) memset(payload, 0, sizeof (payload)); req.emr_cmd = MC_CMD_MAC_STATS; req.emr_in_buf = payload; req.emr_in_length = MC_CMD_MAC_STATS_IN_LEN; req.emr_out_buf = payload; req.emr_out_length = MC_CMD_MAC_STATS_OUT_DMA_LEN; MCDI_IN_POPULATE_DWORD_6(req, MAC_STATS_IN_CMD, MAC_STATS_IN_DMA, upload, MAC_STATS_IN_CLEAR, clear, MAC_STATS_IN_PERIODIC_CHANGE, enable | events | disable, MAC_STATS_IN_PERIODIC_ENABLE, enable | events, MAC_STATS_IN_PERIODIC_NOEVENT, !events, MAC_STATS_IN_PERIOD_MS, (enable | events) ? 1000: 0); if (esmp != NULL) { int bytes = MC_CMD_MAC_NSTATS * sizeof (uint64_t); EFX_STATIC_ASSERT(MC_CMD_MAC_NSTATS * sizeof (uint64_t) <= EFX_MAC_STATS_SIZE); MCDI_IN_SET_DWORD(req, MAC_STATS_IN_DMA_ADDR_LO, EFSYS_MEM_ADDR(esmp) & 0xffffffff); MCDI_IN_SET_DWORD(req, MAC_STATS_IN_DMA_ADDR_HI, EFSYS_MEM_ADDR(esmp) >> 32); MCDI_IN_SET_DWORD(req, MAC_STATS_IN_DMA_LEN, bytes); } else { EFSYS_ASSERT(!upload && !enable && !events); } /* * NOTE: Do not use EVB_PORT_ID_ASSIGNED when disabling periodic stats, * as this may fail (and leave periodic DMA enabled) if the * vadapter has already been deleted. */ MCDI_IN_SET_DWORD(req, MAC_STATS_IN_PORT_ID, (disable ? EVB_PORT_ID_NULL : enp->en_vport_id)); efx_mcdi_execute(enp, &req); if (req.emr_rc != 0) { /* EF10: Expect ENOENT if no DMA queues are initialised */ if ((req.emr_rc != ENOENT) || (enp->en_rx_qcount + enp->en_tx_qcount != 0)) { rc = req.emr_rc; goto fail1; } } return (0); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } __checkReturn efx_rc_t efx_mcdi_mac_stats_clear( __in efx_nic_t *enp) { efx_rc_t rc; if ((rc = efx_mcdi_mac_stats(enp, NULL, EFX_STATS_CLEAR)) != 0) goto fail1; return (0); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } __checkReturn efx_rc_t efx_mcdi_mac_stats_upload( __in efx_nic_t *enp, __in efsys_mem_t *esmp) { efx_rc_t rc; /* * The MC DMAs aggregate statistics for our convenience, so we can * avoid having to pull the statistics buffer into the cache to * maintain cumulative statistics. */ if ((rc = efx_mcdi_mac_stats(enp, esmp, EFX_STATS_UPLOAD)) != 0) goto fail1; return (0); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } __checkReturn efx_rc_t efx_mcdi_mac_stats_periodic( __in efx_nic_t *enp, __in efsys_mem_t *esmp, __in uint16_t period, __in boolean_t events) { efx_rc_t rc; /* * The MC DMAs aggregate statistics for our convenience, so we can * avoid having to pull the statistics buffer into the cache to * maintain cumulative statistics. * Huntington uses a fixed 1sec period, so use that on Siena too. */ if (period == 0) rc = efx_mcdi_mac_stats(enp, NULL, EFX_STATS_DISABLE); else if (events) rc = efx_mcdi_mac_stats(enp, esmp, EFX_STATS_ENABLE_EVENTS); else rc = efx_mcdi_mac_stats(enp, esmp, EFX_STATS_ENABLE_NOEVENTS); if (rc != 0) goto fail1; return (0); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } #endif /* EFSYS_OPT_MAC_STATS */ #if EFSYS_OPT_HUNTINGTON /* * This function returns the pf and vf number of a function. If it is a pf the * vf number is 0xffff. The vf number is the index of the vf on that * function. So if you have 3 vfs on pf 0 the 3 vfs will return (pf=0,vf=0), * (pf=0,vf=1), (pf=0,vf=2) aand the pf will return (pf=0, vf=0xffff). */ __checkReturn efx_rc_t efx_mcdi_get_function_info( __in efx_nic_t *enp, __out uint32_t *pfp, __out_opt uint32_t *vfp) { efx_mcdi_req_t req; uint8_t payload[MAX(MC_CMD_GET_FUNCTION_INFO_IN_LEN, MC_CMD_GET_FUNCTION_INFO_OUT_LEN)]; efx_rc_t rc; (void) memset(payload, 0, sizeof (payload)); req.emr_cmd = MC_CMD_GET_FUNCTION_INFO; req.emr_in_buf = payload; req.emr_in_length = MC_CMD_GET_FUNCTION_INFO_IN_LEN; req.emr_out_buf = payload; req.emr_out_length = MC_CMD_GET_FUNCTION_INFO_OUT_LEN; efx_mcdi_execute(enp, &req); if (req.emr_rc != 0) { rc = req.emr_rc; goto fail1; } if (req.emr_out_length_used < MC_CMD_GET_FUNCTION_INFO_OUT_LEN) { rc = EMSGSIZE; goto fail2; } *pfp = MCDI_OUT_DWORD(req, GET_FUNCTION_INFO_OUT_PF); if (vfp != NULL) *vfp = MCDI_OUT_DWORD(req, GET_FUNCTION_INFO_OUT_VF); return (0); fail2: EFSYS_PROBE(fail2); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } __checkReturn efx_rc_t efx_mcdi_privilege_mask( __in efx_nic_t *enp, __in uint32_t pf, __in uint32_t vf, __out uint32_t *maskp) { efx_mcdi_req_t req; uint8_t payload[MAX(MC_CMD_PRIVILEGE_MASK_IN_LEN, MC_CMD_PRIVILEGE_MASK_OUT_LEN)]; efx_rc_t rc; (void) memset(payload, 0, sizeof (payload)); req.emr_cmd = MC_CMD_PRIVILEGE_MASK; req.emr_in_buf = payload; req.emr_in_length = MC_CMD_PRIVILEGE_MASK_IN_LEN; req.emr_out_buf = payload; req.emr_out_length = MC_CMD_PRIVILEGE_MASK_OUT_LEN; MCDI_IN_POPULATE_DWORD_2(req, PRIVILEGE_MASK_IN_FUNCTION, PRIVILEGE_MASK_IN_FUNCTION_PF, pf, PRIVILEGE_MASK_IN_FUNCTION_VF, vf); efx_mcdi_execute(enp, &req); if (req.emr_rc != 0) { rc = req.emr_rc; goto fail1; } if (req.emr_out_length_used < MC_CMD_PRIVILEGE_MASK_OUT_LEN) { rc = EMSGSIZE; goto fail2; } *maskp = MCDI_OUT_DWORD(req, PRIVILEGE_MASK_OUT_OLD_MASK); return (0); fail2: EFSYS_PROBE(fail2); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } #endif /* EFSYS_OPT_HUNTINGTON */ __checkReturn efx_rc_t efx_mcdi_set_workaround( __in efx_nic_t *enp, __in uint32_t type, __in boolean_t enabled, __out_opt uint32_t *flagsp) { efx_mcdi_req_t req; uint8_t payload[MAX(MC_CMD_WORKAROUND_IN_LEN, MC_CMD_WORKAROUND_EXT_OUT_LEN)]; efx_rc_t rc; (void) memset(payload, 0, sizeof (payload)); req.emr_cmd = MC_CMD_WORKAROUND; req.emr_in_buf = payload; req.emr_in_length = MC_CMD_WORKAROUND_IN_LEN; req.emr_out_buf = payload; req.emr_out_length = MC_CMD_WORKAROUND_OUT_LEN; MCDI_IN_SET_DWORD(req, WORKAROUND_IN_TYPE, type); MCDI_IN_SET_DWORD(req, WORKAROUND_IN_ENABLED, enabled ? 1 : 0); efx_mcdi_execute_quiet(enp, &req); if (req.emr_rc != 0) { rc = req.emr_rc; goto fail1; } if (flagsp != NULL) { if (req.emr_out_length_used >= MC_CMD_WORKAROUND_EXT_OUT_LEN) *flagsp = MCDI_OUT_DWORD(req, WORKAROUND_EXT_OUT_FLAGS); else *flagsp = 0; } return (0); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } __checkReturn efx_rc_t efx_mcdi_get_workarounds( __in efx_nic_t *enp, __out_opt uint32_t *implementedp, __out_opt uint32_t *enabledp) { efx_mcdi_req_t req; uint8_t payload[MC_CMD_GET_WORKAROUNDS_OUT_LEN]; efx_rc_t rc; (void) memset(payload, 0, sizeof (payload)); req.emr_cmd = MC_CMD_GET_WORKAROUNDS; req.emr_in_buf = NULL; req.emr_in_length = 0; req.emr_out_buf = payload; req.emr_out_length = MC_CMD_GET_WORKAROUNDS_OUT_LEN; efx_mcdi_execute(enp, &req); if (req.emr_rc != 0) { rc = req.emr_rc; goto fail1; } if (implementedp != NULL) { *implementedp = MCDI_OUT_DWORD(req, GET_WORKAROUNDS_OUT_IMPLEMENTED); } if (enabledp != NULL) { *enabledp = MCDI_OUT_DWORD(req, GET_WORKAROUNDS_OUT_ENABLED); } return (0); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } #endif /* EFSYS_OPT_MCDI */ Index: user/ngie/more-tests2/sys/dev/sfxge/common/efx_mcdi.h =================================================================== --- user/ngie/more-tests2/sys/dev/sfxge/common/efx_mcdi.h (revision 292053) +++ user/ngie/more-tests2/sys/dev/sfxge/common/efx_mcdi.h (revision 292054) @@ -1,362 +1,398 @@ /*- * Copyright (c) 2009-2015 Solarflare Communications Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are met: * * 1. Redistributions of source code must retain the above copyright notice, * this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright notice, * this list of conditions and the following disclaimer in the documentation * and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR * OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, * EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * The views and conclusions contained in the software and documentation are * those of the authors and should not be interpreted as representing official * policies, either expressed or implied, of the FreeBSD Project. * * $FreeBSD$ */ #ifndef _SYS_EFX_MCDI_H #define _SYS_EFX_MCDI_H #include "efx.h" #include "efx_regs.h" #include "efx_regs_mcdi.h" #ifdef __cplusplus extern "C" { #endif /* * A reboot/assertion causes the MCDI status word to be set after the * command word is set or a REBOOT event is sent. If we notice a reboot * via these mechanisms then wait 10ms for the status word to be set. */ #define EFX_MCDI_STATUS_SLEEP_US 10000 struct efx_mcdi_req_s { boolean_t emr_quiet; /* Inputs: Command #, input buffer and length */ unsigned int emr_cmd; uint8_t *emr_in_buf; size_t emr_in_length; /* Outputs: retcode, buffer, length, and length used*/ int emr_rc; uint8_t *emr_out_buf; size_t emr_out_length; size_t emr_out_length_used; + /* Internals: low level transport details */ + unsigned int emr_err_code; + unsigned int emr_err_arg; +#if EFSYS_OPT_MCDI_PROXY_AUTH + uint32_t emr_proxy_handle; +#endif }; typedef struct efx_mcdi_iface_s { unsigned int emi_port; unsigned int emi_seq; efx_mcdi_req_t *emi_pending_req; boolean_t emi_ev_cpl; boolean_t emi_new_epoch; int emi_aborted; uint32_t emi_poll_cnt; uint32_t emi_mc_reboot_status; } efx_mcdi_iface_t; extern void efx_mcdi_execute( __in efx_nic_t *enp, __inout efx_mcdi_req_t *emrp); extern void efx_mcdi_execute_quiet( __in efx_nic_t *enp, __inout efx_mcdi_req_t *emrp); + extern void +efx_mcdi_read_response_header( + __in efx_nic_t *enp, + __inout efx_mcdi_req_t *emrp); + extern void efx_mcdi_ev_cpl( __in efx_nic_t *enp, __in unsigned int seq, __in unsigned int outlen, __in int errcode); +#if EFSYS_OPT_MCDI_PROXY_AUTH +extern __checkReturn efx_rc_t +efx_mcdi_get_proxy_handle( + __in efx_nic_t *enp, + __in efx_mcdi_req_t *emrp, + __out uint32_t *handlep); + extern void +efx_mcdi_ev_proxy_response( + __in efx_nic_t *enp, + __in unsigned int handle, + __in unsigned int status); +#endif + +extern void efx_mcdi_ev_death( __in efx_nic_t *enp, __in int rc); extern __checkReturn efx_rc_t efx_mcdi_request_errcode( __in unsigned int err); extern void efx_mcdi_raise_exception( __in efx_nic_t *enp, __in_opt efx_mcdi_req_t *emrp, __in int rc); typedef enum efx_mcdi_boot_e { EFX_MCDI_BOOT_PRIMARY, EFX_MCDI_BOOT_SECONDARY, EFX_MCDI_BOOT_ROM, } efx_mcdi_boot_t; extern __checkReturn efx_rc_t efx_mcdi_version( __in efx_nic_t *enp, __out_ecount_opt(4) uint16_t versionp[4], __out_opt uint32_t *buildp, __out_opt efx_mcdi_boot_t *statusp); extern __checkReturn efx_rc_t efx_mcdi_read_assertion( __in efx_nic_t *enp); extern __checkReturn efx_rc_t efx_mcdi_exit_assertion_handler( __in efx_nic_t *enp); extern __checkReturn efx_rc_t efx_mcdi_drv_attach( __in efx_nic_t *enp, __in boolean_t attach); extern __checkReturn efx_rc_t efx_mcdi_get_board_cfg( __in efx_nic_t *enp, __out_opt uint32_t *board_typep, __out_opt efx_dword_t *capabilitiesp, __out_ecount_opt(6) uint8_t mac_addrp[6]); extern __checkReturn efx_rc_t efx_mcdi_get_phy_cfg( __in efx_nic_t *enp); extern __checkReturn efx_rc_t efx_mcdi_firmware_update_supported( __in efx_nic_t *enp, __out boolean_t *supportedp); extern __checkReturn efx_rc_t efx_mcdi_macaddr_change_supported( __in efx_nic_t *enp, __out boolean_t *supportedp); extern __checkReturn efx_rc_t efx_mcdi_link_control_supported( __in efx_nic_t *enp, __out boolean_t *supportedp); +extern __checkReturn efx_rc_t +efx_mcdi_mac_spoofing_supported( + __in efx_nic_t *enp, + __out boolean_t *supportedp); + + #if EFSYS_OPT_BIST #if EFSYS_OPT_HUNTINGTON extern __checkReturn efx_rc_t efx_mcdi_bist_enable_offline( __in efx_nic_t *enp); #endif /* EFSYS_OPT_HUNTINGTON */ extern __checkReturn efx_rc_t efx_mcdi_bist_start( __in efx_nic_t *enp, __in efx_bist_type_t type); #endif /* EFSYS_OPT_BIST */ extern __checkReturn efx_rc_t efx_mcdi_get_resource_limits( __in efx_nic_t *enp, __out_opt uint32_t *nevqp, __out_opt uint32_t *nrxqp, __out_opt uint32_t *ntxqp); extern __checkReturn efx_rc_t efx_mcdi_log_ctrl( __in efx_nic_t *enp); extern __checkReturn efx_rc_t efx_mcdi_mac_stats_clear( __in efx_nic_t *enp); extern __checkReturn efx_rc_t efx_mcdi_mac_stats_upload( __in efx_nic_t *enp, __in efsys_mem_t *esmp); extern __checkReturn efx_rc_t efx_mcdi_mac_stats_periodic( __in efx_nic_t *enp, __in efsys_mem_t *esmp, __in uint16_t period, __in boolean_t events); #if EFSYS_OPT_LOOPBACK extern __checkReturn efx_rc_t efx_mcdi_get_loopback_modes( __in efx_nic_t *enp); #endif /* EFSYS_OPT_LOOPBACK */ #define MCDI_IN(_emr, _type, _ofst) \ ((_type *)((_emr).emr_in_buf + (_ofst))) #define MCDI_IN2(_emr, _type, _ofst) \ MCDI_IN(_emr, _type, MC_CMD_ ## _ofst ## _OFST) #define MCDI_IN_SET_BYTE(_emr, _ofst, _value) \ EFX_POPULATE_BYTE_1(*MCDI_IN2(_emr, efx_byte_t, _ofst), \ EFX_BYTE_0, _value) #define MCDI_IN_SET_WORD(_emr, _ofst, _value) \ EFX_POPULATE_WORD_1(*MCDI_IN2(_emr, efx_word_t, _ofst), \ EFX_WORD_0, _value) #define MCDI_IN_SET_DWORD(_emr, _ofst, _value) \ EFX_POPULATE_DWORD_1(*MCDI_IN2(_emr, efx_dword_t, _ofst), \ EFX_DWORD_0, _value) #define MCDI_IN_SET_DWORD_FIELD(_emr, _ofst, _field, _value) \ EFX_SET_DWORD_FIELD(*MCDI_IN2(_emr, efx_dword_t, _ofst), \ MC_CMD_ ## _field, _value) #define MCDI_IN_POPULATE_DWORD_1(_emr, _ofst, _field1, _value1) \ EFX_POPULATE_DWORD_1(*MCDI_IN2(_emr, efx_dword_t, _ofst), \ MC_CMD_ ## _field1, _value1) #define MCDI_IN_POPULATE_DWORD_2(_emr, _ofst, _field1, _value1, \ _field2, _value2) \ EFX_POPULATE_DWORD_2(*MCDI_IN2(_emr, efx_dword_t, _ofst), \ MC_CMD_ ## _field1, _value1, \ MC_CMD_ ## _field2, _value2) #define MCDI_IN_POPULATE_DWORD_3(_emr, _ofst, _field1, _value1, \ _field2, _value2, _field3, _value3) \ EFX_POPULATE_DWORD_3(*MCDI_IN2(_emr, efx_dword_t, _ofst), \ MC_CMD_ ## _field1, _value1, \ MC_CMD_ ## _field2, _value2, \ MC_CMD_ ## _field3, _value3) #define MCDI_IN_POPULATE_DWORD_4(_emr, _ofst, _field1, _value1, \ _field2, _value2, _field3, _value3, _field4, _value4) \ EFX_POPULATE_DWORD_4(*MCDI_IN2(_emr, efx_dword_t, _ofst), \ MC_CMD_ ## _field1, _value1, \ MC_CMD_ ## _field2, _value2, \ MC_CMD_ ## _field3, _value3, \ MC_CMD_ ## _field4, _value4) #define MCDI_IN_POPULATE_DWORD_5(_emr, _ofst, _field1, _value1, \ _field2, _value2, _field3, _value3, _field4, _value4, \ _field5, _value5) \ EFX_POPULATE_DWORD_5(*MCDI_IN2(_emr, efx_dword_t, _ofst), \ MC_CMD_ ## _field1, _value1, \ MC_CMD_ ## _field2, _value2, \ MC_CMD_ ## _field3, _value3, \ MC_CMD_ ## _field4, _value4, \ MC_CMD_ ## _field5, _value5) #define MCDI_IN_POPULATE_DWORD_6(_emr, _ofst, _field1, _value1, \ _field2, _value2, _field3, _value3, _field4, _value4, \ _field5, _value5, _field6, _value6) \ EFX_POPULATE_DWORD_6(*MCDI_IN2(_emr, efx_dword_t, _ofst), \ MC_CMD_ ## _field1, _value1, \ MC_CMD_ ## _field2, _value2, \ MC_CMD_ ## _field3, _value3, \ MC_CMD_ ## _field4, _value4, \ MC_CMD_ ## _field5, _value5, \ MC_CMD_ ## _field6, _value6) #define MCDI_IN_POPULATE_DWORD_7(_emr, _ofst, _field1, _value1, \ _field2, _value2, _field3, _value3, _field4, _value4, \ _field5, _value5, _field6, _value6, _field7, _value7) \ EFX_POPULATE_DWORD_7(*MCDI_IN2(_emr, efx_dword_t, _ofst), \ MC_CMD_ ## _field1, _value1, \ MC_CMD_ ## _field2, _value2, \ MC_CMD_ ## _field3, _value3, \ MC_CMD_ ## _field4, _value4, \ MC_CMD_ ## _field5, _value5, \ MC_CMD_ ## _field6, _value6, \ MC_CMD_ ## _field7, _value7) #define MCDI_IN_POPULATE_DWORD_8(_emr, _ofst, _field1, _value1, \ _field2, _value2, _field3, _value3, _field4, _value4, \ _field5, _value5, _field6, _value6, _field7, _value7, \ _field8, _value8) \ EFX_POPULATE_DWORD_8(*MCDI_IN2(_emr, efx_dword_t, _ofst), \ MC_CMD_ ## _field1, _value1, \ MC_CMD_ ## _field2, _value2, \ MC_CMD_ ## _field3, _value3, \ MC_CMD_ ## _field4, _value4, \ MC_CMD_ ## _field5, _value5, \ MC_CMD_ ## _field6, _value6, \ MC_CMD_ ## _field7, _value7, \ MC_CMD_ ## _field8, _value8) #define MCDI_IN_POPULATE_DWORD_9(_emr, _ofst, _field1, _value1, \ _field2, _value2, _field3, _value3, _field4, _value4, \ _field5, _value5, _field6, _value6, _field7, _value7, \ _field8, _value8, _field9, _value9) \ EFX_POPULATE_DWORD_9(*MCDI_IN2(_emr, efx_dword_t, _ofst), \ MC_CMD_ ## _field1, _value1, \ MC_CMD_ ## _field2, _value2, \ MC_CMD_ ## _field3, _value3, \ MC_CMD_ ## _field4, _value4, \ MC_CMD_ ## _field5, _value5, \ MC_CMD_ ## _field6, _value6, \ MC_CMD_ ## _field7, _value7, \ MC_CMD_ ## _field8, _value8, \ MC_CMD_ ## _field9, _value9) #define MCDI_IN_POPULATE_DWORD_10(_emr, _ofst, _field1, _value1, \ _field2, _value2, _field3, _value3, _field4, _value4, \ _field5, _value5, _field6, _value6, _field7, _value7, \ _field8, _value8, _field9, _value9, _field10, _value10) \ EFX_POPULATE_DWORD_10(*MCDI_IN2(_emr, efx_dword_t, _ofst), \ MC_CMD_ ## _field1, _value1, \ MC_CMD_ ## _field2, _value2, \ MC_CMD_ ## _field3, _value3, \ MC_CMD_ ## _field4, _value4, \ MC_CMD_ ## _field5, _value5, \ MC_CMD_ ## _field6, _value6, \ MC_CMD_ ## _field7, _value7, \ MC_CMD_ ## _field8, _value8, \ MC_CMD_ ## _field9, _value9, \ MC_CMD_ ## _field10, _value10) #define MCDI_OUT(_emr, _type, _ofst) \ ((_type *)((_emr).emr_out_buf + (_ofst))) #define MCDI_OUT2(_emr, _type, _ofst) \ MCDI_OUT(_emr, _type, MC_CMD_ ## _ofst ## _OFST) #define MCDI_OUT_BYTE(_emr, _ofst) \ EFX_BYTE_FIELD(*MCDI_OUT2(_emr, efx_byte_t, _ofst), \ EFX_BYTE_0) #define MCDI_OUT_WORD(_emr, _ofst) \ EFX_WORD_FIELD(*MCDI_OUT2(_emr, efx_word_t, _ofst), \ EFX_WORD_0) #define MCDI_OUT_DWORD(_emr, _ofst) \ EFX_DWORD_FIELD(*MCDI_OUT2(_emr, efx_dword_t, _ofst), \ EFX_DWORD_0) #define MCDI_OUT_DWORD_FIELD(_emr, _ofst, _field) \ EFX_DWORD_FIELD(*MCDI_OUT2(_emr, efx_dword_t, _ofst), \ MC_CMD_ ## _field) #define MCDI_EV_FIELD(_eqp, _field) \ EFX_QWORD_FIELD(*_eqp, MCDI_EVENT_ ## _field) #define MCDI_CMD_DWORD_FIELD(_edp, _field) \ EFX_DWORD_FIELD(*_edp, MC_CMD_ ## _field) + +#define EFX_MCDI_HAVE_PRIVILEGE(mask, priv) \ + (((mask) & \ + (MC_CMD_PRIVILEGE_MASK_IN_GRP_ ## priv)) == \ + (MC_CMD_PRIVILEGE_MASK_IN_GRP_ ## priv)) #ifdef __cplusplus } #endif #endif /* _SYS_EFX_MCDI_H */ Index: user/ngie/more-tests2/sys/dev/sfxge/common/efx_types.h =================================================================== --- user/ngie/more-tests2/sys/dev/sfxge/common/efx_types.h (revision 292053) +++ user/ngie/more-tests2/sys/dev/sfxge/common/efx_types.h (revision 292054) @@ -1,1649 +1,1649 @@ /*- * Copyright (c) 2007-2015 Solarflare Communications Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are met: * * 1. Redistributions of source code must retain the above copyright notice, * this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright notice, * this list of conditions and the following disclaimer in the documentation * and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR * OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, * EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * The views and conclusions contained in the software and documentation are * those of the authors and should not be interpreted as representing official * policies, either expressed or implied, of the FreeBSD Project. * * Ackowledgement to Fen Systems Ltd. * * $FreeBSD$ */ #ifndef _SYS_EFX_TYPES_H #define _SYS_EFX_TYPES_H #include "efsys.h" #ifdef __cplusplus extern "C" { #endif /* * Bitfield access * * Solarflare NICs make extensive use of bitfields up to 128 bits * wide. Since there is no native 128-bit datatype on most systems, * and since 64-bit datatypes are inefficient on 32-bit systems and * vice versa, we wrap accesses in a way that uses the most efficient * datatype. * * The NICs are PCI devices and therefore little-endian. Since most * of the quantities that we deal with are DMAed to/from host memory, * we define our datatypes (efx_oword_t, efx_qword_t and efx_dword_t) * to be little-endian. * * In the less common case of using PIO for individual register * writes, we construct the little-endian datatype in host memory and * then use non-swapping register access primitives, rather than * constructing a native-endian datatype and relying on implicit * byte-swapping. (We use a similar strategy for register reads.) */ /* * NOTE: Field definitions here and elsewhere are done in terms of a lowest * bit number (LBN) and a width. */ #define EFX_DUMMY_FIELD_LBN 0 #define EFX_DUMMY_FIELD_WIDTH 0 #define EFX_BYTE_0_LBN 0 #define EFX_BYTE_0_WIDTH 8 #define EFX_BYTE_1_LBN 8 #define EFX_BYTE_1_WIDTH 8 #define EFX_BYTE_2_LBN 16 #define EFX_BYTE_2_WIDTH 8 #define EFX_BYTE_3_LBN 24 #define EFX_BYTE_3_WIDTH 8 #define EFX_BYTE_4_LBN 32 #define EFX_BYTE_4_WIDTH 8 #define EFX_BYTE_5_LBN 40 #define EFX_BYTE_5_WIDTH 8 #define EFX_BYTE_6_LBN 48 #define EFX_BYTE_6_WIDTH 8 #define EFX_BYTE_7_LBN 56 #define EFX_BYTE_7_WIDTH 8 #define EFX_WORD_0_LBN 0 #define EFX_WORD_0_WIDTH 16 #define EFX_WORD_1_LBN 16 #define EFX_WORD_1_WIDTH 16 #define EFX_WORD_2_LBN 32 #define EFX_WORD_2_WIDTH 16 #define EFX_WORD_3_LBN 48 #define EFX_WORD_3_WIDTH 16 #define EFX_DWORD_0_LBN 0 #define EFX_DWORD_0_WIDTH 32 #define EFX_DWORD_1_LBN 32 #define EFX_DWORD_1_WIDTH 32 #define EFX_DWORD_2_LBN 64 #define EFX_DWORD_2_WIDTH 32 #define EFX_DWORD_3_LBN 96 #define EFX_DWORD_3_WIDTH 32 /* There are intentionally no EFX_QWORD_0 or EFX_QWORD_1 field definitions * here as the implementaion of EFX_QWORD_FIELD and EFX_OWORD_FIELD do not * support field widths larger than 32 bits. */ /* Specified attribute (i.e. LBN ow WIDTH) of the specified field */ #define EFX_VAL(_field, _attribute) \ _field ## _ ## _attribute /* Lowest bit number of the specified field */ #define EFX_LOW_BIT(_field) \ EFX_VAL(_field, LBN) /* Width of the specified field */ #define EFX_WIDTH(_field) \ EFX_VAL(_field, WIDTH) /* Highest bit number of the specified field */ #define EFX_HIGH_BIT(_field) \ (EFX_LOW_BIT(_field) + EFX_WIDTH(_field) - 1) /* * 64-bit mask equal in width to the specified field. * * For example, a field with width 5 would have a mask of 0x000000000000001f. */ #define EFX_MASK64(_field) \ ((EFX_WIDTH(_field) == 64) ? ~((uint64_t)0) : \ (((((uint64_t)1) << EFX_WIDTH(_field))) - 1)) /* * 32-bit mask equal in width to the specified field. * * For example, a field with width 5 would have a mask of 0x0000001f. */ #define EFX_MASK32(_field) \ ((EFX_WIDTH(_field) == 32) ? ~((uint32_t)0) : \ (((((uint32_t)1) << EFX_WIDTH(_field))) - 1)) /* * 16-bit mask equal in width to the specified field. * * For example, a field with width 5 would have a mask of 0x001f. */ #define EFX_MASK16(_field) \ ((EFX_WIDTH(_field) == 16) ? 0xffffu : \ (uint16_t)((1 << EFX_WIDTH(_field)) - 1)) /* * 8-bit mask equal in width to the specified field. * * For example, a field with width 5 would have a mask of 0x1f. */ #define EFX_MASK8(_field) \ ((uint8_t)((1 << EFX_WIDTH(_field)) - 1)) #pragma pack(1) /* * A byte (i.e. 8-bit) datatype */ typedef union efx_byte_u { uint8_t eb_u8[1]; } efx_byte_t; /* * A word (i.e. 16-bit) datatype * * This datatype is defined to be little-endian. */ typedef union efx_word_u { efx_byte_t ew_byte[2]; uint16_t ew_u16[1]; uint8_t ew_u8[2]; } efx_word_t; /* * A doubleword (i.e. 32-bit) datatype * * This datatype is defined to be little-endian. */ typedef union efx_dword_u { efx_byte_t ed_byte[4]; efx_word_t ed_word[2]; uint32_t ed_u32[1]; uint16_t ed_u16[2]; uint8_t ed_u8[4]; } efx_dword_t; /* * A quadword (i.e. 64-bit) datatype * * This datatype is defined to be little-endian. */ typedef union efx_qword_u { efx_byte_t eq_byte[8]; efx_word_t eq_word[4]; efx_dword_t eq_dword[2]; #if EFSYS_HAS_UINT64 uint64_t eq_u64[1]; #endif uint32_t eq_u32[2]; uint16_t eq_u16[4]; uint8_t eq_u8[8]; } efx_qword_t; /* * An octword (i.e. 128-bit) datatype * * This datatype is defined to be little-endian. */ typedef union efx_oword_u { efx_byte_t eo_byte[16]; efx_word_t eo_word[8]; efx_dword_t eo_dword[4]; efx_qword_t eo_qword[2]; #if EFSYS_HAS_SSE2_M128 __m128i eo_u128[1]; #endif #if EFSYS_HAS_UINT64 uint64_t eo_u64[2]; #endif uint32_t eo_u32[4]; uint16_t eo_u16[8]; uint8_t eo_u8[16]; } efx_oword_t; #pragma pack() #define __SWAP16(_x) \ ((((_x) & 0xff) << 8) | \ (((_x) >> 8) & 0xff)) #define __SWAP32(_x) \ ((__SWAP16((_x) & 0xffff) << 16) | \ __SWAP16(((_x) >> 16) & 0xffff)) #define __SWAP64(_x) \ ((__SWAP32((_x) & 0xffffffff) << 32) | \ __SWAP32(((_x) >> 32) & 0xffffffff)) #define __NOSWAP16(_x) (_x) #define __NOSWAP32(_x) (_x) #define __NOSWAP64(_x) (_x) #if EFSYS_IS_BIG_ENDIAN #define __CPU_TO_LE_16(_x) (uint16_t)__SWAP16(_x) #define __LE_TO_CPU_16(_x) (uint16_t)__SWAP16(_x) #define __CPU_TO_BE_16(_x) (uint16_t)__NOSWAP16(_x) #define __BE_TO_CPU_16(_x) (uint16_t)__NOSWAP16(_x) #define __CPU_TO_LE_32(_x) (uint32_t)__SWAP32(_x) #define __LE_TO_CPU_32(_x) (uint32_t)__SWAP32(_x) #define __CPU_TO_BE_32(_x) (uint32_t)__NOSWAP32(_x) #define __BE_TO_CPU_32(_x) (uint32_t)__NOSWAP32(_x) #define __CPU_TO_LE_64(_x) (uint64_t)__SWAP64(_x) #define __LE_TO_CPU_64(_x) (uint64_t)__SWAP64(_x) #define __CPU_TO_BE_64(_x) (uint64_t)__NOSWAP64(_x) #define __BE_TO_CPU_64(_x) (uint64_t)__NOSWAP64(_x) #elif EFSYS_IS_LITTLE_ENDIAN #define __CPU_TO_LE_16(_x) (uint16_t)__NOSWAP16(_x) #define __LE_TO_CPU_16(_x) (uint16_t)__NOSWAP16(_x) #define __CPU_TO_BE_16(_x) (uint16_t)__SWAP16(_x) #define __BE_TO_CPU_16(_x) (uint16_t)__SWAP16(_x) #define __CPU_TO_LE_32(_x) (uint32_t)__NOSWAP32(_x) #define __LE_TO_CPU_32(_x) (uint32_t)__NOSWAP32(_x) #define __CPU_TO_BE_32(_x) (uint32_t)__SWAP32(_x) #define __BE_TO_CPU_32(_x) (uint32_t)__SWAP32(_x) #define __CPU_TO_LE_64(_x) (uint64_t)__NOSWAP64(_x) #define __LE_TO_CPU_64(_x) (uint64_t)__NOSWAP64(_x) #define __CPU_TO_BE_64(_x) (uint64_t)__SWAP64(_x) #define __BE_TO_CPU_64(_x) (uint64_t)__SWAP64(_x) #else #error "Neither of EFSYS_IS_{BIG,LITTLE}_ENDIAN is set" #endif #define __NATIVE_8(_x) (uint8_t)(_x) /* Format string for printing an efx_byte_t */ #define EFX_BYTE_FMT "0x%02x" /* Format string for printing an efx_word_t */ #define EFX_WORD_FMT "0x%04x" /* Format string for printing an efx_dword_t */ #define EFX_DWORD_FMT "0x%08x" /* Format string for printing an efx_qword_t */ #define EFX_QWORD_FMT "0x%08x:%08x" /* Format string for printing an efx_oword_t */ #define EFX_OWORD_FMT "0x%08x:%08x:%08x:%08x" /* Parameters for printing an efx_byte_t */ #define EFX_BYTE_VAL(_byte) \ ((unsigned int)__NATIVE_8((_byte).eb_u8[0])) /* Parameters for printing an efx_word_t */ #define EFX_WORD_VAL(_word) \ ((unsigned int)__LE_TO_CPU_16((_word).ew_u16[0])) /* Parameters for printing an efx_dword_t */ #define EFX_DWORD_VAL(_dword) \ ((unsigned int)__LE_TO_CPU_32((_dword).ed_u32[0])) /* Parameters for printing an efx_qword_t */ #define EFX_QWORD_VAL(_qword) \ ((unsigned int)__LE_TO_CPU_32((_qword).eq_u32[1])), \ ((unsigned int)__LE_TO_CPU_32((_qword).eq_u32[0])) /* Parameters for printing an efx_oword_t */ #define EFX_OWORD_VAL(_oword) \ ((unsigned int)__LE_TO_CPU_32((_oword).eo_u32[3])), \ ((unsigned int)__LE_TO_CPU_32((_oword).eo_u32[2])), \ ((unsigned int)__LE_TO_CPU_32((_oword).eo_u32[1])), \ ((unsigned int)__LE_TO_CPU_32((_oword).eo_u32[0])) /* * Stop lint complaining about some shifts. */ #ifdef __lint extern int fix_lint; #define FIX_LINT(_x) (_x + fix_lint) #else #define FIX_LINT(_x) (_x) #endif /* * Extract bit field portion [low,high) from the native-endian element * which contains bits [min,max). * * For example, suppose "element" represents the high 32 bits of a * 64-bit value, and we wish to extract the bits belonging to the bit * field occupying bits 28-45 of this 64-bit value. * * Then EFX_EXTRACT(_element, 32, 63, 28, 45) would give * * (_element) << 4 * * The result will contain the relevant bits filled in in the range * [0,high-low), with garbage in bits [high-low+1,...). */ #define EFX_EXTRACT_NATIVE(_element, _min, _max, _low, _high) \ ((FIX_LINT(_low > _max) || FIX_LINT(_high < _min)) ? \ 0U : \ ((_low > _min) ? \ ((_element) >> (_low - _min)) : \ ((_element) << (_min - _low)))) /* * Extract bit field portion [low,high) from the 64-bit little-endian * element which contains bits [min,max) */ #define EFX_EXTRACT64(_element, _min, _max, _low, _high) \ EFX_EXTRACT_NATIVE(__LE_TO_CPU_64(_element), _min, _max, _low, _high) /* * Extract bit field portion [low,high) from the 32-bit little-endian * element which contains bits [min,max) */ #define EFX_EXTRACT32(_element, _min, _max, _low, _high) \ EFX_EXTRACT_NATIVE(__LE_TO_CPU_32(_element), _min, _max, _low, _high) /* * Extract bit field portion [low,high) from the 16-bit little-endian * element which contains bits [min,max) */ #define EFX_EXTRACT16(_element, _min, _max, _low, _high) \ EFX_EXTRACT_NATIVE(__LE_TO_CPU_16(_element), _min, _max, _low, _high) /* * Extract bit field portion [low,high) from the 8-bit * element which contains bits [min,max) */ #define EFX_EXTRACT8(_element, _min, _max, _low, _high) \ EFX_EXTRACT_NATIVE(__NATIVE_8(_element), _min, _max, _low, _high) #define EFX_EXTRACT_OWORD64(_oword, _low, _high) \ (EFX_EXTRACT64((_oword).eo_u64[0], FIX_LINT(0), FIX_LINT(63), \ _low, _high) | \ EFX_EXTRACT64((_oword).eo_u64[1], FIX_LINT(64), FIX_LINT(127), \ _low, _high)) #define EFX_EXTRACT_OWORD32(_oword, _low, _high) \ (EFX_EXTRACT32((_oword).eo_u32[0], FIX_LINT(0), FIX_LINT(31), \ _low, _high) | \ EFX_EXTRACT32((_oword).eo_u32[1], FIX_LINT(32), FIX_LINT(63), \ _low, _high) | \ EFX_EXTRACT32((_oword).eo_u32[2], FIX_LINT(64), FIX_LINT(95), \ _low, _high) | \ EFX_EXTRACT32((_oword).eo_u32[3], FIX_LINT(96), FIX_LINT(127), \ _low, _high)) #define EFX_EXTRACT_QWORD64(_qword, _low, _high) \ (EFX_EXTRACT64((_qword).eq_u64[0], FIX_LINT(0), FIX_LINT(63), \ _low, _high)) #define EFX_EXTRACT_QWORD32(_qword, _low, _high) \ (EFX_EXTRACT32((_qword).eq_u32[0], FIX_LINT(0), FIX_LINT(31), \ _low, _high) | \ EFX_EXTRACT32((_qword).eq_u32[1], FIX_LINT(32), FIX_LINT(63), \ _low, _high)) #define EFX_EXTRACT_DWORD(_dword, _low, _high) \ (EFX_EXTRACT32((_dword).ed_u32[0], FIX_LINT(0), FIX_LINT(31), \ _low, _high)) #define EFX_EXTRACT_WORD(_word, _low, _high) \ (EFX_EXTRACT16((_word).ew_u16[0], FIX_LINT(0), FIX_LINT(15), \ _low, _high)) #define EFX_EXTRACT_BYTE(_byte, _low, _high) \ (EFX_EXTRACT8((_byte).eb_u8[0], FIX_LINT(0), FIX_LINT(7), \ _low, _high)) #define EFX_OWORD_FIELD64(_oword, _field) \ ((uint32_t)EFX_EXTRACT_OWORD64(_oword, EFX_LOW_BIT(_field), \ EFX_HIGH_BIT(_field)) & EFX_MASK32(_field)) #define EFX_OWORD_FIELD32(_oword, _field) \ (EFX_EXTRACT_OWORD32(_oword, EFX_LOW_BIT(_field), \ EFX_HIGH_BIT(_field)) & EFX_MASK32(_field)) #define EFX_QWORD_FIELD64(_qword, _field) \ ((uint32_t)EFX_EXTRACT_QWORD64(_qword, EFX_LOW_BIT(_field), \ EFX_HIGH_BIT(_field)) & EFX_MASK32(_field)) #define EFX_QWORD_FIELD32(_qword, _field) \ (EFX_EXTRACT_QWORD32(_qword, EFX_LOW_BIT(_field), \ EFX_HIGH_BIT(_field)) & EFX_MASK32(_field)) #define EFX_DWORD_FIELD(_dword, _field) \ (EFX_EXTRACT_DWORD(_dword, EFX_LOW_BIT(_field), \ EFX_HIGH_BIT(_field)) & EFX_MASK32(_field)) #define EFX_WORD_FIELD(_word, _field) \ (EFX_EXTRACT_WORD(_word, EFX_LOW_BIT(_field), \ EFX_HIGH_BIT(_field)) & EFX_MASK16(_field)) #define EFX_BYTE_FIELD(_byte, _field) \ (EFX_EXTRACT_BYTE(_byte, EFX_LOW_BIT(_field), \ EFX_HIGH_BIT(_field)) & EFX_MASK8(_field)) #define EFX_OWORD_IS_EQUAL64(_oword_a, _oword_b) \ ((_oword_a).eo_u64[0] == (_oword_b).eo_u64[0] && \ (_oword_a).eo_u64[1] == (_oword_b).eo_u64[1]) #define EFX_OWORD_IS_EQUAL32(_oword_a, _oword_b) \ ((_oword_a).eo_u32[0] == (_oword_b).eo_u32[0] && \ (_oword_a).eo_u32[1] == (_oword_b).eo_u32[1] && \ (_oword_a).eo_u32[2] == (_oword_b).eo_u32[2] && \ (_oword_a).eo_u32[3] == (_oword_b).eo_u32[3]) #define EFX_QWORD_IS_EQUAL64(_qword_a, _qword_b) \ ((_qword_a).eq_u64[0] == (_qword_b).eq_u64[0]) #define EFX_QWORD_IS_EQUAL32(_qword_a, _qword_b) \ ((_qword_a).eq_u32[0] == (_qword_b).eq_u32[0] && \ (_qword_a).eq_u32[1] == (_qword_b).eq_u32[1]) #define EFX_DWORD_IS_EQUAL(_dword_a, _dword_b) \ ((_dword_a).ed_u32[0] == (_dword_b).ed_u32[0]) #define EFX_WORD_IS_EQUAL(_word_a, _word_b) \ ((_word_a).ew_u16[0] == (_word_b).ew_u16[0]) #define EFX_BYTE_IS_EQUAL(_byte_a, _byte_b) \ ((_byte_a).eb_u8[0] == (_byte_b).eb_u8[0]) #define EFX_OWORD_IS_ZERO64(_oword) \ (((_oword).eo_u64[0] | \ (_oword).eo_u64[1]) == 0) #define EFX_OWORD_IS_ZERO32(_oword) \ (((_oword).eo_u32[0] | \ (_oword).eo_u32[1] | \ (_oword).eo_u32[2] | \ (_oword).eo_u32[3]) == 0) #define EFX_QWORD_IS_ZERO64(_qword) \ (((_qword).eq_u64[0]) == 0) #define EFX_QWORD_IS_ZERO32(_qword) \ (((_qword).eq_u32[0] | \ (_qword).eq_u32[1]) == 0) #define EFX_DWORD_IS_ZERO(_dword) \ (((_dword).ed_u32[0]) == 0) #define EFX_WORD_IS_ZERO(_word) \ (((_word).ew_u16[0]) == 0) #define EFX_BYTE_IS_ZERO(_byte) \ (((_byte).eb_u8[0]) == 0) #define EFX_OWORD_IS_SET64(_oword) \ (((_oword).eo_u64[0] & \ (_oword).eo_u64[1]) == ~((uint64_t)0)) #define EFX_OWORD_IS_SET32(_oword) \ (((_oword).eo_u32[0] & \ (_oword).eo_u32[1] & \ (_oword).eo_u32[2] & \ (_oword).eo_u32[3]) == ~((uint32_t)0)) #define EFX_QWORD_IS_SET64(_qword) \ (((_qword).eq_u64[0]) == ~((uint32_t)0)) #define EFX_QWORD_IS_SET32(_qword) \ (((_qword).eq_u32[0] & \ (_qword).eq_u32[1]) == ~((uint32_t)0)) #define EFX_DWORD_IS_SET(_dword) \ ((_dword).ed_u32[0] == ~((uint32_t)0)) #define EFX_WORD_IS_SET(_word) \ ((_word).ew_u16[0] == ~((uint16_t)0)) #define EFX_BYTE_IS_SET(_byte) \ ((_byte).eb_u8[0] == ~((uint8_t)0)) /* * Construct bit field portion * * Creates the portion of the bit field [low,high) that lies within * the range [min,max). */ #define EFX_INSERT_NATIVE64(_min, _max, _low, _high, _value) \ (((_low > _max) || (_high < _min)) ? \ 0U : \ ((_low > _min) ? \ (((uint64_t)(_value)) << (_low - _min)) : \ (((uint64_t)(_value)) >> (_min - _low)))) #define EFX_INSERT_NATIVE32(_min, _max, _low, _high, _value) \ (((_low > _max) || (_high < _min)) ? \ 0U : \ ((_low > _min) ? \ (((uint32_t)(_value)) << (_low - _min)) : \ (((uint32_t)(_value)) >> (_min - _low)))) #define EFX_INSERT_NATIVE16(_min, _max, _low, _high, _value) \ (((_low > _max) || (_high < _min)) ? \ 0U : \ (uint16_t)((_low > _min) ? \ ((_value) << (_low - _min)) : \ ((_value) >> (_min - _low)))) #define EFX_INSERT_NATIVE8(_min, _max, _low, _high, _value) \ (((_low > _max) || (_high < _min)) ? \ 0U : \ (uint8_t)((_low > _min) ? \ ((_value) << (_low - _min)) : \ ((_value) >> (_min - _low)))) /* * Construct bit field portion * * Creates the portion of the named bit field that lies within the * range [min,max). */ #define EFX_INSERT_FIELD_NATIVE64(_min, _max, _field, _value) \ EFX_INSERT_NATIVE64(_min, _max, EFX_LOW_BIT(_field), \ EFX_HIGH_BIT(_field), _value) #define EFX_INSERT_FIELD_NATIVE32(_min, _max, _field, _value) \ EFX_INSERT_NATIVE32(_min, _max, EFX_LOW_BIT(_field), \ EFX_HIGH_BIT(_field), _value) #define EFX_INSERT_FIELD_NATIVE16(_min, _max, _field, _value) \ EFX_INSERT_NATIVE16(_min, _max, EFX_LOW_BIT(_field), \ EFX_HIGH_BIT(_field), _value) #define EFX_INSERT_FIELD_NATIVE8(_min, _max, _field, _value) \ EFX_INSERT_NATIVE8(_min, _max, EFX_LOW_BIT(_field), \ EFX_HIGH_BIT(_field), _value) /* * Construct bit field * * Creates the portion of the named bit fields that lie within the * range [min,max). */ #define EFX_INSERT_FIELDS64(_min, _max, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, _field9, _value9, \ _field10, _value10) \ __CPU_TO_LE_64( \ EFX_INSERT_FIELD_NATIVE64(_min, _max, _field1, _value1) | \ EFX_INSERT_FIELD_NATIVE64(_min, _max, _field2, _value2) | \ EFX_INSERT_FIELD_NATIVE64(_min, _max, _field3, _value3) | \ EFX_INSERT_FIELD_NATIVE64(_min, _max, _field4, _value4) | \ EFX_INSERT_FIELD_NATIVE64(_min, _max, _field5, _value5) | \ EFX_INSERT_FIELD_NATIVE64(_min, _max, _field6, _value6) | \ EFX_INSERT_FIELD_NATIVE64(_min, _max, _field7, _value7) | \ EFX_INSERT_FIELD_NATIVE64(_min, _max, _field8, _value8) | \ EFX_INSERT_FIELD_NATIVE64(_min, _max, _field9, _value9) | \ EFX_INSERT_FIELD_NATIVE64(_min, _max, _field10, _value10)) #define EFX_INSERT_FIELDS32(_min, _max, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, _field9, _value9, \ _field10, _value10) \ __CPU_TO_LE_32( \ EFX_INSERT_FIELD_NATIVE32(_min, _max, _field1, _value1) | \ EFX_INSERT_FIELD_NATIVE32(_min, _max, _field2, _value2) | \ EFX_INSERT_FIELD_NATIVE32(_min, _max, _field3, _value3) | \ EFX_INSERT_FIELD_NATIVE32(_min, _max, _field4, _value4) | \ EFX_INSERT_FIELD_NATIVE32(_min, _max, _field5, _value5) | \ EFX_INSERT_FIELD_NATIVE32(_min, _max, _field6, _value6) | \ EFX_INSERT_FIELD_NATIVE32(_min, _max, _field7, _value7) | \ EFX_INSERT_FIELD_NATIVE32(_min, _max, _field8, _value8) | \ EFX_INSERT_FIELD_NATIVE32(_min, _max, _field9, _value9) | \ EFX_INSERT_FIELD_NATIVE32(_min, _max, _field10, _value10)) #define EFX_INSERT_FIELDS16(_min, _max, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, _field9, _value9, \ _field10, _value10) \ __CPU_TO_LE_16( \ EFX_INSERT_FIELD_NATIVE16(_min, _max, _field1, _value1) | \ EFX_INSERT_FIELD_NATIVE16(_min, _max, _field2, _value2) | \ EFX_INSERT_FIELD_NATIVE16(_min, _max, _field3, _value3) | \ EFX_INSERT_FIELD_NATIVE16(_min, _max, _field4, _value4) | \ EFX_INSERT_FIELD_NATIVE16(_min, _max, _field5, _value5) | \ EFX_INSERT_FIELD_NATIVE16(_min, _max, _field6, _value6) | \ EFX_INSERT_FIELD_NATIVE16(_min, _max, _field7, _value7) | \ EFX_INSERT_FIELD_NATIVE16(_min, _max, _field8, _value8) | \ EFX_INSERT_FIELD_NATIVE16(_min, _max, _field9, _value9) | \ EFX_INSERT_FIELD_NATIVE16(_min, _max, _field10, _value10)) #define EFX_INSERT_FIELDS8(_min, _max, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, _field9, _value9, \ _field10, _value10) \ __NATIVE_8( \ EFX_INSERT_FIELD_NATIVE8(_min, _max, _field1, _value1) | \ EFX_INSERT_FIELD_NATIVE8(_min, _max, _field2, _value2) | \ EFX_INSERT_FIELD_NATIVE8(_min, _max, _field3, _value3) | \ EFX_INSERT_FIELD_NATIVE8(_min, _max, _field4, _value4) | \ EFX_INSERT_FIELD_NATIVE8(_min, _max, _field5, _value5) | \ EFX_INSERT_FIELD_NATIVE8(_min, _max, _field6, _value6) | \ EFX_INSERT_FIELD_NATIVE8(_min, _max, _field7, _value7) | \ EFX_INSERT_FIELD_NATIVE8(_min, _max, _field8, _value8) | \ EFX_INSERT_FIELD_NATIVE8(_min, _max, _field9, _value9) | \ EFX_INSERT_FIELD_NATIVE8(_min, _max, _field10, _value10)) #define EFX_POPULATE_OWORD64(_oword, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, _field9, _value9, \ _field10, _value10) \ do { \ _NOTE(CONSTANTCONDITION) \ (_oword).eo_u64[0] = EFX_INSERT_FIELDS64(0, 63, \ _field1, _value1, _field2, _value2, \ _field3, _value3, _field4, _value4, \ _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, \ _field9, _value9, _field10, _value10); \ _NOTE(CONSTANTCONDITION) \ (_oword).eo_u64[1] = EFX_INSERT_FIELDS64(64, 127, \ _field1, _value1, _field2, _value2, \ _field3, _value3, _field4, _value4, \ _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, \ _field9, _value9, _field10, _value10); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_POPULATE_OWORD32(_oword, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, _field9, _value9, \ _field10, _value10) \ do { \ _NOTE(CONSTANTCONDITION) \ (_oword).eo_u32[0] = EFX_INSERT_FIELDS32(0, 31, \ _field1, _value1, _field2, _value2, \ _field3, _value3, _field4, _value4, \ _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, \ _field9, _value9, _field10, _value10); \ _NOTE(CONSTANTCONDITION) \ (_oword).eo_u32[1] = EFX_INSERT_FIELDS32(32, 63, \ _field1, _value1, _field2, _value2, \ _field3, _value3, _field4, _value4, \ _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, \ _field9, _value9, _field10, _value10); \ _NOTE(CONSTANTCONDITION) \ (_oword).eo_u32[2] = EFX_INSERT_FIELDS32(64, 95, \ _field1, _value1, _field2, _value2, \ _field3, _value3, _field4, _value4, \ _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, \ _field9, _value9, _field10, _value10); \ _NOTE(CONSTANTCONDITION) \ (_oword).eo_u32[3] = EFX_INSERT_FIELDS32(96, 127, \ _field1, _value1, _field2, _value2, \ _field3, _value3, _field4, _value4, \ _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, \ _field9, _value9, _field10, _value10); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_POPULATE_QWORD64(_qword, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, _field9, _value9, \ _field10, _value10) \ do { \ _NOTE(CONSTANTCONDITION) \ (_qword).eq_u64[0] = EFX_INSERT_FIELDS64(0, 63, \ _field1, _value1, _field2, _value2, \ _field3, _value3, _field4, _value4, \ _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, \ _field9, _value9, _field10, _value10); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_POPULATE_QWORD32(_qword, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, _field9, _value9, \ _field10, _value10) \ do { \ _NOTE(CONSTANTCONDITION) \ (_qword).eq_u32[0] = EFX_INSERT_FIELDS32(0, 31, \ _field1, _value1, _field2, _value2, \ _field3, _value3, _field4, _value4, \ _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, \ _field9, _value9, _field10, _value10); \ _NOTE(CONSTANTCONDITION) \ (_qword).eq_u32[1] = EFX_INSERT_FIELDS32(32, 63, \ _field1, _value1, _field2, _value2, \ _field3, _value3, _field4, _value4, \ _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, \ _field9, _value9, _field10, _value10); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_POPULATE_DWORD(_dword, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, _field9, _value9, \ _field10, _value10) \ do { \ _NOTE(CONSTANTCONDITION) \ (_dword).ed_u32[0] = EFX_INSERT_FIELDS32(0, 31, \ _field1, _value1, _field2, _value2, \ _field3, _value3, _field4, _value4, \ _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, \ _field9, _value9, _field10, _value10); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_POPULATE_WORD(_word, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, _field9, _value9, \ _field10, _value10) \ do { \ _NOTE(CONSTANTCONDITION) \ (_word).ew_u16[0] = EFX_INSERT_FIELDS16(0, 15, \ _field1, _value1, _field2, _value2, \ _field3, _value3, _field4, _value4, \ _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, \ _field9, _value9, _field10, _value10); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_POPULATE_BYTE(_byte, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, _field9, _value9, \ _field10, _value10) \ do { \ _NOTE(CONSTANTCONDITION) \ (_byte).eb_u8[0] = EFX_INSERT_FIELDS8(0, 7, \ _field1, _value1, _field2, _value2, \ _field3, _value3, _field4, _value4, \ _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, \ _field9, _value9, _field10, _value10); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) /* Populate an octword field with various numbers of arguments */ #define EFX_POPULATE_OWORD_10 EFX_POPULATE_OWORD #define EFX_POPULATE_OWORD_9(_oword, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, _field9, _value9) \ EFX_POPULATE_OWORD_10(_oword, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, _field9, _value9) #define EFX_POPULATE_OWORD_8(_oword, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8) \ EFX_POPULATE_OWORD_9(_oword, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8) #define EFX_POPULATE_OWORD_7(_oword, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7) \ EFX_POPULATE_OWORD_8(_oword, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7) #define EFX_POPULATE_OWORD_6(_oword, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6) \ EFX_POPULATE_OWORD_7(_oword, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6) #define EFX_POPULATE_OWORD_5(_oword, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5) \ EFX_POPULATE_OWORD_6(_oword, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5) #define EFX_POPULATE_OWORD_4(_oword, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4) \ EFX_POPULATE_OWORD_5(_oword, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4) #define EFX_POPULATE_OWORD_3(_oword, \ _field1, _value1, _field2, _value2, _field3, _value3) \ EFX_POPULATE_OWORD_4(_oword, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3) #define EFX_POPULATE_OWORD_2(_oword, \ _field1, _value1, _field2, _value2) \ EFX_POPULATE_OWORD_3(_oword, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2) #define EFX_POPULATE_OWORD_1(_oword, \ _field1, _value1) \ EFX_POPULATE_OWORD_2(_oword, EFX_DUMMY_FIELD, 0, \ _field1, _value1) #define EFX_ZERO_OWORD(_oword) \ EFX_POPULATE_OWORD_1(_oword, EFX_DUMMY_FIELD, 0) #define EFX_SET_OWORD(_oword) \ EFX_POPULATE_OWORD_4(_oword, \ EFX_DWORD_0, 0xffffffff, EFX_DWORD_1, 0xffffffff, \ EFX_DWORD_2, 0xffffffff, EFX_DWORD_3, 0xffffffff) /* Populate a quadword field with various numbers of arguments */ #define EFX_POPULATE_QWORD_10 EFX_POPULATE_QWORD #define EFX_POPULATE_QWORD_9(_qword, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, _field9, _value9) \ EFX_POPULATE_QWORD_10(_qword, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, _field9, _value9) #define EFX_POPULATE_QWORD_8(_qword, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8) \ EFX_POPULATE_QWORD_9(_qword, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8) #define EFX_POPULATE_QWORD_7(_qword, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7) \ EFX_POPULATE_QWORD_8(_qword, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7) #define EFX_POPULATE_QWORD_6(_qword, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6) \ EFX_POPULATE_QWORD_7(_qword, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6) #define EFX_POPULATE_QWORD_5(_qword, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5) \ EFX_POPULATE_QWORD_6(_qword, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5) #define EFX_POPULATE_QWORD_4(_qword, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4) \ EFX_POPULATE_QWORD_5(_qword, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4) #define EFX_POPULATE_QWORD_3(_qword, \ _field1, _value1, _field2, _value2, _field3, _value3) \ EFX_POPULATE_QWORD_4(_qword, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3) #define EFX_POPULATE_QWORD_2(_qword, \ _field1, _value1, _field2, _value2) \ EFX_POPULATE_QWORD_3(_qword, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2) #define EFX_POPULATE_QWORD_1(_qword, \ _field1, _value1) \ EFX_POPULATE_QWORD_2(_qword, EFX_DUMMY_FIELD, 0, \ _field1, _value1) #define EFX_ZERO_QWORD(_qword) \ EFX_POPULATE_QWORD_1(_qword, EFX_DUMMY_FIELD, 0) #define EFX_SET_QWORD(_qword) \ EFX_POPULATE_QWORD_2(_qword, \ EFX_DWORD_0, 0xffffffff, EFX_DWORD_1, 0xffffffff) /* Populate a dword field with various numbers of arguments */ #define EFX_POPULATE_DWORD_10 EFX_POPULATE_DWORD #define EFX_POPULATE_DWORD_9(_dword, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, _field9, _value9) \ EFX_POPULATE_DWORD_10(_dword, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, _field9, _value9) #define EFX_POPULATE_DWORD_8(_dword, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8) \ EFX_POPULATE_DWORD_9(_dword, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8) #define EFX_POPULATE_DWORD_7(_dword, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7) \ EFX_POPULATE_DWORD_8(_dword, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7) #define EFX_POPULATE_DWORD_6(_dword, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6) \ EFX_POPULATE_DWORD_7(_dword, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6) #define EFX_POPULATE_DWORD_5(_dword, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5) \ EFX_POPULATE_DWORD_6(_dword, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5) #define EFX_POPULATE_DWORD_4(_dword, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4) \ EFX_POPULATE_DWORD_5(_dword, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4) #define EFX_POPULATE_DWORD_3(_dword, \ _field1, _value1, _field2, _value2, _field3, _value3) \ EFX_POPULATE_DWORD_4(_dword, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3) #define EFX_POPULATE_DWORD_2(_dword, \ _field1, _value1, _field2, _value2) \ EFX_POPULATE_DWORD_3(_dword, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2) #define EFX_POPULATE_DWORD_1(_dword, \ _field1, _value1) \ EFX_POPULATE_DWORD_2(_dword, EFX_DUMMY_FIELD, 0, \ _field1, _value1) #define EFX_ZERO_DWORD(_dword) \ EFX_POPULATE_DWORD_1(_dword, EFX_DUMMY_FIELD, 0) #define EFX_SET_DWORD(_dword) \ EFX_POPULATE_DWORD_1(_dword, \ EFX_DWORD_0, 0xffffffff) /* Populate a word field with various numbers of arguments */ #define EFX_POPULATE_WORD_10 EFX_POPULATE_WORD #define EFX_POPULATE_WORD_9(_word, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, _field9, _value9) \ EFX_POPULATE_WORD_10(_word, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, _field9, _value9) #define EFX_POPULATE_WORD_8(_word, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8) \ EFX_POPULATE_WORD_9(_word, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8) #define EFX_POPULATE_WORD_7(_word, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7) \ EFX_POPULATE_WORD_8(_word, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7) #define EFX_POPULATE_WORD_6(_word, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6) \ EFX_POPULATE_WORD_7(_word, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6) #define EFX_POPULATE_WORD_5(_word, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5) \ EFX_POPULATE_WORD_6(_word, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5) #define EFX_POPULATE_WORD_4(_word, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4) \ EFX_POPULATE_WORD_5(_word, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4) #define EFX_POPULATE_WORD_3(_word, \ _field1, _value1, _field2, _value2, _field3, _value3) \ EFX_POPULATE_WORD_4(_word, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3) #define EFX_POPULATE_WORD_2(_word, \ _field1, _value1, _field2, _value2) \ EFX_POPULATE_WORD_3(_word, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2) #define EFX_POPULATE_WORD_1(_word, \ _field1, _value1) \ EFX_POPULATE_WORD_2(_word, EFX_DUMMY_FIELD, 0, \ _field1, _value1) #define EFX_ZERO_WORD(_word) \ EFX_POPULATE_WORD_1(_word, EFX_DUMMY_FIELD, 0) #define EFX_SET_WORD(_word) \ EFX_POPULATE_WORD_1(_word, \ EFX_WORD_0, 0xffff) /* Populate a byte field with various numbers of arguments */ #define EFX_POPULATE_BYTE_10 EFX_POPULATE_BYTE #define EFX_POPULATE_BYTE_9(_byte, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, _field9, _value9) \ EFX_POPULATE_BYTE_10(_byte, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8, _field9, _value9) #define EFX_POPULATE_BYTE_8(_byte, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8) \ EFX_POPULATE_BYTE_9(_byte, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7, _field8, _value8) #define EFX_POPULATE_BYTE_7(_byte, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7) \ EFX_POPULATE_BYTE_8(_byte, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6, \ _field7, _value7) #define EFX_POPULATE_BYTE_6(_byte, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6) \ EFX_POPULATE_BYTE_7(_byte, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5, _field6, _value6) #define EFX_POPULATE_BYTE_5(_byte, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5) \ EFX_POPULATE_BYTE_6(_byte, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4, _field5, _value5) #define EFX_POPULATE_BYTE_4(_byte, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4) \ EFX_POPULATE_BYTE_5(_byte, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3, \ _field4, _value4) #define EFX_POPULATE_BYTE_3(_byte, \ _field1, _value1, _field2, _value2, _field3, _value3) \ EFX_POPULATE_BYTE_4(_byte, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2, _field3, _value3) #define EFX_POPULATE_BYTE_2(_byte, \ _field1, _value1, _field2, _value2) \ EFX_POPULATE_BYTE_3(_byte, EFX_DUMMY_FIELD, 0, \ _field1, _value1, _field2, _value2) #define EFX_POPULATE_BYTE_1(_byte, \ _field1, _value1) \ EFX_POPULATE_BYTE_2(_byte, EFX_DUMMY_FIELD, 0, \ _field1, _value1) #define EFX_ZERO_BYTE(_byte) \ EFX_POPULATE_BYTE_1(_byte, EFX_DUMMY_FIELD, 0) #define EFX_SET_BYTE(_byte) \ EFX_POPULATE_BYTE_1(_byte, \ EFX_BYTE_0, 0xff) /* * Modify a named field within an already-populated structure. Used * for read-modify-write operations. */ #define EFX_INSERT_FIELD64(_min, _max, _field, _value) \ __CPU_TO_LE_64(EFX_INSERT_FIELD_NATIVE64(_min, _max, _field, _value)) #define EFX_INSERT_FIELD32(_min, _max, _field, _value) \ __CPU_TO_LE_32(EFX_INSERT_FIELD_NATIVE32(_min, _max, _field, _value)) #define EFX_INSERT_FIELD16(_min, _max, _field, _value) \ __CPU_TO_LE_16(EFX_INSERT_FIELD_NATIVE16(_min, _max, _field, _value)) #define EFX_INSERT_FIELD8(_min, _max, _field, _value) \ __NATIVE_8(EFX_INSERT_FIELD_NATIVE8(_min, _max, _field, _value)) #define EFX_INPLACE_MASK64(_min, _max, _field) \ EFX_INSERT_FIELD64(_min, _max, _field, EFX_MASK64(_field)) #define EFX_INPLACE_MASK32(_min, _max, _field) \ EFX_INSERT_FIELD32(_min, _max, _field, EFX_MASK32(_field)) #define EFX_INPLACE_MASK16(_min, _max, _field) \ EFX_INSERT_FIELD16(_min, _max, _field, EFX_MASK16(_field)) #define EFX_INPLACE_MASK8(_min, _max, _field) \ EFX_INSERT_FIELD8(_min, _max, _field, EFX_MASK8(_field)) #define EFX_SET_OWORD_FIELD64(_oword, _field, _value) \ do { \ _NOTE(CONSTANTCONDITION) \ (_oword).eo_u64[0] = (((_oword).eo_u64[0] & \ ~EFX_INPLACE_MASK64(0, 63, _field)) | \ EFX_INSERT_FIELD64(0, 63, _field, _value)); \ _NOTE(CONSTANTCONDITION) \ (_oword).eo_u64[1] = (((_oword).eo_u64[1] & \ ~EFX_INPLACE_MASK64(64, 127, _field)) | \ EFX_INSERT_FIELD64(64, 127, _field, _value)); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_SET_OWORD_FIELD32(_oword, _field, _value) \ do { \ _NOTE(CONSTANTCONDITION) \ (_oword).eo_u32[0] = (((_oword).eo_u32[0] & \ ~EFX_INPLACE_MASK32(0, 31, _field)) | \ EFX_INSERT_FIELD32(0, 31, _field, _value)); \ _NOTE(CONSTANTCONDITION) \ (_oword).eo_u32[1] = (((_oword).eo_u32[1] & \ ~EFX_INPLACE_MASK32(32, 63, _field)) | \ EFX_INSERT_FIELD32(32, 63, _field, _value)); \ _NOTE(CONSTANTCONDITION) \ (_oword).eo_u32[2] = (((_oword).eo_u32[2] & \ ~EFX_INPLACE_MASK32(64, 95, _field)) | \ EFX_INSERT_FIELD32(64, 95, _field, _value)); \ _NOTE(CONSTANTCONDITION) \ (_oword).eo_u32[3] = (((_oword).eo_u32[3] & \ ~EFX_INPLACE_MASK32(96, 127, _field)) | \ EFX_INSERT_FIELD32(96, 127, _field, _value)); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_SET_QWORD_FIELD64(_qword, _field, _value) \ do { \ _NOTE(CONSTANTCONDITION) \ (_qword).eq_u64[0] = (((_qword).eq_u64[0] & \ ~EFX_INPLACE_MASK64(0, 63, _field)) | \ EFX_INSERT_FIELD64(0, 63, _field, _value)); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_SET_QWORD_FIELD32(_qword, _field, _value) \ do { \ _NOTE(CONSTANTCONDITION) \ (_qword).eq_u32[0] = (((_qword).eq_u32[0] & \ ~EFX_INPLACE_MASK32(0, 31, _field)) | \ EFX_INSERT_FIELD32(0, 31, _field, _value)); \ _NOTE(CONSTANTCONDITION) \ (_qword).eq_u32[1] = (((_qword).eq_u32[1] & \ ~EFX_INPLACE_MASK32(32, 63, _field)) | \ EFX_INSERT_FIELD32(32, 63, _field, _value)); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_SET_DWORD_FIELD(_dword, _field, _value) \ do { \ _NOTE(CONSTANTCONDITION) \ (_dword).ed_u32[0] = (((_dword).ed_u32[0] & \ ~EFX_INPLACE_MASK32(0, 31, _field)) | \ EFX_INSERT_FIELD32(0, 31, _field, _value)); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_SET_WORD_FIELD(_word, _field, _value) \ do { \ _NOTE(CONSTANTCONDITION) \ (_word).ew_u16[0] = (((_word).ew_u16[0] & \ ~EFX_INPLACE_MASK16(0, 15, _field)) | \ EFX_INSERT_FIELD16(0, 15, _field, _value)); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_SET_BYTE_FIELD(_byte, _field, _value) \ do { \ _NOTE(CONSTANTCONDITION) \ (_byte).eb_u8[0] = (((_byte).eb_u8[0] & \ ~EFX_INPLACE_MASK8(0, 7, _field)) | \ EFX_INSERT_FIELD8(0, 7, _field, _value)); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) /* * Set or clear a numbered bit within an octword. */ #define EFX_SHIFT64(_bit, _base) \ (((_bit) >= (_base) && (_bit) < (_base) + 64) ? \ ((uint64_t)1 << ((_bit) - (_base))) : \ 0U) #define EFX_SHIFT32(_bit, _base) \ (((_bit) >= (_base) && (_bit) < (_base) + 32) ? \ ((uint32_t)1 << ((_bit) - (_base))) : \ 0U) #define EFX_SHIFT16(_bit, _base) \ (((_bit) >= (_base) && (_bit) < (_base) + 16) ? \ (uint16_t)(1 << ((_bit) - (_base))) : \ 0U) #define EFX_SHIFT8(_bit, _base) \ (((_bit) >= (_base) && (_bit) < (_base) + 8) ? \ (uint8_t)(1 << ((_bit) - (_base))) : \ 0U) #define EFX_SET_OWORD_BIT64(_oword, _bit) \ do { \ _NOTE(CONSTANTCONDITION) \ (_oword).eo_u64[0] |= \ __CPU_TO_LE_64(EFX_SHIFT64(_bit, FIX_LINT(0))); \ (_oword).eo_u64[1] |= \ __CPU_TO_LE_64(EFX_SHIFT64(_bit, FIX_LINT(64))); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_SET_OWORD_BIT32(_oword, _bit) \ do { \ _NOTE(CONSTANTCONDITION) \ (_oword).eo_u32[0] |= \ __CPU_TO_LE_32(EFX_SHIFT32(_bit, FIX_LINT(0))); \ (_oword).eo_u32[1] |= \ __CPU_TO_LE_32(EFX_SHIFT32(_bit, FIX_LINT(32))); \ (_oword).eo_u32[2] |= \ __CPU_TO_LE_32(EFX_SHIFT32(_bit, FIX_LINT(64))); \ (_oword).eo_u32[3] |= \ __CPU_TO_LE_32(EFX_SHIFT32(_bit, FIX_LINT(96))); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_CLEAR_OWORD_BIT64(_oword, _bit) \ do { \ _NOTE(CONSTANTCONDITION) \ (_oword).eo_u64[0] &= \ __CPU_TO_LE_64(~EFX_SHIFT64(_bit, FIX_LINT(0))); \ (_oword).eo_u64[1] &= \ __CPU_TO_LE_64(~EFX_SHIFT64(_bit, FIX_LINT(64))); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_CLEAR_OWORD_BIT32(_oword, _bit) \ do { \ _NOTE(CONSTANTCONDITION) \ (_oword).eo_u32[0] &= \ __CPU_TO_LE_32(~EFX_SHIFT32(_bit, FIX_LINT(0))); \ (_oword).eo_u32[1] &= \ __CPU_TO_LE_32(~EFX_SHIFT32(_bit, FIX_LINT(32))); \ (_oword).eo_u32[2] &= \ __CPU_TO_LE_32(~EFX_SHIFT32(_bit, FIX_LINT(64))); \ (_oword).eo_u32[3] &= \ __CPU_TO_LE_32(~EFX_SHIFT32(_bit, FIX_LINT(96))); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_TEST_OWORD_BIT64(_oword, _bit) \ (((_oword).eo_u64[0] & \ __CPU_TO_LE_64(EFX_SHIFT64(_bit, FIX_LINT(0)))) || \ ((_oword).eo_u64[1] & \ __CPU_TO_LE_64(EFX_SHIFT64(_bit, FIX_LINT(64))))) #define EFX_TEST_OWORD_BIT32(_oword, _bit) \ (((_oword).eo_u32[0] & \ __CPU_TO_LE_32(EFX_SHIFT32(_bit, FIX_LINT(0)))) || \ ((_oword).eo_u32[1] & \ __CPU_TO_LE_32(EFX_SHIFT32(_bit, FIX_LINT(32)))) || \ ((_oword).eo_u32[2] & \ __CPU_TO_LE_32(EFX_SHIFT32(_bit, FIX_LINT(64)))) || \ ((_oword).eo_u32[3] & \ __CPU_TO_LE_32(EFX_SHIFT32(_bit, FIX_LINT(96))))) #define EFX_SET_QWORD_BIT64(_qword, _bit) \ do { \ _NOTE(CONSTANTCONDITION) \ (_qword).eq_u64[0] |= \ __CPU_TO_LE_64(EFX_SHIFT64(_bit, FIX_LINT(0))); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_SET_QWORD_BIT32(_qword, _bit) \ do { \ _NOTE(CONSTANTCONDITION) \ (_qword).eq_u32[0] |= \ __CPU_TO_LE_32(EFX_SHIFT32(_bit, FIX_LINT(0))); \ (_qword).eq_u32[1] |= \ __CPU_TO_LE_32(EFX_SHIFT32(_bit, FIX_LINT(32))); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_CLEAR_QWORD_BIT64(_qword, _bit) \ do { \ _NOTE(CONSTANTCONDITION) \ (_qword).eq_u64[0] &= \ __CPU_TO_LE_64(~EFX_SHIFT64(_bit, FIX_LINT(0))); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_CLEAR_QWORD_BIT32(_qword, _bit) \ do { \ _NOTE(CONSTANTCONDITION) \ (_qword).eq_u32[0] &= \ __CPU_TO_LE_32(~EFX_SHIFT32(_bit, FIX_LINT(0))); \ (_qword).eq_u32[1] &= \ __CPU_TO_LE_32(~EFX_SHIFT32(_bit, FIX_LINT(32))); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_TEST_QWORD_BIT64(_qword, _bit) \ (((_qword).eq_u64[0] & \ __CPU_TO_LE_64(EFX_SHIFT64(_bit, FIX_LINT(0)))) != 0) #define EFX_TEST_QWORD_BIT32(_qword, _bit) \ (((_qword).eq_u32[0] & \ __CPU_TO_LE_32(EFX_SHIFT32(_bit, FIX_LINT(0)))) || \ ((_qword).eq_u32[1] & \ __CPU_TO_LE_32(EFX_SHIFT32(_bit, FIX_LINT(32))))) #define EFX_SET_DWORD_BIT(_dword, _bit) \ do { \ (_dword).ed_u32[0] |= \ __CPU_TO_LE_32(EFX_SHIFT32(_bit, FIX_LINT(0))); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_CLEAR_DWORD_BIT(_dword, _bit) \ do { \ (_dword).ed_u32[0] &= \ __CPU_TO_LE_32(~EFX_SHIFT32(_bit, FIX_LINT(0))); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_TEST_DWORD_BIT(_dword, _bit) \ (((_dword).ed_u32[0] & \ __CPU_TO_LE_32(EFX_SHIFT32(_bit, FIX_LINT(0)))) != 0) #define EFX_SET_WORD_BIT(_word, _bit) \ do { \ (_word).ew_u16[0] |= \ __CPU_TO_LE_16(EFX_SHIFT16(_bit, FIX_LINT(0))); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_CLEAR_WORD_BIT(_word, _bit) \ do { \ (_word).ew_u32[0] &= \ __CPU_TO_LE_16(~EFX_SHIFT16(_bit, FIX_LINT(0))); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_TEST_WORD_BIT(_word, _bit) \ (((_word).ew_u16[0] & \ __CPU_TO_LE_16(EFX_SHIFT16(_bit, FIX_LINT(0)))) != 0) #define EFX_SET_BYTE_BIT(_byte, _bit) \ do { \ (_byte).eb_u8[0] |= \ __NATIVE_8(EFX_SHIFT8(_bit, FIX_LINT(0))); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_CLEAR_BYTE_BIT(_byte, _bit) \ do { \ (_byte).eb_u8[0] &= \ __NATIVE_8(~EFX_SHIFT8(_bit, FIX_LINT(0))); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_TEST_BYTE_BIT(_byte, _bit) \ (((_byte).eb_u8[0] & \ __NATIVE_8(EFX_SHIFT8(_bit, FIX_LINT(0)))) != 0) #define EFX_OR_OWORD64(_oword1, _oword2) \ do { \ (_oword1).eo_u64[0] |= (_oword2).eo_u64[0]; \ (_oword1).eo_u64[1] |= (_oword2).eo_u64[1]; \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_OR_OWORD32(_oword1, _oword2) \ do { \ (_oword1).eo_u32[0] |= (_oword2).eo_u32[0]; \ (_oword1).eo_u32[1] |= (_oword2).eo_u32[1]; \ (_oword1).eo_u32[2] |= (_oword2).eo_u32[2]; \ (_oword1).eo_u32[3] |= (_oword2).eo_u32[3]; \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_AND_OWORD64(_oword1, _oword2) \ do { \ (_oword1).eo_u64[0] &= (_oword2).eo_u64[0]; \ (_oword1).eo_u64[1] &= (_oword2).eo_u64[1]; \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_AND_OWORD32(_oword1, _oword2) \ do { \ (_oword1).eo_u32[0] &= (_oword2).eo_u32[0]; \ (_oword1).eo_u32[1] &= (_oword2).eo_u32[1]; \ (_oword1).eo_u32[2] &= (_oword2).eo_u32[2]; \ (_oword1).eo_u32[3] &= (_oword2).eo_u32[3]; \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_OR_QWORD64(_qword1, _qword2) \ do { \ (_qword1).eq_u64[0] |= (_qword2).eq_u64[0]; \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_OR_QWORD32(_qword1, _qword2) \ do { \ (_qword1).eq_u32[0] |= (_qword2).eq_u32[0]; \ (_qword1).eq_u32[1] |= (_qword2).eq_u32[1]; \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_AND_QWORD64(_qword1, _qword2) \ do { \ (_qword1).eq_u64[0] &= (_qword2).eq_u64[0]; \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_AND_QWORD32(_qword1, _qword2) \ do { \ (_qword1).eq_u32[0] &= (_qword2).eq_u32[0]; \ (_qword1).eq_u32[1] &= (_qword2).eq_u32[1]; \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_OR_DWORD(_dword1, _dword2) \ do { \ (_dword1).ed_u32[0] |= (_dword2).ed_u32[0]; \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_AND_DWORD(_dword1, _dword2) \ do { \ (_dword1).ed_u32[0] &= (_dword2).ed_u32[0]; \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_OR_WORD(_word1, _word2) \ do { \ (_word1).ew_u16[0] |= (_word2).ew_u16[0]; \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_AND_WORD(_word1, _word2) \ do { \ (_word1).ew_u16[0] &= (_word2).ew_u16[0]; \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_OR_BYTE(_byte1, _byte2) \ do { \ - (_byte1).eb_u8[0] &= (_byte2).eb_u8[0]; \ + (_byte1).eb_u8[0] |= (_byte2).eb_u8[0]; \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #define EFX_AND_BYTE(_byte1, _byte2) \ do { \ (_byte1).eb_u8[0] &= (_byte2).eb_u8[0]; \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #if EFSYS_USE_UINT64 #define EFX_OWORD_FIELD EFX_OWORD_FIELD64 #define EFX_QWORD_FIELD EFX_QWORD_FIELD64 #define EFX_OWORD_IS_EQUAL EFX_OWORD_IS_EQUAL64 #define EFX_QWORD_IS_EQUAL EFX_QWORD_IS_EQUAL64 #define EFX_OWORD_IS_ZERO EFX_OWORD_IS_ZERO64 #define EFX_QWORD_IS_ZERO EFX_QWORD_IS_ZERO64 #define EFX_OWORD_IS_SET EFX_OWORD_IS_SET64 #define EFX_QWORD_IS_SET EFX_QWORD_IS_SET64 #define EFX_POPULATE_OWORD EFX_POPULATE_OWORD64 #define EFX_POPULATE_QWORD EFX_POPULATE_QWORD64 #define EFX_SET_OWORD_FIELD EFX_SET_OWORD_FIELD64 #define EFX_SET_QWORD_FIELD EFX_SET_QWORD_FIELD64 #define EFX_SET_OWORD_BIT EFX_SET_OWORD_BIT64 #define EFX_CLEAR_OWORD_BIT EFX_CLEAR_OWORD_BIT64 #define EFX_TEST_OWORD_BIT EFX_TEST_OWORD_BIT64 #define EFX_SET_QWORD_BIT EFX_SET_QWORD_BIT64 #define EFX_CLEAR_QWORD_BIT EFX_CLEAR_QWORD_BIT64 #define EFX_TEST_QWORD_BIT EFX_TEST_QWORD_BIT64 #define EFX_OR_OWORD EFX_OR_OWORD64 #define EFX_AND_OWORD EFX_AND_OWORD64 #define EFX_OR_QWORD EFX_OR_QWORD64 #define EFX_AND_QWORD EFX_AND_QWORD64 #else #define EFX_OWORD_FIELD EFX_OWORD_FIELD32 #define EFX_QWORD_FIELD EFX_QWORD_FIELD32 #define EFX_OWORD_IS_EQUAL EFX_OWORD_IS_EQUAL32 #define EFX_QWORD_IS_EQUAL EFX_QWORD_IS_EQUAL32 #define EFX_OWORD_IS_ZERO EFX_OWORD_IS_ZERO32 #define EFX_QWORD_IS_ZERO EFX_QWORD_IS_ZERO32 #define EFX_OWORD_IS_SET EFX_OWORD_IS_SET32 #define EFX_QWORD_IS_SET EFX_QWORD_IS_SET32 #define EFX_POPULATE_OWORD EFX_POPULATE_OWORD32 #define EFX_POPULATE_QWORD EFX_POPULATE_QWORD32 #define EFX_SET_OWORD_FIELD EFX_SET_OWORD_FIELD32 #define EFX_SET_QWORD_FIELD EFX_SET_QWORD_FIELD32 #define EFX_SET_OWORD_BIT EFX_SET_OWORD_BIT32 #define EFX_CLEAR_OWORD_BIT EFX_CLEAR_OWORD_BIT32 #define EFX_TEST_OWORD_BIT EFX_TEST_OWORD_BIT32 #define EFX_SET_QWORD_BIT EFX_SET_QWORD_BIT32 #define EFX_CLEAR_QWORD_BIT EFX_CLEAR_QWORD_BIT32 #define EFX_TEST_QWORD_BIT EFX_TEST_QWORD_BIT32 #define EFX_OR_OWORD EFX_OR_OWORD32 #define EFX_AND_OWORD EFX_AND_OWORD32 #define EFX_OR_QWORD EFX_OR_QWORD32 #define EFX_AND_QWORD EFX_AND_QWORD32 #endif #ifdef __cplusplus } #endif #endif /* _SYS_EFX_TYPES_H */ Index: user/ngie/more-tests2/sys/dev/sfxge/common/hunt_ev.c =================================================================== --- user/ngie/more-tests2/sys/dev/sfxge/common/hunt_ev.c (revision 292053) +++ user/ngie/more-tests2/sys/dev/sfxge/common/hunt_ev.c (revision 292054) @@ -1,1021 +1,1035 @@ /*- * Copyright (c) 2012-2015 Solarflare Communications Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are met: * * 1. Redistributions of source code must retain the above copyright notice, * this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright notice, * this list of conditions and the following disclaimer in the documentation * and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR * OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, * EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * The views and conclusions contained in the software and documentation are * those of the authors and should not be interpreted as representing official * policies, either expressed or implied, of the FreeBSD Project. */ #include __FBSDID("$FreeBSD$"); #include "efsys.h" #include "efx.h" #include "efx_types.h" #include "efx_regs.h" #include "efx_impl.h" #if EFSYS_OPT_MON_STATS #include "mcdi_mon.h" #endif #if EFSYS_OPT_HUNTINGTON #if EFSYS_OPT_QSTATS #define EFX_EV_QSTAT_INCR(_eep, _stat) \ do { \ (_eep)->ee_stat[_stat]++; \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) #else #define EFX_EV_QSTAT_INCR(_eep, _stat) #endif static __checkReturn boolean_t hunt_ev_rx( __in efx_evq_t *eep, __in efx_qword_t *eqp, __in const efx_ev_callbacks_t *eecp, __in_opt void *arg); static __checkReturn boolean_t hunt_ev_tx( __in efx_evq_t *eep, __in efx_qword_t *eqp, __in const efx_ev_callbacks_t *eecp, __in_opt void *arg); static __checkReturn boolean_t hunt_ev_driver( __in efx_evq_t *eep, __in efx_qword_t *eqp, __in const efx_ev_callbacks_t *eecp, __in_opt void *arg); static __checkReturn boolean_t hunt_ev_drv_gen( __in efx_evq_t *eep, __in efx_qword_t *eqp, __in const efx_ev_callbacks_t *eecp, __in_opt void *arg); static __checkReturn boolean_t hunt_ev_mcdi( __in efx_evq_t *eep, __in efx_qword_t *eqp, __in const efx_ev_callbacks_t *eecp, __in_opt void *arg); static __checkReturn efx_rc_t efx_mcdi_init_evq( __in efx_nic_t *enp, __in unsigned int instance, __in efsys_mem_t *esmp, __in size_t nevs, __in uint32_t irq, __out_opt uint32_t *irqp) { efx_mcdi_req_t req; uint8_t payload[ MAX(MC_CMD_INIT_EVQ_IN_LEN(EFX_EVQ_NBUFS(EFX_EVQ_MAXNEVS)), MC_CMD_INIT_EVQ_OUT_LEN)]; efx_qword_t *dma_addr; uint64_t addr; int npages; int i; int supports_rx_batching; efx_rc_t rc; npages = EFX_EVQ_NBUFS(nevs); if (MC_CMD_INIT_EVQ_IN_LEN(npages) > MC_CMD_INIT_EVQ_IN_LENMAX) { rc = EINVAL; goto fail1; } (void) memset(payload, 0, sizeof (payload)); req.emr_cmd = MC_CMD_INIT_EVQ; req.emr_in_buf = payload; req.emr_in_length = MC_CMD_INIT_EVQ_IN_LEN(npages); req.emr_out_buf = payload; req.emr_out_length = MC_CMD_INIT_EVQ_OUT_LEN; MCDI_IN_SET_DWORD(req, INIT_EVQ_IN_SIZE, nevs); MCDI_IN_SET_DWORD(req, INIT_EVQ_IN_INSTANCE, instance); MCDI_IN_SET_DWORD(req, INIT_EVQ_IN_IRQ_NUM, irq); /* * On Huntington RX and TX event batching can only be requested * together (even if the datapath firmware doesn't actually support RX * batching). * Cut through is incompatible with RX batching and so enabling cut * through disables RX batching (but it does not affect TX batching). * * So always enable RX and TX event batching, and enable cut through * if RX event batching isn't supported (i.e. on low latency firmware). */ supports_rx_batching = enp->en_nic_cfg.enc_rx_batching_enabled ? 1 : 0; MCDI_IN_POPULATE_DWORD_6(req, INIT_EVQ_IN_FLAGS, INIT_EVQ_IN_FLAG_INTERRUPTING, 1, INIT_EVQ_IN_FLAG_RPTR_DOS, 0, INIT_EVQ_IN_FLAG_INT_ARMD, 0, INIT_EVQ_IN_FLAG_CUT_THRU, !supports_rx_batching, INIT_EVQ_IN_FLAG_RX_MERGE, 1, INIT_EVQ_IN_FLAG_TX_MERGE, 1); MCDI_IN_SET_DWORD(req, INIT_EVQ_IN_TMR_MODE, MC_CMD_INIT_EVQ_IN_TMR_MODE_DIS); MCDI_IN_SET_DWORD(req, INIT_EVQ_IN_TMR_LOAD, 0); MCDI_IN_SET_DWORD(req, INIT_EVQ_IN_TMR_RELOAD, 0); MCDI_IN_SET_DWORD(req, INIT_EVQ_IN_COUNT_MODE, MC_CMD_INIT_EVQ_IN_COUNT_MODE_DIS); MCDI_IN_SET_DWORD(req, INIT_EVQ_IN_COUNT_THRSHLD, 0); dma_addr = MCDI_IN2(req, efx_qword_t, INIT_EVQ_IN_DMA_ADDR); addr = EFSYS_MEM_ADDR(esmp); for (i = 0; i < npages; i++) { EFX_POPULATE_QWORD_2(*dma_addr, EFX_DWORD_1, (uint32_t)(addr >> 32), EFX_DWORD_0, (uint32_t)(addr & 0xffffffff)); dma_addr++; addr += EFX_BUF_SIZE; } efx_mcdi_execute(enp, &req); if (req.emr_rc != 0) { rc = req.emr_rc; goto fail2; } if (req.emr_out_length_used < MC_CMD_INIT_EVQ_OUT_LEN) { rc = EMSGSIZE; goto fail3; } if (irqp != NULL) *irqp = MCDI_OUT_DWORD(req, INIT_EVQ_OUT_IRQ); return (0); fail3: EFSYS_PROBE(fail3); fail2: EFSYS_PROBE(fail2); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } static __checkReturn efx_rc_t efx_mcdi_fini_evq( __in efx_nic_t *enp, __in uint32_t instance) { efx_mcdi_req_t req; uint8_t payload[MAX(MC_CMD_FINI_EVQ_IN_LEN, MC_CMD_FINI_EVQ_OUT_LEN)]; efx_rc_t rc; (void) memset(payload, 0, sizeof (payload)); req.emr_cmd = MC_CMD_FINI_EVQ; req.emr_in_buf = payload; req.emr_in_length = MC_CMD_FINI_EVQ_IN_LEN; req.emr_out_buf = payload; req.emr_out_length = MC_CMD_FINI_EVQ_OUT_LEN; MCDI_IN_SET_DWORD(req, FINI_EVQ_IN_INSTANCE, instance); efx_mcdi_execute(enp, &req); if (req.emr_rc != 0) { rc = req.emr_rc; goto fail1; } return (0); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } __checkReturn efx_rc_t hunt_ev_init( __in efx_nic_t *enp) { _NOTE(ARGUNUSED(enp)) return (0); } void hunt_ev_fini( __in efx_nic_t *enp) { _NOTE(ARGUNUSED(enp)) } __checkReturn efx_rc_t hunt_ev_qcreate( __in efx_nic_t *enp, __in unsigned int index, __in efsys_mem_t *esmp, __in size_t n, __in uint32_t id, __in efx_evq_t *eep) { efx_nic_cfg_t *encp = &(enp->en_nic_cfg); uint32_t irq; efx_rc_t rc; _NOTE(ARGUNUSED(id)) /* buftbl id managed by MC */ EFX_STATIC_ASSERT(ISP2(EFX_EVQ_MAXNEVS)); EFX_STATIC_ASSERT(ISP2(EFX_EVQ_MINNEVS)); if (!ISP2(n) || (n < EFX_EVQ_MINNEVS) || (n > EFX_EVQ_MAXNEVS)) { rc = EINVAL; goto fail1; } if (index >= encp->enc_evq_limit) { rc = EINVAL; goto fail2; } /* Set up the handler table */ eep->ee_rx = hunt_ev_rx; eep->ee_tx = hunt_ev_tx; eep->ee_driver = hunt_ev_driver; eep->ee_drv_gen = hunt_ev_drv_gen; eep->ee_mcdi = hunt_ev_mcdi; /* * Set up the event queue * NOTE: ignore the returned IRQ param as firmware does not set it. */ irq = index; /* INIT_EVQ expects function-relative vector number */ if ((rc = efx_mcdi_init_evq(enp, index, esmp, n, irq, NULL)) != 0) goto fail3; return (0); fail3: EFSYS_PROBE(fail3); fail2: EFSYS_PROBE(fail2); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } void hunt_ev_qdestroy( __in efx_evq_t *eep) { efx_nic_t *enp = eep->ee_enp; EFSYS_ASSERT(enp->en_family == EFX_FAMILY_HUNTINGTON); (void) efx_mcdi_fini_evq(eep->ee_enp, eep->ee_index); } __checkReturn efx_rc_t hunt_ev_qprime( __in efx_evq_t *eep, __in unsigned int count) { efx_nic_t *enp = eep->ee_enp; uint32_t rptr; efx_dword_t dword; rptr = count & eep->ee_mask; if (enp->en_nic_cfg.enc_bug35388_workaround) { EFX_STATIC_ASSERT(EFX_EVQ_MINNEVS > (1 << ERF_DD_EVQ_IND_RPTR_WIDTH)); EFX_STATIC_ASSERT(EFX_EVQ_MAXNEVS < (1 << 2 * ERF_DD_EVQ_IND_RPTR_WIDTH)); EFX_POPULATE_DWORD_2(dword, ERF_DD_EVQ_IND_RPTR_FLAGS, EFE_DD_EVQ_IND_RPTR_FLAGS_HIGH, ERF_DD_EVQ_IND_RPTR, (rptr >> ERF_DD_EVQ_IND_RPTR_WIDTH)); EFX_BAR_TBL_WRITED(enp, ER_DD_EVQ_INDIRECT, eep->ee_index, &dword, B_FALSE); EFX_POPULATE_DWORD_2(dword, ERF_DD_EVQ_IND_RPTR_FLAGS, EFE_DD_EVQ_IND_RPTR_FLAGS_LOW, ERF_DD_EVQ_IND_RPTR, rptr & ((1 << ERF_DD_EVQ_IND_RPTR_WIDTH) - 1)); EFX_BAR_TBL_WRITED(enp, ER_DD_EVQ_INDIRECT, eep->ee_index, &dword, B_FALSE); } else { EFX_POPULATE_DWORD_1(dword, ERF_DZ_EVQ_RPTR, rptr); EFX_BAR_TBL_WRITED(enp, ER_DZ_EVQ_RPTR_REG, eep->ee_index, &dword, B_FALSE); } return (0); } static __checkReturn efx_rc_t efx_mcdi_driver_event( __in efx_nic_t *enp, __in uint32_t evq, __in efx_qword_t data) { efx_mcdi_req_t req; uint8_t payload[MAX(MC_CMD_DRIVER_EVENT_IN_LEN, MC_CMD_DRIVER_EVENT_OUT_LEN)]; efx_rc_t rc; req.emr_cmd = MC_CMD_DRIVER_EVENT; req.emr_in_buf = payload; req.emr_in_length = MC_CMD_DRIVER_EVENT_IN_LEN; req.emr_out_buf = payload; req.emr_out_length = MC_CMD_DRIVER_EVENT_OUT_LEN; MCDI_IN_SET_DWORD(req, DRIVER_EVENT_IN_EVQ, evq); MCDI_IN_SET_DWORD(req, DRIVER_EVENT_IN_DATA_LO, EFX_QWORD_FIELD(data, EFX_DWORD_0)); MCDI_IN_SET_DWORD(req, DRIVER_EVENT_IN_DATA_HI, EFX_QWORD_FIELD(data, EFX_DWORD_1)); efx_mcdi_execute(enp, &req); if (req.emr_rc != 0) { rc = req.emr_rc; goto fail1; } return (0); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } void hunt_ev_qpost( __in efx_evq_t *eep, __in uint16_t data) { efx_nic_t *enp = eep->ee_enp; efx_qword_t event; EFX_POPULATE_QWORD_3(event, ESF_DZ_DRV_CODE, ESE_DZ_EV_CODE_DRV_GEN_EV, ESF_DZ_DRV_SUB_CODE, 0, ESF_DZ_DRV_SUB_DATA_DW0, (uint32_t)data); (void) efx_mcdi_driver_event(enp, eep->ee_index, event); } __checkReturn efx_rc_t hunt_ev_qmoderate( __in efx_evq_t *eep, __in unsigned int us) { efx_nic_t *enp = eep->ee_enp; efx_nic_cfg_t *encp = &(enp->en_nic_cfg); efx_dword_t dword; uint32_t timer_val, mode; efx_rc_t rc; if (us > encp->enc_evq_timer_max_us) { rc = EINVAL; goto fail1; } /* If the value is zero then disable the timer */ if (us == 0) { timer_val = 0; mode = FFE_CZ_TIMER_MODE_DIS; } else { /* Calculate the timer value in quanta */ timer_val = us * 1000 / encp->enc_evq_timer_quantum_ns; /* Moderation value is base 0 so we need to deduct 1 */ if (timer_val > 0) timer_val--; mode = FFE_CZ_TIMER_MODE_INT_HLDOFF; } if (encp->enc_bug35388_workaround) { EFX_POPULATE_DWORD_3(dword, ERF_DD_EVQ_IND_TIMER_FLAGS, EFE_DD_EVQ_IND_TIMER_FLAGS, ERF_DD_EVQ_IND_TIMER_MODE, mode, ERF_DD_EVQ_IND_TIMER_VAL, timer_val); EFX_BAR_TBL_WRITED(enp, ER_DD_EVQ_INDIRECT, eep->ee_index, &dword, 0); } else { EFX_POPULATE_DWORD_2(dword, FRF_CZ_TC_TIMER_MODE, mode, FRF_CZ_TC_TIMER_VAL, timer_val); EFX_BAR_TBL_WRITED(enp, FR_BZ_TIMER_COMMAND_REGP0, eep->ee_index, &dword, 0); } return (0); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } #if EFSYS_OPT_QSTATS void hunt_ev_qstats_update( __in efx_evq_t *eep, __inout_ecount(EV_NQSTATS) efsys_stat_t *stat) { /* * TBD: Consider a common Siena/Huntington function. The code is * essentially identical. */ unsigned int id; for (id = 0; id < EV_NQSTATS; id++) { efsys_stat_t *essp = &stat[id]; EFSYS_STAT_INCR(essp, eep->ee_stat[id]); eep->ee_stat[id] = 0; } } #endif /* EFSYS_OPT_QSTATS */ static __checkReturn boolean_t hunt_ev_rx( __in efx_evq_t *eep, __in efx_qword_t *eqp, __in const efx_ev_callbacks_t *eecp, __in_opt void *arg) { efx_nic_t *enp = eep->ee_enp; uint32_t size; #if 0 boolean_t parse_err; #endif uint32_t label; uint32_t mcast; uint32_t eth_base_class; uint32_t eth_tag_class; uint32_t l3_class; uint32_t l4_class; uint32_t next_read_lbits; boolean_t soft1, soft2; uint16_t flags; boolean_t should_abort; efx_evq_rxq_state_t *eersp; unsigned int desc_count; unsigned int last_used_id; EFX_EV_QSTAT_INCR(eep, EV_RX); /* Discard events after RXQ/TXQ errors */ if (enp->en_reset_flags & (EFX_RESET_RXQ_ERR | EFX_RESET_TXQ_ERR)) return (B_FALSE); /* * FIXME: likely to be incomplete, incorrect and inefficient. * Improvements in all three areas are required. */ if (EFX_QWORD_FIELD(*eqp, ESF_DZ_RX_DROP_EVENT) != 0) { /* Drop this event */ return (B_FALSE); } flags = 0; size = EFX_QWORD_FIELD(*eqp, ESF_DZ_RX_BYTES); if (EFX_QWORD_FIELD(*eqp, ESF_DZ_RX_CONT) != 0) { /* * This may be part of a scattered frame, or it may be a * truncated frame if scatter is disabled on this RXQ. * Overlength frames can be received if e.g. a VF is configured * for 1500 MTU but connected to a port set to 9000 MTU * (see bug56567). * FIXME: There is not yet any driver that supports scatter on * Huntington. Scatter support is required for OSX. */ flags |= EFX_PKT_CONT; } #if 0 /* TODO What to do if the packet is flagged with parsing error */ parse_err = (EFX_QWORD_FIELD(*eqp, ESF_DZ_RX_PARSE_INCOMPLETE) != 0); #endif label = EFX_QWORD_FIELD(*eqp, ESF_DZ_RX_QLABEL); if (EFX_QWORD_FIELD(*eqp, ESF_DZ_RX_ECRC_ERR) != 0) { /* Ethernet frame CRC bad */ flags |= EFX_DISCARD; } if (EFX_QWORD_FIELD(*eqp, ESF_DZ_RX_CRC0_ERR) != 0) { /* IP+TCP, bad CRC in iSCSI header */ flags |= EFX_DISCARD; } if (EFX_QWORD_FIELD(*eqp, ESF_DZ_RX_CRC1_ERR) != 0) { /* IP+TCP, bad CRC in iSCSI payload or FCoE or FCoIP */ flags |= EFX_DISCARD; } if (EFX_QWORD_FIELD(*eqp, ESF_DZ_RX_ECC_ERR) != 0) { /* ECC memory error */ flags |= EFX_DISCARD; } /* FIXME: do we need soft bits from RXDP firmware ? */ soft1 = (EFX_QWORD_FIELD(*eqp, ESF_DZ_RX_EV_SOFT1) != 0); soft2 = (EFX_QWORD_FIELD(*eqp, ESF_DZ_RX_EV_SOFT2) != 0); mcast = EFX_QWORD_FIELD(*eqp, ESF_DZ_RX_MAC_CLASS); if (mcast == ESE_DZ_MAC_CLASS_UCAST) flags |= EFX_PKT_UNICAST; eth_base_class = EFX_QWORD_FIELD(*eqp, ESF_DZ_RX_ETH_BASE_CLASS); eth_tag_class = EFX_QWORD_FIELD(*eqp, ESF_DZ_RX_ETH_TAG_CLASS); l3_class = EFX_QWORD_FIELD(*eqp, ESF_DZ_RX_L3_CLASS); l4_class = EFX_QWORD_FIELD(*eqp, ESF_DZ_RX_L4_CLASS); /* bottom 4 bits of incremented index (not last desc consumed) */ next_read_lbits = EFX_QWORD_FIELD(*eqp, ESF_DZ_RX_DSC_PTR_LBITS); /* Increment the count of descriptors read */ eersp = &eep->ee_rxq_state[label]; desc_count = (next_read_lbits - eersp->eers_rx_read_ptr) & EFX_MASK32(ESF_DZ_RX_DSC_PTR_LBITS); eersp->eers_rx_read_ptr += desc_count; /* * FIXME: add error checking to make sure this a batched event. * This could also be an aborted scatter, see Bug36629. */ if (desc_count > 1) { EFX_EV_QSTAT_INCR(eep, EV_RX_BATCH); flags |= EFX_PKT_PREFIX_LEN; } /* Calculate the index of the the last descriptor consumed */ last_used_id = (eersp->eers_rx_read_ptr - 1) & eersp->eers_rx_mask; /* EFX_QWORD_FIELD(*eqp, ESF_DZ_RX_OVERRIDE_HOLDOFF); */ switch (eth_base_class) { case ESE_DZ_ETH_BASE_CLASS_LLC_SNAP: case ESE_DZ_ETH_BASE_CLASS_LLC: case ESE_DZ_ETH_BASE_CLASS_ETH2: default: break; } switch (eth_tag_class) { case ESE_DZ_ETH_TAG_CLASS_RSVD7: case ESE_DZ_ETH_TAG_CLASS_RSVD6: case ESE_DZ_ETH_TAG_CLASS_RSVD5: case ESE_DZ_ETH_TAG_CLASS_RSVD4: break; case ESE_DZ_ETH_TAG_CLASS_RSVD3: /* Triple tagged */ case ESE_DZ_ETH_TAG_CLASS_VLAN2: /* Double tagged */ case ESE_DZ_ETH_TAG_CLASS_VLAN1: /* VLAN tagged */ flags |= EFX_PKT_VLAN_TAGGED; break; case ESE_DZ_ETH_TAG_CLASS_NONE: default: break; } switch (l3_class) { case ESE_DZ_L3_CLASS_RSVD7: /* Used by firmware for packet overrun */ #if 0 parse_err = B_TRUE; #endif flags |= EFX_DISCARD; break; case ESE_DZ_L3_CLASS_ARP: case ESE_DZ_L3_CLASS_FCOE: break; case ESE_DZ_L3_CLASS_IP6_FRAG: case ESE_DZ_L3_CLASS_IP6: flags |= EFX_PKT_IPV6; break; case ESE_DZ_L3_CLASS_IP4_FRAG: case ESE_DZ_L3_CLASS_IP4: flags |= EFX_PKT_IPV4; if (EFX_QWORD_FIELD(*eqp, ESF_DZ_RX_IPCKSUM_ERR) == 0) flags |= EFX_CKSUM_IPV4; break; case ESE_DZ_L3_CLASS_UNKNOWN: default: break; } switch (l4_class) { case ESE_DZ_L4_CLASS_RSVD7: case ESE_DZ_L4_CLASS_RSVD6: case ESE_DZ_L4_CLASS_RSVD5: case ESE_DZ_L4_CLASS_RSVD4: case ESE_DZ_L4_CLASS_RSVD3: break; case ESE_DZ_L4_CLASS_UDP: flags |= EFX_PKT_UDP; if (EFX_QWORD_FIELD(*eqp, ESF_DZ_RX_TCPUDP_CKSUM_ERR) == 0) flags |= EFX_CKSUM_TCPUDP; break; case ESE_DZ_L4_CLASS_TCP: flags |= EFX_PKT_TCP; if (EFX_QWORD_FIELD(*eqp, ESF_DZ_RX_TCPUDP_CKSUM_ERR) == 0) flags |= EFX_CKSUM_TCPUDP; break; case ESE_DZ_L4_CLASS_UNKNOWN: default: break; } /* If we're not discarding the packet then it is ok */ if (~flags & EFX_DISCARD) EFX_EV_QSTAT_INCR(eep, EV_RX_OK); EFSYS_ASSERT(eecp->eec_rx != NULL); should_abort = eecp->eec_rx(arg, label, last_used_id, size, flags); return (should_abort); } static __checkReturn boolean_t hunt_ev_tx( __in efx_evq_t *eep, __in efx_qword_t *eqp, __in const efx_ev_callbacks_t *eecp, __in_opt void *arg) { efx_nic_t *enp = eep->ee_enp; uint32_t id; uint32_t label; boolean_t should_abort; EFX_EV_QSTAT_INCR(eep, EV_TX); /* Discard events after RXQ/TXQ errors */ if (enp->en_reset_flags & (EFX_RESET_RXQ_ERR | EFX_RESET_TXQ_ERR)) return (B_FALSE); if (EFX_QWORD_FIELD(*eqp, ESF_DZ_TX_DROP_EVENT) != 0) { /* Drop this event */ return (B_FALSE); } /* Per-packet TX completion (was per-descriptor for Falcon/Siena) */ id = EFX_QWORD_FIELD(*eqp, ESF_DZ_TX_DESCR_INDX); label = EFX_QWORD_FIELD(*eqp, ESF_DZ_TX_QLABEL); EFSYS_PROBE2(tx_complete, uint32_t, label, uint32_t, id); EFSYS_ASSERT(eecp->eec_tx != NULL); should_abort = eecp->eec_tx(arg, label, id); return (should_abort); } static __checkReturn boolean_t hunt_ev_driver( __in efx_evq_t *eep, __in efx_qword_t *eqp, __in const efx_ev_callbacks_t *eecp, __in_opt void *arg) { unsigned int code; boolean_t should_abort; EFX_EV_QSTAT_INCR(eep, EV_DRIVER); should_abort = B_FALSE; code = EFX_QWORD_FIELD(*eqp, ESF_DZ_DRV_SUB_CODE); switch (code) { case ESE_DZ_DRV_TIMER_EV: { uint32_t id; id = EFX_QWORD_FIELD(*eqp, ESF_DZ_DRV_TMR_ID); EFSYS_ASSERT(eecp->eec_timer != NULL); should_abort = eecp->eec_timer(arg, id); break; } case ESE_DZ_DRV_WAKE_UP_EV: { uint32_t id; id = EFX_QWORD_FIELD(*eqp, ESF_DZ_DRV_EVQ_ID); EFSYS_ASSERT(eecp->eec_wake_up != NULL); should_abort = eecp->eec_wake_up(arg, id); break; } case ESE_DZ_DRV_START_UP_EV: EFSYS_ASSERT(eecp->eec_initialized != NULL); should_abort = eecp->eec_initialized(arg); break; default: EFSYS_PROBE3(bad_event, unsigned int, eep->ee_index, uint32_t, EFX_QWORD_FIELD(*eqp, EFX_DWORD_1), uint32_t, EFX_QWORD_FIELD(*eqp, EFX_DWORD_0)); break; } return (should_abort); } static __checkReturn boolean_t hunt_ev_drv_gen( __in efx_evq_t *eep, __in efx_qword_t *eqp, __in const efx_ev_callbacks_t *eecp, __in_opt void *arg) { uint32_t data; boolean_t should_abort; EFX_EV_QSTAT_INCR(eep, EV_DRV_GEN); should_abort = B_FALSE; data = EFX_QWORD_FIELD(*eqp, ESF_DZ_DRV_SUB_DATA_DW0); if (data >= ((uint32_t)1 << 16)) { EFSYS_PROBE3(bad_event, unsigned int, eep->ee_index, uint32_t, EFX_QWORD_FIELD(*eqp, EFX_DWORD_1), uint32_t, EFX_QWORD_FIELD(*eqp, EFX_DWORD_0)); return (B_TRUE); } EFSYS_ASSERT(eecp->eec_software != NULL); should_abort = eecp->eec_software(arg, (uint16_t)data); return (should_abort); } static __checkReturn boolean_t hunt_ev_mcdi( __in efx_evq_t *eep, __in efx_qword_t *eqp, __in const efx_ev_callbacks_t *eecp, __in_opt void *arg) { efx_nic_t *enp = eep->ee_enp; unsigned code; boolean_t should_abort = B_FALSE; EFX_EV_QSTAT_INCR(eep, EV_MCDI_RESPONSE); code = EFX_QWORD_FIELD(*eqp, MCDI_EVENT_CODE); switch (code) { case MCDI_EVENT_CODE_BADSSERT: efx_mcdi_ev_death(enp, EINTR); break; case MCDI_EVENT_CODE_CMDDONE: efx_mcdi_ev_cpl(enp, MCDI_EV_FIELD(eqp, CMDDONE_SEQ), MCDI_EV_FIELD(eqp, CMDDONE_DATALEN), MCDI_EV_FIELD(eqp, CMDDONE_ERRNO)); break; +#if EFSYS_OPT_MCDI_PROXY_AUTH + case MCDI_EVENT_CODE_PROXY_RESPONSE: + /* + * This event notifies a function that an authorization request + * has been processed. If the request was authorized then the + * function can now re-send the original MCDI request. + * See SF-113652-SW "SR-IOV Proxied Network Access Control". + */ + efx_mcdi_ev_proxy_response(enp, + MCDI_EV_FIELD(eqp, PROXY_RESPONSE_HANDLE), + MCDI_EV_FIELD(eqp, PROXY_RESPONSE_RC)); + break; +#endif /* EFSYS_OPT_MCDI_PROXY_AUTH */ + case MCDI_EVENT_CODE_LINKCHANGE: { efx_link_mode_t link_mode; hunt_phy_link_ev(enp, eqp, &link_mode); should_abort = eecp->eec_link_change(arg, link_mode); break; } case MCDI_EVENT_CODE_SENSOREVT: { #if EFSYS_OPT_MON_STATS efx_mon_stat_t id; efx_mon_stat_value_t value; efx_rc_t rc; /* Decode monitor stat for MCDI sensor (if supported) */ if ((rc = mcdi_mon_ev(enp, eqp, &id, &value)) == 0) { /* Report monitor stat change */ should_abort = eecp->eec_monitor(arg, id, value); } else if (rc == ENOTSUP) { should_abort = eecp->eec_exception(arg, EFX_EXCEPTION_UNKNOWN_SENSOREVT, MCDI_EV_FIELD(eqp, DATA)); } else { EFSYS_ASSERT(rc == ENODEV); /* Wrong port */ } #endif break; } case MCDI_EVENT_CODE_SCHEDERR: /* Informational only */ break; case MCDI_EVENT_CODE_REBOOT: /* Falcon/Siena only (should not been seen with Huntington). */ efx_mcdi_ev_death(enp, EIO); break; case MCDI_EVENT_CODE_MC_REBOOT: /* MC_REBOOT event is used for Huntington (EF10) and later. */ efx_mcdi_ev_death(enp, EIO); break; case MCDI_EVENT_CODE_MAC_STATS_DMA: #if EFSYS_OPT_MAC_STATS if (eecp->eec_mac_stats != NULL) { eecp->eec_mac_stats(arg, MCDI_EV_FIELD(eqp, MAC_STATS_DMA_GENERATION)); } #endif break; case MCDI_EVENT_CODE_FWALERT: { uint32_t reason = MCDI_EV_FIELD(eqp, FWALERT_REASON); if (reason == MCDI_EVENT_FWALERT_REASON_SRAM_ACCESS) should_abort = eecp->eec_exception(arg, EFX_EXCEPTION_FWALERT_SRAM, MCDI_EV_FIELD(eqp, FWALERT_DATA)); else should_abort = eecp->eec_exception(arg, EFX_EXCEPTION_UNKNOWN_FWALERT, MCDI_EV_FIELD(eqp, DATA)); break; } case MCDI_EVENT_CODE_TX_ERR: { /* * After a TXQ error is detected, firmware sends a TX_ERR event. * This may be followed by TX completions (which we discard), * and then finally by a TX_FLUSH event. Firmware destroys the * TXQ automatically after sending the TX_FLUSH event. */ enp->en_reset_flags |= EFX_RESET_TXQ_ERR; EFSYS_PROBE1(tx_descq_err, uint32_t, MCDI_EV_FIELD(eqp, DATA)); /* Inform the driver that a reset is required. */ eecp->eec_exception(arg, EFX_EXCEPTION_TX_ERROR, MCDI_EV_FIELD(eqp, TX_ERR_DATA)); break; } case MCDI_EVENT_CODE_TX_FLUSH: { uint32_t txq_index = MCDI_EV_FIELD(eqp, TX_FLUSH_TXQ); /* * EF10 firmware sends two TX_FLUSH events: one to the txq's * event queue, and one to evq 0 (with TX_FLUSH_TO_DRIVER set). * We want to wait for all completions, so ignore the events * with TX_FLUSH_TO_DRIVER. */ if (MCDI_EV_FIELD(eqp, TX_FLUSH_TO_DRIVER) != 0) { should_abort = B_FALSE; break; } EFX_EV_QSTAT_INCR(eep, EV_DRIVER_TX_DESCQ_FLS_DONE); EFSYS_PROBE1(tx_descq_fls_done, uint32_t, txq_index); EFSYS_ASSERT(eecp->eec_txq_flush_done != NULL); should_abort = eecp->eec_txq_flush_done(arg, txq_index); break; } case MCDI_EVENT_CODE_RX_ERR: { /* * After an RXQ error is detected, firmware sends an RX_ERR * event. This may be followed by RX events (which we discard), * and then finally by an RX_FLUSH event. Firmware destroys the * RXQ automatically after sending the RX_FLUSH event. */ enp->en_reset_flags |= EFX_RESET_RXQ_ERR; EFSYS_PROBE1(rx_descq_err, uint32_t, MCDI_EV_FIELD(eqp, DATA)); /* Inform the driver that a reset is required. */ eecp->eec_exception(arg, EFX_EXCEPTION_RX_ERROR, MCDI_EV_FIELD(eqp, RX_ERR_DATA)); break; } case MCDI_EVENT_CODE_RX_FLUSH: { uint32_t rxq_index = MCDI_EV_FIELD(eqp, RX_FLUSH_RXQ); /* * EF10 firmware sends two RX_FLUSH events: one to the rxq's * event queue, and one to evq 0 (with RX_FLUSH_TO_DRIVER set). * We want to wait for all completions, so ignore the events * with RX_FLUSH_TO_DRIVER. */ if (MCDI_EV_FIELD(eqp, RX_FLUSH_TO_DRIVER) != 0) { should_abort = B_FALSE; break; } EFX_EV_QSTAT_INCR(eep, EV_DRIVER_RX_DESCQ_FLS_DONE); EFSYS_PROBE1(rx_descq_fls_done, uint32_t, rxq_index); EFSYS_ASSERT(eecp->eec_rxq_flush_done != NULL); should_abort = eecp->eec_rxq_flush_done(arg, rxq_index); break; } default: EFSYS_PROBE3(bad_event, unsigned int, eep->ee_index, uint32_t, EFX_QWORD_FIELD(*eqp, EFX_DWORD_1), uint32_t, EFX_QWORD_FIELD(*eqp, EFX_DWORD_0)); break; } return (should_abort); } void hunt_ev_rxlabel_init( __in efx_evq_t *eep, __in efx_rxq_t *erp, __in unsigned int label) { efx_evq_rxq_state_t *eersp; EFSYS_ASSERT3U(label, <, EFX_ARRAY_SIZE(eep->ee_rxq_state)); eersp = &eep->ee_rxq_state[label]; EFSYS_ASSERT3U(eersp->eers_rx_mask, ==, 0); eersp->eers_rx_read_ptr = 0; eersp->eers_rx_mask = erp->er_mask; } void hunt_ev_rxlabel_fini( __in efx_evq_t *eep, __in unsigned int label) { efx_evq_rxq_state_t *eersp; EFSYS_ASSERT3U(label, <, EFX_ARRAY_SIZE(eep->ee_rxq_state)); eersp = &eep->ee_rxq_state[label]; EFSYS_ASSERT3U(eersp->eers_rx_mask, !=, 0); eersp->eers_rx_read_ptr = 0; eersp->eers_rx_mask = 0; } #endif /* EFSYS_OPT_HUNTINGTON */ Index: user/ngie/more-tests2/sys/dev/sfxge/common/hunt_impl.h =================================================================== --- user/ngie/more-tests2/sys/dev/sfxge/common/hunt_impl.h (revision 292053) +++ user/ngie/more-tests2/sys/dev/sfxge/common/hunt_impl.h (revision 292054) @@ -1,1051 +1,1057 @@ /*- * Copyright (c) 2012-2015 Solarflare Communications Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are met: * * 1. Redistributions of source code must retain the above copyright notice, * this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright notice, * this list of conditions and the following disclaimer in the documentation * and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR * OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, * EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * The views and conclusions contained in the software and documentation are * those of the authors and should not be interpreted as representing official * policies, either expressed or implied, of the FreeBSD Project. * * $FreeBSD$ */ #ifndef _SYS_HUNT_IMPL_H #define _SYS_HUNT_IMPL_H #include "efx.h" #include "efx_regs.h" #include "efx_regs_ef10.h" #include "efx_mcdi.h" #ifdef __cplusplus extern "C" { #endif #define HUNTINGTON_NVRAM_CHUNK 0x80 /* Alignment requirement for value written to RX WPTR: * the WPTR must be aligned to an 8 descriptor boundary */ #define HUNTINGTON_RX_WPTR_ALIGN 8 /* Invalid RSS context handle */ #define HUNTINGTON_RSS_CONTEXT_INVALID (0xffffffff) /* EV */ __checkReturn efx_rc_t hunt_ev_init( __in efx_nic_t *enp); void hunt_ev_fini( __in efx_nic_t *enp); __checkReturn efx_rc_t hunt_ev_qcreate( __in efx_nic_t *enp, __in unsigned int index, __in efsys_mem_t *esmp, __in size_t n, __in uint32_t id, __in efx_evq_t *eep); void hunt_ev_qdestroy( __in efx_evq_t *eep); __checkReturn efx_rc_t hunt_ev_qprime( __in efx_evq_t *eep, __in unsigned int count); void hunt_ev_qpost( __in efx_evq_t *eep, __in uint16_t data); __checkReturn efx_rc_t hunt_ev_qmoderate( __in efx_evq_t *eep, __in unsigned int us); #if EFSYS_OPT_QSTATS void hunt_ev_qstats_update( __in efx_evq_t *eep, __inout_ecount(EV_NQSTATS) efsys_stat_t *stat); #endif /* EFSYS_OPT_QSTATS */ void hunt_ev_rxlabel_init( __in efx_evq_t *eep, __in efx_rxq_t *erp, __in unsigned int label); void hunt_ev_rxlabel_fini( __in efx_evq_t *eep, __in unsigned int label); /* INTR */ __checkReturn efx_rc_t hunt_intr_init( __in efx_nic_t *enp, __in efx_intr_type_t type, __in efsys_mem_t *esmp); void hunt_intr_enable( __in efx_nic_t *enp); void hunt_intr_disable( __in efx_nic_t *enp); void hunt_intr_disable_unlocked( __in efx_nic_t *enp); __checkReturn efx_rc_t hunt_intr_trigger( __in efx_nic_t *enp, __in unsigned int level); void hunt_intr_fini( __in efx_nic_t *enp); /* NIC */ extern __checkReturn efx_rc_t hunt_nic_probe( __in efx_nic_t *enp); extern __checkReturn efx_rc_t hunt_nic_set_drv_limits( __inout efx_nic_t *enp, __in efx_drv_limits_t *edlp); extern __checkReturn efx_rc_t hunt_nic_get_vi_pool( __in efx_nic_t *enp, __out uint32_t *vi_countp); extern __checkReturn efx_rc_t hunt_nic_get_bar_region( __in efx_nic_t *enp, __in efx_nic_region_t region, __out uint32_t *offsetp, __out size_t *sizep); extern __checkReturn efx_rc_t hunt_nic_reset( __in efx_nic_t *enp); extern __checkReturn efx_rc_t hunt_nic_init( __in efx_nic_t *enp); #if EFSYS_OPT_DIAG extern __checkReturn efx_rc_t hunt_nic_register_test( __in efx_nic_t *enp); #endif /* EFSYS_OPT_DIAG */ extern void hunt_nic_fini( __in efx_nic_t *enp); extern void hunt_nic_unprobe( __in efx_nic_t *enp); /* MAC */ extern __checkReturn efx_rc_t hunt_mac_poll( __in efx_nic_t *enp, __out efx_link_mode_t *link_modep); extern __checkReturn efx_rc_t hunt_mac_up( __in efx_nic_t *enp, __out boolean_t *mac_upp); extern __checkReturn efx_rc_t hunt_mac_addr_set( __in efx_nic_t *enp); extern __checkReturn efx_rc_t hunt_mac_reconfigure( __in efx_nic_t *enp); extern __checkReturn efx_rc_t hunt_mac_multicast_list_set( __in efx_nic_t *enp); extern __checkReturn efx_rc_t hunt_mac_filter_default_rxq_set( __in efx_nic_t *enp, __in efx_rxq_t *erp, __in boolean_t using_rss); extern void hunt_mac_filter_default_rxq_clear( __in efx_nic_t *enp); #if EFSYS_OPT_LOOPBACK extern __checkReturn efx_rc_t hunt_mac_loopback_set( __in efx_nic_t *enp, __in efx_link_mode_t link_mode, __in efx_loopback_type_t loopback_type); #endif /* EFSYS_OPT_LOOPBACK */ #if EFSYS_OPT_MAC_STATS extern __checkReturn efx_rc_t hunt_mac_stats_update( __in efx_nic_t *enp, __in efsys_mem_t *esmp, __inout_ecount(EFX_MAC_NSTATS) efsys_stat_t *stat, __inout_opt uint32_t *generationp); #endif /* EFSYS_OPT_MAC_STATS */ /* MCDI */ #if EFSYS_OPT_MCDI extern __checkReturn efx_rc_t hunt_mcdi_init( __in efx_nic_t *enp, __in const efx_mcdi_transport_t *mtp); extern void hunt_mcdi_fini( __in efx_nic_t *enp); extern void hunt_mcdi_request_copyin( __in efx_nic_t *enp, __in efx_mcdi_req_t *emrp, __in unsigned int seq, __in boolean_t ev_cpl, __in boolean_t new_epoch); extern void hunt_mcdi_read_response( __in efx_nic_t *enp, __out void *bufferp, __in size_t offset, __in size_t length); extern __checkReturn boolean_t hunt_mcdi_request_poll( __in efx_nic_t *enp); extern void hunt_mcdi_request_copyout( __in efx_nic_t *enp, __in efx_mcdi_req_t *emrp); extern efx_rc_t hunt_mcdi_poll_reboot( __in efx_nic_t *enp); extern __checkReturn efx_rc_t hunt_mcdi_fw_update_supported( __in efx_nic_t *enp, __out boolean_t *supportedp); extern __checkReturn efx_rc_t hunt_mcdi_macaddr_change_supported( __in efx_nic_t *enp, __out boolean_t *supportedp); extern __checkReturn efx_rc_t hunt_mcdi_link_control_supported( __in efx_nic_t *enp, __out boolean_t *supportedp); +extern __checkReturn efx_rc_t +hunt_mcdi_mac_spoofing_supported( + __in efx_nic_t *enp, + __out boolean_t *supportedp); + + #endif /* EFSYS_OPT_MCDI */ /* NVRAM */ #if EFSYS_OPT_NVRAM || EFSYS_OPT_VPD extern __checkReturn efx_rc_t hunt_nvram_buf_read_tlv( __in efx_nic_t *enp, __in_bcount(max_seg_size) caddr_t seg_data, __in size_t max_seg_size, __in uint32_t tag, __deref_out_bcount_opt(*sizep) caddr_t *datap, __out size_t *sizep); extern __checkReturn efx_rc_t hunt_nvram_buf_write_tlv( __inout_bcount(partn_size) caddr_t partn_data, __in size_t partn_size, __in uint32_t tag, __in_bcount(tag_size) caddr_t tag_data, __in size_t tag_size, __out size_t *total_lengthp); extern __checkReturn efx_rc_t hunt_nvram_partn_read_tlv( __in efx_nic_t *enp, __in uint32_t partn, __in uint32_t tag, __deref_out_bcount_opt(*sizep) caddr_t *datap, __out size_t *sizep); extern __checkReturn efx_rc_t hunt_nvram_partn_write_tlv( __in efx_nic_t *enp, __in uint32_t partn, __in uint32_t tag, __in_bcount(size) caddr_t data, __in size_t size); extern __checkReturn efx_rc_t hunt_nvram_partn_write_segment_tlv( __in efx_nic_t *enp, __in uint32_t partn, __in uint32_t tag, __in_bcount(size) caddr_t data, __in size_t size, __in boolean_t all_segments); extern __checkReturn efx_rc_t hunt_nvram_partn_size( __in efx_nic_t *enp, __in unsigned int partn, __out size_t *sizep); extern __checkReturn efx_rc_t hunt_nvram_partn_lock( __in efx_nic_t *enp, __in unsigned int partn); extern __checkReturn efx_rc_t hunt_nvram_partn_read( __in efx_nic_t *enp, __in unsigned int partn, __in unsigned int offset, __out_bcount(size) caddr_t data, __in size_t size); extern __checkReturn efx_rc_t hunt_nvram_partn_erase( __in efx_nic_t *enp, __in unsigned int partn, __in unsigned int offset, __in size_t size); extern __checkReturn efx_rc_t hunt_nvram_partn_write( __in efx_nic_t *enp, __in unsigned int partn, __in unsigned int offset, __out_bcount(size) caddr_t data, __in size_t size); extern void hunt_nvram_partn_unlock( __in efx_nic_t *enp, __in unsigned int partn); #endif /* EFSYS_OPT_NVRAM || EFSYS_OPT_VPD */ #if EFSYS_OPT_NVRAM #if EFSYS_OPT_DIAG extern __checkReturn efx_rc_t hunt_nvram_test( __in efx_nic_t *enp); #endif /* EFSYS_OPT_DIAG */ extern __checkReturn efx_rc_t hunt_nvram_size( __in efx_nic_t *enp, __in efx_nvram_type_t type, __out size_t *sizep); extern __checkReturn efx_rc_t hunt_nvram_get_version( __in efx_nic_t *enp, __in efx_nvram_type_t type, __out uint32_t *subtypep, __out_ecount(4) uint16_t version[4]); extern __checkReturn efx_rc_t hunt_nvram_rw_start( __in efx_nic_t *enp, __in efx_nvram_type_t type, __out size_t *pref_chunkp); extern __checkReturn efx_rc_t hunt_nvram_read_chunk( __in efx_nic_t *enp, __in efx_nvram_type_t type, __in unsigned int offset, __out_bcount(size) caddr_t data, __in size_t size); extern __checkReturn efx_rc_t hunt_nvram_erase( __in efx_nic_t *enp, __in efx_nvram_type_t type); extern __checkReturn efx_rc_t hunt_nvram_write_chunk( __in efx_nic_t *enp, __in efx_nvram_type_t type, __in unsigned int offset, __in_bcount(size) caddr_t data, __in size_t size); extern void hunt_nvram_rw_finish( __in efx_nic_t *enp, __in efx_nvram_type_t type); extern __checkReturn efx_rc_t hunt_nvram_partn_set_version( __in efx_nic_t *enp, __in unsigned int partn, __in_ecount(4) uint16_t version[4]); extern __checkReturn efx_rc_t hunt_nvram_set_version( __in efx_nic_t *enp, __in efx_nvram_type_t type, __in_ecount(4) uint16_t version[4]); #endif /* EFSYS_OPT_NVRAM */ /* PHY */ typedef struct hunt_link_state_s { uint32_t hls_adv_cap_mask; uint32_t hls_lp_cap_mask; unsigned int hls_fcntl; efx_link_mode_t hls_link_mode; #if EFSYS_OPT_LOOPBACK efx_loopback_type_t hls_loopback; #endif boolean_t hls_mac_up; } hunt_link_state_t; extern void hunt_phy_link_ev( __in efx_nic_t *enp, __in efx_qword_t *eqp, __out efx_link_mode_t *link_modep); extern __checkReturn efx_rc_t hunt_phy_get_link( __in efx_nic_t *enp, __out hunt_link_state_t *hlsp); extern __checkReturn efx_rc_t hunt_phy_power( __in efx_nic_t *enp, __in boolean_t on); extern __checkReturn efx_rc_t hunt_phy_reconfigure( __in efx_nic_t *enp); extern __checkReturn efx_rc_t hunt_phy_verify( __in efx_nic_t *enp); extern __checkReturn efx_rc_t hunt_phy_oui_get( __in efx_nic_t *enp, __out uint32_t *ouip); #if EFSYS_OPT_PHY_STATS extern __checkReturn efx_rc_t hunt_phy_stats_update( __in efx_nic_t *enp, __in efsys_mem_t *esmp, __inout_ecount(EFX_PHY_NSTATS) uint32_t *stat); #endif /* EFSYS_OPT_PHY_STATS */ #if EFSYS_OPT_PHY_PROPS #if EFSYS_OPT_NAMES extern const char * hunt_phy_prop_name( __in efx_nic_t *enp, __in unsigned int id); #endif /* EFSYS_OPT_NAMES */ extern __checkReturn efx_rc_t hunt_phy_prop_get( __in efx_nic_t *enp, __in unsigned int id, __in uint32_t flags, __out uint32_t *valp); extern __checkReturn efx_rc_t hunt_phy_prop_set( __in efx_nic_t *enp, __in unsigned int id, __in uint32_t val); #endif /* EFSYS_OPT_PHY_PROPS */ #if EFSYS_OPT_BIST extern __checkReturn efx_rc_t hunt_bist_enable_offline( __in efx_nic_t *enp); extern __checkReturn efx_rc_t hunt_bist_start( __in efx_nic_t *enp, __in efx_bist_type_t type); extern __checkReturn efx_rc_t hunt_bist_poll( __in efx_nic_t *enp, __in efx_bist_type_t type, __out efx_bist_result_t *resultp, __out_opt __drv_when(count > 0, __notnull) uint32_t *value_maskp, __out_ecount_opt(count) __drv_when(count > 0, __notnull) unsigned long *valuesp, __in size_t count); extern void hunt_bist_stop( __in efx_nic_t *enp, __in efx_bist_type_t type); #endif /* EFSYS_OPT_BIST */ /* SRAM */ #if EFSYS_OPT_DIAG extern __checkReturn efx_rc_t hunt_sram_test( __in efx_nic_t *enp, __in efx_sram_pattern_fn_t func); #endif /* EFSYS_OPT_DIAG */ /* TX */ extern __checkReturn efx_rc_t hunt_tx_init( __in efx_nic_t *enp); extern void hunt_tx_fini( __in efx_nic_t *enp); extern __checkReturn efx_rc_t hunt_tx_qcreate( __in efx_nic_t *enp, __in unsigned int index, __in unsigned int label, __in efsys_mem_t *esmp, __in size_t n, __in uint32_t id, __in uint16_t flags, __in efx_evq_t *eep, __in efx_txq_t *etp, __out unsigned int *addedp); extern void hunt_tx_qdestroy( __in efx_txq_t *etp); extern __checkReturn efx_rc_t hunt_tx_qpost( __in efx_txq_t *etp, __in_ecount(n) efx_buffer_t *eb, __in unsigned int n, __in unsigned int completed, __inout unsigned int *addedp); extern void hunt_tx_qpush( __in efx_txq_t *etp, __in unsigned int added, __in unsigned int pushed); extern __checkReturn efx_rc_t hunt_tx_qpace( __in efx_txq_t *etp, __in unsigned int ns); extern __checkReturn efx_rc_t hunt_tx_qflush( __in efx_txq_t *etp); extern void hunt_tx_qenable( __in efx_txq_t *etp); extern __checkReturn efx_rc_t hunt_tx_qpio_enable( __in efx_txq_t *etp); extern void hunt_tx_qpio_disable( __in efx_txq_t *etp); extern __checkReturn efx_rc_t hunt_tx_qpio_write( __in efx_txq_t *etp, __in_ecount(buf_length) uint8_t *buffer, __in size_t buf_length, __in size_t pio_buf_offset); extern __checkReturn efx_rc_t hunt_tx_qpio_post( __in efx_txq_t *etp, __in size_t pkt_length, __in unsigned int completed, __inout unsigned int *addedp); extern __checkReturn efx_rc_t hunt_tx_qdesc_post( __in efx_txq_t *etp, __in_ecount(n) efx_desc_t *ed, __in unsigned int n, __in unsigned int completed, __inout unsigned int *addedp); extern void hunt_tx_qdesc_dma_create( __in efx_txq_t *etp, __in efsys_dma_addr_t addr, __in size_t size, __in boolean_t eop, __out efx_desc_t *edp); extern void hunt_tx_qdesc_tso_create( __in efx_txq_t *etp, __in uint16_t ipv4_id, __in uint32_t tcp_seq, __in uint8_t tcp_flags, __out efx_desc_t *edp); extern void hunt_tx_qdesc_vlantci_create( __in efx_txq_t *etp, __in uint16_t vlan_tci, __out efx_desc_t *edp); #if EFSYS_OPT_QSTATS extern void hunt_tx_qstats_update( __in efx_txq_t *etp, __inout_ecount(TX_NQSTATS) efsys_stat_t *stat); #endif /* EFSYS_OPT_QSTATS */ /* PIO */ /* Missing register definitions */ #ifndef ER_DZ_TX_PIOBUF_OFST #define ER_DZ_TX_PIOBUF_OFST 0x00001000 #endif #ifndef ER_DZ_TX_PIOBUF_STEP #define ER_DZ_TX_PIOBUF_STEP 8192 #endif #ifndef ER_DZ_TX_PIOBUF_ROWS #define ER_DZ_TX_PIOBUF_ROWS 2048 #endif #ifndef ER_DZ_TX_PIOBUF_SIZE #define ER_DZ_TX_PIOBUF_SIZE 2048 #endif #define HUNT_PIOBUF_NBUFS (16) #define HUNT_PIOBUF_SIZE (ER_DZ_TX_PIOBUF_SIZE) #define HUNT_MIN_PIO_ALLOC_SIZE (HUNT_PIOBUF_SIZE / 32) #define HUNT_LEGACY_PF_PRIVILEGE_MASK \ (MC_CMD_PRIVILEGE_MASK_IN_GRP_ADMIN | \ MC_CMD_PRIVILEGE_MASK_IN_GRP_LINK | \ MC_CMD_PRIVILEGE_MASK_IN_GRP_ONLOAD | \ MC_CMD_PRIVILEGE_MASK_IN_GRP_PTP | \ MC_CMD_PRIVILEGE_MASK_IN_GRP_INSECURE_FILTERS | \ MC_CMD_PRIVILEGE_MASK_IN_GRP_MAC_SPOOFING | \ MC_CMD_PRIVILEGE_MASK_IN_GRP_UNICAST | \ MC_CMD_PRIVILEGE_MASK_IN_GRP_MULTICAST | \ MC_CMD_PRIVILEGE_MASK_IN_GRP_BROADCAST | \ MC_CMD_PRIVILEGE_MASK_IN_GRP_ALL_MULTICAST | \ MC_CMD_PRIVILEGE_MASK_IN_GRP_PROMISCUOUS) #define HUNT_LEGACY_VF_PRIVILEGE_MASK 0 typedef uint32_t efx_piobuf_handle_t; #define EFX_PIOBUF_HANDLE_INVALID ((efx_piobuf_handle_t) -1) extern __checkReturn efx_rc_t hunt_nic_pio_alloc( __inout efx_nic_t *enp, __out uint32_t *bufnump, __out efx_piobuf_handle_t *handlep, __out uint32_t *blknump, __out uint32_t *offsetp, __out size_t *sizep); extern __checkReturn efx_rc_t hunt_nic_pio_free( __inout efx_nic_t *enp, __in uint32_t bufnum, __in uint32_t blknum); extern __checkReturn efx_rc_t hunt_nic_pio_link( __inout efx_nic_t *enp, __in uint32_t vi_index, __in efx_piobuf_handle_t handle); extern __checkReturn efx_rc_t hunt_nic_pio_unlink( __inout efx_nic_t *enp, __in uint32_t vi_index); /* VPD */ #if EFSYS_OPT_VPD extern __checkReturn efx_rc_t hunt_vpd_init( __in efx_nic_t *enp); extern __checkReturn efx_rc_t hunt_vpd_size( __in efx_nic_t *enp, __out size_t *sizep); extern __checkReturn efx_rc_t hunt_vpd_read( __in efx_nic_t *enp, __out_bcount(size) caddr_t data, __in size_t size); extern __checkReturn efx_rc_t hunt_vpd_verify( __in efx_nic_t *enp, __in_bcount(size) caddr_t data, __in size_t size); extern __checkReturn efx_rc_t hunt_vpd_reinit( __in efx_nic_t *enp, __in_bcount(size) caddr_t data, __in size_t size); extern __checkReturn efx_rc_t hunt_vpd_get( __in efx_nic_t *enp, __in_bcount(size) caddr_t data, __in size_t size, __inout efx_vpd_value_t *evvp); extern __checkReturn efx_rc_t hunt_vpd_set( __in efx_nic_t *enp, __in_bcount(size) caddr_t data, __in size_t size, __in efx_vpd_value_t *evvp); extern __checkReturn efx_rc_t hunt_vpd_next( __in efx_nic_t *enp, __in_bcount(size) caddr_t data, __in size_t size, __out efx_vpd_value_t *evvp, __inout unsigned int *contp); extern __checkReturn efx_rc_t hunt_vpd_write( __in efx_nic_t *enp, __in_bcount(size) caddr_t data, __in size_t size); extern void hunt_vpd_fini( __in efx_nic_t *enp); #endif /* EFSYS_OPT_VPD */ /* RX */ extern __checkReturn efx_rc_t hunt_rx_init( __in efx_nic_t *enp); #if EFSYS_OPT_RX_HDR_SPLIT extern __checkReturn efx_rc_t hunt_rx_hdr_split_enable( __in efx_nic_t *enp, __in unsigned int hdr_buf_size, __in unsigned int pld_buf_size); #endif /* EFSYS_OPT_RX_HDR_SPLIT */ #if EFSYS_OPT_RX_SCATTER extern __checkReturn efx_rc_t hunt_rx_scatter_enable( __in efx_nic_t *enp, __in unsigned int buf_size); #endif /* EFSYS_OPT_RX_SCATTER */ #if EFSYS_OPT_RX_SCALE extern __checkReturn efx_rc_t hunt_rx_scale_mode_set( __in efx_nic_t *enp, __in efx_rx_hash_alg_t alg, __in efx_rx_hash_type_t type, __in boolean_t insert); extern __checkReturn efx_rc_t hunt_rx_scale_key_set( __in efx_nic_t *enp, __in_ecount(n) uint8_t *key, __in size_t n); extern __checkReturn efx_rc_t hunt_rx_scale_tbl_set( __in efx_nic_t *enp, __in_ecount(n) unsigned int *table, __in size_t n); #endif /* EFSYS_OPT_RX_SCALE */ extern void hunt_rx_qpost( __in efx_rxq_t *erp, __in_ecount(n) efsys_dma_addr_t *addrp, __in size_t size, __in unsigned int n, __in unsigned int completed, __in unsigned int added); extern void hunt_rx_qpush( __in efx_rxq_t *erp, __in unsigned int added, __inout unsigned int *pushedp); extern __checkReturn efx_rc_t hunt_rx_qflush( __in efx_rxq_t *erp); extern void hunt_rx_qenable( __in efx_rxq_t *erp); extern __checkReturn efx_rc_t hunt_rx_qcreate( __in efx_nic_t *enp, __in unsigned int index, __in unsigned int label, __in efx_rxq_type_t type, __in efsys_mem_t *esmp, __in size_t n, __in uint32_t id, __in efx_evq_t *eep, __in efx_rxq_t *erp); extern void hunt_rx_qdestroy( __in efx_rxq_t *erp); extern void hunt_rx_fini( __in efx_nic_t *enp); #if EFSYS_OPT_FILTER typedef struct hunt_filter_handle_s { uint32_t hfh_lo; uint32_t hfh_hi; } hunt_filter_handle_t; typedef struct hunt_filter_entry_s { uintptr_t hfe_spec; /* pointer to filter spec plus busy bit */ hunt_filter_handle_t hfe_handle; } hunt_filter_entry_t; /* * BUSY flag indicates that an update is in progress. * AUTO_OLD flag is used to mark and sweep MAC packet filters. */ #define EFX_HUNT_FILTER_FLAG_BUSY 1U #define EFX_HUNT_FILTER_FLAG_AUTO_OLD 2U #define EFX_HUNT_FILTER_FLAGS 3U #define EFX_HUNT_FILTER_TBL_ROWS 8192 /* Allow for the broadcast address to be added to the multicast list */ #define EFX_HUNT_FILTER_MULTICAST_FILTERS_MAX (EFX_MAC_MULTICAST_LIST_MAX + 1) typedef struct hunt_filter_table_s { hunt_filter_entry_t hft_entry[EFX_HUNT_FILTER_TBL_ROWS]; efx_rxq_t * hft_default_rxq; boolean_t hft_using_rss; uint32_t hft_unicst_filter_index; boolean_t hft_unicst_filter_set; uint32_t hft_mulcst_filter_indexes[ EFX_HUNT_FILTER_MULTICAST_FILTERS_MAX]; uint32_t hft_mulcst_filter_count; } hunt_filter_table_t; __checkReturn efx_rc_t hunt_filter_init( __in efx_nic_t *enp); void hunt_filter_fini( __in efx_nic_t *enp); __checkReturn efx_rc_t hunt_filter_restore( __in efx_nic_t *enp); __checkReturn efx_rc_t hunt_filter_add( __in efx_nic_t *enp, __inout efx_filter_spec_t *spec, __in boolean_t may_replace); __checkReturn efx_rc_t hunt_filter_delete( __in efx_nic_t *enp, __inout efx_filter_spec_t *spec); extern __checkReturn efx_rc_t hunt_filter_supported_filters( __in efx_nic_t *enp, __out uint32_t *list, __out size_t *length); extern __checkReturn efx_rc_t hunt_filter_reconfigure( __in efx_nic_t *enp, __in_ecount(6) uint8_t const *mac_addr, __in boolean_t all_unicst, __in boolean_t mulcst, __in boolean_t all_mulcst, __in boolean_t brdcst, __in_ecount(6*count) uint8_t const *addrs, __in int count); extern void hunt_filter_get_default_rxq( __in efx_nic_t *enp, __out efx_rxq_t **erpp, __out boolean_t *using_rss); extern void hunt_filter_default_rxq_set( __in efx_nic_t *enp, __in efx_rxq_t *erp, __in boolean_t using_rss); extern void hunt_filter_default_rxq_clear( __in efx_nic_t *enp); #endif /* EFSYS_OPT_FILTER */ extern __checkReturn efx_rc_t hunt_pktfilter_set( __in efx_nic_t *enp, __in boolean_t unicst, __in boolean_t brdcst); #if EFSYS_OPT_MCAST_FILTER_LIST extern __checkReturn efx_rc_t hunt_pktfilter_mcast_set( __in efx_nic_t *enp, __in uint8_t const *addrs, __in int count); #endif /* EFSYS_OPT_MCAST_FILTER_LIST */ extern __checkReturn efx_rc_t hunt_pktfilter_mcast_all( __in efx_nic_t *enp); extern __checkReturn efx_rc_t efx_mcdi_get_function_info( __in efx_nic_t *enp, __out uint32_t *pfp, __out_opt uint32_t *vfp); extern __checkReturn efx_rc_t efx_mcdi_privilege_mask( __in efx_nic_t *enp, __in uint32_t pf, __in uint32_t vf, __out uint32_t *maskp); #ifdef __cplusplus } #endif #endif /* _SYS_HUNT_IMPL_H */ Index: user/ngie/more-tests2/sys/dev/sfxge/common/hunt_mcdi.c =================================================================== --- user/ngie/more-tests2/sys/dev/sfxge/common/hunt_mcdi.c (revision 292053) +++ user/ngie/more-tests2/sys/dev/sfxge/common/hunt_mcdi.c (revision 292054) @@ -1,555 +1,494 @@ /*- * Copyright (c) 2012-2015 Solarflare Communications Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are met: * * 1. Redistributions of source code must retain the above copyright notice, * this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright notice, * this list of conditions and the following disclaimer in the documentation * and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR * OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, * EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * The views and conclusions contained in the software and documentation are * those of the authors and should not be interpreted as representing official * policies, either expressed or implied, of the FreeBSD Project. */ #include __FBSDID("$FreeBSD$"); #include "efsys.h" #include "efx.h" #include "efx_impl.h" #if EFSYS_OPT_HUNTINGTON #if EFSYS_OPT_MCDI #ifndef WITH_MCDI_V2 #error "WITH_MCDI_V2 required for Huntington MCDIv2 commands." #endif typedef enum efx_mcdi_header_type_e { EFX_MCDI_HEADER_TYPE_V1, /* MCDIv0 (BootROM), MCDIv1 commands */ EFX_MCDI_HEADER_TYPE_V2, /* MCDIv2 commands */ } efx_mcdi_header_type_t; /* * Return the header format to use for sending an MCDI request. * * An MCDIv1 (Siena compatible) command should use MCDIv2 encapsulation if the * request input buffer or response output buffer are too large for the MCDIv1 * format. An MCDIv2 command must always be sent using MCDIv2 encapsulation. */ #define EFX_MCDI_HEADER_TYPE(_cmd, _length) \ ((((_cmd) & ~EFX_MASK32(MCDI_HEADER_CODE)) || \ ((_length) & ~EFX_MASK32(MCDI_HEADER_DATALEN))) ? \ EFX_MCDI_HEADER_TYPE_V2 : EFX_MCDI_HEADER_TYPE_V1) /* * MCDI Header NOT_EPOCH flag * ========================== * A new epoch begins at initial startup or after an MC reboot, and defines when * the MC should reject stale MCDI requests. * * The first MCDI request sent by the host should contain NOT_EPOCH=0, and all * subsequent requests (until the next MC reboot) should contain NOT_EPOCH=1. * * After rebooting the MC will fail all requests with NOT_EPOCH=1 by writing a * response with ERROR=1 and DATALEN=0 until a request is seen with NOT_EPOCH=0. */ __checkReturn efx_rc_t hunt_mcdi_init( __in efx_nic_t *enp, __in const efx_mcdi_transport_t *emtp) { efsys_mem_t *esmp = emtp->emt_dma_mem; efx_dword_t dword; efx_rc_t rc; EFSYS_ASSERT(enp->en_family == EFX_FAMILY_HUNTINGTON); EFSYS_ASSERT(enp->en_features & EFX_FEATURE_MCDI_DMA); /* A host DMA buffer is required for Huntington MCDI */ if (esmp == NULL) { rc = EINVAL; goto fail1; } /* * Ensure that the MC doorbell is in a known state before issuing MCDI * commands. The recovery algorithm requires that the MC command buffer * must be 256 byte aligned. See bug24769. */ if ((EFSYS_MEM_ADDR(esmp) & 0xFF) != 0) { rc = EINVAL; goto fail2; } EFX_POPULATE_DWORD_1(dword, EFX_DWORD_0, 1); EFX_BAR_WRITED(enp, ER_DZ_MC_DB_HWRD_REG, &dword, B_FALSE); /* Save initial MC reboot status */ (void) hunt_mcdi_poll_reboot(enp); /* Start a new epoch (allow fresh MCDI requests to succeed) */ efx_mcdi_new_epoch(enp); return (0); fail2: EFSYS_PROBE(fail2); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } void hunt_mcdi_fini( __in efx_nic_t *enp) { efx_mcdi_iface_t *emip = &(enp->en_mcdi.em_emip); emip->emi_new_epoch = B_FALSE; } void hunt_mcdi_request_copyin( __in efx_nic_t *enp, __in efx_mcdi_req_t *emrp, __in unsigned int seq, __in boolean_t ev_cpl, __in boolean_t new_epoch) { const efx_mcdi_transport_t *emtp = enp->en_mcdi.em_emtp; efsys_mem_t *esmp = emtp->emt_dma_mem; efx_mcdi_header_type_t hdr_type; efx_dword_t dword; efx_dword_t hdr[2]; unsigned int xflags; unsigned int pos; size_t offset; EFSYS_ASSERT3U(enp->en_family, ==, EFX_FAMILY_HUNTINGTON); xflags = 0; if (ev_cpl) xflags |= MCDI_HEADER_XFLAGS_EVREQ; offset = 0; hdr_type = EFX_MCDI_HEADER_TYPE(emrp->emr_cmd, MAX(emrp->emr_in_length, emrp->emr_out_length)); if (hdr_type == EFX_MCDI_HEADER_TYPE_V2) { /* Construct MCDI v2 header */ EFX_POPULATE_DWORD_8(hdr[0], MCDI_HEADER_CODE, MC_CMD_V2_EXTN, MCDI_HEADER_RESYNC, 1, MCDI_HEADER_DATALEN, 0, MCDI_HEADER_SEQ, seq, MCDI_HEADER_NOT_EPOCH, new_epoch ? 0 : 1, MCDI_HEADER_ERROR, 0, MCDI_HEADER_RESPONSE, 0, MCDI_HEADER_XFLAGS, xflags); EFSYS_MEM_WRITED(esmp, offset, &hdr[0]); offset += sizeof (efx_dword_t); EFX_POPULATE_DWORD_2(hdr[1], MC_CMD_V2_EXTN_IN_EXTENDED_CMD, emrp->emr_cmd, MC_CMD_V2_EXTN_IN_ACTUAL_LEN, emrp->emr_in_length); EFSYS_MEM_WRITED(esmp, offset, &hdr[1]); offset += sizeof (efx_dword_t); } else { /* Construct MCDI v1 header */ EFX_POPULATE_DWORD_8(hdr[0], MCDI_HEADER_CODE, emrp->emr_cmd, MCDI_HEADER_RESYNC, 1, MCDI_HEADER_DATALEN, emrp->emr_in_length, MCDI_HEADER_SEQ, seq, MCDI_HEADER_NOT_EPOCH, new_epoch ? 0 : 1, MCDI_HEADER_ERROR, 0, MCDI_HEADER_RESPONSE, 0, MCDI_HEADER_XFLAGS, xflags); EFSYS_MEM_WRITED(esmp, 0, &hdr[0]); offset += sizeof (efx_dword_t); } #if EFSYS_OPT_MCDI_LOGGING if (emtp->emt_logger != NULL) { emtp->emt_logger(emtp->emt_context, EFX_LOG_MCDI_REQUEST, &hdr, offset, emrp->emr_in_buf, emrp->emr_in_length); } #endif /* EFSYS_OPT_MCDI_LOGGING */ /* Construct the payload */ for (pos = 0; pos < emrp->emr_in_length; pos += sizeof (efx_dword_t)) { memcpy(&dword, MCDI_IN(*emrp, efx_dword_t, pos), MIN(sizeof (dword), emrp->emr_in_length - pos)); EFSYS_MEM_WRITED(esmp, offset + pos, &dword); } /* Ring the doorbell to post the command DMA address to the MC */ EFSYS_ASSERT((EFSYS_MEM_ADDR(esmp) & 0xFF) == 0); /* Guarantee ordering of memory (MCDI request) and PIO (MC doorbell) */ EFSYS_DMA_SYNC_FOR_DEVICE(esmp, 0, offset + emrp->emr_in_length); EFSYS_PIO_WRITE_BARRIER(); EFX_POPULATE_DWORD_1(dword, EFX_DWORD_0, EFSYS_MEM_ADDR(esmp) >> 32); EFX_BAR_WRITED(enp, ER_DZ_MC_DB_LWRD_REG, &dword, B_FALSE); EFX_POPULATE_DWORD_1(dword, EFX_DWORD_0, EFSYS_MEM_ADDR(esmp) & 0xffffffff); EFX_BAR_WRITED(enp, ER_DZ_MC_DB_HWRD_REG, &dword, B_FALSE); } void hunt_mcdi_request_copyout( __in efx_nic_t *enp, __in efx_mcdi_req_t *emrp) { #if EFSYS_OPT_MCDI_LOGGING const efx_mcdi_transport_t *emtp = enp->en_mcdi.em_emtp; #endif /* EFSYS_OPT_MCDI_LOGGING */ efx_dword_t hdr[2]; unsigned int hdr_len; size_t bytes; if (emrp->emr_out_buf == NULL) return; /* Read the command header to detect MCDI response format */ hdr_len = sizeof (hdr[0]); hunt_mcdi_read_response(enp, &hdr[0], 0, hdr_len); if (EFX_DWORD_FIELD(hdr[0], MCDI_HEADER_CODE) == MC_CMD_V2_EXTN) { /* * Read the actual payload length. The length given in the event * is only correct for responses with the V1 format. */ hunt_mcdi_read_response(enp, &hdr[1], hdr_len, sizeof (hdr[1])); hdr_len += sizeof (hdr[1]); emrp->emr_out_length_used = EFX_DWORD_FIELD(hdr[1], MC_CMD_V2_EXTN_IN_ACTUAL_LEN); } /* Copy payload out into caller supplied buffer */ bytes = MIN(emrp->emr_out_length_used, emrp->emr_out_length); hunt_mcdi_read_response(enp, emrp->emr_out_buf, hdr_len, bytes); #if EFSYS_OPT_MCDI_LOGGING if (emtp->emt_logger != NULL) { emtp->emt_logger(emtp->emt_context, EFX_LOG_MCDI_RESPONSE, &hdr, hdr_len, emrp->emr_out_buf, bytes); } #endif /* EFSYS_OPT_MCDI_LOGGING */ } static __checkReturn boolean_t hunt_mcdi_poll_response( __in efx_nic_t *enp) { const efx_mcdi_transport_t *emtp = enp->en_mcdi.em_emtp; efsys_mem_t *esmp = emtp->emt_dma_mem; efx_dword_t hdr; EFSYS_MEM_READD(esmp, 0, &hdr); return (EFX_DWORD_FIELD(hdr, MCDI_HEADER_RESPONSE) ? B_TRUE : B_FALSE); } void hunt_mcdi_read_response( __in efx_nic_t *enp, __out void *bufferp, __in size_t offset, __in size_t length) { const efx_mcdi_transport_t *emtp = enp->en_mcdi.em_emtp; efsys_mem_t *esmp = emtp->emt_dma_mem; unsigned int pos; efx_dword_t data; for (pos = 0; pos < length; pos += sizeof (efx_dword_t)) { EFSYS_MEM_READD(esmp, offset + pos, &data); memcpy((uint8_t *)bufferp + pos, &data, MIN(sizeof (data), length - pos)); } } __checkReturn boolean_t hunt_mcdi_request_poll( __in efx_nic_t *enp) { -#if EFSYS_OPT_MCDI_LOGGING - const efx_mcdi_transport_t *emtp = enp->en_mcdi.em_emtp; -#endif /* EFSYS_OPT_MCDI_LOGGING */ efx_mcdi_iface_t *emip = &(enp->en_mcdi.em_emip); efx_mcdi_req_t *emrp; - efx_dword_t hdr[2]; - unsigned int hdr_len; - unsigned int data_len; - unsigned int seq; - unsigned int cmd; int state; efx_rc_t rc; EFSYS_ASSERT3U(enp->en_family, ==, EFX_FAMILY_HUNTINGTON); /* Serialise against post-watchdog efx_mcdi_ev* */ EFSYS_LOCK(enp->en_eslp, state); EFSYS_ASSERT(emip->emi_pending_req != NULL); EFSYS_ASSERT(!emip->emi_ev_cpl); emrp = emip->emi_pending_req; /* Check if a response is available */ if (hunt_mcdi_poll_response(enp) == B_FALSE) { EFSYS_UNLOCK(enp->en_eslp, state); return (B_FALSE); } /* Read the response header */ - hdr_len = sizeof (hdr[0]); - hunt_mcdi_read_response(enp, &hdr[0], 0, hdr_len); + efx_mcdi_read_response_header(enp, emrp); - if (EFX_DWORD_FIELD(hdr[0], MCDI_HEADER_CODE) == MC_CMD_V2_EXTN) { - hunt_mcdi_read_response(enp, &hdr[1], hdr_len, sizeof (hdr[1])); - hdr_len += sizeof (hdr[1]); - - cmd = EFX_DWORD_FIELD(hdr[1], MC_CMD_V2_EXTN_IN_EXTENDED_CMD); - data_len = - EFX_DWORD_FIELD(hdr[1], MC_CMD_V2_EXTN_IN_ACTUAL_LEN); - } else { - cmd = EFX_DWORD_FIELD(hdr[0], MCDI_HEADER_CODE); - data_len = EFX_DWORD_FIELD(hdr[0], MCDI_HEADER_DATALEN); - } - /* Request complete */ emip->emi_pending_req = NULL; - seq = (emip->emi_seq - 1) & EFX_MASK32(MCDI_HEADER_SEQ); - /* Check for synchronous reboot */ - if (EFX_DWORD_FIELD(hdr[0], MCDI_HEADER_ERROR) != 0 && data_len == 0) { - /* The MC has rebooted since the request was sent. */ - EFSYS_SPIN(EFX_MCDI_STATUS_SLEEP_US); - hunt_mcdi_poll_reboot(enp); - - EFSYS_UNLOCK(enp->en_eslp, state); - rc = EIO; - goto fail1; - } - /* Ensure stale MCDI requests fail after an MC reboot. */ emip->emi_new_epoch = B_FALSE; EFSYS_UNLOCK(enp->en_eslp, state); - /* Check that the returned data is consistent */ - if (cmd != emrp->emr_cmd || - EFX_DWORD_FIELD(hdr[0], MCDI_HEADER_SEQ) != seq) { - /* Response is for a different request */ - rc = EIO; - goto fail2; - } - if (EFX_DWORD_FIELD(hdr[0], MCDI_HEADER_ERROR)) { - efx_dword_t err[2]; - unsigned int err_len = MIN(data_len, sizeof (err)); - int err_code = MC_CMD_ERR_EPROTO; - int err_arg = 0; + if ((rc = emrp->emr_rc) != 0) + goto fail1; - /* Read error code (and arg num for MCDI v2 commands) */ - hunt_mcdi_read_response(enp, &err[0], hdr_len, err_len); - - if (err_len >= MC_CMD_ERR_CODE_OFST + sizeof (efx_dword_t)) - err_code = EFX_DWORD_FIELD(err[0], EFX_DWORD_0); - - if (err_len >= MC_CMD_ERR_ARG_OFST + sizeof (efx_dword_t)) - err_arg = EFX_DWORD_FIELD(err[1], EFX_DWORD_0); - -#if EFSYS_OPT_MCDI_LOGGING - if (emtp->emt_logger != NULL) { - emtp->emt_logger(emtp->emt_context, - EFX_LOG_MCDI_RESPONSE, - &hdr, hdr_len, - &err, err_len); - } -#endif /* EFSYS_OPT_MCDI_LOGGING */ - - rc = efx_mcdi_request_errcode(err_code); - if (!emrp->emr_quiet) { - EFSYS_PROBE3(mcdi_err_arg, int, emrp->emr_cmd, - int, err_code, int, err_arg); - } - goto fail3; - - } else { - emrp->emr_out_length_used = data_len; - emrp->emr_rc = 0; - hunt_mcdi_request_copyout(enp, emrp); - } - + hunt_mcdi_request_copyout(enp, emrp); goto out; -fail3: - if (!emrp->emr_quiet) - EFSYS_PROBE(fail3); -fail2: - if (!emrp->emr_quiet) - EFSYS_PROBE(fail2); fail1: if (!emrp->emr_quiet) EFSYS_PROBE1(fail1, efx_rc_t, rc); - /* Fill out error state */ - emrp->emr_rc = rc; - emrp->emr_out_length_used = 0; - /* Reboot/Assertion */ if (rc == EIO || rc == EINTR) efx_mcdi_raise_exception(enp, emrp, rc); out: return (B_TRUE); } efx_rc_t hunt_mcdi_poll_reboot( __in efx_nic_t *enp) { efx_mcdi_iface_t *emip = &(enp->en_mcdi.em_emip); efx_dword_t dword; uint32_t old_status; uint32_t new_status; efx_rc_t rc; old_status = emip->emi_mc_reboot_status; /* Update MC reboot status word */ EFX_BAR_TBL_READD(enp, ER_DZ_BIU_MC_SFT_STATUS_REG, 0, &dword, B_FALSE); new_status = dword.ed_u32[0]; /* MC has rebooted if the value has changed */ if (new_status != old_status) { emip->emi_mc_reboot_status = new_status; /* * FIXME: Ignore detected MC REBOOT for now. * * The Siena support for checking for MC reboot from status * flags is broken - see comments in siena_mcdi_poll_reboot(). * As the generic MCDI code is shared the Huntington reboot * detection suffers similar problems. * * Do not report an error when the boot status changes until * this can be handled by common code drivers (and reworked to * support Siena too). */ if (B_FALSE) { rc = EIO; goto fail1; } } return (0); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } __checkReturn efx_rc_t hunt_mcdi_fw_update_supported( __in efx_nic_t *enp, __out boolean_t *supportedp) { efx_nic_cfg_t *encp = &(enp->en_nic_cfg); EFSYS_ASSERT3U(enp->en_family, ==, EFX_FAMILY_HUNTINGTON); /* * Use privilege mask state at MCDI attach. * Admin privilege must be used prior to introduction of * specific flag. */ - *supportedp = (encp->enc_privilege_mask & - MC_CMD_PRIVILEGE_MASK_IN_GRP_ADMIN) - == MC_CMD_PRIVILEGE_MASK_IN_GRP_ADMIN; + *supportedp = + EFX_MCDI_HAVE_PRIVILEGE(encp->enc_privilege_mask, ADMIN); return (0); } __checkReturn efx_rc_t hunt_mcdi_macaddr_change_supported( __in efx_nic_t *enp, __out boolean_t *supportedp) { efx_nic_cfg_t *encp = &(enp->en_nic_cfg); uint32_t privilege_mask = encp->enc_privilege_mask; EFSYS_ASSERT3U(enp->en_family, ==, EFX_FAMILY_HUNTINGTON); /* * Use privilege mask state at MCDI attach. * Admin privilege must be used prior to introduction of - * specific flag (at v4.6). + * mac spoofing privilege (at v4.6), which is used up to + * introduction of change mac spoofing privilege (at v4.7) */ *supportedp = - ((privilege_mask & MC_CMD_PRIVILEGE_MASK_IN_GRP_MAC_SPOOFING) == - MC_CMD_PRIVILEGE_MASK_IN_GRP_MAC_SPOOFING) || - ((privilege_mask & MC_CMD_PRIVILEGE_MASK_IN_GRP_ADMIN) == - MC_CMD_PRIVILEGE_MASK_IN_GRP_ADMIN); + EFX_MCDI_HAVE_PRIVILEGE(privilege_mask, CHANGE_MAC) || + EFX_MCDI_HAVE_PRIVILEGE(privilege_mask, MAC_SPOOFING) || + EFX_MCDI_HAVE_PRIVILEGE(privilege_mask, ADMIN); return (0); } __checkReturn efx_rc_t +hunt_mcdi_mac_spoofing_supported( + __in efx_nic_t *enp, + __out boolean_t *supportedp) +{ + efx_nic_cfg_t *encp = &(enp->en_nic_cfg); + uint32_t privilege_mask = encp->enc_privilege_mask; + + EFSYS_ASSERT3U(enp->en_family, ==, EFX_FAMILY_HUNTINGTON); + + /* + * Use privilege mask state at MCDI attach. + * Admin privilege must be used prior to introduction of + * mac spoofing privilege (at v4.6), which is used up to + * introduction of mac spoofing TX privilege (at v4.7) + */ + *supportedp = + EFX_MCDI_HAVE_PRIVILEGE(privilege_mask, MAC_SPOOFING_TX) || + EFX_MCDI_HAVE_PRIVILEGE(privilege_mask, MAC_SPOOFING) || + EFX_MCDI_HAVE_PRIVILEGE(privilege_mask, ADMIN); + + return (0); +} + + + __checkReturn efx_rc_t hunt_mcdi_link_control_supported( __in efx_nic_t *enp, __out boolean_t *supportedp) { efx_nic_cfg_t *encp = &(enp->en_nic_cfg); uint32_t privilege_mask = encp->enc_privilege_mask; EFSYS_ASSERT3U(enp->en_family, ==, EFX_FAMILY_HUNTINGTON); /* * Use privilege mask state at MCDI attach. * Admin privilege used prior to introduction of * specific flag. */ *supportedp = - ((privilege_mask & MC_CMD_PRIVILEGE_MASK_IN_GRP_LINK) == - MC_CMD_PRIVILEGE_MASK_IN_GRP_LINK) || - ((privilege_mask & MC_CMD_PRIVILEGE_MASK_IN_GRP_ADMIN) == - MC_CMD_PRIVILEGE_MASK_IN_GRP_ADMIN); + EFX_MCDI_HAVE_PRIVILEGE(privilege_mask, LINK) || + EFX_MCDI_HAVE_PRIVILEGE(privilege_mask, ADMIN); return (0); } #endif /* EFSYS_OPT_MCDI */ #endif /* EFSYS_OPT_HUNTINGTON */ Index: user/ngie/more-tests2/sys/dev/sfxge/common/siena_mcdi.c =================================================================== --- user/ngie/more-tests2/sys/dev/sfxge/common/siena_mcdi.c (revision 292053) +++ user/ngie/more-tests2/sys/dev/sfxge/common/siena_mcdi.c (revision 292054) @@ -1,435 +1,367 @@ /*- * Copyright (c) 2012-2015 Solarflare Communications Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are met: * * 1. Redistributions of source code must retain the above copyright notice, * this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright notice, * this list of conditions and the following disclaimer in the documentation * and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR * OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, * EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * The views and conclusions contained in the software and documentation are * those of the authors and should not be interpreted as representing official * policies, either expressed or implied, of the FreeBSD Project. */ #include __FBSDID("$FreeBSD$"); #include "efsys.h" #include "efx.h" #include "efx_impl.h" #if EFSYS_OPT_SIENA && EFSYS_OPT_MCDI #define SIENA_MCDI_PDU(_emip) \ (((emip)->emi_port == 1) \ ? MC_SMEM_P0_PDU_OFST >> 2 \ : MC_SMEM_P1_PDU_OFST >> 2) #define SIENA_MCDI_DOORBELL(_emip) \ (((emip)->emi_port == 1) \ ? MC_SMEM_P0_DOORBELL_OFST >> 2 \ : MC_SMEM_P1_DOORBELL_OFST >> 2) #define SIENA_MCDI_STATUS(_emip) \ (((emip)->emi_port == 1) \ ? MC_SMEM_P0_STATUS_OFST >> 2 \ : MC_SMEM_P1_STATUS_OFST >> 2) void siena_mcdi_request_copyin( __in efx_nic_t *enp, __in efx_mcdi_req_t *emrp, __in unsigned int seq, __in boolean_t ev_cpl, __in boolean_t new_epoch) { #if EFSYS_OPT_MCDI_LOGGING const efx_mcdi_transport_t *emtp = enp->en_mcdi.em_emtp; #endif efx_mcdi_iface_t *emip = &(enp->en_mcdi.em_emip); efx_dword_t hdr; efx_dword_t dword; unsigned int xflags; unsigned int pdur; unsigned int dbr; unsigned int pos; EFSYS_ASSERT(enp->en_family == EFX_FAMILY_SIENA); _NOTE(ARGUNUSED(new_epoch)) EFSYS_ASSERT(emip->emi_port == 1 || emip->emi_port == 2); pdur = SIENA_MCDI_PDU(emip); dbr = SIENA_MCDI_DOORBELL(emip); xflags = 0; if (ev_cpl) xflags |= MCDI_HEADER_XFLAGS_EVREQ; /* Construct the header in shared memory */ EFX_POPULATE_DWORD_6(hdr, MCDI_HEADER_CODE, emrp->emr_cmd, MCDI_HEADER_RESYNC, 1, MCDI_HEADER_DATALEN, emrp->emr_in_length, MCDI_HEADER_SEQ, seq, MCDI_HEADER_RESPONSE, 0, MCDI_HEADER_XFLAGS, xflags); EFX_BAR_TBL_WRITED(enp, FR_CZ_MC_TREG_SMEM, pdur, &hdr, B_TRUE); #if EFSYS_OPT_MCDI_LOGGING if (emtp->emt_logger != NULL) { emtp->emt_logger(emtp->emt_context, EFX_LOG_MCDI_REQUEST, &hdr, sizeof (hdr), emrp->emr_in_buf, emrp->emr_in_length); } #endif /* EFSYS_OPT_MCDI_LOGGING */ /* Construct the payload */ for (pos = 0; pos < emrp->emr_in_length; pos += sizeof (efx_dword_t)) { memcpy(&dword, MCDI_IN(*emrp, efx_dword_t, pos), MIN(sizeof (dword), emrp->emr_in_length - pos)); EFX_BAR_TBL_WRITED(enp, FR_CZ_MC_TREG_SMEM, pdur + 1 + (pos >> 2), &dword, B_FALSE); } /* Ring the doorbell */ EFX_POPULATE_DWORD_1(dword, EFX_DWORD_0, 0xd004be11); EFX_BAR_TBL_WRITED(enp, FR_CZ_MC_TREG_SMEM, dbr, &dword, B_FALSE); } void siena_mcdi_request_copyout( __in efx_nic_t *enp, __in efx_mcdi_req_t *emrp) { #if EFSYS_OPT_MCDI_LOGGING const efx_mcdi_transport_t *emtp = enp->en_mcdi.em_emtp; efx_dword_t hdr; #endif size_t bytes = MIN(emrp->emr_out_length_used, emrp->emr_out_length); /* Copy payload out if caller supplied buffer */ if (emrp->emr_out_buf != NULL) { siena_mcdi_read_response(enp, emrp->emr_out_buf, sizeof (efx_dword_t), bytes); } #if EFSYS_OPT_MCDI_LOGGING if (emtp->emt_logger != NULL) { siena_mcdi_read_response(enp, &hdr, 0, sizeof (hdr)); emtp->emt_logger(emtp->emt_context, EFX_LOG_MCDI_RESPONSE, &hdr, sizeof (hdr), emrp->emr_out_buf, bytes); } #endif /* EFSYS_OPT_MCDI_LOGGING */ } efx_rc_t siena_mcdi_poll_reboot( __in efx_nic_t *enp) { #ifndef EFX_GRACEFUL_MC_REBOOT /* * This function is not being used properly. * Until its callers are fixed, it should always return 0. */ _NOTE(ARGUNUSED(enp)) return (0); #else efx_mcdi_iface_t *emip = &(enp->en_mcdi.em_emip); unsigned int rebootr; efx_dword_t dword; uint32_t value; EFSYS_ASSERT(enp->en_family == EFX_FAMILY_SIENA); EFSYS_ASSERT(emip->emi_port == 1 || emip->emi_port == 2); rebootr = SIENA_MCDI_STATUS(emip); EFX_BAR_TBL_READD(enp, FR_CZ_MC_TREG_SMEM, rebootr, &dword, B_FALSE); value = EFX_DWORD_FIELD(dword, EFX_DWORD_0); if (value == 0) return (0); EFX_ZERO_DWORD(dword); EFX_BAR_TBL_WRITED(enp, FR_CZ_MC_TREG_SMEM, rebootr, &dword, B_FALSE); if (value == MC_STATUS_DWORD_ASSERT) return (EINTR); else return (EIO); #endif } static __checkReturn boolean_t siena_mcdi_poll_response( __in efx_nic_t *enp) { efx_mcdi_iface_t *emip = &(enp->en_mcdi.em_emip); efx_dword_t hdr; unsigned int pdur; EFSYS_ASSERT(emip->emi_port == 1 || emip->emi_port == 2); pdur = SIENA_MCDI_PDU(emip); EFX_BAR_TBL_READD(enp, FR_CZ_MC_TREG_SMEM, pdur, &hdr, B_FALSE); return (EFX_DWORD_FIELD(hdr, MCDI_HEADER_RESPONSE) ? B_TRUE : B_FALSE); } void siena_mcdi_read_response( __in efx_nic_t *enp, __out void *bufferp, __in size_t offset, __in size_t length) { efx_mcdi_iface_t *emip = &(enp->en_mcdi.em_emip); unsigned int pdur; unsigned int pos; efx_dword_t data; EFSYS_ASSERT(emip->emi_port == 1 || emip->emi_port == 2); pdur = SIENA_MCDI_PDU(emip); for (pos = 0; pos < length; pos += sizeof (efx_dword_t)) { EFX_BAR_TBL_READD(enp, FR_CZ_MC_TREG_SMEM, pdur + ((offset + pos) >> 2), &data, B_FALSE); memcpy((uint8_t *)bufferp + pos, &data, MIN(sizeof (data), length - pos)); } } __checkReturn boolean_t siena_mcdi_request_poll( __in efx_nic_t *enp) { -#if EFSYS_OPT_MCDI_LOGGING - const efx_mcdi_transport_t *emtp = enp->en_mcdi.em_emtp; -#endif efx_mcdi_iface_t *emip = &(enp->en_mcdi.em_emip); efx_mcdi_req_t *emrp; - efx_dword_t hdr; - unsigned int hdr_len; - unsigned int data_len; - unsigned int seq; int state; efx_rc_t rc; EFSYS_ASSERT3U(enp->en_family, ==, EFX_FAMILY_SIENA); /* Serialise against post-watchdog efx_mcdi_ev* */ EFSYS_LOCK(enp->en_eslp, state); EFSYS_ASSERT(emip->emi_pending_req != NULL); EFSYS_ASSERT(!emip->emi_ev_cpl); emrp = emip->emi_pending_req; /* Check for reboot atomically w.r.t efx_mcdi_request_start */ if (emip->emi_poll_cnt++ == 0) { if ((rc = siena_mcdi_poll_reboot(enp)) != 0) { emip->emi_pending_req = NULL; EFSYS_UNLOCK(enp->en_eslp, state); goto fail1; } } /* Check if a response is available */ if (siena_mcdi_poll_response(enp) == B_FALSE) { EFSYS_UNLOCK(enp->en_eslp, state); return (B_FALSE); } /* Read the response header */ - hdr_len = sizeof (hdr); - siena_mcdi_read_response(enp, &hdr, 0, hdr_len); + efx_mcdi_read_response_header(enp, emrp); /* Request complete */ emip->emi_pending_req = NULL; - seq = (emip->emi_seq - 1) & EFX_MASK32(MCDI_HEADER_SEQ); - /* Check for synchronous reboot */ - if (EFX_DWORD_FIELD(hdr, MCDI_HEADER_ERROR) != 0 && - EFX_DWORD_FIELD(hdr, MCDI_HEADER_DATALEN) == 0) { - /* Consume status word */ - EFSYS_SPIN(EFX_MCDI_STATUS_SLEEP_US); - siena_mcdi_poll_reboot(enp); - EFSYS_UNLOCK(enp->en_eslp, state); - rc = EIO; - goto fail2; - } - EFSYS_UNLOCK(enp->en_eslp, state); - /* Check that the returned data is consistent */ - if (EFX_DWORD_FIELD(hdr, MCDI_HEADER_CODE) != emrp->emr_cmd || - EFX_DWORD_FIELD(hdr, MCDI_HEADER_SEQ) != seq) { - /* Response is for a different request */ - rc = EIO; - goto fail3; - } + if ((rc = emrp->emr_rc) != 0) + goto fail2; - data_len = EFX_DWORD_FIELD(hdr, MCDI_HEADER_DATALEN); - if (EFX_DWORD_FIELD(hdr, MCDI_HEADER_ERROR)) { - efx_dword_t err; - int err_code = MC_CMD_ERR_EPROTO; - unsigned int err_len = MIN(data_len, sizeof (err)); - - /* Read error code */ - siena_mcdi_read_response(enp, &err, hdr_len, err_len); - - if (err_len >= MC_CMD_ERR_CODE_OFST + sizeof (efx_dword_t)) - err_code = EFX_DWORD_FIELD(err, EFX_DWORD_0); - -#if EFSYS_OPT_MCDI_LOGGING - if (emtp->emt_logger != NULL) { - emtp->emt_logger(emtp->emt_context, - EFX_LOG_MCDI_RESPONSE, - &hdr, hdr_len, - &err, err_len); - } -#endif /* EFSYS_OPT_MCDI_LOGGING */ - - rc = efx_mcdi_request_errcode(err_code); - if (!emrp->emr_quiet) { - EFSYS_PROBE2(mcdi_err, int, emrp->emr_cmd, - int, err_code); - } - goto fail4; - - } else { - emrp->emr_out_length_used = data_len; - emrp->emr_rc = 0; - siena_mcdi_request_copyout(enp, emrp); - } - + siena_mcdi_request_copyout(enp, emrp); goto out; -fail4: - if (!emrp->emr_quiet) - EFSYS_PROBE(fail4); -fail3: - if (!emrp->emr_quiet) - EFSYS_PROBE(fail3); fail2: if (!emrp->emr_quiet) EFSYS_PROBE(fail2); fail1: if (!emrp->emr_quiet) EFSYS_PROBE1(fail1, efx_rc_t, rc); - - /* Fill out error state */ - emrp->emr_rc = rc; - emrp->emr_out_length_used = 0; /* Reboot/Assertion */ if (rc == EIO || rc == EINTR) efx_mcdi_raise_exception(enp, emrp, rc); out: return (B_TRUE); } __checkReturn efx_rc_t siena_mcdi_init( __in efx_nic_t *enp, __in const efx_mcdi_transport_t *mtp) { efx_mcdi_iface_t *emip = &(enp->en_mcdi.em_emip); efx_oword_t oword; unsigned int portnum; efx_rc_t rc; EFSYS_ASSERT(enp->en_family == EFX_FAMILY_SIENA); /* Determine the port number to use for MCDI */ EFX_BAR_READO(enp, FR_AZ_CS_DEBUG_REG, &oword); portnum = EFX_OWORD_FIELD(oword, FRF_CZ_CS_PORT_NUM); if (portnum == 0) { /* Presumably booted from ROM; only MCDI port 1 will work */ emip->emi_port = 1; } else if (portnum <= 2) { emip->emi_port = portnum; } else { rc = EINVAL; goto fail1; } /* * Wipe the atomic reboot status so subsequent MCDI requests succeed. * BOOT_STATUS is preserved so eno_nic_probe() can boot out of the * assertion handler. */ (void) siena_mcdi_poll_reboot(enp); return (0); fail1: EFSYS_PROBE1(fail1, efx_rc_t, rc); return (rc); } void siena_mcdi_fini( __in efx_nic_t *enp) { } __checkReturn efx_rc_t siena_mcdi_fw_update_supported( __in efx_nic_t *enp, __out boolean_t *supportedp) { EFSYS_ASSERT3U(enp->en_family, ==, EFX_FAMILY_SIENA); *supportedp = B_TRUE; return (0); } __checkReturn efx_rc_t siena_mcdi_macaddr_change_supported( __in efx_nic_t *enp, __out boolean_t *supportedp) { EFSYS_ASSERT3U(enp->en_family, ==, EFX_FAMILY_SIENA); *supportedp = B_TRUE; return (0); } __checkReturn efx_rc_t siena_mcdi_link_control_supported( __in efx_nic_t *enp, __out boolean_t *supportedp) { EFSYS_ASSERT3U(enp->en_family, ==, EFX_FAMILY_SIENA); *supportedp = B_TRUE; return (0); } #endif /* EFSYS_OPT_SIENA && EFSYS_OPT_MCDI */ Index: user/ngie/more-tests2/sys/dev/usb/wlan/if_urtwn.c =================================================================== --- user/ngie/more-tests2/sys/dev/usb/wlan/if_urtwn.c (revision 292053) +++ user/ngie/more-tests2/sys/dev/usb/wlan/if_urtwn.c (revision 292054) @@ -1,4108 +1,4160 @@ /* $OpenBSD: if_urtwn.c,v 1.16 2011/02/10 17:26:40 jakemsr Exp $ */ /*- * Copyright (c) 2010 Damien Bergamini * Copyright (c) 2014 Kevin Lo * * Permission to use, copy, modify, and distribute this software for any * purpose with or without fee is hereby granted, provided that the above * copyright notice and this permission notice appear in all copies. * * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ #include __FBSDID("$FreeBSD$"); /* * Driver for Realtek RTL8188CE-VAU/RTL8188CUS/RTL8188EU/RTL8188RU/RTL8192CU. */ #include "opt_wlan.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "usbdevs.h" #define USB_DEBUG_VAR urtwn_debug #include #include #include #ifdef USB_DEBUG static int urtwn_debug = 0; SYSCTL_NODE(_hw_usb, OID_AUTO, urtwn, CTLFLAG_RW, 0, "USB urtwn"); SYSCTL_INT(_hw_usb_urtwn, OID_AUTO, debug, CTLFLAG_RWTUN, &urtwn_debug, 0, "Debug level"); #endif #define IEEE80211_HAS_ADDR4(wh) IEEE80211_IS_DSTODS(wh) /* various supported device vendors/products */ static const STRUCT_USB_HOST_ID urtwn_devs[] = { #define URTWN_DEV(v,p) { USB_VP(USB_VENDOR_##v, USB_PRODUCT_##v##_##p) } #define URTWN_RTL8188E_DEV(v,p) \ { USB_VPI(USB_VENDOR_##v, USB_PRODUCT_##v##_##p, URTWN_RTL8188E) } #define URTWN_RTL8188E 1 URTWN_DEV(ABOCOM, RTL8188CU_1), URTWN_DEV(ABOCOM, RTL8188CU_2), URTWN_DEV(ABOCOM, RTL8192CU), URTWN_DEV(ASUS, RTL8192CU), URTWN_DEV(ASUS, USBN10NANO), URTWN_DEV(AZUREWAVE, RTL8188CE_1), URTWN_DEV(AZUREWAVE, RTL8188CE_2), URTWN_DEV(AZUREWAVE, RTL8188CU), URTWN_DEV(BELKIN, F7D2102), URTWN_DEV(BELKIN, RTL8188CU), URTWN_DEV(BELKIN, RTL8192CU), URTWN_DEV(CHICONY, RTL8188CUS_1), URTWN_DEV(CHICONY, RTL8188CUS_2), URTWN_DEV(CHICONY, RTL8188CUS_3), URTWN_DEV(CHICONY, RTL8188CUS_4), URTWN_DEV(CHICONY, RTL8188CUS_5), URTWN_DEV(COREGA, RTL8192CU), URTWN_DEV(DLINK, RTL8188CU), URTWN_DEV(DLINK, RTL8192CU_1), URTWN_DEV(DLINK, RTL8192CU_2), URTWN_DEV(DLINK, RTL8192CU_3), URTWN_DEV(DLINK, DWA131B), URTWN_DEV(EDIMAX, EW7811UN), URTWN_DEV(EDIMAX, RTL8192CU), URTWN_DEV(FEIXUN, RTL8188CU), URTWN_DEV(FEIXUN, RTL8192CU), URTWN_DEV(GUILLEMOT, HWNUP150), URTWN_DEV(HAWKING, RTL8192CU), URTWN_DEV(HP3, RTL8188CU), URTWN_DEV(NETGEAR, WNA1000M), URTWN_DEV(NETGEAR, RTL8192CU), URTWN_DEV(NETGEAR4, RTL8188CU), URTWN_DEV(NOVATECH, RTL8188CU), URTWN_DEV(PLANEX2, RTL8188CU_1), URTWN_DEV(PLANEX2, RTL8188CU_2), URTWN_DEV(PLANEX2, RTL8188CU_3), URTWN_DEV(PLANEX2, RTL8188CU_4), URTWN_DEV(PLANEX2, RTL8188CUS), URTWN_DEV(PLANEX2, RTL8192CU), URTWN_DEV(REALTEK, RTL8188CE_0), URTWN_DEV(REALTEK, RTL8188CE_1), URTWN_DEV(REALTEK, RTL8188CTV), URTWN_DEV(REALTEK, RTL8188CU_0), URTWN_DEV(REALTEK, RTL8188CU_1), URTWN_DEV(REALTEK, RTL8188CU_2), URTWN_DEV(REALTEK, RTL8188CU_3), URTWN_DEV(REALTEK, RTL8188CU_COMBO), URTWN_DEV(REALTEK, RTL8188CUS), URTWN_DEV(REALTEK, RTL8188RU_1), URTWN_DEV(REALTEK, RTL8188RU_2), URTWN_DEV(REALTEK, RTL8188RU_3), URTWN_DEV(REALTEK, RTL8191CU), URTWN_DEV(REALTEK, RTL8192CE), URTWN_DEV(REALTEK, RTL8192CU), URTWN_DEV(SITECOMEU, RTL8188CU_1), URTWN_DEV(SITECOMEU, RTL8188CU_2), URTWN_DEV(SITECOMEU, RTL8192CU), URTWN_DEV(TRENDNET, RTL8188CU), URTWN_DEV(TRENDNET, RTL8192CU), URTWN_DEV(ZYXEL, RTL8192CU), /* URTWN_RTL8188E */ URTWN_RTL8188E_DEV(DLINK, DWA123D1), URTWN_RTL8188E_DEV(DLINK, DWA125D1), URTWN_RTL8188E_DEV(ELECOM, WDC150SU2M), URTWN_RTL8188E_DEV(REALTEK, RTL8188ETV), URTWN_RTL8188E_DEV(REALTEK, RTL8188EU), #undef URTWN_RTL8188E_DEV #undef URTWN_DEV }; static device_probe_t urtwn_match; static device_attach_t urtwn_attach; static device_detach_t urtwn_detach; static usb_callback_t urtwn_bulk_tx_callback; static usb_callback_t urtwn_bulk_rx_callback; static void urtwn_drain_mbufq(struct urtwn_softc *sc); static usb_error_t urtwn_do_request(struct urtwn_softc *, struct usb_device_request *, void *); static struct ieee80211vap *urtwn_vap_create(struct ieee80211com *, const char [IFNAMSIZ], int, enum ieee80211_opmode, int, const uint8_t [IEEE80211_ADDR_LEN], const uint8_t [IEEE80211_ADDR_LEN]); static void urtwn_vap_delete(struct ieee80211vap *); static struct mbuf * urtwn_rx_frame(struct urtwn_softc *, uint8_t *, int, int *); static struct mbuf * urtwn_rxeof(struct usb_xfer *, struct urtwn_data *, int *, int8_t *); static void urtwn_txeof(struct urtwn_softc *, struct urtwn_data *, int); static int urtwn_alloc_list(struct urtwn_softc *, struct urtwn_data[], int, int); static int urtwn_alloc_rx_list(struct urtwn_softc *); static int urtwn_alloc_tx_list(struct urtwn_softc *); static void urtwn_free_list(struct urtwn_softc *, struct urtwn_data data[], int); static void urtwn_free_rx_list(struct urtwn_softc *); static void urtwn_free_tx_list(struct urtwn_softc *); static struct urtwn_data * _urtwn_getbuf(struct urtwn_softc *); static struct urtwn_data * urtwn_getbuf(struct urtwn_softc *); static usb_error_t urtwn_write_region_1(struct urtwn_softc *, uint16_t, uint8_t *, int); static usb_error_t urtwn_write_1(struct urtwn_softc *, uint16_t, uint8_t); static usb_error_t urtwn_write_2(struct urtwn_softc *, uint16_t, uint16_t); static usb_error_t urtwn_write_4(struct urtwn_softc *, uint16_t, uint32_t); static usb_error_t urtwn_read_region_1(struct urtwn_softc *, uint16_t, uint8_t *, int); static uint8_t urtwn_read_1(struct urtwn_softc *, uint16_t); static uint16_t urtwn_read_2(struct urtwn_softc *, uint16_t); static uint32_t urtwn_read_4(struct urtwn_softc *, uint16_t); static int urtwn_fw_cmd(struct urtwn_softc *, uint8_t, const void *, int); static void urtwn_r92c_rf_write(struct urtwn_softc *, int, uint8_t, uint32_t); static void urtwn_r88e_rf_write(struct urtwn_softc *, int, uint8_t, uint32_t); static uint32_t urtwn_rf_read(struct urtwn_softc *, int, uint8_t); static int urtwn_llt_write(struct urtwn_softc *, uint32_t, uint32_t); static int urtwn_efuse_read_next(struct urtwn_softc *, uint8_t *); static int urtwn_efuse_read_data(struct urtwn_softc *, uint8_t *, uint8_t, uint8_t); #ifdef URTWN_DEBUG static void urtwn_dump_rom_contents(struct urtwn_softc *, uint8_t *, uint16_t); #endif static int urtwn_efuse_read(struct urtwn_softc *, uint8_t *, uint16_t); static int urtwn_efuse_switch_power(struct urtwn_softc *); static int urtwn_read_chipid(struct urtwn_softc *); static int urtwn_read_rom(struct urtwn_softc *); static int urtwn_r88e_read_rom(struct urtwn_softc *); static int urtwn_ra_init(struct urtwn_softc *); static void urtwn_init_beacon(struct urtwn_softc *, struct urtwn_vap *); static int urtwn_setup_beacon(struct urtwn_softc *, struct ieee80211_node *); static void urtwn_update_beacon(struct ieee80211vap *, int); static int urtwn_tx_beacon(struct urtwn_softc *sc, struct urtwn_vap *); static void urtwn_tsf_task_adhoc(void *, int); static void urtwn_tsf_sync_enable(struct urtwn_softc *, struct ieee80211vap *); static void urtwn_set_led(struct urtwn_softc *, int, int); static void urtwn_set_mode(struct urtwn_softc *, uint8_t); static void urtwn_ibss_recv_mgmt(struct ieee80211_node *, struct mbuf *, int, const struct ieee80211_rx_stats *, int, int); static int urtwn_newstate(struct ieee80211vap *, enum ieee80211_state, int); static void urtwn_watchdog(void *); static void urtwn_update_avgrssi(struct urtwn_softc *, int, int8_t); static int8_t urtwn_get_rssi(struct urtwn_softc *, int, void *); static int8_t urtwn_r88e_get_rssi(struct urtwn_softc *, int, void *); static int urtwn_tx_data(struct urtwn_softc *, struct ieee80211_node *, struct mbuf *, struct urtwn_data *); static void urtwn_tx_start(struct urtwn_softc *, struct mbuf *, uint8_t, struct urtwn_data *); static int urtwn_transmit(struct ieee80211com *, struct mbuf *); static void urtwn_start(struct urtwn_softc *); static void urtwn_parent(struct ieee80211com *); static int urtwn_r92c_power_on(struct urtwn_softc *); static int urtwn_r88e_power_on(struct urtwn_softc *); static int urtwn_llt_init(struct urtwn_softc *); static void urtwn_fw_reset(struct urtwn_softc *); static void urtwn_r88e_fw_reset(struct urtwn_softc *); static int urtwn_fw_loadpage(struct urtwn_softc *, int, const uint8_t *, int); static int urtwn_load_firmware(struct urtwn_softc *); static int urtwn_dma_init(struct urtwn_softc *); static int urtwn_mac_init(struct urtwn_softc *); static void urtwn_bb_init(struct urtwn_softc *); static void urtwn_rf_init(struct urtwn_softc *); static void urtwn_cam_init(struct urtwn_softc *); static void urtwn_pa_bias_init(struct urtwn_softc *); static void urtwn_rxfilter_init(struct urtwn_softc *); static void urtwn_edca_init(struct urtwn_softc *); static void urtwn_write_txpower(struct urtwn_softc *, int, uint16_t[]); static void urtwn_get_txpower(struct urtwn_softc *, int, struct ieee80211_channel *, struct ieee80211_channel *, uint16_t[]); static void urtwn_r88e_get_txpower(struct urtwn_softc *, int, struct ieee80211_channel *, struct ieee80211_channel *, uint16_t[]); static void urtwn_set_txpower(struct urtwn_softc *, struct ieee80211_channel *, struct ieee80211_channel *); static void urtwn_set_rx_bssid_all(struct urtwn_softc *, int); static void urtwn_set_gain(struct urtwn_softc *, uint8_t); static void urtwn_scan_start(struct ieee80211com *); static void urtwn_scan_end(struct ieee80211com *); static void urtwn_set_channel(struct ieee80211com *); +static int urtwn_wme_update(struct ieee80211com *); static void urtwn_set_promisc(struct urtwn_softc *); static void urtwn_update_promisc(struct ieee80211com *); static void urtwn_update_mcast(struct ieee80211com *); static void urtwn_set_chan(struct urtwn_softc *, struct ieee80211_channel *, struct ieee80211_channel *); static void urtwn_iq_calib(struct urtwn_softc *); static void urtwn_lc_calib(struct urtwn_softc *); static int urtwn_init(struct urtwn_softc *); static void urtwn_stop(struct urtwn_softc *); static void urtwn_abort_xfers(struct urtwn_softc *); static int urtwn_raw_xmit(struct ieee80211_node *, struct mbuf *, const struct ieee80211_bpf_params *); static void urtwn_ms_delay(struct urtwn_softc *); /* Aliases. */ #define urtwn_bb_write urtwn_write_4 #define urtwn_bb_read urtwn_read_4 static const struct usb_config urtwn_config[URTWN_N_TRANSFER] = { [URTWN_BULK_RX] = { .type = UE_BULK, .endpoint = UE_ADDR_ANY, .direction = UE_DIR_IN, .bufsize = URTWN_RXBUFSZ, .flags = { .pipe_bof = 1, .short_xfer_ok = 1 }, .callback = urtwn_bulk_rx_callback, }, [URTWN_BULK_TX_BE] = { .type = UE_BULK, .endpoint = 0x03, .direction = UE_DIR_OUT, .bufsize = URTWN_TXBUFSZ, .flags = { .ext_buffer = 1, .pipe_bof = 1, .force_short_xfer = 1 }, .callback = urtwn_bulk_tx_callback, .timeout = URTWN_TX_TIMEOUT, /* ms */ }, [URTWN_BULK_TX_BK] = { .type = UE_BULK, .endpoint = 0x03, .direction = UE_DIR_OUT, .bufsize = URTWN_TXBUFSZ, .flags = { .ext_buffer = 1, .pipe_bof = 1, .force_short_xfer = 1, }, .callback = urtwn_bulk_tx_callback, .timeout = URTWN_TX_TIMEOUT, /* ms */ }, [URTWN_BULK_TX_VI] = { .type = UE_BULK, .endpoint = 0x02, .direction = UE_DIR_OUT, .bufsize = URTWN_TXBUFSZ, .flags = { .ext_buffer = 1, .pipe_bof = 1, .force_short_xfer = 1 }, .callback = urtwn_bulk_tx_callback, .timeout = URTWN_TX_TIMEOUT, /* ms */ }, [URTWN_BULK_TX_VO] = { .type = UE_BULK, .endpoint = 0x02, .direction = UE_DIR_OUT, .bufsize = URTWN_TXBUFSZ, .flags = { .ext_buffer = 1, .pipe_bof = 1, .force_short_xfer = 1 }, .callback = urtwn_bulk_tx_callback, .timeout = URTWN_TX_TIMEOUT, /* ms */ }, }; +static const struct wme_to_queue { + uint16_t reg; + uint8_t qid; +} wme2queue[WME_NUM_AC] = { + { R92C_EDCA_BE_PARAM, URTWN_BULK_TX_BE}, + { R92C_EDCA_BK_PARAM, URTWN_BULK_TX_BK}, + { R92C_EDCA_VI_PARAM, URTWN_BULK_TX_VI}, + { R92C_EDCA_VO_PARAM, URTWN_BULK_TX_VO} +}; + static int urtwn_match(device_t self) { struct usb_attach_arg *uaa = device_get_ivars(self); if (uaa->usb_mode != USB_MODE_HOST) return (ENXIO); if (uaa->info.bConfigIndex != URTWN_CONFIG_INDEX) return (ENXIO); if (uaa->info.bIfaceIndex != URTWN_IFACE_INDEX) return (ENXIO); return (usbd_lookup_id_by_uaa(urtwn_devs, sizeof(urtwn_devs), uaa)); } static int urtwn_attach(device_t self) { struct usb_attach_arg *uaa = device_get_ivars(self); struct urtwn_softc *sc = device_get_softc(self); struct ieee80211com *ic = &sc->sc_ic; uint8_t bands; int error; device_set_usb_desc(self); sc->sc_udev = uaa->device; sc->sc_dev = self; if (USB_GET_DRIVER_INFO(uaa) == URTWN_RTL8188E) sc->chip |= URTWN_CHIP_88E; mtx_init(&sc->sc_mtx, device_get_nameunit(self), MTX_NETWORK_LOCK, MTX_DEF); callout_init(&sc->sc_watchdog_ch, 0); mbufq_init(&sc->sc_snd, ifqmaxlen); sc->sc_iface_index = URTWN_IFACE_INDEX; error = usbd_transfer_setup(uaa->device, &sc->sc_iface_index, sc->sc_xfer, urtwn_config, URTWN_N_TRANSFER, sc, &sc->sc_mtx); if (error) { device_printf(self, "could not allocate USB transfers, " "err=%s\n", usbd_errstr(error)); goto detach; } URTWN_LOCK(sc); error = urtwn_read_chipid(sc); if (error) { device_printf(sc->sc_dev, "unsupported test chip\n"); URTWN_UNLOCK(sc); goto detach; } /* Determine number of Tx/Rx chains. */ if (sc->chip & URTWN_CHIP_92C) { sc->ntxchains = (sc->chip & URTWN_CHIP_92C_1T2R) ? 1 : 2; sc->nrxchains = 2; } else { sc->ntxchains = 1; sc->nrxchains = 1; } if (sc->chip & URTWN_CHIP_88E) error = urtwn_r88e_read_rom(sc); else error = urtwn_read_rom(sc); if (error != 0) { device_printf(sc->sc_dev, "%s: cannot read rom, error %d\n", __func__, error); URTWN_UNLOCK(sc); goto detach; } device_printf(sc->sc_dev, "MAC/BB RTL%s, RF 6052 %dT%dR\n", (sc->chip & URTWN_CHIP_92C) ? "8192CU" : (sc->chip & URTWN_CHIP_88E) ? "8188EU" : (sc->board_type == R92C_BOARD_TYPE_HIGHPA) ? "8188RU" : (sc->board_type == R92C_BOARD_TYPE_MINICARD) ? "8188CE-VAU" : "8188CUS", sc->ntxchains, sc->nrxchains); URTWN_UNLOCK(sc); ic->ic_softc = sc; ic->ic_name = device_get_nameunit(self); ic->ic_phytype = IEEE80211_T_OFDM; /* not only, but not used */ ic->ic_opmode = IEEE80211_M_STA; /* default to BSS mode */ /* set device capabilities */ ic->ic_caps = IEEE80211_C_STA /* station mode */ | IEEE80211_C_MONITOR /* monitor mode */ | IEEE80211_C_IBSS /* adhoc mode */ | IEEE80211_C_HOSTAP /* hostap mode */ | IEEE80211_C_SHPREAMBLE /* short preamble supported */ | IEEE80211_C_SHSLOT /* short slot time supported */ | IEEE80211_C_BGSCAN /* capable of bg scanning */ | IEEE80211_C_WPA /* 802.11i */ + | IEEE80211_C_WME /* 802.11e */ ; bands = 0; setbit(&bands, IEEE80211_MODE_11B); setbit(&bands, IEEE80211_MODE_11G); ieee80211_init_channels(ic, NULL, &bands); ieee80211_ifattach(ic); ic->ic_raw_xmit = urtwn_raw_xmit; ic->ic_scan_start = urtwn_scan_start; ic->ic_scan_end = urtwn_scan_end; ic->ic_set_channel = urtwn_set_channel; ic->ic_transmit = urtwn_transmit; ic->ic_parent = urtwn_parent; ic->ic_vap_create = urtwn_vap_create; ic->ic_vap_delete = urtwn_vap_delete; + ic->ic_wme.wme_update = urtwn_wme_update; ic->ic_update_promisc = urtwn_update_promisc; ic->ic_update_mcast = urtwn_update_mcast; ieee80211_radiotap_attach(ic, &sc->sc_txtap.wt_ihdr, sizeof(sc->sc_txtap), URTWN_TX_RADIOTAP_PRESENT, &sc->sc_rxtap.wr_ihdr, sizeof(sc->sc_rxtap), URTWN_RX_RADIOTAP_PRESENT); if (bootverbose) ieee80211_announce(ic); return (0); detach: urtwn_detach(self); return (ENXIO); /* failure */ } static int urtwn_detach(device_t self) { struct urtwn_softc *sc = device_get_softc(self); struct ieee80211com *ic = &sc->sc_ic; unsigned int x; /* Prevent further ioctls. */ URTWN_LOCK(sc); sc->sc_flags |= URTWN_DETACHED; URTWN_UNLOCK(sc); urtwn_stop(sc); callout_drain(&sc->sc_watchdog_ch); /* stop all USB transfers */ usbd_transfer_unsetup(sc->sc_xfer, URTWN_N_TRANSFER); /* Prevent further allocations from RX/TX data lists. */ URTWN_LOCK(sc); STAILQ_INIT(&sc->sc_tx_active); STAILQ_INIT(&sc->sc_tx_inactive); STAILQ_INIT(&sc->sc_tx_pending); STAILQ_INIT(&sc->sc_rx_active); STAILQ_INIT(&sc->sc_rx_inactive); URTWN_UNLOCK(sc); /* drain USB transfers */ for (x = 0; x != URTWN_N_TRANSFER; x++) usbd_transfer_drain(sc->sc_xfer[x]); /* Free data buffers. */ URTWN_LOCK(sc); urtwn_free_tx_list(sc); urtwn_free_rx_list(sc); URTWN_UNLOCK(sc); ieee80211_ifdetach(ic); mtx_destroy(&sc->sc_mtx); return (0); } static void urtwn_drain_mbufq(struct urtwn_softc *sc) { struct mbuf *m; struct ieee80211_node *ni; URTWN_ASSERT_LOCKED(sc); while ((m = mbufq_dequeue(&sc->sc_snd)) != NULL) { ni = (struct ieee80211_node *)m->m_pkthdr.rcvif; m->m_pkthdr.rcvif = NULL; ieee80211_free_node(ni); m_freem(m); } } static usb_error_t urtwn_do_request(struct urtwn_softc *sc, struct usb_device_request *req, void *data) { usb_error_t err; int ntries = 10; URTWN_ASSERT_LOCKED(sc); while (ntries--) { err = usbd_do_request_flags(sc->sc_udev, &sc->sc_mtx, req, data, 0, NULL, 250 /* ms */); if (err == 0) break; DPRINTFN(1, "Control request failed, %s (retrying)\n", usbd_errstr(err)); usb_pause_mtx(&sc->sc_mtx, hz / 100); } return (err); } static struct ieee80211vap * urtwn_vap_create(struct ieee80211com *ic, const char name[IFNAMSIZ], int unit, enum ieee80211_opmode opmode, int flags, const uint8_t bssid[IEEE80211_ADDR_LEN], const uint8_t mac[IEEE80211_ADDR_LEN]) { struct urtwn_softc *sc = ic->ic_softc; struct urtwn_vap *uvp; struct ieee80211vap *vap; if (!TAILQ_EMPTY(&ic->ic_vaps)) /* only one at a time */ return (NULL); uvp = malloc(sizeof(struct urtwn_vap), M_80211_VAP, M_WAITOK | M_ZERO); vap = &uvp->vap; /* enable s/w bmiss handling for sta mode */ if (ieee80211_vap_setup(ic, vap, name, unit, opmode, flags | IEEE80211_CLONE_NOBEACONS, bssid) != 0) { /* out of memory */ free(uvp, M_80211_VAP); return (NULL); } if (opmode == IEEE80211_M_HOSTAP || opmode == IEEE80211_M_IBSS) urtwn_init_beacon(sc, uvp); /* override state transition machine */ uvp->newstate = vap->iv_newstate; vap->iv_newstate = urtwn_newstate; vap->iv_update_beacon = urtwn_update_beacon; if (opmode == IEEE80211_M_IBSS) { uvp->recv_mgmt = vap->iv_recv_mgmt; vap->iv_recv_mgmt = urtwn_ibss_recv_mgmt; TASK_INIT(&uvp->tsf_task_adhoc, 0, urtwn_tsf_task_adhoc, vap); } /* complete setup */ ieee80211_vap_attach(vap, ieee80211_media_change, ieee80211_media_status, mac); ic->ic_opmode = opmode; return (vap); } static void urtwn_vap_delete(struct ieee80211vap *vap) { struct ieee80211com *ic = vap->iv_ic; struct urtwn_vap *uvp = URTWN_VAP(vap); if (uvp->bcn_mbuf != NULL) m_freem(uvp->bcn_mbuf); if (vap->iv_opmode == IEEE80211_M_IBSS) ieee80211_draintask(ic, &uvp->tsf_task_adhoc); ieee80211_vap_detach(vap); free(uvp, M_80211_VAP); } static struct mbuf * urtwn_rx_frame(struct urtwn_softc *sc, uint8_t *buf, int pktlen, int *rssi_p) { struct ieee80211com *ic = &sc->sc_ic; struct ieee80211_frame *wh; struct mbuf *m; struct r92c_rx_stat *stat; uint32_t rxdw0, rxdw3; uint8_t rate; int8_t rssi = 0; int infosz; /* * don't pass packets to the ieee80211 framework if the driver isn't * RUNNING. */ if (!(sc->sc_flags & URTWN_RUNNING)) return (NULL); stat = (struct r92c_rx_stat *)buf; rxdw0 = le32toh(stat->rxdw0); rxdw3 = le32toh(stat->rxdw3); if (rxdw0 & (R92C_RXDW0_CRCERR | R92C_RXDW0_ICVERR)) { /* * This should not happen since we setup our Rx filter * to not receive these frames. */ counter_u64_add(ic->ic_ierrors, 1); return (NULL); } if (pktlen < sizeof(struct ieee80211_frame_ack) || pktlen > MCLBYTES) { counter_u64_add(ic->ic_ierrors, 1); return (NULL); } rate = MS(rxdw3, R92C_RXDW3_RATE); infosz = MS(rxdw0, R92C_RXDW0_INFOSZ) * 8; /* Get RSSI from PHY status descriptor if present. */ if (infosz != 0 && (rxdw0 & R92C_RXDW0_PHYST)) { if (sc->chip & URTWN_CHIP_88E) rssi = urtwn_r88e_get_rssi(sc, rate, &stat[1]); else rssi = urtwn_get_rssi(sc, rate, &stat[1]); /* Update our average RSSI. */ urtwn_update_avgrssi(sc, rate, rssi); } m = m_getcl(M_NOWAIT, MT_DATA, M_PKTHDR); if (m == NULL) { device_printf(sc->sc_dev, "could not create RX mbuf\n"); return (NULL); } /* Finalize mbuf. */ wh = (struct ieee80211_frame *)((uint8_t *)&stat[1] + infosz); memcpy(mtod(m, uint8_t *), wh, pktlen); m->m_pkthdr.len = m->m_len = pktlen; if (ieee80211_radiotap_active(ic)) { struct urtwn_rx_radiotap_header *tap = &sc->sc_rxtap; tap->wr_flags = 0; /* Map HW rate index to 802.11 rate. */ if (!(rxdw3 & R92C_RXDW3_HT)) { tap->wr_rate = ridx2rate[rate]; } else if (rate >= 12) { /* MCS0~15. */ /* Bit 7 set means HT MCS instead of rate. */ tap->wr_rate = 0x80 | (rate - 12); } tap->wr_dbm_antsignal = rssi; tap->wr_dbm_antnoise = URTWN_NOISE_FLOOR; tap->wr_chan_freq = htole16(ic->ic_curchan->ic_freq); tap->wr_chan_flags = htole16(ic->ic_curchan->ic_flags); } *rssi_p = rssi; return (m); } static struct mbuf * urtwn_rxeof(struct usb_xfer *xfer, struct urtwn_data *data, int *rssi, int8_t *nf) { struct urtwn_softc *sc = data->sc; struct ieee80211com *ic = &sc->sc_ic; struct r92c_rx_stat *stat; struct mbuf *m, *m0 = NULL, *prevm = NULL; uint32_t rxdw0; uint8_t *buf; int len, totlen, pktlen, infosz, npkts; usbd_xfer_status(xfer, &len, NULL, NULL, NULL); if (len < sizeof(*stat)) { counter_u64_add(ic->ic_ierrors, 1); return (NULL); } buf = data->buf; /* Get the number of encapsulated frames. */ stat = (struct r92c_rx_stat *)buf; npkts = MS(le32toh(stat->rxdw2), R92C_RXDW2_PKTCNT); DPRINTFN(6, "Rx %d frames in one chunk\n", npkts); /* Process all of them. */ while (npkts-- > 0) { if (len < sizeof(*stat)) break; stat = (struct r92c_rx_stat *)buf; rxdw0 = le32toh(stat->rxdw0); pktlen = MS(rxdw0, R92C_RXDW0_PKTLEN); if (pktlen == 0) break; infosz = MS(rxdw0, R92C_RXDW0_INFOSZ) * 8; /* Make sure everything fits in xfer. */ totlen = sizeof(*stat) + infosz + pktlen; if (totlen > len) break; m = urtwn_rx_frame(sc, buf, pktlen, rssi); if (m0 == NULL) m0 = m; if (prevm == NULL) prevm = m; else { prevm->m_next = m; prevm = m; } /* Next chunk is 128-byte aligned. */ totlen = (totlen + 127) & ~127; buf += totlen; len -= totlen; } return (m0); } static void urtwn_bulk_rx_callback(struct usb_xfer *xfer, usb_error_t error) { struct urtwn_softc *sc = usbd_xfer_softc(xfer); struct ieee80211com *ic = &sc->sc_ic; struct ieee80211_frame_min *wh; struct ieee80211_node *ni; struct mbuf *m = NULL, *next; struct urtwn_data *data; int8_t nf; int rssi = 1; URTWN_ASSERT_LOCKED(sc); switch (USB_GET_STATE(xfer)) { case USB_ST_TRANSFERRED: data = STAILQ_FIRST(&sc->sc_rx_active); if (data == NULL) goto tr_setup; STAILQ_REMOVE_HEAD(&sc->sc_rx_active, next); m = urtwn_rxeof(xfer, data, &rssi, &nf); STAILQ_INSERT_TAIL(&sc->sc_rx_inactive, data, next); /* FALLTHROUGH */ case USB_ST_SETUP: tr_setup: data = STAILQ_FIRST(&sc->sc_rx_inactive); if (data == NULL) { KASSERT(m == NULL, ("mbuf isn't NULL")); return; } STAILQ_REMOVE_HEAD(&sc->sc_rx_inactive, next); STAILQ_INSERT_TAIL(&sc->sc_rx_active, data, next); usbd_xfer_set_frame_data(xfer, 0, data->buf, usbd_xfer_max_len(xfer)); usbd_transfer_submit(xfer); /* * To avoid LOR we should unlock our private mutex here to call * ieee80211_input() because here is at the end of a USB * callback and safe to unlock. */ URTWN_UNLOCK(sc); while (m != NULL) { next = m->m_next; m->m_next = NULL; wh = mtod(m, struct ieee80211_frame_min *); if (m->m_len >= sizeof(*wh)) ni = ieee80211_find_rxnode(ic, wh); else ni = NULL; nf = URTWN_NOISE_FLOOR; if (ni != NULL) { (void)ieee80211_input(ni, m, rssi - nf, nf); ieee80211_free_node(ni); } else { (void)ieee80211_input_all(ic, m, rssi - nf, nf); } m = next; } URTWN_LOCK(sc); break; default: /* needs it to the inactive queue due to a error. */ data = STAILQ_FIRST(&sc->sc_rx_active); if (data != NULL) { STAILQ_REMOVE_HEAD(&sc->sc_rx_active, next); STAILQ_INSERT_TAIL(&sc->sc_rx_inactive, data, next); } if (error != USB_ERR_CANCELLED) { usbd_xfer_set_stall(xfer); counter_u64_add(ic->ic_ierrors, 1); goto tr_setup; } break; } } static void urtwn_txeof(struct urtwn_softc *sc, struct urtwn_data *data, int status) { URTWN_ASSERT_LOCKED(sc); if (data->ni != NULL) /* not a beacon frame */ ieee80211_tx_complete(data->ni, data->m, status); data->ni = NULL; data->m = NULL; sc->sc_txtimer = 0; STAILQ_INSERT_TAIL(&sc->sc_tx_inactive, data, next); } static int urtwn_alloc_list(struct urtwn_softc *sc, struct urtwn_data data[], int ndata, int maxsz) { int i, error; for (i = 0; i < ndata; i++) { struct urtwn_data *dp = &data[i]; dp->sc = sc; dp->m = NULL; dp->buf = malloc(maxsz, M_USBDEV, M_NOWAIT); if (dp->buf == NULL) { device_printf(sc->sc_dev, "could not allocate buffer\n"); error = ENOMEM; goto fail; } dp->ni = NULL; } return (0); fail: urtwn_free_list(sc, data, ndata); return (error); } static int urtwn_alloc_rx_list(struct urtwn_softc *sc) { int error, i; error = urtwn_alloc_list(sc, sc->sc_rx, URTWN_RX_LIST_COUNT, URTWN_RXBUFSZ); if (error != 0) return (error); STAILQ_INIT(&sc->sc_rx_active); STAILQ_INIT(&sc->sc_rx_inactive); for (i = 0; i < URTWN_RX_LIST_COUNT; i++) STAILQ_INSERT_HEAD(&sc->sc_rx_inactive, &sc->sc_rx[i], next); return (0); } static int urtwn_alloc_tx_list(struct urtwn_softc *sc) { int error, i; error = urtwn_alloc_list(sc, sc->sc_tx, URTWN_TX_LIST_COUNT, URTWN_TXBUFSZ); if (error != 0) return (error); STAILQ_INIT(&sc->sc_tx_active); STAILQ_INIT(&sc->sc_tx_inactive); STAILQ_INIT(&sc->sc_tx_pending); for (i = 0; i < URTWN_TX_LIST_COUNT; i++) STAILQ_INSERT_HEAD(&sc->sc_tx_inactive, &sc->sc_tx[i], next); return (0); } static void urtwn_free_list(struct urtwn_softc *sc, struct urtwn_data data[], int ndata) { int i; for (i = 0; i < ndata; i++) { struct urtwn_data *dp = &data[i]; if (dp->buf != NULL) { free(dp->buf, M_USBDEV); dp->buf = NULL; } if (dp->ni != NULL) { ieee80211_free_node(dp->ni); dp->ni = NULL; } } } static void urtwn_free_rx_list(struct urtwn_softc *sc) { urtwn_free_list(sc, sc->sc_rx, URTWN_RX_LIST_COUNT); } static void urtwn_free_tx_list(struct urtwn_softc *sc) { urtwn_free_list(sc, sc->sc_tx, URTWN_TX_LIST_COUNT); } static void urtwn_bulk_tx_callback(struct usb_xfer *xfer, usb_error_t error) { struct urtwn_softc *sc = usbd_xfer_softc(xfer); struct urtwn_data *data; URTWN_ASSERT_LOCKED(sc); switch (USB_GET_STATE(xfer)){ case USB_ST_TRANSFERRED: data = STAILQ_FIRST(&sc->sc_tx_active); if (data == NULL) goto tr_setup; STAILQ_REMOVE_HEAD(&sc->sc_tx_active, next); urtwn_txeof(sc, data, 0); /* FALLTHROUGH */ case USB_ST_SETUP: tr_setup: data = STAILQ_FIRST(&sc->sc_tx_pending); if (data == NULL) { DPRINTF("%s: empty pending queue\n", __func__); goto finish; } STAILQ_REMOVE_HEAD(&sc->sc_tx_pending, next); STAILQ_INSERT_TAIL(&sc->sc_tx_active, data, next); usbd_xfer_set_frame_data(xfer, 0, data->buf, data->buflen); usbd_transfer_submit(xfer); break; default: data = STAILQ_FIRST(&sc->sc_tx_active); if (data == NULL) goto tr_setup; STAILQ_REMOVE_HEAD(&sc->sc_tx_active, next); urtwn_txeof(sc, data, 1); if (error != USB_ERR_CANCELLED) { usbd_xfer_set_stall(xfer); goto tr_setup; } break; } finish: /* Kick-start more transmit */ urtwn_start(sc); } static struct urtwn_data * _urtwn_getbuf(struct urtwn_softc *sc) { struct urtwn_data *bf; bf = STAILQ_FIRST(&sc->sc_tx_inactive); if (bf != NULL) STAILQ_REMOVE_HEAD(&sc->sc_tx_inactive, next); else DPRINTF("%s: %s\n", __func__, "out of xmit buffers"); return (bf); } static struct urtwn_data * urtwn_getbuf(struct urtwn_softc *sc) { struct urtwn_data *bf; URTWN_ASSERT_LOCKED(sc); bf = _urtwn_getbuf(sc); if (bf == NULL) DPRINTF("%s: stop queue\n", __func__); return (bf); } static usb_error_t urtwn_write_region_1(struct urtwn_softc *sc, uint16_t addr, uint8_t *buf, int len) { usb_device_request_t req; req.bmRequestType = UT_WRITE_VENDOR_DEVICE; req.bRequest = R92C_REQ_REGS; USETW(req.wValue, addr); USETW(req.wIndex, 0); USETW(req.wLength, len); return (urtwn_do_request(sc, &req, buf)); } static usb_error_t urtwn_write_1(struct urtwn_softc *sc, uint16_t addr, uint8_t val) { return (urtwn_write_region_1(sc, addr, &val, sizeof(val))); } static usb_error_t urtwn_write_2(struct urtwn_softc *sc, uint16_t addr, uint16_t val) { val = htole16(val); return (urtwn_write_region_1(sc, addr, (uint8_t *)&val, sizeof(val))); } static usb_error_t urtwn_write_4(struct urtwn_softc *sc, uint16_t addr, uint32_t val) { val = htole32(val); return (urtwn_write_region_1(sc, addr, (uint8_t *)&val, sizeof(val))); } static usb_error_t urtwn_read_region_1(struct urtwn_softc *sc, uint16_t addr, uint8_t *buf, int len) { usb_device_request_t req; req.bmRequestType = UT_READ_VENDOR_DEVICE; req.bRequest = R92C_REQ_REGS; USETW(req.wValue, addr); USETW(req.wIndex, 0); USETW(req.wLength, len); return (urtwn_do_request(sc, &req, buf)); } static uint8_t urtwn_read_1(struct urtwn_softc *sc, uint16_t addr) { uint8_t val; if (urtwn_read_region_1(sc, addr, &val, 1) != 0) return (0xff); return (val); } static uint16_t urtwn_read_2(struct urtwn_softc *sc, uint16_t addr) { uint16_t val; if (urtwn_read_region_1(sc, addr, (uint8_t *)&val, 2) != 0) return (0xffff); return (le16toh(val)); } static uint32_t urtwn_read_4(struct urtwn_softc *sc, uint16_t addr) { uint32_t val; if (urtwn_read_region_1(sc, addr, (uint8_t *)&val, 4) != 0) return (0xffffffff); return (le32toh(val)); } static int urtwn_fw_cmd(struct urtwn_softc *sc, uint8_t id, const void *buf, int len) { struct r92c_fw_cmd cmd; usb_error_t error; int ntries; /* Wait for current FW box to be empty. */ for (ntries = 0; ntries < 100; ntries++) { if (!(urtwn_read_1(sc, R92C_HMETFR) & (1 << sc->fwcur))) break; urtwn_ms_delay(sc); } if (ntries == 100) { device_printf(sc->sc_dev, "could not send firmware command\n"); return (ETIMEDOUT); } memset(&cmd, 0, sizeof(cmd)); cmd.id = id; if (len > 3) cmd.id |= R92C_CMD_FLAG_EXT; KASSERT(len <= sizeof(cmd.msg), ("urtwn_fw_cmd\n")); memcpy(cmd.msg, buf, len); /* Write the first word last since that will trigger the FW. */ error = urtwn_write_region_1(sc, R92C_HMEBOX_EXT(sc->fwcur), (uint8_t *)&cmd + 4, 2); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); error = urtwn_write_region_1(sc, R92C_HMEBOX(sc->fwcur), (uint8_t *)&cmd + 0, 4); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); sc->fwcur = (sc->fwcur + 1) % R92C_H2C_NBOX; return (0); } static __inline void urtwn_rf_write(struct urtwn_softc *sc, int chain, uint8_t addr, uint32_t val) { sc->sc_rf_write(sc, chain, addr, val); } static void urtwn_r92c_rf_write(struct urtwn_softc *sc, int chain, uint8_t addr, uint32_t val) { urtwn_bb_write(sc, R92C_LSSI_PARAM(chain), SM(R92C_LSSI_PARAM_ADDR, addr) | SM(R92C_LSSI_PARAM_DATA, val)); } static void urtwn_r88e_rf_write(struct urtwn_softc *sc, int chain, uint8_t addr, uint32_t val) { urtwn_bb_write(sc, R92C_LSSI_PARAM(chain), SM(R88E_LSSI_PARAM_ADDR, addr) | SM(R92C_LSSI_PARAM_DATA, val)); } static uint32_t urtwn_rf_read(struct urtwn_softc *sc, int chain, uint8_t addr) { uint32_t reg[R92C_MAX_CHAINS], val; reg[0] = urtwn_bb_read(sc, R92C_HSSI_PARAM2(0)); if (chain != 0) reg[chain] = urtwn_bb_read(sc, R92C_HSSI_PARAM2(chain)); urtwn_bb_write(sc, R92C_HSSI_PARAM2(0), reg[0] & ~R92C_HSSI_PARAM2_READ_EDGE); urtwn_ms_delay(sc); urtwn_bb_write(sc, R92C_HSSI_PARAM2(chain), RW(reg[chain], R92C_HSSI_PARAM2_READ_ADDR, addr) | R92C_HSSI_PARAM2_READ_EDGE); urtwn_ms_delay(sc); urtwn_bb_write(sc, R92C_HSSI_PARAM2(0), reg[0] | R92C_HSSI_PARAM2_READ_EDGE); urtwn_ms_delay(sc); if (urtwn_bb_read(sc, R92C_HSSI_PARAM1(chain)) & R92C_HSSI_PARAM1_PI) val = urtwn_bb_read(sc, R92C_HSPI_READBACK(chain)); else val = urtwn_bb_read(sc, R92C_LSSI_READBACK(chain)); return (MS(val, R92C_LSSI_READBACK_DATA)); } static int urtwn_llt_write(struct urtwn_softc *sc, uint32_t addr, uint32_t data) { usb_error_t error; int ntries; error = urtwn_write_4(sc, R92C_LLT_INIT, SM(R92C_LLT_INIT_OP, R92C_LLT_INIT_OP_WRITE) | SM(R92C_LLT_INIT_ADDR, addr) | SM(R92C_LLT_INIT_DATA, data)); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); /* Wait for write operation to complete. */ for (ntries = 0; ntries < 20; ntries++) { if (MS(urtwn_read_4(sc, R92C_LLT_INIT), R92C_LLT_INIT_OP) == R92C_LLT_INIT_OP_NO_ACTIVE) return (0); urtwn_ms_delay(sc); } return (ETIMEDOUT); } static int urtwn_efuse_read_next(struct urtwn_softc *sc, uint8_t *val) { uint32_t reg; usb_error_t error; int ntries; if (sc->last_rom_addr >= URTWN_EFUSE_MAX_LEN) return (EFAULT); reg = urtwn_read_4(sc, R92C_EFUSE_CTRL); reg = RW(reg, R92C_EFUSE_CTRL_ADDR, sc->last_rom_addr); reg &= ~R92C_EFUSE_CTRL_VALID; error = urtwn_write_4(sc, R92C_EFUSE_CTRL, reg); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); /* Wait for read operation to complete. */ for (ntries = 0; ntries < 100; ntries++) { reg = urtwn_read_4(sc, R92C_EFUSE_CTRL); if (reg & R92C_EFUSE_CTRL_VALID) break; urtwn_ms_delay(sc); } if (ntries == 100) { device_printf(sc->sc_dev, "could not read efuse byte at address 0x%x\n", sc->last_rom_addr); return (ETIMEDOUT); } *val = MS(reg, R92C_EFUSE_CTRL_DATA); sc->last_rom_addr++; return (0); } static int urtwn_efuse_read_data(struct urtwn_softc *sc, uint8_t *rom, uint8_t off, uint8_t msk) { uint8_t reg; int i, error; for (i = 0; i < 4; i++) { if (msk & (1 << i)) continue; error = urtwn_efuse_read_next(sc, ®); if (error != 0) return (error); DPRINTF("rom[0x%03X] == 0x%02X\n", off * 8 + i * 2, reg); rom[off * 8 + i * 2 + 0] = reg; error = urtwn_efuse_read_next(sc, ®); if (error != 0) return (error); DPRINTF("rom[0x%03X] == 0x%02X\n", off * 8 + i * 2 + 1, reg); rom[off * 8 + i * 2 + 1] = reg; } return (0); } #ifdef URTWN_DEBUG static void urtwn_dump_rom_contents(struct urtwn_softc *sc, uint8_t *rom, uint16_t size) { int i; /* Dump ROM contents. */ device_printf(sc->sc_dev, "%s:", __func__); for (i = 0; i < size; i++) { if (i % 32 == 0) printf("\n%03X: ", i); else if (i % 4 == 0) printf(" "); printf("%02X", rom[i]); } printf("\n"); } #endif static int urtwn_efuse_read(struct urtwn_softc *sc, uint8_t *rom, uint16_t size) { #define URTWN_CHK(res) do { \ if ((error = res) != 0) \ goto end; \ } while(0) uint8_t msk, off, reg; int error; URTWN_CHK(urtwn_efuse_switch_power(sc)); /* Read full ROM image. */ sc->last_rom_addr = 0; memset(rom, 0xff, size); URTWN_CHK(urtwn_efuse_read_next(sc, ®)); while (reg != 0xff) { /* check for extended header */ if ((sc->chip & URTWN_CHIP_88E) && (reg & 0x1f) == 0x0f) { off = reg >> 5; URTWN_CHK(urtwn_efuse_read_next(sc, ®)); if ((reg & 0x0f) != 0x0f) off = ((reg & 0xf0) >> 1) | off; else continue; } else off = reg >> 4; msk = reg & 0xf; URTWN_CHK(urtwn_efuse_read_data(sc, rom, off, msk)); URTWN_CHK(urtwn_efuse_read_next(sc, ®)); } end: #ifdef URTWN_DEBUG if (urtwn_debug >= 2) urtwn_dump_rom_contents(sc, rom, size); #endif urtwn_write_1(sc, R92C_EFUSE_ACCESS, R92C_EFUSE_ACCESS_OFF); if (error != 0) { device_printf(sc->sc_dev, "%s: error while reading ROM\n", __func__); } return (error); #undef URTWN_CHK } static int urtwn_efuse_switch_power(struct urtwn_softc *sc) { usb_error_t error; uint32_t reg; error = urtwn_write_1(sc, R92C_EFUSE_ACCESS, R92C_EFUSE_ACCESS_ON); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); reg = urtwn_read_2(sc, R92C_SYS_ISO_CTRL); if (!(reg & R92C_SYS_ISO_CTRL_PWC_EV12V)) { error = urtwn_write_2(sc, R92C_SYS_ISO_CTRL, reg | R92C_SYS_ISO_CTRL_PWC_EV12V); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); } reg = urtwn_read_2(sc, R92C_SYS_FUNC_EN); if (!(reg & R92C_SYS_FUNC_EN_ELDR)) { error = urtwn_write_2(sc, R92C_SYS_FUNC_EN, reg | R92C_SYS_FUNC_EN_ELDR); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); } reg = urtwn_read_2(sc, R92C_SYS_CLKR); if ((reg & (R92C_SYS_CLKR_LOADER_EN | R92C_SYS_CLKR_ANA8M)) != (R92C_SYS_CLKR_LOADER_EN | R92C_SYS_CLKR_ANA8M)) { error = urtwn_write_2(sc, R92C_SYS_CLKR, reg | R92C_SYS_CLKR_LOADER_EN | R92C_SYS_CLKR_ANA8M); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); } return (0); } static int urtwn_read_chipid(struct urtwn_softc *sc) { uint32_t reg; if (sc->chip & URTWN_CHIP_88E) return (0); reg = urtwn_read_4(sc, R92C_SYS_CFG); if (reg & R92C_SYS_CFG_TRP_VAUX_EN) return (EIO); if (reg & R92C_SYS_CFG_TYPE_92C) { sc->chip |= URTWN_CHIP_92C; /* Check if it is a castrated 8192C. */ if (MS(urtwn_read_4(sc, R92C_HPON_FSM), R92C_HPON_FSM_CHIP_BONDING_ID) == R92C_HPON_FSM_CHIP_BONDING_ID_92C_1T2R) sc->chip |= URTWN_CHIP_92C_1T2R; } if (reg & R92C_SYS_CFG_VENDOR_UMC) { sc->chip |= URTWN_CHIP_UMC; if (MS(reg, R92C_SYS_CFG_CHIP_VER_RTL) == 0) sc->chip |= URTWN_CHIP_UMC_A_CUT; } return (0); } static int urtwn_read_rom(struct urtwn_softc *sc) { struct r92c_rom *rom = &sc->rom.r92c_rom; int error; /* Read full ROM image. */ error = urtwn_efuse_read(sc, (uint8_t *)rom, sizeof(*rom)); if (error != 0) return (error); /* XXX Weird but this is what the vendor driver does. */ sc->last_rom_addr = 0x1fa; error = urtwn_efuse_read_next(sc, &sc->pa_setting); if (error != 0) return (error); DPRINTF("PA setting=0x%x\n", sc->pa_setting); sc->board_type = MS(rom->rf_opt1, R92C_ROM_RF1_BOARD_TYPE); sc->regulatory = MS(rom->rf_opt1, R92C_ROM_RF1_REGULATORY); DPRINTF("regulatory type=%d\n", sc->regulatory); IEEE80211_ADDR_COPY(sc->sc_ic.ic_macaddr, rom->macaddr); sc->sc_rf_write = urtwn_r92c_rf_write; sc->sc_power_on = urtwn_r92c_power_on; return (0); } static int urtwn_r88e_read_rom(struct urtwn_softc *sc) { uint8_t *rom = sc->rom.r88e_rom; uint16_t addr; int error, i; error = urtwn_efuse_read(sc, rom, sizeof(sc->rom.r88e_rom)); if (error != 0) return (error); addr = 0x10; for (i = 0; i < 6; i++) sc->cck_tx_pwr[i] = rom[addr++]; for (i = 0; i < 5; i++) sc->ht40_tx_pwr[i] = rom[addr++]; sc->bw20_tx_pwr_diff = (rom[addr] & 0xf0) >> 4; if (sc->bw20_tx_pwr_diff & 0x08) sc->bw20_tx_pwr_diff |= 0xf0; sc->ofdm_tx_pwr_diff = (rom[addr] & 0xf); if (sc->ofdm_tx_pwr_diff & 0x08) sc->ofdm_tx_pwr_diff |= 0xf0; sc->regulatory = MS(rom[0xc1], R92C_ROM_RF1_REGULATORY); IEEE80211_ADDR_COPY(sc->sc_ic.ic_macaddr, &rom[0xd7]); sc->sc_rf_write = urtwn_r88e_rf_write; sc->sc_power_on = urtwn_r88e_power_on; return (0); } /* * Initialize rate adaptation in firmware. */ static int urtwn_ra_init(struct urtwn_softc *sc) { struct ieee80211com *ic = &sc->sc_ic; struct ieee80211vap *vap = TAILQ_FIRST(&ic->ic_vaps); struct ieee80211_node *ni; struct ieee80211_rateset *rs; struct r92c_fw_cmd_macid_cfg cmd; uint32_t rates, basicrates; uint8_t mode; int maxrate, maxbasicrate, error, i, j; ni = ieee80211_ref_node(vap->iv_bss); rs = &ni->ni_rates; /* Get normal and basic rates mask. */ rates = basicrates = 0; maxrate = maxbasicrate = 0; for (i = 0; i < rs->rs_nrates; i++) { /* Convert 802.11 rate to HW rate index. */ for (j = 0; j < nitems(ridx2rate); j++) if ((rs->rs_rates[i] & IEEE80211_RATE_VAL) == ridx2rate[j]) break; if (j == nitems(ridx2rate)) /* Unknown rate, skip. */ continue; rates |= 1 << j; if (j > maxrate) maxrate = j; if (rs->rs_rates[i] & IEEE80211_RATE_BASIC) { basicrates |= 1 << j; if (j > maxbasicrate) maxbasicrate = j; } } if (ic->ic_curmode == IEEE80211_MODE_11B) mode = R92C_RAID_11B; else mode = R92C_RAID_11BG; DPRINTF("mode=0x%x rates=0x%08x, basicrates=0x%08x\n", mode, rates, basicrates); /* Set rates mask for group addressed frames. */ cmd.macid = URTWN_MACID_BC | URTWN_MACID_VALID; cmd.mask = htole32(mode << 28 | basicrates); error = urtwn_fw_cmd(sc, R92C_CMD_MACID_CONFIG, &cmd, sizeof(cmd)); if (error != 0) { ieee80211_free_node(ni); device_printf(sc->sc_dev, "could not add broadcast station\n"); return (error); } /* Set initial MRR rate. */ DPRINTF("maxbasicrate=%d\n", maxbasicrate); urtwn_write_1(sc, R92C_INIDATA_RATE_SEL(URTWN_MACID_BC), maxbasicrate); /* Set rates mask for unicast frames. */ cmd.macid = URTWN_MACID_BSS | URTWN_MACID_VALID; cmd.mask = htole32(mode << 28 | rates); error = urtwn_fw_cmd(sc, R92C_CMD_MACID_CONFIG, &cmd, sizeof(cmd)); if (error != 0) { ieee80211_free_node(ni); device_printf(sc->sc_dev, "could not add BSS station\n"); return (error); } /* Set initial MRR rate. */ DPRINTF("maxrate=%d\n", maxrate); urtwn_write_1(sc, R92C_INIDATA_RATE_SEL(URTWN_MACID_BSS), maxrate); /* Indicate highest supported rate. */ ni->ni_txrate = rs->rs_rates[rs->rs_nrates - 1]; ieee80211_free_node(ni); return (0); } static void urtwn_init_beacon(struct urtwn_softc *sc, struct urtwn_vap *uvp) { struct r92c_tx_desc *txd = &uvp->bcn_desc; txd->txdw0 = htole32( SM(R92C_TXDW0_OFFSET, sizeof(*txd)) | R92C_TXDW0_BMCAST | R92C_TXDW0_OWN | R92C_TXDW0_FSG | R92C_TXDW0_LSG); txd->txdw1 = htole32( SM(R92C_TXDW1_QSEL, R92C_TXDW1_QSEL_BEACON) | SM(R92C_TXDW1_RAID, R92C_RAID_11B)); if (sc->chip & URTWN_CHIP_88E) { txd->txdw1 |= htole32(SM(R88E_TXDW1_MACID, URTWN_MACID_BC)); txd->txdseq |= htole16(R88E_TXDSEQ_HWSEQ_EN); } else { txd->txdw1 |= htole32(SM(R92C_TXDW1_MACID, URTWN_MACID_BC)); txd->txdw4 |= htole32(R92C_TXDW4_HWSEQ_EN); } txd->txdw4 = htole32(R92C_TXDW4_DRVRATE); txd->txdw5 = htole32(SM(R92C_TXDW5_DATARATE, URTWN_RIDX_CCK1)); } static int urtwn_setup_beacon(struct urtwn_softc *sc, struct ieee80211_node *ni) { struct ieee80211vap *vap = ni->ni_vap; struct urtwn_vap *uvp = URTWN_VAP(vap); struct mbuf *m; int error; URTWN_ASSERT_LOCKED(sc); if (ni->ni_chan == IEEE80211_CHAN_ANYC) return (EINVAL); m = ieee80211_beacon_alloc(ni); if (m == NULL) { device_printf(sc->sc_dev, "%s: could not allocate beacon frame\n", __func__); return (ENOMEM); } if (uvp->bcn_mbuf != NULL) m_freem(uvp->bcn_mbuf); uvp->bcn_mbuf = m; if ((error = urtwn_tx_beacon(sc, uvp)) != 0) return (error); /* XXX bcnq stuck workaround */ if ((error = urtwn_tx_beacon(sc, uvp)) != 0) return (error); return (0); } static void urtwn_update_beacon(struct ieee80211vap *vap, int item) { struct urtwn_softc *sc = vap->iv_ic->ic_softc; struct urtwn_vap *uvp = URTWN_VAP(vap); struct ieee80211_beacon_offsets *bo = &vap->iv_bcn_off; struct ieee80211_node *ni = vap->iv_bss; int mcast = 0; URTWN_LOCK(sc); if (uvp->bcn_mbuf == NULL) { uvp->bcn_mbuf = ieee80211_beacon_alloc(ni); if (uvp->bcn_mbuf == NULL) { device_printf(sc->sc_dev, "%s: could not allocate beacon frame\n", __func__); URTWN_UNLOCK(sc); return; } } URTWN_UNLOCK(sc); if (item == IEEE80211_BEACON_TIM) mcast = 1; /* XXX */ setbit(bo->bo_flags, item); ieee80211_beacon_update(ni, uvp->bcn_mbuf, mcast); URTWN_LOCK(sc); urtwn_tx_beacon(sc, uvp); URTWN_UNLOCK(sc); } /* * Push a beacon frame into the chip. Beacon will * be repeated by the chip every R92C_BCN_INTERVAL. */ static int urtwn_tx_beacon(struct urtwn_softc *sc, struct urtwn_vap *uvp) { struct r92c_tx_desc *desc = &uvp->bcn_desc; struct urtwn_data *bf; URTWN_ASSERT_LOCKED(sc); bf = urtwn_getbuf(sc); if (bf == NULL) return (ENOMEM); memcpy(bf->buf, desc, sizeof(*desc)); urtwn_tx_start(sc, uvp->bcn_mbuf, IEEE80211_FC0_TYPE_MGT, bf); sc->sc_txtimer = 5; callout_reset(&sc->sc_watchdog_ch, hz, urtwn_watchdog, sc); return (0); } static void urtwn_tsf_task_adhoc(void *arg, int pending) { struct ieee80211vap *vap = arg; struct urtwn_softc *sc = vap->iv_ic->ic_softc; struct ieee80211_node *ni; uint32_t reg; URTWN_LOCK(sc); ni = ieee80211_ref_node(vap->iv_bss); reg = urtwn_read_1(sc, R92C_BCN_CTRL); /* Accept beacons with the same BSSID. */ urtwn_set_rx_bssid_all(sc, 0); /* Enable synchronization. */ reg &= ~R92C_BCN_CTRL_DIS_TSF_UDT0; urtwn_write_1(sc, R92C_BCN_CTRL, reg); /* Synchronize. */ usb_pause_mtx(&sc->sc_mtx, hz * ni->ni_intval * 5 / 1000); /* Disable synchronization. */ reg |= R92C_BCN_CTRL_DIS_TSF_UDT0; urtwn_write_1(sc, R92C_BCN_CTRL, reg); /* Remove beacon filter. */ urtwn_set_rx_bssid_all(sc, 1); /* Enable beaconing. */ urtwn_write_1(sc, R92C_MBID_NUM, urtwn_read_1(sc, R92C_MBID_NUM) | R92C_MBID_TXBCN_RPT0); reg |= R92C_BCN_CTRL_EN_BCN; urtwn_write_1(sc, R92C_BCN_CTRL, reg); ieee80211_free_node(ni); URTWN_UNLOCK(sc); } static void urtwn_tsf_sync_enable(struct urtwn_softc *sc, struct ieee80211vap *vap) { struct ieee80211com *ic = &sc->sc_ic; struct urtwn_vap *uvp = URTWN_VAP(vap); /* Reset TSF. */ urtwn_write_1(sc, R92C_DUAL_TSF_RST, R92C_DUAL_TSF_RST0); switch (vap->iv_opmode) { case IEEE80211_M_STA: /* Enable TSF synchronization. */ urtwn_write_1(sc, R92C_BCN_CTRL, urtwn_read_1(sc, R92C_BCN_CTRL) & ~R92C_BCN_CTRL_DIS_TSF_UDT0); break; case IEEE80211_M_IBSS: ieee80211_runtask(ic, &uvp->tsf_task_adhoc); break; case IEEE80211_M_HOSTAP: /* Enable beaconing. */ urtwn_write_1(sc, R92C_MBID_NUM, urtwn_read_1(sc, R92C_MBID_NUM) | R92C_MBID_TXBCN_RPT0); urtwn_write_1(sc, R92C_BCN_CTRL, urtwn_read_1(sc, R92C_BCN_CTRL) | R92C_BCN_CTRL_EN_BCN); break; default: device_printf(sc->sc_dev, "undefined opmode %d\n", vap->iv_opmode); return; } } static void urtwn_set_led(struct urtwn_softc *sc, int led, int on) { uint8_t reg; if (led == URTWN_LED_LINK) { if (sc->chip & URTWN_CHIP_88E) { reg = urtwn_read_1(sc, R92C_LEDCFG2) & 0xf0; urtwn_write_1(sc, R92C_LEDCFG2, reg | 0x60); if (!on) { reg = urtwn_read_1(sc, R92C_LEDCFG2) & 0x90; urtwn_write_1(sc, R92C_LEDCFG2, reg | R92C_LEDCFG0_DIS); urtwn_write_1(sc, R92C_MAC_PINMUX_CFG, urtwn_read_1(sc, R92C_MAC_PINMUX_CFG) & 0xfe); } } else { reg = urtwn_read_1(sc, R92C_LEDCFG0) & 0x70; if (!on) reg |= R92C_LEDCFG0_DIS; urtwn_write_1(sc, R92C_LEDCFG0, reg); } sc->ledlink = on; /* Save LED state. */ } } static void urtwn_set_mode(struct urtwn_softc *sc, uint8_t mode) { uint8_t reg; reg = urtwn_read_1(sc, R92C_MSR); reg = (reg & ~R92C_MSR_MASK) | mode; urtwn_write_1(sc, R92C_MSR, reg); } static void urtwn_ibss_recv_mgmt(struct ieee80211_node *ni, struct mbuf *m, int subtype, const struct ieee80211_rx_stats *rxs, int rssi, int nf) { struct ieee80211vap *vap = ni->ni_vap; struct urtwn_softc *sc = vap->iv_ic->ic_softc; struct urtwn_vap *uvp = URTWN_VAP(vap); uint64_t ni_tstamp, curr_tstamp; uvp->recv_mgmt(ni, m, subtype, rxs, rssi, nf); if (vap->iv_state == IEEE80211_S_RUN && (subtype == IEEE80211_FC0_SUBTYPE_BEACON || subtype == IEEE80211_FC0_SUBTYPE_PROBE_RESP)) { ni_tstamp = le64toh(ni->ni_tstamp.tsf); #ifdef D3831 URTWN_LOCK(sc); urtwn_get_tsf(sc, &curr_tstamp); URTWN_UNLOCK(sc); curr_tstamp = le64toh(curr_tstamp); if (ni_tstamp >= curr_tstamp) (void) ieee80211_ibss_merge(ni); #else (void) sc; (void) curr_tstamp; #endif } } static int urtwn_newstate(struct ieee80211vap *vap, enum ieee80211_state nstate, int arg) { struct urtwn_vap *uvp = URTWN_VAP(vap); struct ieee80211com *ic = vap->iv_ic; struct urtwn_softc *sc = ic->ic_softc; struct ieee80211_node *ni; enum ieee80211_state ostate; uint32_t reg; uint8_t mode; int error = 0; ostate = vap->iv_state; DPRINTF("%s -> %s\n", ieee80211_state_name[ostate], ieee80211_state_name[nstate]); IEEE80211_UNLOCK(ic); URTWN_LOCK(sc); callout_stop(&sc->sc_watchdog_ch); if (ostate == IEEE80211_S_RUN) { /* Turn link LED off. */ urtwn_set_led(sc, URTWN_LED_LINK, 0); /* Set media status to 'No Link'. */ urtwn_set_mode(sc, R92C_MSR_NOLINK); /* Stop Rx of data frames. */ urtwn_write_2(sc, R92C_RXFLTMAP2, 0); /* Disable TSF synchronization. */ urtwn_write_1(sc, R92C_BCN_CTRL, (urtwn_read_1(sc, R92C_BCN_CTRL) & ~R92C_BCN_CTRL_EN_BCN) | R92C_BCN_CTRL_DIS_TSF_UDT0); /* Disable beaconing. */ urtwn_write_1(sc, R92C_MBID_NUM, urtwn_read_1(sc, R92C_MBID_NUM) & ~R92C_MBID_TXBCN_RPT0); /* Reset TSF. */ urtwn_write_1(sc, R92C_DUAL_TSF_RST, R92C_DUAL_TSF_RST0); /* Reset EDCA parameters. */ urtwn_write_4(sc, R92C_EDCA_VO_PARAM, 0x002f3217); urtwn_write_4(sc, R92C_EDCA_VI_PARAM, 0x005e4317); urtwn_write_4(sc, R92C_EDCA_BE_PARAM, 0x00105320); urtwn_write_4(sc, R92C_EDCA_BK_PARAM, 0x0000a444); } switch (nstate) { case IEEE80211_S_INIT: /* Turn link LED off. */ urtwn_set_led(sc, URTWN_LED_LINK, 0); break; case IEEE80211_S_SCAN: /* Pause AC Tx queues. */ urtwn_write_1(sc, R92C_TXPAUSE, urtwn_read_1(sc, R92C_TXPAUSE) | 0x0f); break; case IEEE80211_S_AUTH: urtwn_set_chan(sc, ic->ic_curchan, NULL); break; case IEEE80211_S_RUN: if (vap->iv_opmode == IEEE80211_M_MONITOR) { /* Turn link LED on. */ urtwn_set_led(sc, URTWN_LED_LINK, 1); break; } ni = ieee80211_ref_node(vap->iv_bss); if (ic->ic_bsschan == IEEE80211_CHAN_ANYC || ni->ni_chan == IEEE80211_CHAN_ANYC) { device_printf(sc->sc_dev, "%s: could not move to RUN state\n", __func__); error = EINVAL; goto end_run; } switch (vap->iv_opmode) { case IEEE80211_M_STA: mode = R92C_MSR_INFRA; break; case IEEE80211_M_IBSS: mode = R92C_MSR_ADHOC; break; case IEEE80211_M_HOSTAP: mode = R92C_MSR_AP; break; default: device_printf(sc->sc_dev, "undefined opmode %d\n", vap->iv_opmode); error = EINVAL; goto end_run; } /* Set media status to 'Associated'. */ urtwn_set_mode(sc, mode); /* Set BSSID. */ urtwn_write_4(sc, R92C_BSSID + 0, LE_READ_4(&ni->ni_bssid[0])); urtwn_write_4(sc, R92C_BSSID + 4, LE_READ_2(&ni->ni_bssid[4])); if (ic->ic_curmode == IEEE80211_MODE_11B) urtwn_write_1(sc, R92C_INIRTS_RATE_SEL, 0); else /* 802.11b/g */ urtwn_write_1(sc, R92C_INIRTS_RATE_SEL, 3); /* Enable Rx of data frames. */ urtwn_write_2(sc, R92C_RXFLTMAP2, 0xffff); /* Flush all AC queues. */ urtwn_write_1(sc, R92C_TXPAUSE, 0); /* Set beacon interval. */ urtwn_write_2(sc, R92C_BCN_INTERVAL, ni->ni_intval); /* Allow Rx from our BSSID only. */ if (ic->ic_promisc == 0) { reg = urtwn_read_4(sc, R92C_RCR); if (vap->iv_opmode != IEEE80211_M_HOSTAP) reg |= R92C_RCR_CBSSID_DATA; if (vap->iv_opmode != IEEE80211_M_IBSS) reg |= R92C_RCR_CBSSID_BCN; urtwn_write_4(sc, R92C_RCR, reg); } if (vap->iv_opmode == IEEE80211_M_HOSTAP || vap->iv_opmode == IEEE80211_M_IBSS) { error = urtwn_setup_beacon(sc, ni); if (error != 0) { device_printf(sc->sc_dev, "unable to push beacon into the chip, " "error %d\n", error); goto end_run; } } /* Enable TSF synchronization. */ urtwn_tsf_sync_enable(sc, vap); urtwn_write_1(sc, R92C_SIFS_CCK + 1, 10); urtwn_write_1(sc, R92C_SIFS_OFDM + 1, 10); urtwn_write_1(sc, R92C_SPEC_SIFS + 1, 10); urtwn_write_1(sc, R92C_MAC_SPEC_SIFS + 1, 10); urtwn_write_1(sc, R92C_R2T_SIFS + 1, 10); urtwn_write_1(sc, R92C_T2T_SIFS + 1, 10); /* Intialize rate adaptation. */ if (sc->chip & URTWN_CHIP_88E) ni->ni_txrate = ni->ni_rates.rs_rates[ni->ni_rates.rs_nrates-1]; else urtwn_ra_init(sc); /* Turn link LED on. */ urtwn_set_led(sc, URTWN_LED_LINK, 1); sc->avg_pwdb = -1; /* Reset average RSSI. */ /* Reset temperature calibration state machine. */ sc->thcal_state = 0; sc->thcal_lctemp = 0; end_run: ieee80211_free_node(ni); break; default: break; } URTWN_UNLOCK(sc); IEEE80211_LOCK(ic); return (error != 0 ? error : uvp->newstate(vap, nstate, arg)); } static void urtwn_watchdog(void *arg) { struct urtwn_softc *sc = arg; if (sc->sc_txtimer > 0) { if (--sc->sc_txtimer == 0) { device_printf(sc->sc_dev, "device timeout\n"); counter_u64_add(sc->sc_ic.ic_oerrors, 1); return; } callout_reset(&sc->sc_watchdog_ch, hz, urtwn_watchdog, sc); } } static void urtwn_update_avgrssi(struct urtwn_softc *sc, int rate, int8_t rssi) { int pwdb; /* Convert antenna signal to percentage. */ if (rssi <= -100 || rssi >= 20) pwdb = 0; else if (rssi >= 0) pwdb = 100; else pwdb = 100 + rssi; if (!(sc->chip & URTWN_CHIP_88E)) { if (rate <= URTWN_RIDX_CCK11) { /* CCK gain is smaller than OFDM/MCS gain. */ pwdb += 6; if (pwdb > 100) pwdb = 100; if (pwdb <= 14) pwdb -= 4; else if (pwdb <= 26) pwdb -= 8; else if (pwdb <= 34) pwdb -= 6; else if (pwdb <= 42) pwdb -= 2; } } if (sc->avg_pwdb == -1) /* Init. */ sc->avg_pwdb = pwdb; else if (sc->avg_pwdb < pwdb) sc->avg_pwdb = ((sc->avg_pwdb * 19 + pwdb) / 20) + 1; else sc->avg_pwdb = ((sc->avg_pwdb * 19 + pwdb) / 20); DPRINTFN(4, "PWDB=%d EMA=%d\n", pwdb, sc->avg_pwdb); } static int8_t urtwn_get_rssi(struct urtwn_softc *sc, int rate, void *physt) { static const int8_t cckoff[] = { 16, -12, -26, -46 }; struct r92c_rx_phystat *phy; struct r92c_rx_cck *cck; uint8_t rpt; int8_t rssi; if (rate <= URTWN_RIDX_CCK11) { cck = (struct r92c_rx_cck *)physt; if (sc->sc_flags & URTWN_FLAG_CCK_HIPWR) { rpt = (cck->agc_rpt >> 5) & 0x3; rssi = (cck->agc_rpt & 0x1f) << 1; } else { rpt = (cck->agc_rpt >> 6) & 0x3; rssi = cck->agc_rpt & 0x3e; } rssi = cckoff[rpt] - rssi; } else { /* OFDM/HT. */ phy = (struct r92c_rx_phystat *)physt; rssi = ((le32toh(phy->phydw1) >> 1) & 0x7f) - 110; } return (rssi); } static int8_t urtwn_r88e_get_rssi(struct urtwn_softc *sc, int rate, void *physt) { struct r92c_rx_phystat *phy; struct r88e_rx_cck *cck; uint8_t cck_agc_rpt, lna_idx, vga_idx; int8_t rssi; rssi = 0; if (rate <= URTWN_RIDX_CCK11) { cck = (struct r88e_rx_cck *)physt; cck_agc_rpt = cck->agc_rpt; lna_idx = (cck_agc_rpt & 0xe0) >> 5; vga_idx = cck_agc_rpt & 0x1f; switch (lna_idx) { case 7: if (vga_idx <= 27) rssi = -100 + 2* (27 - vga_idx); else rssi = -100; break; case 6: rssi = -48 + 2 * (2 - vga_idx); break; case 5: rssi = -42 + 2 * (7 - vga_idx); break; case 4: rssi = -36 + 2 * (7 - vga_idx); break; case 3: rssi = -24 + 2 * (7 - vga_idx); break; case 2: rssi = -12 + 2 * (5 - vga_idx); break; case 1: rssi = 8 - (2 * vga_idx); break; case 0: rssi = 14 - (2 * vga_idx); break; } rssi += 6; } else { /* OFDM/HT. */ phy = (struct r92c_rx_phystat *)physt; rssi = ((le32toh(phy->phydw1) >> 1) & 0x7f) - 110; } return (rssi); } static int urtwn_tx_data(struct urtwn_softc *sc, struct ieee80211_node *ni, struct mbuf *m, struct urtwn_data *data) { struct ieee80211_frame *wh; struct ieee80211_key *k = NULL; struct ieee80211com *ic = &sc->sc_ic; struct ieee80211vap *vap = ni->ni_vap; struct r92c_tx_desc *txd; - uint8_t macid, raid, ridx, subtype, type, qsel; - int ismcast; + uint8_t macid, raid, ridx, subtype, type, tid, qsel; + int hasqos, ismcast; URTWN_ASSERT_LOCKED(sc); /* * Software crypto. */ wh = mtod(m, struct ieee80211_frame *); type = wh->i_fc[0] & IEEE80211_FC0_TYPE_MASK; subtype = wh->i_fc[0] & IEEE80211_FC0_SUBTYPE_MASK; + hasqos = IEEE80211_QOS_HAS_SEQ(wh); ismcast = IEEE80211_IS_MULTICAST(wh->i_addr1); + /* Select TX ring for this frame. */ + if (hasqos) { + tid = ((const struct ieee80211_qosframe *)wh)->i_qos[0]; + tid &= IEEE80211_QOS_TID; + } else + tid = 0; + if (wh->i_fc[1] & IEEE80211_FC1_PROTECTED) { k = ieee80211_crypto_encap(ni, m); if (k == NULL) { device_printf(sc->sc_dev, "ieee80211_crypto_encap returns NULL.\n"); return (ENOBUFS); } /* in case packet header moved, reset pointer */ wh = mtod(m, struct ieee80211_frame *); } /* Fill Tx descriptor. */ txd = (struct r92c_tx_desc *)data->buf; memset(txd, 0, sizeof(*txd)); txd->txdw0 |= htole32( SM(R92C_TXDW0_OFFSET, sizeof(*txd)) | R92C_TXDW0_OWN | R92C_TXDW0_FSG | R92C_TXDW0_LSG); if (ismcast) txd->txdw0 |= htole32(R92C_TXDW0_BMCAST); raid = R92C_RAID_11B; /* by default */ ridx = URTWN_RIDX_CCK1; if (!ismcast) { macid = URTWN_MACID_BSS; if (type == IEEE80211_FC0_TYPE_DATA) { - qsel = R92C_TXDW1_QSEL_BE; + qsel = tid % URTWN_MAX_TID; if (!(m->m_flags & M_EAPOL)) { if (ic->ic_curmode != IEEE80211_MODE_11B) { raid = R92C_RAID_11BG; ridx = URTWN_RIDX_OFDM54; } else ridx = URTWN_RIDX_CCK11; } if (sc->chip & URTWN_CHIP_88E) txd->txdw2 |= htole32(R88E_TXDW2_AGGBK); else txd->txdw1 |= htole32(R92C_TXDW1_AGGBK); if (ic->ic_flags & IEEE80211_F_USEPROT) { switch (ic->ic_protmode) { case IEEE80211_PROT_CTSONLY: txd->txdw4 |= htole32( R92C_TXDW4_CTS2SELF | R92C_TXDW4_HWRTSEN); break; case IEEE80211_PROT_RTSCTS: txd->txdw4 |= htole32( R92C_TXDW4_RTSEN | R92C_TXDW4_HWRTSEN); break; default: break; } } txd->txdw4 |= htole32(SM(R92C_TXDW4_RTSRATE, URTWN_RIDX_OFDM24)); txd->txdw5 |= htole32(0x0001ff00); } else /* IEEE80211_FC0_TYPE_MGT */ qsel = R92C_TXDW1_QSEL_MGNT; } else { macid = URTWN_MACID_BC; qsel = R92C_TXDW1_QSEL_MGNT; } txd->txdw1 |= htole32( SM(R92C_TXDW1_QSEL, qsel) | SM(R92C_TXDW1_RAID, raid)); if (sc->chip & URTWN_CHIP_88E) txd->txdw1 |= htole32(SM(R88E_TXDW1_MACID, macid)); else txd->txdw1 |= htole32(SM(R92C_TXDW1_MACID, macid)); txd->txdw5 |= htole32(SM(R92C_TXDW5_DATARATE, ridx)); /* Force this rate if needed. */ if (ismcast || type != IEEE80211_FC0_TYPE_DATA || (m->m_flags & M_EAPOL)) txd->txdw4 |= htole32(R92C_TXDW4_DRVRATE); - if (!IEEE80211_QOS_HAS_SEQ(wh)) { + if (!hasqos) { /* Use HW sequence numbering for non-QoS frames. */ if (sc->chip & URTWN_CHIP_88E) txd->txdseq = htole16(R88E_TXDSEQ_HWSEQ_EN); else txd->txdw4 |= htole32(R92C_TXDW4_HWSEQ_EN); } else { /* Set sequence number. */ txd->txdseq = htole16(M_SEQNO_GET(m) % IEEE80211_SEQ_RANGE); } if (ieee80211_radiotap_active_vap(vap)) { struct urtwn_tx_radiotap_header *tap = &sc->sc_txtap; tap->wt_flags = 0; tap->wt_chan_freq = htole16(ic->ic_curchan->ic_freq); tap->wt_chan_flags = htole16(ic->ic_curchan->ic_flags); if (k != NULL) tap->wt_flags |= IEEE80211_RADIOTAP_F_WEP; ieee80211_radiotap_tx(vap, m); } data->ni = ni; urtwn_tx_start(sc, m, type, data); return (0); } static void urtwn_tx_start(struct urtwn_softc *sc, struct mbuf *m, uint8_t type, struct urtwn_data *data) { struct usb_xfer *xfer; struct r92c_tx_desc *txd; uint16_t ac, sum; int i, xferlen; - struct usb_xfer *urtwn_pipes[WME_NUM_AC] = { - sc->sc_xfer[URTWN_BULK_TX_BE], - sc->sc_xfer[URTWN_BULK_TX_BK], - sc->sc_xfer[URTWN_BULK_TX_VI], - sc->sc_xfer[URTWN_BULK_TX_VO] - }; URTWN_ASSERT_LOCKED(sc); ac = M_WME_GETAC(m); switch (type) { case IEEE80211_FC0_TYPE_CTL: case IEEE80211_FC0_TYPE_MGT: xfer = sc->sc_xfer[URTWN_BULK_TX_VO]; break; default: - xfer = urtwn_pipes[ac]; + xfer = sc->sc_xfer[wme2queue[ac].qid]; break; } txd = (struct r92c_tx_desc *)data->buf; txd->txdw0 |= htole32(SM(R92C_TXDW0_PKTLEN, m->m_pkthdr.len)); /* Compute Tx descriptor checksum. */ sum = 0; for (i = 0; i < sizeof(*txd) / 2; i++) sum ^= ((uint16_t *)txd)[i]; txd->txdsum = sum; /* NB: already little endian. */ xferlen = sizeof(*txd) + m->m_pkthdr.len; m_copydata(m, 0, m->m_pkthdr.len, (caddr_t)&txd[1]); data->buflen = xferlen; data->m = m; STAILQ_INSERT_TAIL(&sc->sc_tx_pending, data, next); usbd_transfer_start(xfer); } static int urtwn_transmit(struct ieee80211com *ic, struct mbuf *m) { struct urtwn_softc *sc = ic->ic_softc; int error; URTWN_LOCK(sc); if ((sc->sc_flags & URTWN_RUNNING) == 0) { URTWN_UNLOCK(sc); return (ENXIO); } error = mbufq_enqueue(&sc->sc_snd, m); if (error) { URTWN_UNLOCK(sc); return (error); } urtwn_start(sc); URTWN_UNLOCK(sc); return (0); } static void urtwn_start(struct urtwn_softc *sc) { struct ieee80211_node *ni; struct mbuf *m; struct urtwn_data *bf; URTWN_ASSERT_LOCKED(sc); while ((m = mbufq_dequeue(&sc->sc_snd)) != NULL) { bf = urtwn_getbuf(sc); if (bf == NULL) { mbufq_prepend(&sc->sc_snd, m); break; } ni = (struct ieee80211_node *)m->m_pkthdr.rcvif; m->m_pkthdr.rcvif = NULL; if (urtwn_tx_data(sc, ni, m, bf) != 0) { if_inc_counter(ni->ni_vap->iv_ifp, IFCOUNTER_OERRORS, 1); STAILQ_INSERT_HEAD(&sc->sc_tx_inactive, bf, next); m_freem(m); ieee80211_free_node(ni); break; } sc->sc_txtimer = 5; callout_reset(&sc->sc_watchdog_ch, hz, urtwn_watchdog, sc); } } static void urtwn_parent(struct ieee80211com *ic) { struct urtwn_softc *sc = ic->ic_softc; URTWN_LOCK(sc); if (sc->sc_flags & URTWN_DETACHED) { URTWN_UNLOCK(sc); return; } URTWN_UNLOCK(sc); if (ic->ic_nrunning > 0) { if (urtwn_init(sc) != 0) { struct ieee80211vap *vap = TAILQ_FIRST(&ic->ic_vaps); if (vap != NULL) ieee80211_stop(vap); } else ieee80211_start_all(ic); } else urtwn_stop(sc); } static __inline int urtwn_power_on(struct urtwn_softc *sc) { return sc->sc_power_on(sc); } static int urtwn_r92c_power_on(struct urtwn_softc *sc) { uint32_t reg; usb_error_t error; int ntries; /* Wait for autoload done bit. */ for (ntries = 0; ntries < 1000; ntries++) { if (urtwn_read_1(sc, R92C_APS_FSMCO) & R92C_APS_FSMCO_PFM_ALDN) break; urtwn_ms_delay(sc); } if (ntries == 1000) { device_printf(sc->sc_dev, "timeout waiting for chip autoload\n"); return (ETIMEDOUT); } /* Unlock ISO/CLK/Power control register. */ error = urtwn_write_1(sc, R92C_RSV_CTRL, 0); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); /* Move SPS into PWM mode. */ error = urtwn_write_1(sc, R92C_SPS0_CTRL, 0x2b); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); urtwn_ms_delay(sc); reg = urtwn_read_1(sc, R92C_LDOV12D_CTRL); if (!(reg & R92C_LDOV12D_CTRL_LDV12_EN)) { error = urtwn_write_1(sc, R92C_LDOV12D_CTRL, reg | R92C_LDOV12D_CTRL_LDV12_EN); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); urtwn_ms_delay(sc); error = urtwn_write_1(sc, R92C_SYS_ISO_CTRL, urtwn_read_1(sc, R92C_SYS_ISO_CTRL) & ~R92C_SYS_ISO_CTRL_MD2PP); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); } /* Auto enable WLAN. */ error = urtwn_write_2(sc, R92C_APS_FSMCO, urtwn_read_2(sc, R92C_APS_FSMCO) | R92C_APS_FSMCO_APFM_ONMAC); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); for (ntries = 0; ntries < 1000; ntries++) { if (!(urtwn_read_2(sc, R92C_APS_FSMCO) & R92C_APS_FSMCO_APFM_ONMAC)) break; urtwn_ms_delay(sc); } if (ntries == 1000) { device_printf(sc->sc_dev, "timeout waiting for MAC auto ON\n"); return (ETIMEDOUT); } /* Enable radio, GPIO and LED functions. */ error = urtwn_write_2(sc, R92C_APS_FSMCO, R92C_APS_FSMCO_AFSM_HSUS | R92C_APS_FSMCO_PDN_EN | R92C_APS_FSMCO_PFM_ALDN); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); /* Release RF digital isolation. */ error = urtwn_write_2(sc, R92C_SYS_ISO_CTRL, urtwn_read_2(sc, R92C_SYS_ISO_CTRL) & ~R92C_SYS_ISO_CTRL_DIOR); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); /* Initialize MAC. */ error = urtwn_write_1(sc, R92C_APSD_CTRL, urtwn_read_1(sc, R92C_APSD_CTRL) & ~R92C_APSD_CTRL_OFF); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); for (ntries = 0; ntries < 200; ntries++) { if (!(urtwn_read_1(sc, R92C_APSD_CTRL) & R92C_APSD_CTRL_OFF_STATUS)) break; urtwn_ms_delay(sc); } if (ntries == 200) { device_printf(sc->sc_dev, "timeout waiting for MAC initialization\n"); return (ETIMEDOUT); } /* Enable MAC DMA/WMAC/SCHEDULE/SEC blocks. */ reg = urtwn_read_2(sc, R92C_CR); reg |= R92C_CR_HCI_TXDMA_EN | R92C_CR_HCI_RXDMA_EN | R92C_CR_TXDMA_EN | R92C_CR_RXDMA_EN | R92C_CR_PROTOCOL_EN | R92C_CR_SCHEDULE_EN | R92C_CR_MACTXEN | R92C_CR_MACRXEN | R92C_CR_ENSEC; error = urtwn_write_2(sc, R92C_CR, reg); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); error = urtwn_write_1(sc, 0xfe10, 0x19); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); return (0); } static int urtwn_r88e_power_on(struct urtwn_softc *sc) { uint32_t reg; usb_error_t error; int ntries; /* Wait for power ready bit. */ for (ntries = 0; ntries < 5000; ntries++) { if (urtwn_read_4(sc, R92C_APS_FSMCO) & R92C_APS_FSMCO_SUS_HOST) break; urtwn_ms_delay(sc); } if (ntries == 5000) { device_printf(sc->sc_dev, "timeout waiting for chip power up\n"); return (ETIMEDOUT); } /* Reset BB. */ error = urtwn_write_1(sc, R92C_SYS_FUNC_EN, urtwn_read_1(sc, R92C_SYS_FUNC_EN) & ~(R92C_SYS_FUNC_EN_BBRSTB | R92C_SYS_FUNC_EN_BB_GLB_RST)); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); error = urtwn_write_1(sc, R92C_AFE_XTAL_CTRL + 2, urtwn_read_1(sc, R92C_AFE_XTAL_CTRL + 2) | 0x80); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); /* Disable HWPDN. */ error = urtwn_write_2(sc, R92C_APS_FSMCO, urtwn_read_2(sc, R92C_APS_FSMCO) & ~R92C_APS_FSMCO_APDM_HPDN); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); /* Disable WL suspend. */ error = urtwn_write_2(sc, R92C_APS_FSMCO, urtwn_read_2(sc, R92C_APS_FSMCO) & ~(R92C_APS_FSMCO_AFSM_HSUS | R92C_APS_FSMCO_AFSM_PCIE)); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); error = urtwn_write_2(sc, R92C_APS_FSMCO, urtwn_read_2(sc, R92C_APS_FSMCO) | R92C_APS_FSMCO_APFM_ONMAC); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); for (ntries = 0; ntries < 5000; ntries++) { if (!(urtwn_read_2(sc, R92C_APS_FSMCO) & R92C_APS_FSMCO_APFM_ONMAC)) break; urtwn_ms_delay(sc); } if (ntries == 5000) return (ETIMEDOUT); /* Enable LDO normal mode. */ error = urtwn_write_1(sc, R92C_LPLDO_CTRL, urtwn_read_1(sc, R92C_LPLDO_CTRL) & ~0x10); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); /* Enable MAC DMA/WMAC/SCHEDULE/SEC blocks. */ error = urtwn_write_2(sc, R92C_CR, 0); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); reg = urtwn_read_2(sc, R92C_CR); reg |= R92C_CR_HCI_TXDMA_EN | R92C_CR_HCI_RXDMA_EN | R92C_CR_TXDMA_EN | R92C_CR_RXDMA_EN | R92C_CR_PROTOCOL_EN | R92C_CR_SCHEDULE_EN | R92C_CR_ENSEC | R92C_CR_CALTMR_EN; error = urtwn_write_2(sc, R92C_CR, reg); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); return (0); } static int urtwn_llt_init(struct urtwn_softc *sc) { int i, error, page_count, pktbuf_count; page_count = (sc->chip & URTWN_CHIP_88E) ? R88E_TX_PAGE_COUNT : R92C_TX_PAGE_COUNT; pktbuf_count = (sc->chip & URTWN_CHIP_88E) ? R88E_TXPKTBUF_COUNT : R92C_TXPKTBUF_COUNT; /* Reserve pages [0; page_count]. */ for (i = 0; i < page_count; i++) { if ((error = urtwn_llt_write(sc, i, i + 1)) != 0) return (error); } /* NB: 0xff indicates end-of-list. */ if ((error = urtwn_llt_write(sc, i, 0xff)) != 0) return (error); /* * Use pages [page_count + 1; pktbuf_count - 1] * as ring buffer. */ for (++i; i < pktbuf_count - 1; i++) { if ((error = urtwn_llt_write(sc, i, i + 1)) != 0) return (error); } /* Make the last page point to the beginning of the ring buffer. */ error = urtwn_llt_write(sc, i, page_count + 1); return (error); } static void urtwn_fw_reset(struct urtwn_softc *sc) { uint16_t reg; int ntries; /* Tell 8051 to reset itself. */ urtwn_write_1(sc, R92C_HMETFR + 3, 0x20); /* Wait until 8051 resets by itself. */ for (ntries = 0; ntries < 100; ntries++) { reg = urtwn_read_2(sc, R92C_SYS_FUNC_EN); if (!(reg & R92C_SYS_FUNC_EN_CPUEN)) return; urtwn_ms_delay(sc); } /* Force 8051 reset. */ urtwn_write_2(sc, R92C_SYS_FUNC_EN, reg & ~R92C_SYS_FUNC_EN_CPUEN); } static void urtwn_r88e_fw_reset(struct urtwn_softc *sc) { uint16_t reg; reg = urtwn_read_2(sc, R92C_SYS_FUNC_EN); urtwn_write_2(sc, R92C_SYS_FUNC_EN, reg & ~R92C_SYS_FUNC_EN_CPUEN); urtwn_write_2(sc, R92C_SYS_FUNC_EN, reg | R92C_SYS_FUNC_EN_CPUEN); } static int urtwn_fw_loadpage(struct urtwn_softc *sc, int page, const uint8_t *buf, int len) { uint32_t reg; usb_error_t error = USB_ERR_NORMAL_COMPLETION; int off, mlen; reg = urtwn_read_4(sc, R92C_MCUFWDL); reg = RW(reg, R92C_MCUFWDL_PAGE, page); urtwn_write_4(sc, R92C_MCUFWDL, reg); off = R92C_FW_START_ADDR; while (len > 0) { if (len > 196) mlen = 196; else if (len > 4) mlen = 4; else mlen = 1; /* XXX fix this deconst */ error = urtwn_write_region_1(sc, off, __DECONST(uint8_t *, buf), mlen); if (error != USB_ERR_NORMAL_COMPLETION) break; off += mlen; buf += mlen; len -= mlen; } return (error); } static int urtwn_load_firmware(struct urtwn_softc *sc) { const struct firmware *fw; const struct r92c_fw_hdr *hdr; const char *imagename; const u_char *ptr; size_t len; uint32_t reg; int mlen, ntries, page, error; URTWN_UNLOCK(sc); /* Read firmware image from the filesystem. */ if (sc->chip & URTWN_CHIP_88E) imagename = "urtwn-rtl8188eufw"; else if ((sc->chip & (URTWN_CHIP_UMC_A_CUT | URTWN_CHIP_92C)) == URTWN_CHIP_UMC_A_CUT) imagename = "urtwn-rtl8192cfwU"; else imagename = "urtwn-rtl8192cfwT"; fw = firmware_get(imagename); URTWN_LOCK(sc); if (fw == NULL) { device_printf(sc->sc_dev, "failed loadfirmware of file %s\n", imagename); return (ENOENT); } len = fw->datasize; if (len < sizeof(*hdr)) { device_printf(sc->sc_dev, "firmware too short\n"); error = EINVAL; goto fail; } ptr = fw->data; hdr = (const struct r92c_fw_hdr *)ptr; /* Check if there is a valid FW header and skip it. */ if ((le16toh(hdr->signature) >> 4) == 0x88c || (le16toh(hdr->signature) >> 4) == 0x88e || (le16toh(hdr->signature) >> 4) == 0x92c) { DPRINTF("FW V%d.%d %02d-%02d %02d:%02d\n", le16toh(hdr->version), le16toh(hdr->subversion), hdr->month, hdr->date, hdr->hour, hdr->minute); ptr += sizeof(*hdr); len -= sizeof(*hdr); } if (urtwn_read_1(sc, R92C_MCUFWDL) & R92C_MCUFWDL_RAM_DL_SEL) { if (sc->chip & URTWN_CHIP_88E) urtwn_r88e_fw_reset(sc); else urtwn_fw_reset(sc); urtwn_write_1(sc, R92C_MCUFWDL, 0); } if (!(sc->chip & URTWN_CHIP_88E)) { urtwn_write_2(sc, R92C_SYS_FUNC_EN, urtwn_read_2(sc, R92C_SYS_FUNC_EN) | R92C_SYS_FUNC_EN_CPUEN); } urtwn_write_1(sc, R92C_MCUFWDL, urtwn_read_1(sc, R92C_MCUFWDL) | R92C_MCUFWDL_EN); urtwn_write_1(sc, R92C_MCUFWDL + 2, urtwn_read_1(sc, R92C_MCUFWDL + 2) & ~0x08); /* Reset the FWDL checksum. */ urtwn_write_1(sc, R92C_MCUFWDL, urtwn_read_1(sc, R92C_MCUFWDL) | R92C_MCUFWDL_CHKSUM_RPT); for (page = 0; len > 0; page++) { mlen = min(len, R92C_FW_PAGE_SIZE); error = urtwn_fw_loadpage(sc, page, ptr, mlen); if (error != 0) { device_printf(sc->sc_dev, "could not load firmware page\n"); goto fail; } ptr += mlen; len -= mlen; } urtwn_write_1(sc, R92C_MCUFWDL, urtwn_read_1(sc, R92C_MCUFWDL) & ~R92C_MCUFWDL_EN); urtwn_write_1(sc, R92C_MCUFWDL + 1, 0); /* Wait for checksum report. */ for (ntries = 0; ntries < 1000; ntries++) { if (urtwn_read_4(sc, R92C_MCUFWDL) & R92C_MCUFWDL_CHKSUM_RPT) break; urtwn_ms_delay(sc); } if (ntries == 1000) { device_printf(sc->sc_dev, "timeout waiting for checksum report\n"); error = ETIMEDOUT; goto fail; } reg = urtwn_read_4(sc, R92C_MCUFWDL); reg = (reg & ~R92C_MCUFWDL_WINTINI_RDY) | R92C_MCUFWDL_RDY; urtwn_write_4(sc, R92C_MCUFWDL, reg); if (sc->chip & URTWN_CHIP_88E) urtwn_r88e_fw_reset(sc); /* Wait for firmware readiness. */ for (ntries = 0; ntries < 1000; ntries++) { if (urtwn_read_4(sc, R92C_MCUFWDL) & R92C_MCUFWDL_WINTINI_RDY) break; urtwn_ms_delay(sc); } if (ntries == 1000) { device_printf(sc->sc_dev, "timeout waiting for firmware readiness\n"); error = ETIMEDOUT; goto fail; } fail: firmware_put(fw, FIRMWARE_UNLOAD); return (error); } static int urtwn_dma_init(struct urtwn_softc *sc) { struct usb_endpoint *ep, *ep_end; usb_error_t usb_err; uint32_t reg; int hashq, hasnq, haslq, nqueues, ntx; int error, pagecount, npubqpages, nqpages, nrempages, tx_boundary; /* Initialize LLT table. */ error = urtwn_llt_init(sc); if (error != 0) return (error); /* Determine the number of bulk-out pipes. */ ntx = 0; ep = sc->sc_udev->endpoints; ep_end = sc->sc_udev->endpoints + sc->sc_udev->endpoints_max; for (; ep != ep_end; ep++) { if ((ep->edesc == NULL) || (ep->iface_index != sc->sc_iface_index)) continue; if (UE_GET_DIR(ep->edesc->bEndpointAddress) == UE_DIR_OUT) ntx++; } if (ntx == 0) { device_printf(sc->sc_dev, "%d: invalid number of Tx bulk pipes\n", ntx); return (EIO); } /* Get Tx queues to USB endpoints mapping. */ hashq = hasnq = haslq = nqueues = 0; switch (ntx) { case 1: hashq = 1; break; case 2: hashq = hasnq = 1; break; case 3: case 4: hashq = hasnq = haslq = 1; break; } nqueues = hashq + hasnq + haslq; if (nqueues == 0) return (EIO); npubqpages = nqpages = nrempages = pagecount = 0; if (sc->chip & URTWN_CHIP_88E) tx_boundary = R88E_TX_PAGE_BOUNDARY; else { pagecount = R92C_TX_PAGE_COUNT; npubqpages = R92C_PUBQ_NPAGES; tx_boundary = R92C_TX_PAGE_BOUNDARY; } /* Set number of pages for normal priority queue. */ if (sc->chip & URTWN_CHIP_88E) { usb_err = urtwn_write_2(sc, R92C_RQPN_NPQ, 0xd); if (usb_err != USB_ERR_NORMAL_COMPLETION) return (EIO); usb_err = urtwn_write_4(sc, R92C_RQPN, 0x808e000d); if (usb_err != USB_ERR_NORMAL_COMPLETION) return (EIO); } else { /* Get the number of pages for each queue. */ nqpages = (pagecount - npubqpages) / nqueues; /* * The remaining pages are assigned to the high priority * queue. */ nrempages = (pagecount - npubqpages) % nqueues; usb_err = urtwn_write_1(sc, R92C_RQPN_NPQ, hasnq ? nqpages : 0); if (usb_err != USB_ERR_NORMAL_COMPLETION) return (EIO); usb_err = urtwn_write_4(sc, R92C_RQPN, /* Set number of pages for public queue. */ SM(R92C_RQPN_PUBQ, npubqpages) | /* Set number of pages for high priority queue. */ SM(R92C_RQPN_HPQ, hashq ? nqpages + nrempages : 0) | /* Set number of pages for low priority queue. */ SM(R92C_RQPN_LPQ, haslq ? nqpages : 0) | /* Load values. */ R92C_RQPN_LD); if (usb_err != USB_ERR_NORMAL_COMPLETION) return (EIO); } usb_err = urtwn_write_1(sc, R92C_TXPKTBUF_BCNQ_BDNY, tx_boundary); if (usb_err != USB_ERR_NORMAL_COMPLETION) return (EIO); usb_err = urtwn_write_1(sc, R92C_TXPKTBUF_MGQ_BDNY, tx_boundary); if (usb_err != USB_ERR_NORMAL_COMPLETION) return (EIO); usb_err = urtwn_write_1(sc, R92C_TXPKTBUF_WMAC_LBK_BF_HD, tx_boundary); if (usb_err != USB_ERR_NORMAL_COMPLETION) return (EIO); usb_err = urtwn_write_1(sc, R92C_TRXFF_BNDY, tx_boundary); if (usb_err != USB_ERR_NORMAL_COMPLETION) return (EIO); usb_err = urtwn_write_1(sc, R92C_TDECTRL + 1, tx_boundary); if (usb_err != USB_ERR_NORMAL_COMPLETION) return (EIO); /* Set queue to USB pipe mapping. */ reg = urtwn_read_2(sc, R92C_TRXDMA_CTRL); reg &= ~R92C_TRXDMA_CTRL_QMAP_M; if (nqueues == 1) { if (hashq) reg |= R92C_TRXDMA_CTRL_QMAP_HQ; else if (hasnq) reg |= R92C_TRXDMA_CTRL_QMAP_NQ; else reg |= R92C_TRXDMA_CTRL_QMAP_LQ; } else if (nqueues == 2) { /* All 2-endpoints configs have a high priority queue. */ if (!hashq) return (EIO); if (hasnq) reg |= R92C_TRXDMA_CTRL_QMAP_HQ_NQ; else reg |= R92C_TRXDMA_CTRL_QMAP_HQ_LQ; } else reg |= R92C_TRXDMA_CTRL_QMAP_3EP; usb_err = urtwn_write_2(sc, R92C_TRXDMA_CTRL, reg); if (usb_err != USB_ERR_NORMAL_COMPLETION) return (EIO); /* Set Tx/Rx transfer page boundary. */ usb_err = urtwn_write_2(sc, R92C_TRXFF_BNDY + 2, (sc->chip & URTWN_CHIP_88E) ? 0x23ff : 0x27ff); if (usb_err != USB_ERR_NORMAL_COMPLETION) return (EIO); /* Set Tx/Rx transfer page size. */ usb_err = urtwn_write_1(sc, R92C_PBP, SM(R92C_PBP_PSRX, R92C_PBP_128) | SM(R92C_PBP_PSTX, R92C_PBP_128)); if (usb_err != USB_ERR_NORMAL_COMPLETION) return (EIO); return (0); } static int urtwn_mac_init(struct urtwn_softc *sc) { usb_error_t error; int i; /* Write MAC initialization values. */ if (sc->chip & URTWN_CHIP_88E) { for (i = 0; i < nitems(rtl8188eu_mac); i++) { error = urtwn_write_1(sc, rtl8188eu_mac[i].reg, rtl8188eu_mac[i].val); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); } urtwn_write_1(sc, R92C_MAX_AGGR_NUM, 0x07); } else { for (i = 0; i < nitems(rtl8192cu_mac); i++) error = urtwn_write_1(sc, rtl8192cu_mac[i].reg, rtl8192cu_mac[i].val); if (error != USB_ERR_NORMAL_COMPLETION) return (EIO); } return (0); } static void urtwn_bb_init(struct urtwn_softc *sc) { const struct urtwn_bb_prog *prog; uint32_t reg; uint8_t crystalcap; int i; /* Enable BB and RF. */ urtwn_write_2(sc, R92C_SYS_FUNC_EN, urtwn_read_2(sc, R92C_SYS_FUNC_EN) | R92C_SYS_FUNC_EN_BBRSTB | R92C_SYS_FUNC_EN_BB_GLB_RST | R92C_SYS_FUNC_EN_DIO_RF); if (!(sc->chip & URTWN_CHIP_88E)) urtwn_write_2(sc, R92C_AFE_PLL_CTRL, 0xdb83); urtwn_write_1(sc, R92C_RF_CTRL, R92C_RF_CTRL_EN | R92C_RF_CTRL_RSTB | R92C_RF_CTRL_SDMRSTB); urtwn_write_1(sc, R92C_SYS_FUNC_EN, R92C_SYS_FUNC_EN_USBA | R92C_SYS_FUNC_EN_USBD | R92C_SYS_FUNC_EN_BB_GLB_RST | R92C_SYS_FUNC_EN_BBRSTB); if (!(sc->chip & URTWN_CHIP_88E)) { urtwn_write_1(sc, R92C_LDOHCI12_CTRL, 0x0f); urtwn_write_1(sc, 0x15, 0xe9); urtwn_write_1(sc, R92C_AFE_XTAL_CTRL + 1, 0x80); } /* Select BB programming based on board type. */ if (sc->chip & URTWN_CHIP_88E) prog = &rtl8188eu_bb_prog; else if (!(sc->chip & URTWN_CHIP_92C)) { if (sc->board_type == R92C_BOARD_TYPE_MINICARD) prog = &rtl8188ce_bb_prog; else if (sc->board_type == R92C_BOARD_TYPE_HIGHPA) prog = &rtl8188ru_bb_prog; else prog = &rtl8188cu_bb_prog; } else { if (sc->board_type == R92C_BOARD_TYPE_MINICARD) prog = &rtl8192ce_bb_prog; else prog = &rtl8192cu_bb_prog; } /* Write BB initialization values. */ for (i = 0; i < prog->count; i++) { urtwn_bb_write(sc, prog->regs[i], prog->vals[i]); urtwn_ms_delay(sc); } if (sc->chip & URTWN_CHIP_92C_1T2R) { /* 8192C 1T only configuration. */ reg = urtwn_bb_read(sc, R92C_FPGA0_TXINFO); reg = (reg & ~0x00000003) | 0x2; urtwn_bb_write(sc, R92C_FPGA0_TXINFO, reg); reg = urtwn_bb_read(sc, R92C_FPGA1_TXINFO); reg = (reg & ~0x00300033) | 0x00200022; urtwn_bb_write(sc, R92C_FPGA1_TXINFO, reg); reg = urtwn_bb_read(sc, R92C_CCK0_AFESETTING); reg = (reg & ~0xff000000) | 0x45 << 24; urtwn_bb_write(sc, R92C_CCK0_AFESETTING, reg); reg = urtwn_bb_read(sc, R92C_OFDM0_TRXPATHENA); reg = (reg & ~0x000000ff) | 0x23; urtwn_bb_write(sc, R92C_OFDM0_TRXPATHENA, reg); reg = urtwn_bb_read(sc, R92C_OFDM0_AGCPARAM1); reg = (reg & ~0x00000030) | 1 << 4; urtwn_bb_write(sc, R92C_OFDM0_AGCPARAM1, reg); reg = urtwn_bb_read(sc, 0xe74); reg = (reg & ~0x0c000000) | 2 << 26; urtwn_bb_write(sc, 0xe74, reg); reg = urtwn_bb_read(sc, 0xe78); reg = (reg & ~0x0c000000) | 2 << 26; urtwn_bb_write(sc, 0xe78, reg); reg = urtwn_bb_read(sc, 0xe7c); reg = (reg & ~0x0c000000) | 2 << 26; urtwn_bb_write(sc, 0xe7c, reg); reg = urtwn_bb_read(sc, 0xe80); reg = (reg & ~0x0c000000) | 2 << 26; urtwn_bb_write(sc, 0xe80, reg); reg = urtwn_bb_read(sc, 0xe88); reg = (reg & ~0x0c000000) | 2 << 26; urtwn_bb_write(sc, 0xe88, reg); } /* Write AGC values. */ for (i = 0; i < prog->agccount; i++) { urtwn_bb_write(sc, R92C_OFDM0_AGCRSSITABLE, prog->agcvals[i]); urtwn_ms_delay(sc); } if (sc->chip & URTWN_CHIP_88E) { urtwn_bb_write(sc, R92C_OFDM0_AGCCORE1(0), 0x69553422); urtwn_ms_delay(sc); urtwn_bb_write(sc, R92C_OFDM0_AGCCORE1(0), 0x69553420); urtwn_ms_delay(sc); crystalcap = sc->rom.r88e_rom[0xb9]; if (crystalcap == 0xff) crystalcap = 0x20; crystalcap &= 0x3f; reg = urtwn_bb_read(sc, R92C_AFE_XTAL_CTRL); urtwn_bb_write(sc, R92C_AFE_XTAL_CTRL, RW(reg, R92C_AFE_XTAL_CTRL_ADDR, crystalcap | crystalcap << 6)); } else { if (urtwn_bb_read(sc, R92C_HSSI_PARAM2(0)) & R92C_HSSI_PARAM2_CCK_HIPWR) sc->sc_flags |= URTWN_FLAG_CCK_HIPWR; } } static void urtwn_rf_init(struct urtwn_softc *sc) { const struct urtwn_rf_prog *prog; uint32_t reg, type; int i, j, idx, off; /* Select RF programming based on board type. */ if (sc->chip & URTWN_CHIP_88E) prog = rtl8188eu_rf_prog; else if (!(sc->chip & URTWN_CHIP_92C)) { if (sc->board_type == R92C_BOARD_TYPE_MINICARD) prog = rtl8188ce_rf_prog; else if (sc->board_type == R92C_BOARD_TYPE_HIGHPA) prog = rtl8188ru_rf_prog; else prog = rtl8188cu_rf_prog; } else prog = rtl8192ce_rf_prog; for (i = 0; i < sc->nrxchains; i++) { /* Save RF_ENV control type. */ idx = i / 2; off = (i % 2) * 16; reg = urtwn_bb_read(sc, R92C_FPGA0_RFIFACESW(idx)); type = (reg >> off) & 0x10; /* Set RF_ENV enable. */ reg = urtwn_bb_read(sc, R92C_FPGA0_RFIFACEOE(i)); reg |= 0x100000; urtwn_bb_write(sc, R92C_FPGA0_RFIFACEOE(i), reg); urtwn_ms_delay(sc); /* Set RF_ENV output high. */ reg = urtwn_bb_read(sc, R92C_FPGA0_RFIFACEOE(i)); reg |= 0x10; urtwn_bb_write(sc, R92C_FPGA0_RFIFACEOE(i), reg); urtwn_ms_delay(sc); /* Set address and data lengths of RF registers. */ reg = urtwn_bb_read(sc, R92C_HSSI_PARAM2(i)); reg &= ~R92C_HSSI_PARAM2_ADDR_LENGTH; urtwn_bb_write(sc, R92C_HSSI_PARAM2(i), reg); urtwn_ms_delay(sc); reg = urtwn_bb_read(sc, R92C_HSSI_PARAM2(i)); reg &= ~R92C_HSSI_PARAM2_DATA_LENGTH; urtwn_bb_write(sc, R92C_HSSI_PARAM2(i), reg); urtwn_ms_delay(sc); /* Write RF initialization values for this chain. */ for (j = 0; j < prog[i].count; j++) { if (prog[i].regs[j] >= 0xf9 && prog[i].regs[j] <= 0xfe) { /* * These are fake RF registers offsets that * indicate a delay is required. */ usb_pause_mtx(&sc->sc_mtx, hz / 20); /* 50ms */ continue; } urtwn_rf_write(sc, i, prog[i].regs[j], prog[i].vals[j]); urtwn_ms_delay(sc); } /* Restore RF_ENV control type. */ reg = urtwn_bb_read(sc, R92C_FPGA0_RFIFACESW(idx)); reg &= ~(0x10 << off) | (type << off); urtwn_bb_write(sc, R92C_FPGA0_RFIFACESW(idx), reg); /* Cache RF register CHNLBW. */ sc->rf_chnlbw[i] = urtwn_rf_read(sc, i, R92C_RF_CHNLBW); } if ((sc->chip & (URTWN_CHIP_UMC_A_CUT | URTWN_CHIP_92C)) == URTWN_CHIP_UMC_A_CUT) { urtwn_rf_write(sc, 0, R92C_RF_RX_G1, 0x30255); urtwn_rf_write(sc, 0, R92C_RF_RX_G2, 0x50a00); } } static void urtwn_cam_init(struct urtwn_softc *sc) { /* Invalidate all CAM entries. */ urtwn_write_4(sc, R92C_CAMCMD, R92C_CAMCMD_POLLING | R92C_CAMCMD_CLR); } static void urtwn_pa_bias_init(struct urtwn_softc *sc) { uint8_t reg; int i; for (i = 0; i < sc->nrxchains; i++) { if (sc->pa_setting & (1 << i)) continue; urtwn_rf_write(sc, i, R92C_RF_IPA, 0x0f406); urtwn_rf_write(sc, i, R92C_RF_IPA, 0x4f406); urtwn_rf_write(sc, i, R92C_RF_IPA, 0x8f406); urtwn_rf_write(sc, i, R92C_RF_IPA, 0xcf406); } if (!(sc->pa_setting & 0x10)) { reg = urtwn_read_1(sc, 0x16); reg = (reg & ~0xf0) | 0x90; urtwn_write_1(sc, 0x16, reg); } } static void urtwn_rxfilter_init(struct urtwn_softc *sc) { struct ieee80211com *ic = &sc->sc_ic; struct ieee80211vap *vap = TAILQ_FIRST(&ic->ic_vaps); uint32_t rcr; uint16_t filter; URTWN_ASSERT_LOCKED(sc); /* Accept all multicast frames. */ urtwn_write_4(sc, R92C_MAR + 0, 0xffffffff); urtwn_write_4(sc, R92C_MAR + 4, 0xffffffff); /* Filter for management frames. */ filter = 0x7f3f; switch (vap->iv_opmode) { case IEEE80211_M_STA: filter &= ~( R92C_RXFLTMAP_SUBTYPE(IEEE80211_FC0_SUBTYPE_ASSOC_REQ) | R92C_RXFLTMAP_SUBTYPE(IEEE80211_FC0_SUBTYPE_REASSOC_REQ) | R92C_RXFLTMAP_SUBTYPE(IEEE80211_FC0_SUBTYPE_PROBE_REQ)); break; case IEEE80211_M_HOSTAP: filter &= ~( R92C_RXFLTMAP_SUBTYPE(IEEE80211_FC0_SUBTYPE_ASSOC_RESP) | R92C_RXFLTMAP_SUBTYPE(IEEE80211_FC0_SUBTYPE_REASSOC_RESP) | R92C_RXFLTMAP_SUBTYPE(IEEE80211_FC0_SUBTYPE_BEACON)); break; case IEEE80211_M_MONITOR: case IEEE80211_M_IBSS: break; default: device_printf(sc->sc_dev, "%s: undefined opmode %d\n", __func__, vap->iv_opmode); break; } urtwn_write_2(sc, R92C_RXFLTMAP0, filter); /* Reject all control frames. */ urtwn_write_2(sc, R92C_RXFLTMAP1, 0x0000); /* Reject all data frames. */ urtwn_write_2(sc, R92C_RXFLTMAP2, 0x0000); rcr = R92C_RCR_AM | R92C_RCR_AB | R92C_RCR_APM | R92C_RCR_HTC_LOC_CTRL | R92C_RCR_APP_PHYSTS | R92C_RCR_APP_ICV | R92C_RCR_APP_MIC; if (vap->iv_opmode == IEEE80211_M_MONITOR) { /* Accept all frames. */ rcr |= R92C_RCR_ACF | R92C_RCR_ADF | R92C_RCR_AMF | R92C_RCR_AAP; } /* Set Rx filter. */ urtwn_write_4(sc, R92C_RCR, rcr); if (ic->ic_promisc != 0) { /* Update Rx filter. */ urtwn_set_promisc(sc); } } static void urtwn_edca_init(struct urtwn_softc *sc) { urtwn_write_2(sc, R92C_SPEC_SIFS, 0x100a); urtwn_write_2(sc, R92C_MAC_SPEC_SIFS, 0x100a); urtwn_write_2(sc, R92C_SIFS_CCK, 0x100a); urtwn_write_2(sc, R92C_SIFS_OFDM, 0x100a); urtwn_write_4(sc, R92C_EDCA_BE_PARAM, 0x005ea42b); urtwn_write_4(sc, R92C_EDCA_BK_PARAM, 0x0000a44f); urtwn_write_4(sc, R92C_EDCA_VI_PARAM, 0x005ea324); urtwn_write_4(sc, R92C_EDCA_VO_PARAM, 0x002fa226); } static void urtwn_write_txpower(struct urtwn_softc *sc, int chain, uint16_t power[URTWN_RIDX_COUNT]) { uint32_t reg; /* Write per-CCK rate Tx power. */ if (chain == 0) { reg = urtwn_bb_read(sc, R92C_TXAGC_A_CCK1_MCS32); reg = RW(reg, R92C_TXAGC_A_CCK1, power[0]); urtwn_bb_write(sc, R92C_TXAGC_A_CCK1_MCS32, reg); reg = urtwn_bb_read(sc, R92C_TXAGC_B_CCK11_A_CCK2_11); reg = RW(reg, R92C_TXAGC_A_CCK2, power[1]); reg = RW(reg, R92C_TXAGC_A_CCK55, power[2]); reg = RW(reg, R92C_TXAGC_A_CCK11, power[3]); urtwn_bb_write(sc, R92C_TXAGC_B_CCK11_A_CCK2_11, reg); } else { reg = urtwn_bb_read(sc, R92C_TXAGC_B_CCK1_55_MCS32); reg = RW(reg, R92C_TXAGC_B_CCK1, power[0]); reg = RW(reg, R92C_TXAGC_B_CCK2, power[1]); reg = RW(reg, R92C_TXAGC_B_CCK55, power[2]); urtwn_bb_write(sc, R92C_TXAGC_B_CCK1_55_MCS32, reg); reg = urtwn_bb_read(sc, R92C_TXAGC_B_CCK11_A_CCK2_11); reg = RW(reg, R92C_TXAGC_B_CCK11, power[3]); urtwn_bb_write(sc, R92C_TXAGC_B_CCK11_A_CCK2_11, reg); } /* Write per-OFDM rate Tx power. */ urtwn_bb_write(sc, R92C_TXAGC_RATE18_06(chain), SM(R92C_TXAGC_RATE06, power[ 4]) | SM(R92C_TXAGC_RATE09, power[ 5]) | SM(R92C_TXAGC_RATE12, power[ 6]) | SM(R92C_TXAGC_RATE18, power[ 7])); urtwn_bb_write(sc, R92C_TXAGC_RATE54_24(chain), SM(R92C_TXAGC_RATE24, power[ 8]) | SM(R92C_TXAGC_RATE36, power[ 9]) | SM(R92C_TXAGC_RATE48, power[10]) | SM(R92C_TXAGC_RATE54, power[11])); /* Write per-MCS Tx power. */ urtwn_bb_write(sc, R92C_TXAGC_MCS03_MCS00(chain), SM(R92C_TXAGC_MCS00, power[12]) | SM(R92C_TXAGC_MCS01, power[13]) | SM(R92C_TXAGC_MCS02, power[14]) | SM(R92C_TXAGC_MCS03, power[15])); urtwn_bb_write(sc, R92C_TXAGC_MCS07_MCS04(chain), SM(R92C_TXAGC_MCS04, power[16]) | SM(R92C_TXAGC_MCS05, power[17]) | SM(R92C_TXAGC_MCS06, power[18]) | SM(R92C_TXAGC_MCS07, power[19])); urtwn_bb_write(sc, R92C_TXAGC_MCS11_MCS08(chain), SM(R92C_TXAGC_MCS08, power[20]) | SM(R92C_TXAGC_MCS09, power[21]) | SM(R92C_TXAGC_MCS10, power[22]) | SM(R92C_TXAGC_MCS11, power[23])); urtwn_bb_write(sc, R92C_TXAGC_MCS15_MCS12(chain), SM(R92C_TXAGC_MCS12, power[24]) | SM(R92C_TXAGC_MCS13, power[25]) | SM(R92C_TXAGC_MCS14, power[26]) | SM(R92C_TXAGC_MCS15, power[27])); } static void urtwn_get_txpower(struct urtwn_softc *sc, int chain, struct ieee80211_channel *c, struct ieee80211_channel *extc, uint16_t power[URTWN_RIDX_COUNT]) { struct ieee80211com *ic = &sc->sc_ic; struct r92c_rom *rom = &sc->rom.r92c_rom; uint16_t cckpow, ofdmpow, htpow, diff, max; const struct urtwn_txpwr *base; int ridx, chan, group; /* Determine channel group. */ chan = ieee80211_chan2ieee(ic, c); /* XXX center freq! */ if (chan <= 3) group = 0; else if (chan <= 9) group = 1; else group = 2; /* Get original Tx power based on board type and RF chain. */ if (!(sc->chip & URTWN_CHIP_92C)) { if (sc->board_type == R92C_BOARD_TYPE_HIGHPA) base = &rtl8188ru_txagc[chain]; else base = &rtl8192cu_txagc[chain]; } else base = &rtl8192cu_txagc[chain]; memset(power, 0, URTWN_RIDX_COUNT * sizeof(power[0])); if (sc->regulatory == 0) { for (ridx = URTWN_RIDX_CCK1; ridx <= URTWN_RIDX_CCK11; ridx++) power[ridx] = base->pwr[0][ridx]; } for (ridx = URTWN_RIDX_OFDM6; ridx < URTWN_RIDX_COUNT; ridx++) { if (sc->regulatory == 3) { power[ridx] = base->pwr[0][ridx]; /* Apply vendor limits. */ if (extc != NULL) max = rom->ht40_max_pwr[group]; else max = rom->ht20_max_pwr[group]; max = (max >> (chain * 4)) & 0xf; if (power[ridx] > max) power[ridx] = max; } else if (sc->regulatory == 1) { if (extc == NULL) power[ridx] = base->pwr[group][ridx]; } else if (sc->regulatory != 2) power[ridx] = base->pwr[0][ridx]; } /* Compute per-CCK rate Tx power. */ cckpow = rom->cck_tx_pwr[chain][group]; for (ridx = URTWN_RIDX_CCK1; ridx <= URTWN_RIDX_CCK11; ridx++) { power[ridx] += cckpow; if (power[ridx] > R92C_MAX_TX_PWR) power[ridx] = R92C_MAX_TX_PWR; } htpow = rom->ht40_1s_tx_pwr[chain][group]; if (sc->ntxchains > 1) { /* Apply reduction for 2 spatial streams. */ diff = rom->ht40_2s_tx_pwr_diff[group]; diff = (diff >> (chain * 4)) & 0xf; htpow = (htpow > diff) ? htpow - diff : 0; } /* Compute per-OFDM rate Tx power. */ diff = rom->ofdm_tx_pwr_diff[group]; diff = (diff >> (chain * 4)) & 0xf; ofdmpow = htpow + diff; /* HT->OFDM correction. */ for (ridx = URTWN_RIDX_OFDM6; ridx <= URTWN_RIDX_OFDM54; ridx++) { power[ridx] += ofdmpow; if (power[ridx] > R92C_MAX_TX_PWR) power[ridx] = R92C_MAX_TX_PWR; } /* Compute per-MCS Tx power. */ if (extc == NULL) { diff = rom->ht20_tx_pwr_diff[group]; diff = (diff >> (chain * 4)) & 0xf; htpow += diff; /* HT40->HT20 correction. */ } for (ridx = 12; ridx <= 27; ridx++) { power[ridx] += htpow; if (power[ridx] > R92C_MAX_TX_PWR) power[ridx] = R92C_MAX_TX_PWR; } #ifdef URTWN_DEBUG if (urtwn_debug >= 4) { /* Dump per-rate Tx power values. */ printf("Tx power for chain %d:\n", chain); for (ridx = URTWN_RIDX_CCK1; ridx < URTWN_RIDX_COUNT; ridx++) printf("Rate %d = %u\n", ridx, power[ridx]); } #endif } static void urtwn_r88e_get_txpower(struct urtwn_softc *sc, int chain, struct ieee80211_channel *c, struct ieee80211_channel *extc, uint16_t power[URTWN_RIDX_COUNT]) { struct ieee80211com *ic = &sc->sc_ic; uint16_t cckpow, ofdmpow, bw20pow, htpow; const struct urtwn_r88e_txpwr *base; int ridx, chan, group; /* Determine channel group. */ chan = ieee80211_chan2ieee(ic, c); /* XXX center freq! */ if (chan <= 2) group = 0; else if (chan <= 5) group = 1; else if (chan <= 8) group = 2; else if (chan <= 11) group = 3; else if (chan <= 13) group = 4; else group = 5; /* Get original Tx power based on board type and RF chain. */ base = &rtl8188eu_txagc[chain]; memset(power, 0, URTWN_RIDX_COUNT * sizeof(power[0])); if (sc->regulatory == 0) { for (ridx = URTWN_RIDX_CCK1; ridx <= URTWN_RIDX_CCK11; ridx++) power[ridx] = base->pwr[0][ridx]; } for (ridx = URTWN_RIDX_OFDM6; ridx < URTWN_RIDX_COUNT; ridx++) { if (sc->regulatory == 3) power[ridx] = base->pwr[0][ridx]; else if (sc->regulatory == 1) { if (extc == NULL) power[ridx] = base->pwr[group][ridx]; } else if (sc->regulatory != 2) power[ridx] = base->pwr[0][ridx]; } /* Compute per-CCK rate Tx power. */ cckpow = sc->cck_tx_pwr[group]; for (ridx = URTWN_RIDX_CCK1; ridx <= URTWN_RIDX_CCK11; ridx++) { power[ridx] += cckpow; if (power[ridx] > R92C_MAX_TX_PWR) power[ridx] = R92C_MAX_TX_PWR; } htpow = sc->ht40_tx_pwr[group]; /* Compute per-OFDM rate Tx power. */ ofdmpow = htpow + sc->ofdm_tx_pwr_diff; for (ridx = URTWN_RIDX_OFDM6; ridx <= URTWN_RIDX_OFDM54; ridx++) { power[ridx] += ofdmpow; if (power[ridx] > R92C_MAX_TX_PWR) power[ridx] = R92C_MAX_TX_PWR; } bw20pow = htpow + sc->bw20_tx_pwr_diff; for (ridx = 12; ridx <= 27; ridx++) { power[ridx] += bw20pow; if (power[ridx] > R92C_MAX_TX_PWR) power[ridx] = R92C_MAX_TX_PWR; } } static void urtwn_set_txpower(struct urtwn_softc *sc, struct ieee80211_channel *c, struct ieee80211_channel *extc) { uint16_t power[URTWN_RIDX_COUNT]; int i; for (i = 0; i < sc->ntxchains; i++) { /* Compute per-rate Tx power values. */ if (sc->chip & URTWN_CHIP_88E) urtwn_r88e_get_txpower(sc, i, c, extc, power); else urtwn_get_txpower(sc, i, c, extc, power); /* Write per-rate Tx power values to hardware. */ urtwn_write_txpower(sc, i, power); } } static void urtwn_set_rx_bssid_all(struct urtwn_softc *sc, int enable) { uint32_t reg; reg = urtwn_read_4(sc, R92C_RCR); if (enable) reg &= ~R92C_RCR_CBSSID_BCN; else reg |= R92C_RCR_CBSSID_BCN; urtwn_write_4(sc, R92C_RCR, reg); } static void urtwn_set_gain(struct urtwn_softc *sc, uint8_t gain) { uint32_t reg; reg = urtwn_bb_read(sc, R92C_OFDM0_AGCCORE1(0)); reg = RW(reg, R92C_OFDM0_AGCCORE1_GAIN, gain); urtwn_bb_write(sc, R92C_OFDM0_AGCCORE1(0), reg); if (!(sc->chip & URTWN_CHIP_88E)) { reg = urtwn_bb_read(sc, R92C_OFDM0_AGCCORE1(1)); reg = RW(reg, R92C_OFDM0_AGCCORE1_GAIN, gain); urtwn_bb_write(sc, R92C_OFDM0_AGCCORE1(1), reg); } } static void urtwn_scan_start(struct ieee80211com *ic) { struct urtwn_softc *sc = ic->ic_softc; URTWN_LOCK(sc); /* Receive beacons / probe responses from any BSSID. */ if (ic->ic_opmode != IEEE80211_M_IBSS) urtwn_set_rx_bssid_all(sc, 1); /* Set gain for scanning. */ urtwn_set_gain(sc, 0x20); URTWN_UNLOCK(sc); } static void urtwn_scan_end(struct ieee80211com *ic) { struct urtwn_softc *sc = ic->ic_softc; URTWN_LOCK(sc); /* Restore limitations. */ if (ic->ic_promisc == 0 && ic->ic_opmode != IEEE80211_M_IBSS) urtwn_set_rx_bssid_all(sc, 0); /* Set gain under link. */ urtwn_set_gain(sc, 0x32); URTWN_UNLOCK(sc); } static void urtwn_set_channel(struct ieee80211com *ic) { struct urtwn_softc *sc = ic->ic_softc; struct ieee80211vap *vap = TAILQ_FIRST(&ic->ic_vaps); URTWN_LOCK(sc); if (vap->iv_state == IEEE80211_S_SCAN) { /* Make link LED blink during scan. */ urtwn_set_led(sc, URTWN_LED_LINK, !sc->ledlink); } urtwn_set_chan(sc, ic->ic_curchan, NULL); URTWN_UNLOCK(sc); +} + +static int +urtwn_wme_update(struct ieee80211com *ic) +{ + const struct wmeParams *wmep = + ic->ic_wme.wme_chanParams.cap_wmeParams; + struct urtwn_softc *sc = ic->ic_softc; + uint8_t aifs, acm, slottime; + int ac; + + acm = 0; + slottime = (ic->ic_flags & IEEE80211_F_SHSLOT) ? + IEEE80211_DUR_SHSLOT : IEEE80211_DUR_SLOT; + + URTWN_LOCK(sc); + for (ac = WME_AC_BE; ac < WME_NUM_AC; ac++) { + /* AIFS[AC] = AIFSN[AC] * aSlotTime + aSIFSTime. */ + aifs = wmep[ac].wmep_aifsn * slottime + IEEE80211_DUR_SIFS; + urtwn_write_4(sc, wme2queue[ac].reg, + SM(R92C_EDCA_PARAM_TXOP, wmep[ac].wmep_txopLimit) | + SM(R92C_EDCA_PARAM_ECWMIN, wmep[ac].wmep_logcwmin) | + SM(R92C_EDCA_PARAM_ECWMAX, wmep[ac].wmep_logcwmax) | + SM(R92C_EDCA_PARAM_AIFS, aifs)); + if (ac != WME_AC_BE) + acm |= wmep[ac].wmep_acm << ac; + } + + if (acm != 0) + acm |= R92C_ACMHWCTRL_EN; + urtwn_write_1(sc, R92C_ACMHWCTRL, + (urtwn_read_1(sc, R92C_ACMHWCTRL) & ~R92C_ACMHWCTRL_ACM_MASK) | + acm); + + URTWN_UNLOCK(sc); + + return 0; } static void urtwn_set_promisc(struct urtwn_softc *sc) { struct ieee80211com *ic = &sc->sc_ic; struct ieee80211vap *vap = TAILQ_FIRST(&ic->ic_vaps); uint32_t rcr, mask1, mask2; URTWN_ASSERT_LOCKED(sc); if (vap->iv_opmode == IEEE80211_M_MONITOR) return; mask1 = R92C_RCR_ACF | R92C_RCR_ADF | R92C_RCR_AMF | R92C_RCR_AAP; mask2 = R92C_RCR_APM; if (vap->iv_state == IEEE80211_S_RUN) { switch (vap->iv_opmode) { case IEEE80211_M_STA: mask2 |= R92C_RCR_CBSSID_DATA; /* FALLTHROUGH */ case IEEE80211_M_HOSTAP: mask2 |= R92C_RCR_CBSSID_BCN; break; case IEEE80211_M_IBSS: mask2 |= R92C_RCR_CBSSID_DATA; break; default: device_printf(sc->sc_dev, "%s: undefined opmode %d\n", __func__, vap->iv_opmode); return; } } rcr = urtwn_read_4(sc, R92C_RCR); if (ic->ic_promisc == 0) rcr = (rcr & ~mask1) | mask2; else rcr = (rcr & ~mask2) | mask1; urtwn_write_4(sc, R92C_RCR, rcr); } static void urtwn_update_promisc(struct ieee80211com *ic) { struct urtwn_softc *sc = ic->ic_softc; URTWN_LOCK(sc); if (sc->sc_flags & URTWN_RUNNING) urtwn_set_promisc(sc); URTWN_UNLOCK(sc); } static void urtwn_update_mcast(struct ieee80211com *ic) { /* XXX do nothing? */ } static void urtwn_set_chan(struct urtwn_softc *sc, struct ieee80211_channel *c, struct ieee80211_channel *extc) { struct ieee80211com *ic = &sc->sc_ic; uint32_t reg; u_int chan; int i; chan = ieee80211_chan2ieee(ic, c); /* XXX center freq! */ if (chan == 0 || chan == IEEE80211_CHAN_ANY) { device_printf(sc->sc_dev, "%s: invalid channel %x\n", __func__, chan); return; } /* Set Tx power for this new channel. */ urtwn_set_txpower(sc, c, extc); for (i = 0; i < sc->nrxchains; i++) { urtwn_rf_write(sc, i, R92C_RF_CHNLBW, RW(sc->rf_chnlbw[i], R92C_RF_CHNLBW_CHNL, chan)); } #ifndef IEEE80211_NO_HT if (extc != NULL) { /* Is secondary channel below or above primary? */ int prichlo = c->ic_freq < extc->ic_freq; urtwn_write_1(sc, R92C_BWOPMODE, urtwn_read_1(sc, R92C_BWOPMODE) & ~R92C_BWOPMODE_20MHZ); reg = urtwn_read_1(sc, R92C_RRSR + 2); reg = (reg & ~0x6f) | (prichlo ? 1 : 2) << 5; urtwn_write_1(sc, R92C_RRSR + 2, reg); urtwn_bb_write(sc, R92C_FPGA0_RFMOD, urtwn_bb_read(sc, R92C_FPGA0_RFMOD) | R92C_RFMOD_40MHZ); urtwn_bb_write(sc, R92C_FPGA1_RFMOD, urtwn_bb_read(sc, R92C_FPGA1_RFMOD) | R92C_RFMOD_40MHZ); /* Set CCK side band. */ reg = urtwn_bb_read(sc, R92C_CCK0_SYSTEM); reg = (reg & ~0x00000010) | (prichlo ? 0 : 1) << 4; urtwn_bb_write(sc, R92C_CCK0_SYSTEM, reg); reg = urtwn_bb_read(sc, R92C_OFDM1_LSTF); reg = (reg & ~0x00000c00) | (prichlo ? 1 : 2) << 10; urtwn_bb_write(sc, R92C_OFDM1_LSTF, reg); urtwn_bb_write(sc, R92C_FPGA0_ANAPARAM2, urtwn_bb_read(sc, R92C_FPGA0_ANAPARAM2) & ~R92C_FPGA0_ANAPARAM2_CBW20); reg = urtwn_bb_read(sc, 0x818); reg = (reg & ~0x0c000000) | (prichlo ? 2 : 1) << 26; urtwn_bb_write(sc, 0x818, reg); /* Select 40MHz bandwidth. */ urtwn_rf_write(sc, 0, R92C_RF_CHNLBW, (sc->rf_chnlbw[0] & ~0xfff) | chan); } else #endif { urtwn_write_1(sc, R92C_BWOPMODE, urtwn_read_1(sc, R92C_BWOPMODE) | R92C_BWOPMODE_20MHZ); urtwn_bb_write(sc, R92C_FPGA0_RFMOD, urtwn_bb_read(sc, R92C_FPGA0_RFMOD) & ~R92C_RFMOD_40MHZ); urtwn_bb_write(sc, R92C_FPGA1_RFMOD, urtwn_bb_read(sc, R92C_FPGA1_RFMOD) & ~R92C_RFMOD_40MHZ); if (!(sc->chip & URTWN_CHIP_88E)) { urtwn_bb_write(sc, R92C_FPGA0_ANAPARAM2, urtwn_bb_read(sc, R92C_FPGA0_ANAPARAM2) | R92C_FPGA0_ANAPARAM2_CBW20); } /* Select 20MHz bandwidth. */ urtwn_rf_write(sc, 0, R92C_RF_CHNLBW, (sc->rf_chnlbw[0] & ~0xfff) | chan | ((sc->chip & URTWN_CHIP_88E) ? R88E_RF_CHNLBW_BW20 : R92C_RF_CHNLBW_BW20)); } } static void urtwn_iq_calib(struct urtwn_softc *sc) { /* TODO */ } static void urtwn_lc_calib(struct urtwn_softc *sc) { uint32_t rf_ac[2]; uint8_t txmode; int i; txmode = urtwn_read_1(sc, R92C_OFDM1_LSTF + 3); if ((txmode & 0x70) != 0) { /* Disable all continuous Tx. */ urtwn_write_1(sc, R92C_OFDM1_LSTF + 3, txmode & ~0x70); /* Set RF mode to standby mode. */ for (i = 0; i < sc->nrxchains; i++) { rf_ac[i] = urtwn_rf_read(sc, i, R92C_RF_AC); urtwn_rf_write(sc, i, R92C_RF_AC, RW(rf_ac[i], R92C_RF_AC_MODE, R92C_RF_AC_MODE_STANDBY)); } } else { /* Block all Tx queues. */ urtwn_write_1(sc, R92C_TXPAUSE, 0xff); } /* Start calibration. */ urtwn_rf_write(sc, 0, R92C_RF_CHNLBW, urtwn_rf_read(sc, 0, R92C_RF_CHNLBW) | R92C_RF_CHNLBW_LCSTART); /* Give calibration the time to complete. */ usb_pause_mtx(&sc->sc_mtx, hz / 10); /* 100ms */ /* Restore configuration. */ if ((txmode & 0x70) != 0) { /* Restore Tx mode. */ urtwn_write_1(sc, R92C_OFDM1_LSTF + 3, txmode); /* Restore RF mode. */ for (i = 0; i < sc->nrxchains; i++) urtwn_rf_write(sc, i, R92C_RF_AC, rf_ac[i]); } else { /* Unblock all Tx queues. */ urtwn_write_1(sc, R92C_TXPAUSE, 0x00); } } static int urtwn_init(struct urtwn_softc *sc) { struct ieee80211com *ic = &sc->sc_ic; struct ieee80211vap *vap = TAILQ_FIRST(&ic->ic_vaps); uint8_t macaddr[IEEE80211_ADDR_LEN]; uint32_t reg; usb_error_t usb_err = USB_ERR_NORMAL_COMPLETION; int error; URTWN_LOCK(sc); if (sc->sc_flags & URTWN_RUNNING) { URTWN_UNLOCK(sc); return (0); } /* Init firmware commands ring. */ sc->fwcur = 0; /* Allocate Tx/Rx buffers. */ error = urtwn_alloc_rx_list(sc); if (error != 0) goto fail; error = urtwn_alloc_tx_list(sc); if (error != 0) goto fail; /* Power on adapter. */ error = urtwn_power_on(sc); if (error != 0) goto fail; /* Initialize DMA. */ error = urtwn_dma_init(sc); if (error != 0) goto fail; /* Set info size in Rx descriptors (in 64-bit words). */ urtwn_write_1(sc, R92C_RX_DRVINFO_SZ, 4); /* Init interrupts. */ if (sc->chip & URTWN_CHIP_88E) { usb_err = urtwn_write_4(sc, R88E_HISR, 0xffffffff); if (usb_err != USB_ERR_NORMAL_COMPLETION) goto fail; usb_err = urtwn_write_4(sc, R88E_HIMR, R88E_HIMR_CPWM | R88E_HIMR_CPWM2 | R88E_HIMR_TBDER | R88E_HIMR_PSTIMEOUT); if (usb_err != USB_ERR_NORMAL_COMPLETION) goto fail; usb_err = urtwn_write_4(sc, R88E_HIMRE, R88E_HIMRE_RXFOVW | R88E_HIMRE_TXFOVW | R88E_HIMRE_RXERR | R88E_HIMRE_TXERR); if (usb_err != USB_ERR_NORMAL_COMPLETION) goto fail; usb_err = urtwn_write_1(sc, R92C_USB_SPECIAL_OPTION, urtwn_read_1(sc, R92C_USB_SPECIAL_OPTION) | R92C_USB_SPECIAL_OPTION_INT_BULK_SEL); if (usb_err != USB_ERR_NORMAL_COMPLETION) goto fail; } else { usb_err = urtwn_write_4(sc, R92C_HISR, 0xffffffff); if (usb_err != USB_ERR_NORMAL_COMPLETION) goto fail; usb_err = urtwn_write_4(sc, R92C_HIMR, 0xffffffff); if (usb_err != USB_ERR_NORMAL_COMPLETION) goto fail; } /* Set MAC address. */ IEEE80211_ADDR_COPY(macaddr, vap ? vap->iv_myaddr : ic->ic_macaddr); usb_err = urtwn_write_region_1(sc, R92C_MACID, macaddr, IEEE80211_ADDR_LEN); if (usb_err != USB_ERR_NORMAL_COMPLETION) goto fail; /* Set initial network type. */ urtwn_set_mode(sc, R92C_MSR_INFRA); /* Initialize Rx filter. */ urtwn_rxfilter_init(sc); /* Set response rate. */ reg = urtwn_read_4(sc, R92C_RRSR); reg = RW(reg, R92C_RRSR_RATE_BITMAP, R92C_RRSR_RATE_CCK_ONLY_1M); urtwn_write_4(sc, R92C_RRSR, reg); /* Set short/long retry limits. */ urtwn_write_2(sc, R92C_RL, SM(R92C_RL_SRL, 0x30) | SM(R92C_RL_LRL, 0x30)); /* Initialize EDCA parameters. */ urtwn_edca_init(sc); /* Setup rate fallback. */ if (!(sc->chip & URTWN_CHIP_88E)) { urtwn_write_4(sc, R92C_DARFRC + 0, 0x00000000); urtwn_write_4(sc, R92C_DARFRC + 4, 0x10080404); urtwn_write_4(sc, R92C_RARFRC + 0, 0x04030201); urtwn_write_4(sc, R92C_RARFRC + 4, 0x08070605); } urtwn_write_1(sc, R92C_FWHW_TXQ_CTRL, urtwn_read_1(sc, R92C_FWHW_TXQ_CTRL) | R92C_FWHW_TXQ_CTRL_AMPDU_RTY_NEW); /* Set ACK timeout. */ urtwn_write_1(sc, R92C_ACKTO, 0x40); /* Setup USB aggregation. */ reg = urtwn_read_4(sc, R92C_TDECTRL); reg = RW(reg, R92C_TDECTRL_BLK_DESC_NUM, 6); urtwn_write_4(sc, R92C_TDECTRL, reg); urtwn_write_1(sc, R92C_TRXDMA_CTRL, urtwn_read_1(sc, R92C_TRXDMA_CTRL) | R92C_TRXDMA_CTRL_RXDMA_AGG_EN); urtwn_write_1(sc, R92C_RXDMA_AGG_PG_TH, 48); if (sc->chip & URTWN_CHIP_88E) urtwn_write_1(sc, R92C_RXDMA_AGG_PG_TH + 1, 4); else { urtwn_write_1(sc, R92C_USB_DMA_AGG_TO, 4); urtwn_write_1(sc, R92C_USB_SPECIAL_OPTION, urtwn_read_1(sc, R92C_USB_SPECIAL_OPTION) | R92C_USB_SPECIAL_OPTION_AGG_EN); urtwn_write_1(sc, R92C_USB_AGG_TH, 8); urtwn_write_1(sc, R92C_USB_AGG_TO, 6); } /* Initialize beacon parameters. */ urtwn_write_2(sc, R92C_BCN_CTRL, 0x1010); urtwn_write_2(sc, R92C_TBTT_PROHIBIT, 0x6404); urtwn_write_1(sc, R92C_DRVERLYINT, 0x05); urtwn_write_1(sc, R92C_BCNDMATIM, 0x02); urtwn_write_2(sc, R92C_BCNTCFG, 0x660f); if (!(sc->chip & URTWN_CHIP_88E)) { /* Setup AMPDU aggregation. */ urtwn_write_4(sc, R92C_AGGLEN_LMT, 0x99997631); /* MCS7~0 */ urtwn_write_1(sc, R92C_AGGR_BREAK_TIME, 0x16); urtwn_write_2(sc, R92C_MAX_AGGR_NUM, 0x0708); urtwn_write_1(sc, R92C_BCN_MAX_ERR, 0xff); } /* Load 8051 microcode. */ error = urtwn_load_firmware(sc); if (error != 0) goto fail; /* Initialize MAC/BB/RF blocks. */ error = urtwn_mac_init(sc); if (error != 0) { device_printf(sc->sc_dev, "%s: error while initializing MAC block\n", __func__); goto fail; } urtwn_bb_init(sc); urtwn_rf_init(sc); /* Reinitialize Rx filter (D3845 is not committed yet). */ urtwn_rxfilter_init(sc); if (sc->chip & URTWN_CHIP_88E) { urtwn_write_2(sc, R92C_CR, urtwn_read_2(sc, R92C_CR) | R92C_CR_MACTXEN | R92C_CR_MACRXEN); } /* Turn CCK and OFDM blocks on. */ reg = urtwn_bb_read(sc, R92C_FPGA0_RFMOD); reg |= R92C_RFMOD_CCK_EN; usb_err = urtwn_bb_write(sc, R92C_FPGA0_RFMOD, reg); if (usb_err != USB_ERR_NORMAL_COMPLETION) goto fail; reg = urtwn_bb_read(sc, R92C_FPGA0_RFMOD); reg |= R92C_RFMOD_OFDM_EN; usb_err = urtwn_bb_write(sc, R92C_FPGA0_RFMOD, reg); if (usb_err != USB_ERR_NORMAL_COMPLETION) goto fail; /* Clear per-station keys table. */ urtwn_cam_init(sc); /* Enable hardware sequence numbering. */ urtwn_write_1(sc, R92C_HWSEQ_CTRL, 0xff); /* Perform LO and IQ calibrations. */ urtwn_iq_calib(sc); /* Perform LC calibration. */ urtwn_lc_calib(sc); /* Fix USB interference issue. */ if (!(sc->chip & URTWN_CHIP_88E)) { urtwn_write_1(sc, 0xfe40, 0xe0); urtwn_write_1(sc, 0xfe41, 0x8d); urtwn_write_1(sc, 0xfe42, 0x80); urtwn_pa_bias_init(sc); } /* Initialize GPIO setting. */ urtwn_write_1(sc, R92C_GPIO_MUXCFG, urtwn_read_1(sc, R92C_GPIO_MUXCFG) & ~R92C_GPIO_MUXCFG_ENBT); /* Fix for lower temperature. */ if (!(sc->chip & URTWN_CHIP_88E)) urtwn_write_1(sc, 0x15, 0xe9); usbd_transfer_start(sc->sc_xfer[URTWN_BULK_RX]); sc->sc_flags |= URTWN_RUNNING; callout_reset(&sc->sc_watchdog_ch, hz, urtwn_watchdog, sc); fail: if (usb_err != USB_ERR_NORMAL_COMPLETION) error = EIO; URTWN_UNLOCK(sc); return (error); } static void urtwn_stop(struct urtwn_softc *sc) { URTWN_LOCK(sc); if (!(sc->sc_flags & URTWN_RUNNING)) { URTWN_UNLOCK(sc); return; } sc->sc_flags &= ~URTWN_RUNNING; callout_stop(&sc->sc_watchdog_ch); urtwn_abort_xfers(sc); urtwn_drain_mbufq(sc); URTWN_UNLOCK(sc); } static void urtwn_abort_xfers(struct urtwn_softc *sc) { int i; URTWN_ASSERT_LOCKED(sc); /* abort any pending transfers */ for (i = 0; i < URTWN_N_TRANSFER; i++) usbd_transfer_stop(sc->sc_xfer[i]); } static int urtwn_raw_xmit(struct ieee80211_node *ni, struct mbuf *m, const struct ieee80211_bpf_params *params) { struct ieee80211com *ic = ni->ni_ic; struct urtwn_softc *sc = ic->ic_softc; struct urtwn_data *bf; int error; /* prevent management frames from being sent if we're not ready */ URTWN_LOCK(sc); if (!(sc->sc_flags & URTWN_RUNNING)) { error = ENETDOWN; goto end; } bf = urtwn_getbuf(sc); if (bf == NULL) { error = ENOBUFS; goto end; } if ((error = urtwn_tx_data(sc, ni, m, bf)) != 0) { STAILQ_INSERT_HEAD(&sc->sc_tx_inactive, bf, next); goto end; } sc->sc_txtimer = 5; callout_reset(&sc->sc_watchdog_ch, hz, urtwn_watchdog, sc); end: if (error != 0) m_freem(m); URTWN_UNLOCK(sc); return (error); } static void urtwn_ms_delay(struct urtwn_softc *sc) { usb_pause_mtx(&sc->sc_mtx, hz / 1000); } static device_method_t urtwn_methods[] = { /* Device interface */ DEVMETHOD(device_probe, urtwn_match), DEVMETHOD(device_attach, urtwn_attach), DEVMETHOD(device_detach, urtwn_detach), DEVMETHOD_END }; static driver_t urtwn_driver = { "urtwn", urtwn_methods, sizeof(struct urtwn_softc) }; static devclass_t urtwn_devclass; DRIVER_MODULE(urtwn, uhub, urtwn_driver, urtwn_devclass, NULL, NULL); MODULE_DEPEND(urtwn, usb, 1, 1, 1); MODULE_DEPEND(urtwn, wlan, 1, 1, 1); MODULE_DEPEND(urtwn, firmware, 1, 1, 1); MODULE_VERSION(urtwn, 1); Index: user/ngie/more-tests2/sys/dev/usb/wlan/if_urtwnreg.h =================================================================== --- user/ngie/more-tests2/sys/dev/usb/wlan/if_urtwnreg.h (revision 292053) +++ user/ngie/more-tests2/sys/dev/usb/wlan/if_urtwnreg.h (revision 292054) @@ -1,2030 +1,2037 @@ /*- * Copyright (c) 2010 Damien Bergamini * * Permission to use, copy, modify, and distribute this software for any * purpose with or without fee is hereby granted, provided that the above * copyright notice and this permission notice appear in all copies. * * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. * * $OpenBSD: if_urtwnreg.h,v 1.3 2010/11/16 18:02:59 damien Exp $ * $FreeBSD$ */ #define URTWN_CONFIG_INDEX 0 #define URTWN_IFACE_INDEX 0 #define URTWN_NOISE_FLOOR -95 #define R92C_MAX_CHAINS 2 /* Maximum number of output pipes is 3. */ #define R92C_MAX_EPOUT 3 #define R92C_MAX_TX_PWR 0x3f #define R92C_PUBQ_NPAGES 231 #define R92C_TXPKTBUF_COUNT 256 #define R92C_TX_PAGE_COUNT 248 #define R92C_TX_PAGE_BOUNDARY (R92C_TX_PAGE_COUNT + 1) #define R88E_TXPKTBUF_COUNT 177 #define R88E_TX_PAGE_COUNT 169 #define R88E_TX_PAGE_BOUNDARY (R88E_TX_PAGE_COUNT + 1) #define R92C_H2C_NBOX 4 /* USB Requests. */ #define R92C_REQ_REGS 0x05 /* * MAC registers. */ /* System Configuration. */ #define R92C_SYS_ISO_CTRL 0x000 #define R92C_SYS_FUNC_EN 0x002 #define R92C_APS_FSMCO 0x004 #define R92C_SYS_CLKR 0x008 #define R92C_AFE_MISC 0x010 #define R92C_SPS0_CTRL 0x011 #define R92C_SPS_OCP_CFG 0x018 #define R92C_RSV_CTRL 0x01c #define R92C_RF_CTRL 0x01f #define R92C_LDOA15_CTRL 0x020 #define R92C_LDOV12D_CTRL 0x021 #define R92C_LDOHCI12_CTRL 0x022 #define R92C_LPLDO_CTRL 0x023 #define R92C_AFE_XTAL_CTRL 0x024 #define R92C_AFE_PLL_CTRL 0x028 #define R92C_EFUSE_CTRL 0x030 #define R92C_EFUSE_TEST 0x034 #define R92C_PWR_DATA 0x038 #define R92C_CAL_TIMER 0x03c #define R92C_ACLK_MON 0x03e #define R92C_GPIO_MUXCFG 0x040 #define R92C_GPIO_IO_SEL 0x042 #define R92C_MAC_PINMUX_CFG 0x043 #define R92C_GPIO_PIN_CTRL 0x044 #define R92C_GPIO_INTM 0x048 #define R92C_LEDCFG0 0x04c #define R92C_LEDCFG1 0x04d #define R92C_LEDCFG2 0x04e #define R92C_LEDCFG3 0x04f #define R92C_FSIMR 0x050 #define R92C_FSISR 0x054 #define R92C_HSIMR 0x058 #define R92C_HSISR 0x05c #define R92C_MCUFWDL 0x080 #define R92C_HMEBOX_EXT(idx) (0x088 + (idx) * 2) #define R88E_HIMR 0x0b0 #define R88E_HISR 0x0b4 #define R88E_HIMRE 0x0b8 #define R88E_HISRE 0x0bc #define R92C_EFUSE_ACCESS 0x0cf #define R92C_BIST_SCAN 0x0d0 #define R92C_BIST_RPT 0x0d4 #define R92C_BIST_ROM_RPT 0x0d8 #define R92C_USB_SIE_INTF 0x0e0 #define R92C_PCIE_MIO_INTF 0x0e4 #define R92C_PCIE_MIO_INTD 0x0e8 #define R92C_HPON_FSM 0x0ec #define R92C_SYS_CFG 0x0f0 /* MAC General Configuration. */ #define R92C_CR 0x100 #define R92C_MSR 0x102 #define R92C_PBP 0x104 #define R92C_TRXDMA_CTRL 0x10c #define R92C_TRXFF_BNDY 0x114 #define R92C_TRXFF_STATUS 0x118 #define R92C_RXFF_PTR 0x11c #define R92C_HIMR 0x120 #define R92C_HISR 0x124 #define R92C_HIMRE 0x128 #define R92C_HISRE 0x12c #define R92C_CPWM 0x12f #define R92C_FWIMR 0x130 #define R92C_FWISR 0x134 #define R92C_PKTBUF_DBG_CTRL 0x140 #define R92C_PKTBUF_DBG_DATA_L 0x144 #define R92C_PKTBUF_DBG_DATA_H 0x148 #define R92C_TC0_CTRL(i) (0x150 + (i) * 4) #define R92C_TCUNIT_BASE 0x164 #define R92C_MBIST_START 0x174 #define R92C_MBIST_DONE 0x178 #define R92C_MBIST_FAIL 0x17c #define R92C_C2HEVT_MSG_NORMAL 0x1a0 #define R92C_C2HEVT_MSG_TEST 0x1b8 #define R92C_C2HEVT_CLEAR 0x1bf #define R92C_MCUTST_1 0x1c0 #define R92C_FMETHR 0x1c8 #define R92C_HMETFR 0x1cc #define R92C_HMEBOX(idx) (0x1d0 + (idx) * 4) #define R92C_LLT_INIT 0x1e0 #define R92C_BB_ACCESS_CTRL 0x1e8 #define R92C_BB_ACCESS_DATA 0x1ec #define R88E_HMEBOX_EXT(idx) (0x1f0 + (idx) * 4) /* Tx DMA Configuration. */ #define R92C_RQPN 0x200 #define R92C_FIFOPAGE 0x204 #define R92C_TDECTRL 0x208 #define R92C_TXDMA_OFFSET_CHK 0x20c #define R92C_TXDMA_STATUS 0x210 #define R92C_RQPN_NPQ 0x214 /* Rx DMA Configuration. */ #define R92C_RXDMA_AGG_PG_TH 0x280 #define R92C_RXPKT_NUM 0x284 #define R92C_RXDMA_STATUS 0x288 /* Protocol Configuration. */ #define R92C_FWHW_TXQ_CTRL 0x420 #define R92C_HWSEQ_CTRL 0x423 #define R92C_TXPKTBUF_BCNQ_BDNY 0x424 #define R92C_TXPKTBUF_MGQ_BDNY 0x425 #define R92C_SPEC_SIFS 0x428 #define R92C_RL 0x42a #define R92C_DARFRC 0x430 #define R92C_RARFRC 0x438 #define R92C_RRSR 0x440 #define R92C_ARFR(i) (0x444 + (i) * 4) #define R92C_AGGLEN_LMT 0x458 #define R92C_AMPDU_MIN_SPACE 0x45c #define R92C_TXPKTBUF_WMAC_LBK_BF_HD 0x45d #define R92C_FAST_EDCA_CTRL 0x460 #define R92C_RD_RESP_PKT_TH 0x463 #define R92C_INIRTS_RATE_SEL 0x480 #define R92C_INIDATA_RATE_SEL(macid) (0x484 + (macid)) #define R92C_MAX_AGGR_NUM 0x4ca /* EDCA Configuration. */ #define R92C_EDCA_VO_PARAM 0x500 #define R92C_EDCA_VI_PARAM 0x504 #define R92C_EDCA_BE_PARAM 0x508 #define R92C_EDCA_BK_PARAM 0x50c #define R92C_BCNTCFG 0x510 #define R92C_PIFS 0x512 #define R92C_RDG_PIFS 0x513 #define R92C_SIFS_CCK 0x514 #define R92C_SIFS_OFDM 0x516 #define R92C_AGGR_BREAK_TIME 0x51a #define R92C_SLOT 0x51b #define R92C_TX_PTCL_CTRL 0x520 #define R92C_TXPAUSE 0x522 #define R92C_DIS_TXREQ_CLR 0x523 #define R92C_RD_CTRL 0x524 #define R92C_TBTT_PROHIBIT 0x540 #define R92C_RD_NAV_NXT 0x544 #define R92C_NAV_PROT_LEN 0x546 #define R92C_BCN_CTRL 0x550 #define R92C_MBID_NUM 0x552 #define R92C_DUAL_TSF_RST 0x553 #define R92C_BCN_INTERVAL 0x554 #define R92C_DRVERLYINT 0x558 #define R92C_BCNDMATIM 0x559 #define R92C_ATIMWND 0x55a #define R92C_USTIME_TSF 0x55c #define R92C_BCN_MAX_ERR 0x55d #define R92C_RXTSF_OFFSET_CCK 0x55e #define R92C_RXTSF_OFFSET_OFDM 0x55f #define R92C_TSFTR 0x560 #define R92C_INIT_TSFTR 0x564 #define R92C_PSTIMER 0x580 #define R92C_TIMER0 0x584 #define R92C_TIMER1 0x588 #define R92C_ACMHWCTRL 0x5c0 #define R92C_ACMRSTCTRL 0x5c1 #define R92C_ACMAVG 0x5c2 #define R92C_VO_ADMTIME 0x5c4 #define R92C_VI_ADMTIME 0x5c6 #define R92C_BE_ADMTIME 0x5c8 #define R92C_EDCA_RANDOM_GEN 0x5cc #define R92C_SCH_TXCMD 0x5d0 /* WMAC Configuration. */ #define R92C_APSD_CTRL 0x600 #define R92C_BWOPMODE 0x603 #define R92C_RCR 0x608 #define R92C_RX_DRVINFO_SZ 0x60f #define R92C_MACID 0x610 #define R92C_BSSID 0x618 #define R92C_MAR 0x620 #define R92C_MAC_SPEC_SIFS 0x63a #define R92C_R2T_SIFS 0x63c #define R92C_T2T_SIFS 0x63e #define R92C_ACKTO 0x640 #define R92C_CAMCMD 0x670 #define R92C_CAMWRITE 0x674 #define R92C_CAMREAD 0x678 #define R92C_CAMDBG 0x67c #define R92C_SECCFG 0x680 #define R92C_RXFLTMAP0 0x6a0 #define R92C_RXFLTMAP1 0x6a2 #define R92C_RXFLTMAP2 0x6a4 /* Bits for R92C_SYS_ISO_CTRL. */ #define R92C_SYS_ISO_CTRL_MD2PP 0x0001 #define R92C_SYS_ISO_CTRL_UA2USB 0x0002 #define R92C_SYS_ISO_CTRL_UD2CORE 0x0004 #define R92C_SYS_ISO_CTRL_PA2PCIE 0x0008 #define R92C_SYS_ISO_CTRL_PD2CORE 0x0010 #define R92C_SYS_ISO_CTRL_IP2MAC 0x0020 #define R92C_SYS_ISO_CTRL_DIOP 0x0040 #define R92C_SYS_ISO_CTRL_DIOE 0x0080 #define R92C_SYS_ISO_CTRL_EB2CORE 0x0100 #define R92C_SYS_ISO_CTRL_DIOR 0x0200 #define R92C_SYS_ISO_CTRL_PWC_EV25V 0x4000 #define R92C_SYS_ISO_CTRL_PWC_EV12V 0x8000 /* Bits for R92C_SYS_FUNC_EN. */ #define R92C_SYS_FUNC_EN_BBRSTB 0x0001 #define R92C_SYS_FUNC_EN_BB_GLB_RST 0x0002 #define R92C_SYS_FUNC_EN_USBA 0x0004 #define R92C_SYS_FUNC_EN_UPLL 0x0008 #define R92C_SYS_FUNC_EN_USBD 0x0010 #define R92C_SYS_FUNC_EN_DIO_PCIE 0x0020 #define R92C_SYS_FUNC_EN_PCIEA 0x0040 #define R92C_SYS_FUNC_EN_PPLL 0x0080 #define R92C_SYS_FUNC_EN_PCIED 0x0100 #define R92C_SYS_FUNC_EN_DIOE 0x0200 #define R92C_SYS_FUNC_EN_CPUEN 0x0400 #define R92C_SYS_FUNC_EN_DCORE 0x0800 #define R92C_SYS_FUNC_EN_ELDR 0x1000 #define R92C_SYS_FUNC_EN_DIO_RF 0x2000 #define R92C_SYS_FUNC_EN_HWPDN 0x4000 #define R92C_SYS_FUNC_EN_MREGEN 0x8000 /* Bits for R92C_APS_FSMCO. */ #define R92C_APS_FSMCO_PFM_LDALL 0x00000001 #define R92C_APS_FSMCO_PFM_ALDN 0x00000002 #define R92C_APS_FSMCO_PFM_LDKP 0x00000004 #define R92C_APS_FSMCO_PFM_WOWL 0x00000008 #define R92C_APS_FSMCO_PDN_EN 0x00000010 #define R92C_APS_FSMCO_PDN_PL 0x00000020 #define R92C_APS_FSMCO_APFM_ONMAC 0x00000100 #define R92C_APS_FSMCO_APFM_OFF 0x00000200 #define R92C_APS_FSMCO_APFM_RSM 0x00000400 #define R92C_APS_FSMCO_AFSM_HSUS 0x00000800 #define R92C_APS_FSMCO_AFSM_PCIE 0x00001000 #define R92C_APS_FSMCO_APDM_MAC 0x00002000 #define R92C_APS_FSMCO_APDM_HOST 0x00004000 #define R92C_APS_FSMCO_APDM_HPDN 0x00008000 #define R92C_APS_FSMCO_RDY_MACON 0x00010000 #define R92C_APS_FSMCO_SUS_HOST 0x00020000 #define R92C_APS_FSMCO_ROP_ALD 0x00100000 #define R92C_APS_FSMCO_ROP_PWR 0x00200000 #define R92C_APS_FSMCO_ROP_SPS 0x00400000 #define R92C_APS_FSMCO_SOP_MRST 0x02000000 #define R92C_APS_FSMCO_SOP_FUSE 0x04000000 #define R92C_APS_FSMCO_SOP_ABG 0x08000000 #define R92C_APS_FSMCO_SOP_AMB 0x10000000 #define R92C_APS_FSMCO_SOP_RCK 0x20000000 #define R92C_APS_FSMCO_SOP_A8M 0x40000000 #define R92C_APS_FSMCO_XOP_BTCK 0x80000000 /* Bits for R92C_SYS_CLKR. */ #define R92C_SYS_CLKR_ANAD16V_EN 0x00000001 #define R92C_SYS_CLKR_ANA8M 0x00000002 #define R92C_SYS_CLKR_MACSLP 0x00000010 #define R92C_SYS_CLKR_LOADER_EN 0x00000020 #define R92C_SYS_CLKR_80M_SSC_DIS 0x00000080 #define R92C_SYS_CLKR_80M_SSC_EN_HO 0x00000100 #define R92C_SYS_CLKR_PHY_SSC_RSTB 0x00000200 #define R92C_SYS_CLKR_SEC_EN 0x00000400 #define R92C_SYS_CLKR_MAC_EN 0x00000800 #define R92C_SYS_CLKR_SYS_EN 0x00001000 #define R92C_SYS_CLKR_RING_EN 0x00002000 /* Bits for R92C_RF_CTRL. */ #define R92C_RF_CTRL_EN 0x01 #define R92C_RF_CTRL_RSTB 0x02 #define R92C_RF_CTRL_SDMRSTB 0x04 /* Bits for R92C_LDOV12D_CTRL. */ #define R92C_LDOV12D_CTRL_LDV12_EN 0x01 /* Bits for R92C_AFE_XTAL_CTRL. */ #define R92C_AFE_XTAL_CTRL_ADDR_M 0x007ff800 #define R92C_AFE_XTAL_CTRL_ADDR_S 11 /* Bits for R92C_EFUSE_CTRL. */ #define R92C_EFUSE_CTRL_DATA_M 0x000000ff #define R92C_EFUSE_CTRL_DATA_S 0 #define R92C_EFUSE_CTRL_ADDR_M 0x0003ff00 #define R92C_EFUSE_CTRL_ADDR_S 8 #define R92C_EFUSE_CTRL_VALID 0x80000000 /* Bits for R92C_GPIO_MUXCFG. */ #define R92C_GPIO_MUXCFG_ENBT 0x0020 /* Bits for R92C_LEDCFG0. */ #define R92C_LEDCFG0_DIS 0x08 /* Bits for R92C_MCUFWDL. */ #define R92C_MCUFWDL_EN 0x00000001 #define R92C_MCUFWDL_RDY 0x00000002 #define R92C_MCUFWDL_CHKSUM_RPT 0x00000004 #define R92C_MCUFWDL_MACINI_RDY 0x00000008 #define R92C_MCUFWDL_BBINI_RDY 0x00000010 #define R92C_MCUFWDL_RFINI_RDY 0x00000020 #define R92C_MCUFWDL_WINTINI_RDY 0x00000040 #define R92C_MCUFWDL_RAM_DL_SEL 0x00000080 #define R92C_MCUFWDL_PAGE_M 0x00070000 #define R92C_MCUFWDL_PAGE_S 16 #define R92C_MCUFWDL_CPRST 0x00800000 /* Bits for R88E_HIMR. */ #define R88E_HIMR_CPWM 0x00000100 #define R88E_HIMR_CPWM2 0x00000200 #define R88E_HIMR_TBDER 0x04000000 #define R88E_HIMR_PSTIMEOUT 0x20000000 /* Bits for R88E_HIMRE.*/ #define R88E_HIMRE_RXFOVW 0x00000100 #define R88E_HIMRE_TXFOVW 0x00000200 #define R88E_HIMRE_RXERR 0x00000400 #define R88E_HIMRE_TXERR 0x00000800 /* Bits for R92C_EFUSE_ACCESS. */ #define R92C_EFUSE_ACCESS_OFF 0x00 #define R92C_EFUSE_ACCESS_ON 0x69 /* Bits for R92C_HPON_FSM. */ #define R92C_HPON_FSM_CHIP_BONDING_ID_S 22 #define R92C_HPON_FSM_CHIP_BONDING_ID_M 0x00c00000 #define R92C_HPON_FSM_CHIP_BONDING_ID_92C_1T2R 1 /* Bits for R92C_SYS_CFG. */ #define R92C_SYS_CFG_XCLK_VLD 0x00000001 #define R92C_SYS_CFG_ACLK_VLD 0x00000002 #define R92C_SYS_CFG_UCLK_VLD 0x00000004 #define R92C_SYS_CFG_PCLK_VLD 0x00000008 #define R92C_SYS_CFG_PCIRSTB 0x00000010 #define R92C_SYS_CFG_V15_VLD 0x00000020 #define R92C_SYS_CFG_TRP_B15V_EN 0x00000080 #define R92C_SYS_CFG_SIC_IDLE 0x00000100 #define R92C_SYS_CFG_BD_MAC2 0x00000200 #define R92C_SYS_CFG_BD_MAC1 0x00000400 #define R92C_SYS_CFG_IC_MACPHY_MODE 0x00000800 #define R92C_SYS_CFG_CHIP_VER_RTL_M 0x0000f000 #define R92C_SYS_CFG_CHIP_VER_RTL_S 12 #define R92C_SYS_CFG_BT_FUNC 0x00010000 #define R92C_SYS_CFG_VENDOR_UMC 0x00080000 #define R92C_SYS_CFG_PAD_HWPD_IDN 0x00400000 #define R92C_SYS_CFG_TRP_VAUX_EN 0x00800000 #define R92C_SYS_CFG_TRP_BT_EN 0x01000000 #define R92C_SYS_CFG_BD_PKG_SEL 0x02000000 #define R92C_SYS_CFG_BD_HCI_SEL 0x04000000 #define R92C_SYS_CFG_TYPE_92C 0x08000000 /* Bits for R92C_CR. */ #define R92C_CR_HCI_TXDMA_EN 0x0001 #define R92C_CR_HCI_RXDMA_EN 0x0002 #define R92C_CR_TXDMA_EN 0x0004 #define R92C_CR_RXDMA_EN 0x0008 #define R92C_CR_PROTOCOL_EN 0x0010 #define R92C_CR_SCHEDULE_EN 0x0020 #define R92C_CR_MACTXEN 0x0040 #define R92C_CR_MACRXEN 0x0080 #define R92C_CR_ENSEC 0x0200 #define R92C_CR_CALTMR_EN 0x0400 /* Bits for R92C_MSR. */ #define R92C_MSR_NOLINK 0x00 #define R92C_MSR_ADHOC 0x01 #define R92C_MSR_INFRA 0x02 #define R92C_MSR_AP 0x03 #define R92C_MSR_MASK (R92C_MSR_AP) /* Bits for R92C_PBP. */ #define R92C_PBP_PSRX_M 0x0f #define R92C_PBP_PSRX_S 0 #define R92C_PBP_PSTX_M 0xf0 #define R92C_PBP_PSTX_S 4 #define R92C_PBP_64 0 #define R92C_PBP_128 1 #define R92C_PBP_256 2 #define R92C_PBP_512 3 #define R92C_PBP_1024 4 /* Bits for R92C_TRXDMA_CTRL. */ #define R92C_TRXDMA_CTRL_RXDMA_AGG_EN 0x0004 #define R92C_TRXDMA_CTRL_TXDMA_VOQ_MAP_M 0x0030 #define R92C_TRXDMA_CTRL_TXDMA_VOQ_MAP_S 4 #define R92C_TRXDMA_CTRL_TXDMA_VIQ_MAP_M 0x00c0 #define R92C_TRXDMA_CTRL_TXDMA_VIQ_MAP_S 6 #define R92C_TRXDMA_CTRL_TXDMA_BEQ_MAP_M 0x0300 #define R92C_TRXDMA_CTRL_TXDMA_BEQ_MAP_S 8 #define R92C_TRXDMA_CTRL_TXDMA_BKQ_MAP_M 0x0c00 #define R92C_TRXDMA_CTRL_TXDMA_BKQ_MAP_S 10 #define R92C_TRXDMA_CTRL_TXDMA_MGQ_MAP_M 0x3000 #define R92C_TRXDMA_CTRL_TXDMA_MGQ_MAP_S 12 #define R92C_TRXDMA_CTRL_TXDMA_HIQ_MAP_M 0xc000 #define R92C_TRXDMA_CTRL_TXDMA_HIQ_MAP_S 14 #define R92C_TRXDMA_CTRL_QUEUE_LOW 1 #define R92C_TRXDMA_CTRL_QUEUE_NORMAL 2 #define R92C_TRXDMA_CTRL_QUEUE_HIGH 3 #define R92C_TRXDMA_CTRL_QMAP_M 0xfff0 /* Shortcuts. */ #define R92C_TRXDMA_CTRL_QMAP_3EP 0xf5b0 #define R92C_TRXDMA_CTRL_QMAP_HQ_LQ 0xf5f0 #define R92C_TRXDMA_CTRL_QMAP_HQ_NQ 0xfaf0 #define R92C_TRXDMA_CTRL_QMAP_LQ 0x5550 #define R92C_TRXDMA_CTRL_QMAP_NQ 0xaaa0 #define R92C_TRXDMA_CTRL_QMAP_HQ 0xfff0 /* Bits for R92C_LLT_INIT. */ #define R92C_LLT_INIT_DATA_M 0x000000ff #define R92C_LLT_INIT_DATA_S 0 #define R92C_LLT_INIT_ADDR_M 0x0000ff00 #define R92C_LLT_INIT_ADDR_S 8 #define R92C_LLT_INIT_OP_M 0xc0000000 #define R92C_LLT_INIT_OP_S 30 #define R92C_LLT_INIT_OP_NO_ACTIVE 0 #define R92C_LLT_INIT_OP_WRITE 1 /* Bits for R92C_RQPN. */ #define R92C_RQPN_HPQ_M 0x000000ff #define R92C_RQPN_HPQ_S 0 #define R92C_RQPN_LPQ_M 0x0000ff00 #define R92C_RQPN_LPQ_S 8 #define R92C_RQPN_PUBQ_M 0x00ff0000 #define R92C_RQPN_PUBQ_S 16 #define R92C_RQPN_LD 0x80000000 /* Bits for R92C_TDECTRL. */ #define R92C_TDECTRL_BLK_DESC_NUM_M 0x000000f0 #define R92C_TDECTRL_BLK_DESC_NUM_S 4 /* Bits for R92C_FWHW_TXQ_CTRL. */ #define R92C_FWHW_TXQ_CTRL_AMPDU_RTY_NEW 0x80 /* Bits for R92C_SPEC_SIFS. */ #define R92C_SPEC_SIFS_CCK_M 0x00ff #define R92C_SPEC_SIFS_CCK_S 0 #define R92C_SPEC_SIFS_OFDM_M 0xff00 #define R92C_SPEC_SIFS_OFDM_S 8 /* Bits for R92C_RL. */ #define R92C_RL_LRL_M 0x003f #define R92C_RL_LRL_S 0 #define R92C_RL_SRL_M 0x3f00 #define R92C_RL_SRL_S 8 /* Bits for R92C_RRSR. */ #define R92C_RRSR_RATE_BITMAP_M 0x000fffff #define R92C_RRSR_RATE_BITMAP_S 0 #define R92C_RRSR_RATE_CCK_ONLY_1M 0xffff1 #define R92C_RRSR_RSC_LOWSUBCHNL 0x00200000 #define R92C_RRSR_RSC_UPSUBCHNL 0x00400000 #define R92C_RRSR_SHORT 0x00800000 /* Bits for R92C_EDCA_XX_PARAM. */ #define R92C_EDCA_PARAM_AIFS_M 0x000000ff #define R92C_EDCA_PARAM_AIFS_S 0 #define R92C_EDCA_PARAM_ECWMIN_M 0x00000f00 #define R92C_EDCA_PARAM_ECWMIN_S 8 #define R92C_EDCA_PARAM_ECWMAX_M 0x0000f000 #define R92C_EDCA_PARAM_ECWMAX_S 12 #define R92C_EDCA_PARAM_TXOP_M 0xffff0000 #define R92C_EDCA_PARAM_TXOP_S 16 /* Bits for R92C_BCN_CTRL. */ #define R92C_BCN_CTRL_EN_MBSSID 0x02 #define R92C_BCN_CTRL_TXBCN_RPT 0x04 #define R92C_BCN_CTRL_EN_BCN 0x08 #define R92C_BCN_CTRL_DIS_TSF_UDT0 0x10 /* Bits for R92C_MBID_NUM. */ #define R92C_MBID_TXBCN_RPT0 0x08 #define R92C_MBID_TXBCN_RPT1 0x10 /* Bits for R92C_DUAL_TSF_RST. */ #define R92C_DUAL_TSF_RST0 0x01 #define R92C_DUAL_TSF_RST1 0x02 +/* Bits for R92C_ACMHWCTRL. */ +#define R92C_ACMHWCTRL_EN 0x01 +#define R92C_ACMHWCTRL_BE 0x02 +#define R92C_ACMHWCTRL_VI 0x04 +#define R92C_ACMHWCTRL_VO 0x08 +#define R92C_ACMHWCTRL_ACM_MASK 0x0f + /* Bits for R92C_APSD_CTRL. */ #define R92C_APSD_CTRL_OFF 0x40 #define R92C_APSD_CTRL_OFF_STATUS 0x80 /* Bits for R92C_BWOPMODE. */ #define R92C_BWOPMODE_11J 0x01 #define R92C_BWOPMODE_5G 0x02 #define R92C_BWOPMODE_20MHZ 0x04 /* Bits for R92C_RCR. */ #define R92C_RCR_AAP 0x00000001 #define R92C_RCR_APM 0x00000002 #define R92C_RCR_AM 0x00000004 #define R92C_RCR_AB 0x00000008 #define R92C_RCR_ADD3 0x00000010 #define R92C_RCR_APWRMGT 0x00000020 #define R92C_RCR_CBSSID_DATA 0x00000040 #define R92C_RCR_CBSSID_BCN 0x00000080 #define R92C_RCR_ACRC32 0x00000100 #define R92C_RCR_AICV 0x00000200 #define R92C_RCR_ADF 0x00000800 #define R92C_RCR_ACF 0x00001000 #define R92C_RCR_AMF 0x00002000 #define R92C_RCR_HTC_LOC_CTRL 0x00004000 #define R92C_RCR_MFBEN 0x00400000 #define R92C_RCR_LSIGEN 0x00800000 #define R92C_RCR_ENMBID 0x01000000 #define R92C_RCR_APP_BA_SSN 0x08000000 #define R92C_RCR_APP_PHYSTS 0x10000000 #define R92C_RCR_APP_ICV 0x20000000 #define R92C_RCR_APP_MIC 0x40000000 #define R92C_RCR_APPFCS 0x80000000 /* Bits for R92C_CAMCMD. */ #define R92C_CAMCMD_ADDR_M 0x0000ffff #define R92C_CAMCMD_ADDR_S 0 #define R92C_CAMCMD_WRITE 0x00010000 #define R92C_CAMCMD_CLR 0x40000000 #define R92C_CAMCMD_POLLING 0x80000000 /* Bits for R92C_RXFLTMAP*. */ #define R92C_RXFLTMAP_SUBTYPE(subtype) \ (1 << ((subtype) >> IEEE80211_FC0_SUBTYPE_SHIFT)) /* * Baseband registers. */ #define R92C_FPGA0_RFMOD 0x800 #define R92C_FPGA0_TXINFO 0x804 #define R92C_HSSI_PARAM1(chain) (0x820 + (chain) * 8) #define R92C_HSSI_PARAM2(chain) (0x824 + (chain) * 8) #define R92C_TXAGC_RATE18_06(i) (((i) == 0) ? 0xe00 : 0x830) #define R92C_TXAGC_RATE54_24(i) (((i) == 0) ? 0xe04 : 0x834) #define R92C_TXAGC_A_CCK1_MCS32 0xe08 #define R92C_TXAGC_B_CCK1_55_MCS32 0x838 #define R92C_TXAGC_B_CCK11_A_CCK2_11 0x86c #define R92C_TXAGC_MCS03_MCS00(i) (((i) == 0) ? 0xe10 : 0x83c) #define R92C_TXAGC_MCS07_MCS04(i) (((i) == 0) ? 0xe14 : 0x848) #define R92C_TXAGC_MCS11_MCS08(i) (((i) == 0) ? 0xe18 : 0x84c) #define R92C_TXAGC_MCS15_MCS12(i) (((i) == 0) ? 0xe1c : 0x868) #define R92C_LSSI_PARAM(chain) (0x840 + (chain) * 4) #define R92C_FPGA0_RFIFACEOE(chain) (0x860 + (chain) * 4) #define R92C_FPGA0_RFIFACESW(idx) (0x870 + (idx) * 4) #define R92C_FPGA0_RFPARAM(idx) (0x878 + (idx) * 4) #define R92C_FPGA0_ANAPARAM2 0x884 #define R92C_LSSI_READBACK(chain) (0x8a0 + (chain) * 4) #define R92C_HSPI_READBACK(chain) (0x8b8 + (chain) * 4) #define R92C_FPGA1_RFMOD 0x900 #define R92C_FPGA1_TXINFO 0x90c #define R92C_CCK0_SYSTEM 0xa00 #define R92C_CCK0_AFESETTING 0xa04 #define R92C_OFDM0_TRXPATHENA 0xc04 #define R92C_OFDM0_TRMUXPAR 0xc08 #define R92C_OFDM0_AGCCORE1(chain) (0xc50 + (chain) * 8) #define R92C_OFDM0_AGCPARAM1 0xc70 #define R92C_OFDM0_AGCRSSITABLE 0xc78 #define R92C_OFDM1_LSTF 0xd00 /* Bits for R92C_FPGA[01]_RFMOD. */ #define R92C_RFMOD_40MHZ 0x00000001 #define R92C_RFMOD_JAPAN 0x00000002 #define R92C_RFMOD_CCK_TXSC 0x00000030 #define R92C_RFMOD_CCK_EN 0x01000000 #define R92C_RFMOD_OFDM_EN 0x02000000 /* Bits for R92C_HSSI_PARAM1(i). */ #define R92C_HSSI_PARAM1_PI 0x00000100 /* Bits for R92C_HSSI_PARAM2(i). */ #define R92C_HSSI_PARAM2_CCK_HIPWR 0x00000200 #define R92C_HSSI_PARAM2_ADDR_LENGTH 0x00000400 #define R92C_HSSI_PARAM2_DATA_LENGTH 0x00000800 #define R92C_HSSI_PARAM2_READ_ADDR_M 0x7f800000 #define R92C_HSSI_PARAM2_READ_ADDR_S 23 #define R92C_HSSI_PARAM2_READ_EDGE 0x80000000 /* Bits for R92C_TXAGC_A_CCK1_MCS32. */ #define R92C_TXAGC_A_CCK1_M 0x0000ff00 #define R92C_TXAGC_A_CCK1_S 8 /* Bits for R92C_TXAGC_B_CCK11_A_CCK2_11. */ #define R92C_TXAGC_B_CCK11_M 0x000000ff #define R92C_TXAGC_B_CCK11_S 0 #define R92C_TXAGC_A_CCK2_M 0x0000ff00 #define R92C_TXAGC_A_CCK2_S 8 #define R92C_TXAGC_A_CCK55_M 0x00ff0000 #define R92C_TXAGC_A_CCK55_S 16 #define R92C_TXAGC_A_CCK11_M 0xff000000 #define R92C_TXAGC_A_CCK11_S 24 /* Bits for R92C_TXAGC_B_CCK1_55_MCS32. */ #define R92C_TXAGC_B_CCK1_M 0x0000ff00 #define R92C_TXAGC_B_CCK1_S 8 #define R92C_TXAGC_B_CCK2_M 0x00ff0000 #define R92C_TXAGC_B_CCK2_S 16 #define R92C_TXAGC_B_CCK55_M 0xff000000 #define R92C_TXAGC_B_CCK55_S 24 /* Bits for R92C_TXAGC_RATE18_06(x). */ #define R92C_TXAGC_RATE06_M 0x000000ff #define R92C_TXAGC_RATE06_S 0 #define R92C_TXAGC_RATE09_M 0x0000ff00 #define R92C_TXAGC_RATE09_S 8 #define R92C_TXAGC_RATE12_M 0x00ff0000 #define R92C_TXAGC_RATE12_S 16 #define R92C_TXAGC_RATE18_M 0xff000000 #define R92C_TXAGC_RATE18_S 24 /* Bits for R92C_TXAGC_RATE54_24(x). */ #define R92C_TXAGC_RATE24_M 0x000000ff #define R92C_TXAGC_RATE24_S 0 #define R92C_TXAGC_RATE36_M 0x0000ff00 #define R92C_TXAGC_RATE36_S 8 #define R92C_TXAGC_RATE48_M 0x00ff0000 #define R92C_TXAGC_RATE48_S 16 #define R92C_TXAGC_RATE54_M 0xff000000 #define R92C_TXAGC_RATE54_S 24 /* Bits for R92C_TXAGC_MCS03_MCS00(x). */ #define R92C_TXAGC_MCS00_M 0x000000ff #define R92C_TXAGC_MCS00_S 0 #define R92C_TXAGC_MCS01_M 0x0000ff00 #define R92C_TXAGC_MCS01_S 8 #define R92C_TXAGC_MCS02_M 0x00ff0000 #define R92C_TXAGC_MCS02_S 16 #define R92C_TXAGC_MCS03_M 0xff000000 #define R92C_TXAGC_MCS03_S 24 /* Bits for R92C_TXAGC_MCS07_MCS04(x). */ #define R92C_TXAGC_MCS04_M 0x000000ff #define R92C_TXAGC_MCS04_S 0 #define R92C_TXAGC_MCS05_M 0x0000ff00 #define R92C_TXAGC_MCS05_S 8 #define R92C_TXAGC_MCS06_M 0x00ff0000 #define R92C_TXAGC_MCS06_S 16 #define R92C_TXAGC_MCS07_M 0xff000000 #define R92C_TXAGC_MCS07_S 24 /* Bits for R92C_TXAGC_MCS11_MCS08(x). */ #define R92C_TXAGC_MCS08_M 0x000000ff #define R92C_TXAGC_MCS08_S 0 #define R92C_TXAGC_MCS09_M 0x0000ff00 #define R92C_TXAGC_MCS09_S 8 #define R92C_TXAGC_MCS10_M 0x00ff0000 #define R92C_TXAGC_MCS10_S 16 #define R92C_TXAGC_MCS11_M 0xff000000 #define R92C_TXAGC_MCS11_S 24 /* Bits for R92C_TXAGC_MCS15_MCS12(x). */ #define R92C_TXAGC_MCS12_M 0x000000ff #define R92C_TXAGC_MCS12_S 0 #define R92C_TXAGC_MCS13_M 0x0000ff00 #define R92C_TXAGC_MCS13_S 8 #define R92C_TXAGC_MCS14_M 0x00ff0000 #define R92C_TXAGC_MCS14_S 16 #define R92C_TXAGC_MCS15_M 0xff000000 #define R92C_TXAGC_MCS15_S 24 /* Bits for R92C_LSSI_PARAM(i). */ #define R92C_LSSI_PARAM_DATA_M 0x000fffff #define R92C_LSSI_PARAM_DATA_S 0 #define R92C_LSSI_PARAM_ADDR_M 0x03f00000 #define R92C_LSSI_PARAM_ADDR_S 20 #define R88E_LSSI_PARAM_ADDR_M 0x0ff00000 #define R88E_LSSI_PARAM_ADDR_S 20 /* Bits for R92C_FPGA0_ANAPARAM2. */ #define R92C_FPGA0_ANAPARAM2_CBW20 0x00000400 /* Bits for R92C_LSSI_READBACK(i). */ #define R92C_LSSI_READBACK_DATA_M 0x000fffff #define R92C_LSSI_READBACK_DATA_S 0 /* Bits for R92C_OFDM0_AGCCORE1(i). */ #define R92C_OFDM0_AGCCORE1_GAIN_M 0x0000007f #define R92C_OFDM0_AGCCORE1_GAIN_S 0 /* * USB registers. */ #define R92C_USB_INFO 0xfe17 #define R92C_USB_SPECIAL_OPTION 0xfe55 #define R92C_USB_HCPWM 0xfe57 #define R92C_USB_HRPWM 0xfe58 #define R92C_USB_DMA_AGG_TO 0xfe5b #define R92C_USB_AGG_TO 0xfe5c #define R92C_USB_AGG_TH 0xfe5d #define R92C_USB_VID 0xfe60 #define R92C_USB_PID 0xfe62 #define R92C_USB_OPTIONAL 0xfe64 #define R92C_USB_EP 0xfe65 #define R92C_USB_PHY 0xfe68 #define R92C_USB_MAC_ADDR 0xfe70 #define R92C_USB_STRING 0xfe80 /* Bits for R92C_USB_SPECIAL_OPTION. */ #define R92C_USB_SPECIAL_OPTION_AGG_EN 0x08 #define R92C_USB_SPECIAL_OPTION_INT_BULK_SEL 0x10 /* Bits for R92C_USB_EP. */ #define R92C_USB_EP_HQ_M 0x000f #define R92C_USB_EP_HQ_S 0 #define R92C_USB_EP_NQ_M 0x00f0 #define R92C_USB_EP_NQ_S 4 #define R92C_USB_EP_LQ_M 0x0f00 #define R92C_USB_EP_LQ_S 8 /* * Firmware base address. */ #define R92C_FW_START_ADDR 0x1000 #define R92C_FW_PAGE_SIZE 4096 /* * RF (6052) registers. */ #define R92C_RF_AC 0x00 #define R92C_RF_IQADJ_G(i) (0x01 + (i)) #define R92C_RF_POW_TRSW 0x05 #define R92C_RF_GAIN_RX 0x06 #define R92C_RF_GAIN_TX 0x07 #define R92C_RF_TXM_IDAC 0x08 #define R92C_RF_BS_IQGEN 0x0f #define R92C_RF_MODE1 0x10 #define R92C_RF_MODE2 0x11 #define R92C_RF_RX_AGC_HP 0x12 #define R92C_RF_TX_AGC 0x13 #define R92C_RF_BIAS 0x14 #define R92C_RF_IPA 0x15 #define R92C_RF_POW_ABILITY 0x17 #define R92C_RF_CHNLBW 0x18 #define R92C_RF_RX_G1 0x1a #define R92C_RF_RX_G2 0x1b #define R92C_RF_RX_BB2 0x1c #define R92C_RF_RX_BB1 0x1d #define R92C_RF_RCK1 0x1e #define R92C_RF_RCK2 0x1f #define R92C_RF_TX_G(i) (0x20 + (i)) #define R92C_RF_TX_BB1 0x23 #define R92C_RF_T_METER 0x24 #define R92C_RF_SYN_G(i) (0x25 + (i)) #define R92C_RF_RCK_OS 0x30 #define R92C_RF_TXPA_G(i) (0x31 + (i)) /* Bits for R92C_RF_AC. */ #define R92C_RF_AC_MODE_M 0x70000 #define R92C_RF_AC_MODE_S 16 #define R92C_RF_AC_MODE_STANDBY 1 /* Bits for R92C_RF_CHNLBW. */ #define R92C_RF_CHNLBW_CHNL_M 0x003ff #define R92C_RF_CHNLBW_CHNL_S 0 #define R92C_RF_CHNLBW_BW20 0x00400 #define R88E_RF_CHNLBW_BW20 0x00c00 #define R92C_RF_CHNLBW_LCSTART 0x08000 /* * CAM entries. */ #define R92C_CAM_ENTRY_COUNT 32 #define R92C_CAM_CTL0(entry) ((entry) * 8 + 0) #define R92C_CAM_CTL1(entry) ((entry) * 8 + 1) #define R92C_CAM_KEY(entry, i) ((entry) * 8 + 2 + (i)) /* Bits for R92C_CAM_CTL0(i). */ #define R92C_CAM_KEYID_M 0x00000003 #define R92C_CAM_KEYID_S 0 #define R92C_CAM_ALGO_M 0x0000001c #define R92C_CAM_ALGO_S 2 #define R92C_CAM_ALGO_NONE 0 #define R92C_CAM_ALGO_WEP40 1 #define R92C_CAM_ALGO_TKIP 2 #define R92C_CAM_ALGO_AES 4 #define R92C_CAM_ALGO_WEP104 5 #define R92C_CAM_VALID 0x00008000 #define R92C_CAM_MACLO_M 0xffff0000 #define R92C_CAM_MACLO_S 16 /* Rate adaptation modes. */ #define R92C_RAID_11GN 1 #define R92C_RAID_11N 3 #define R92C_RAID_11BG 4 #define R92C_RAID_11G 5 /* "pure" 11g */ #define R92C_RAID_11B 6 /* * Macros to access subfields in registers. */ /* Mask and Shift (getter). */ #define MS(val, field) \ (((val) & field##_M) >> field##_S) /* Shift and Mask (setter). */ #define SM(field, val) \ (((val) << field##_S) & field##_M) /* Rewrite. */ #define RW(var, field, val) \ (((var) & ~field##_M) | SM(field, val)) /* * Firmware image header. */ struct r92c_fw_hdr { /* QWORD0 */ uint16_t signature; uint8_t category; uint8_t function; uint16_t version; uint16_t subversion; /* QWORD1 */ uint8_t month; uint8_t date; uint8_t hour; uint8_t minute; uint16_t ramcodesize; uint16_t reserved2; /* QWORD2 */ uint32_t svnidx; uint32_t reserved3; /* QWORD3 */ uint32_t reserved4; uint32_t reserved5; } __packed; /* * Host to firmware commands. */ struct r92c_fw_cmd { uint8_t id; #define R92C_CMD_AP_OFFLOAD 0 #define R92C_CMD_SET_PWRMODE 1 #define R92C_CMD_JOINBSS_RPT 2 #define R92C_CMD_RSVD_PAGE 3 #define R92C_CMD_RSSI 4 #define R92C_CMD_RSSI_SETTING 5 #define R92C_CMD_MACID_CONFIG 6 #define R92C_CMD_MACID_PS_MODE 7 #define R92C_CMD_P2P_PS_OFFLOAD 8 #define R92C_CMD_SELECTIVE_SUSPEND 9 #define R92C_CMD_FLAG_EXT 0x80 uint8_t msg[5]; } __packed; /* Structure for R92C_CMD_RSSI_SETTING. */ struct r92c_fw_cmd_rssi { uint8_t macid; uint8_t reserved; uint8_t pwdb; } __packed; /* Structure for R92C_CMD_MACID_CONFIG. */ struct r92c_fw_cmd_macid_cfg { uint32_t mask; uint8_t macid; #define URTWN_MACID_BSS 0 #define URTWN_MACID_BC 4 /* Broadcast. */ #define URTWN_MACID_VALID 0x80 } __packed; /* * RTL8192CU ROM image. */ struct r92c_rom { uint16_t id; /* 0x8192 */ uint8_t reserved1[5]; uint8_t dbg_sel; uint16_t reserved2; uint16_t vid; uint16_t pid; uint8_t usb_opt; uint8_t ep_setting; uint16_t reserved3; uint8_t usb_phy; uint8_t reserved4[3]; uint8_t macaddr[6]; uint8_t string[61]; /* "Realtek" */ uint8_t subcustomer_id; uint8_t cck_tx_pwr[R92C_MAX_CHAINS][3]; uint8_t ht40_1s_tx_pwr[R92C_MAX_CHAINS][3]; uint8_t ht40_2s_tx_pwr_diff[3]; uint8_t ht20_tx_pwr_diff[3]; uint8_t ofdm_tx_pwr_diff[3]; uint8_t ht40_max_pwr[3]; uint8_t ht20_max_pwr[3]; uint8_t xtal_calib; uint8_t tssi[R92C_MAX_CHAINS]; uint8_t thermal_meter; uint8_t rf_opt1; #define R92C_ROM_RF1_REGULATORY_M 0x07 #define R92C_ROM_RF1_REGULATORY_S 0 #define R92C_ROM_RF1_BOARD_TYPE_M 0xe0 #define R92C_ROM_RF1_BOARD_TYPE_S 5 #define R92C_BOARD_TYPE_DONGLE 0 #define R92C_BOARD_TYPE_HIGHPA 1 #define R92C_BOARD_TYPE_MINICARD 2 #define R92C_BOARD_TYPE_SOLO 3 #define R92C_BOARD_TYPE_COMBO 4 uint8_t rf_opt2; uint8_t rf_opt3; uint8_t rf_opt4; uint8_t channel_plan; uint8_t version; uint8_t curstomer_id; } __packed; #define URTWN_EFUSE_MAX_LEN 512 /* Rx MAC descriptor. */ struct r92c_rx_stat { uint32_t rxdw0; #define R92C_RXDW0_PKTLEN_M 0x00003fff #define R92C_RXDW0_PKTLEN_S 0 #define R92C_RXDW0_CRCERR 0x00004000 #define R92C_RXDW0_ICVERR 0x00008000 #define R92C_RXDW0_INFOSZ_M 0x000f0000 #define R92C_RXDW0_INFOSZ_S 16 #define R92C_RXDW0_QOS 0x00800000 #define R92C_RXDW0_SHIFT_M 0x03000000 #define R92C_RXDW0_SHIFT_S 24 #define R92C_RXDW0_PHYST 0x04000000 #define R92C_RXDW0_DECRYPTED 0x08000000 uint32_t rxdw1; uint32_t rxdw2; #define R92C_RXDW2_PKTCNT_M 0x00ff0000 #define R92C_RXDW2_PKTCNT_S 16 uint32_t rxdw3; #define R92C_RXDW3_RATE_M 0x0000003f #define R92C_RXDW3_RATE_S 0 #define R92C_RXDW3_HT 0x00000040 #define R92C_RXDW3_HTC 0x00000400 uint32_t rxdw4; uint32_t rxdw5; } __packed __attribute__((aligned(4))); /* Rx PHY descriptor. */ struct r92c_rx_phystat { uint32_t phydw0; uint32_t phydw1; uint32_t phydw2; uint32_t phydw3; uint32_t phydw4; uint32_t phydw5; uint32_t phydw6; uint32_t phydw7; } __packed __attribute__((aligned(4))); /* Rx PHY CCK descriptor. */ struct r92c_rx_cck { uint8_t adc_pwdb[4]; uint8_t sq_rpt; uint8_t agc_rpt; } __packed; struct r88e_rx_cck { uint8_t path_agc[2]; uint8_t chan; uint8_t reserved1; uint8_t sig_qual; uint8_t agc_rpt; uint8_t rpt_b; uint8_t reserved2; uint8_t noise_power; uint8_t path_cfotail[2]; uint8_t pcts_mask[2]; uint8_t stream_rxevm[2]; uint8_t path_rxsnr[2]; uint8_t noise_power_db_lsb; uint8_t reserved3[3]; uint8_t stream_csi[2]; uint8_t stream_target_csi[2]; uint8_t sig_evm; } __packed; /* Tx MAC descriptor. */ struct r92c_tx_desc { uint32_t txdw0; #define R92C_TXDW0_PKTLEN_M 0x0000ffff #define R92C_TXDW0_PKTLEN_S 0 #define R92C_TXDW0_OFFSET_M 0x00ff0000 #define R92C_TXDW0_OFFSET_S 16 #define R92C_TXDW0_BMCAST 0x01000000 #define R92C_TXDW0_LSG 0x04000000 #define R92C_TXDW0_FSG 0x08000000 #define R92C_TXDW0_OWN 0x80000000 uint32_t txdw1; #define R92C_TXDW1_MACID_M 0x0000001f #define R92C_TXDW1_MACID_S 0 #define R88E_TXDW1_MACID_M 0x0000003f #define R88E_TXDW1_MACID_S 0 #define R92C_TXDW1_AGGEN 0x00000020 #define R92C_TXDW1_AGGBK 0x00000040 #define R92C_TXDW1_QSEL_M 0x00001f00 #define R92C_TXDW1_QSEL_S 8 #define R92C_TXDW1_QSEL_BE 0x00 /* or 0x03 */ #define R92C_TXDW1_QSEL_BK 0x01 /* or 0x02 */ #define R92C_TXDW1_QSEL_VI 0x04 /* or 0x05 */ #define R92C_TXDW1_QSEL_VO 0x06 /* or 0x07 */ #define URTWN_MAX_TID 8 #define R92C_TXDW1_QSEL_BEACON 0x10 #define R92C_TXDW1_QSEL_MGNT 0x12 #define R92C_TXDW1_RAID_M 0x000f0000 #define R92C_TXDW1_RAID_S 16 #define R92C_TXDW1_CIPHER_M 0x00c00000 #define R92C_TXDW1_CIPHER_S 22 #define R92C_TXDW1_CIPHER_NONE 0 #define R92C_TXDW1_CIPHER_RC4 1 #define R92C_TXDW1_CIPHER_AES 3 #define R92C_TXDW1_PKTOFF_M 0x7c000000 #define R92C_TXDW1_PKTOFF_S 26 uint32_t txdw2; #define R88E_TXDW2_AGGBK 0x00010000 uint16_t txdw3; uint16_t txdseq; #define R88E_TXDSEQ_HWSEQ_EN 0x8000 uint32_t txdw4; #define R92C_TXDW4_RTSRATE_M 0x0000003f #define R92C_TXDW4_RTSRATE_S 0 #define R92C_TXDW4_HWSEQ_QOS 0x00000040 #define R92C_TXDW4_HWSEQ_EN 0x00000080 #define R92C_TXDW4_DRVRATE 0x00000100 #define R92C_TXDW4_CTS2SELF 0x00000800 #define R92C_TXDW4_RTSEN 0x00001000 #define R92C_TXDW4_HWRTSEN 0x00002000 #define R92C_TXDW4_SCO_M 0x003f0000 #define R92C_TXDW4_SCO_S 20 #define R92C_TXDW4_SCO_SCA 1 #define R92C_TXDW4_SCO_SCB 2 #define R92C_TXDW4_40MHZ 0x02000000 uint32_t txdw5; #define R92C_TXDW5_DATARATE_M 0x0000003f #define R92C_TXDW5_DATARATE_S 0 #define R92C_TXDW5_SGI 0x00000040 #define R92C_TXDW5_AGGNUM_M 0xff000000 #define R92C_TXDW5_AGGNUM_S 24 uint32_t txdw6; uint16_t txdsum; uint16_t pad; } __packed __attribute__((aligned(4))); static const uint8_t ridx2rate[] = { 2, 4, 11, 22, 12, 18, 24, 36, 48, 72, 96, 108 }; /* HW rate indices. */ #define URTWN_RIDX_CCK1 0 #define URTWN_RIDX_CCK11 3 #define URTWN_RIDX_OFDM6 4 #define URTWN_RIDX_OFDM24 8 #define URTWN_RIDX_OFDM54 11 #define URTWN_RIDX_COUNT 28 /* * MAC initialization values. */ static const struct { uint16_t reg; uint8_t val; } rtl8188eu_mac[] = { { 0x026, 0x41 }, { 0x027, 0x35 }, { 0x040, 0x00 }, { 0x428, 0x0a }, { 0x429, 0x10 }, { 0x430, 0x00 }, { 0x431, 0x01 }, { 0x432, 0x02 }, { 0x433, 0x04 }, { 0x434, 0x05 }, { 0x435, 0x06 }, { 0x436, 0x07 }, { 0x437, 0x08 }, { 0x438, 0x00 }, { 0x439, 0x00 }, { 0x43a, 0x01 }, { 0x43b, 0x02 }, { 0x43c, 0x04 }, { 0x43d, 0x05 }, { 0x43e, 0x06 }, { 0x43f, 0x07 }, { 0x440, 0x5d }, { 0x441, 0x01 }, { 0x442, 0x00 }, { 0x444, 0x15 }, { 0x445, 0xf0 }, { 0x446, 0x0f }, { 0x447, 0x00 }, { 0x458, 0x41 }, { 0x459, 0xa8 }, { 0x45a, 0x72 }, { 0x45b, 0xb9 }, { 0x460, 0x66 }, { 0x461, 0x66 }, { 0x480, 0x08 }, { 0x4c8, 0xff }, { 0x4c9, 0x08 }, { 0x4cc, 0xff }, { 0x4cd, 0xff }, { 0x4ce, 0x01 }, { 0x4d3, 0x01 }, { 0x500, 0x26 }, { 0x501, 0xa2 }, { 0x502, 0x2f }, { 0x503, 0x00 }, { 0x504, 0x28 }, { 0x505, 0xa3 }, { 0x506, 0x5e }, { 0x507, 0x00 }, { 0x508, 0x2b }, { 0x509, 0xa4 }, { 0x50a, 0x5e }, { 0x50b, 0x00 }, { 0x50c, 0x4f }, { 0x50d, 0xa4 }, { 0x50e, 0x00 }, { 0x50f, 0x00 }, { 0x512, 0x1c }, { 0x514, 0x0a }, { 0x516, 0x0a }, { 0x525, 0x4f }, { 0x550, 0x10 }, { 0x551, 0x10 }, { 0x559, 0x02 }, { 0x55d, 0xff }, { 0x605, 0x30 }, { 0x608, 0x0e }, { 0x609, 0x2a }, { 0x620, 0xff }, { 0x621, 0xff }, { 0x622, 0xff }, { 0x623, 0xff }, { 0x624, 0xff }, { 0x625, 0xff }, { 0x626, 0xff }, { 0x627, 0xff }, { 0x652, 0x20 }, { 0x63c, 0x0a }, { 0x63d, 0x0a }, { 0x63e, 0x0e }, { 0x63f, 0x0e }, { 0x640, 0x40 }, { 0x66e, 0x05 }, { 0x700, 0x21 }, { 0x701, 0x43 }, { 0x702, 0x65 }, { 0x703, 0x87 }, { 0x708, 0x21 }, { 0x709, 0x43 }, { 0x70a, 0x65 }, { 0x70b, 0x87 } }, rtl8192cu_mac[] = { { 0x420, 0x80 }, { 0x423, 0x00 }, { 0x430, 0x00 }, { 0x431, 0x00 }, { 0x432, 0x00 }, { 0x433, 0x01 }, { 0x434, 0x04 }, { 0x435, 0x05 }, { 0x436, 0x06 }, { 0x437, 0x07 }, { 0x438, 0x00 }, { 0x439, 0x00 }, { 0x43a, 0x00 }, { 0x43b, 0x01 }, { 0x43c, 0x04 }, { 0x43d, 0x05 }, { 0x43e, 0x06 }, { 0x43f, 0x07 }, { 0x440, 0x5d }, { 0x441, 0x01 }, { 0x442, 0x00 }, { 0x444, 0x15 }, { 0x445, 0xf0 }, { 0x446, 0x0f }, { 0x447, 0x00 }, { 0x458, 0x41 }, { 0x459, 0xa8 }, { 0x45a, 0x72 }, { 0x45b, 0xb9 }, { 0x460, 0x66 }, { 0x461, 0x66 }, { 0x462, 0x08 }, { 0x463, 0x03 }, { 0x4c8, 0xff }, { 0x4c9, 0x08 }, { 0x4cc, 0xff }, { 0x4cd, 0xff }, { 0x4ce, 0x01 }, { 0x500, 0x26 }, { 0x501, 0xa2 }, { 0x502, 0x2f }, { 0x503, 0x00 }, { 0x504, 0x28 }, { 0x505, 0xa3 }, { 0x506, 0x5e }, { 0x507, 0x00 }, { 0x508, 0x2b }, { 0x509, 0xa4 }, { 0x50a, 0x5e }, { 0x50b, 0x00 }, { 0x50c, 0x4f }, { 0x50d, 0xa4 }, { 0x50e, 0x00 }, { 0x50f, 0x00 }, { 0x512, 0x1c }, { 0x514, 0x0a }, { 0x515, 0x10 }, { 0x516, 0x0a }, { 0x517, 0x10 }, { 0x51a, 0x16 }, { 0x524, 0x0f }, { 0x525, 0x4f }, { 0x546, 0x40 }, { 0x547, 0x00 }, { 0x550, 0x10 }, { 0x551, 0x10 }, { 0x559, 0x02 }, { 0x55a, 0x02 }, { 0x55d, 0xff }, { 0x605, 0x30 }, { 0x608, 0x0e }, { 0x609, 0x2a }, { 0x652, 0x20 }, { 0x63c, 0x0a }, { 0x63d, 0x0e }, { 0x63e, 0x0a }, { 0x63f, 0x0e }, { 0x66e, 0x05 }, { 0x700, 0x21 }, { 0x701, 0x43 }, { 0x702, 0x65 }, { 0x703, 0x87 }, { 0x708, 0x21 }, { 0x709, 0x43 }, { 0x70a, 0x65 }, { 0x70b, 0x87 } }; /* * Baseband initialization values. */ struct urtwn_bb_prog { int count; const uint16_t *regs; const uint32_t *vals; int agccount; const uint32_t *agcvals; }; /* * RTL8192CU and RTL8192CE-VAU. */ static const uint16_t rtl8192ce_bb_regs[] = { 0x024, 0x028, 0x800, 0x804, 0x808, 0x80c, 0x810, 0x814, 0x818, 0x81c, 0x820, 0x824, 0x828, 0x82c, 0x830, 0x834, 0x838, 0x83c, 0x840, 0x844, 0x848, 0x84c, 0x850, 0x854, 0x858, 0x85c, 0x860, 0x864, 0x868, 0x86c, 0x870, 0x874, 0x878, 0x87c, 0x880, 0x884, 0x888, 0x88c, 0x890, 0x894, 0x898, 0x89c, 0x900, 0x904, 0x908, 0x90c, 0xa00, 0xa04, 0xa08, 0xa0c, 0xa10, 0xa14, 0xa18, 0xa1c, 0xa20, 0xa24, 0xa28, 0xa2c, 0xa70, 0xa74, 0xc00, 0xc04, 0xc08, 0xc0c, 0xc10, 0xc14, 0xc18, 0xc1c, 0xc20, 0xc24, 0xc28, 0xc2c, 0xc30, 0xc34, 0xc38, 0xc3c, 0xc40, 0xc44, 0xc48, 0xc4c, 0xc50, 0xc54, 0xc58, 0xc5c, 0xc60, 0xc64, 0xc68, 0xc6c, 0xc70, 0xc74, 0xc78, 0xc7c, 0xc80, 0xc84, 0xc88, 0xc8c, 0xc90, 0xc94, 0xc98, 0xc9c, 0xca0, 0xca4, 0xca8, 0xcac, 0xcb0, 0xcb4, 0xcb8, 0xcbc, 0xcc0, 0xcc4, 0xcc8, 0xccc, 0xcd0, 0xcd4, 0xcd8, 0xcdc, 0xce0, 0xce4, 0xce8, 0xcec, 0xd00, 0xd04, 0xd08, 0xd0c, 0xd10, 0xd14, 0xd18, 0xd2c, 0xd30, 0xd34, 0xd38, 0xd3c, 0xd40, 0xd44, 0xd48, 0xd4c, 0xd50, 0xd54, 0xd58, 0xd5c, 0xd60, 0xd64, 0xd68, 0xd6c, 0xd70, 0xd74, 0xd78, 0xe00, 0xe04, 0xe08, 0xe10, 0xe14, 0xe18, 0xe1c, 0xe28, 0xe30, 0xe34, 0xe38, 0xe3c, 0xe40, 0xe44, 0xe48, 0xe4c, 0xe50, 0xe54, 0xe58, 0xe5c, 0xe60, 0xe68, 0xe6c, 0xe70, 0xe74, 0xe78, 0xe7c, 0xe80, 0xe84, 0xe88, 0xe8c, 0xed0, 0xed4, 0xed8, 0xedc, 0xee0, 0xeec, 0xf14, 0xf4c, 0xf00 }; static const uint32_t rtl8192ce_bb_vals[] = { 0x0011800d, 0x00ffdb83, 0x80040002, 0x00000003, 0x0000fc00, 0x0000000a, 0x10005388, 0x020c3d10, 0x02200385, 0x00000000, 0x01000100, 0x00390004, 0x01000100, 0x00390004, 0x27272727, 0x27272727, 0x27272727, 0x27272727, 0x00010000, 0x00010000, 0x27272727, 0x27272727, 0x00000000, 0x00000000, 0x569a569a, 0x0c1b25a4, 0x66e60230, 0x061f0130, 0x27272727, 0x2b2b2b27, 0x07000700, 0x22184000, 0x08080808, 0x00000000, 0xc0083070, 0x000004d5, 0x00000000, 0xcc0000c0, 0x00000800, 0xfffffffe, 0x40302010, 0x00706050, 0x00000000, 0x00000023, 0x00000000, 0x81121313, 0x00d047c8, 0x80ff000c, 0x8c838300, 0x2e68120f, 0x9500bb78, 0x11144028, 0x00881117, 0x89140f00, 0x1a1b0000, 0x090e1317, 0x00000204, 0x00d30000, 0x101fbf00, 0x00000007, 0x48071d40, 0x03a05633, 0x000000e4, 0x6c6c6c6c, 0x08800000, 0x40000100, 0x08800000, 0x40000100, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x69e9ac44, 0x469652cf, 0x49795994, 0x0a97971c, 0x1f7c403f, 0x000100b7, 0xec020107, 0x007f037f, 0x6954341e, 0x43bc0094, 0x6954341e, 0x433c0094, 0x00000000, 0x5116848b, 0x47c00bff, 0x00000036, 0x2c7f000d, 0x018610db, 0x0000001f, 0x00b91612, 0x40000100, 0x20f60000, 0x40000100, 0x20200000, 0x00121820, 0x00000000, 0x00121820, 0x00007f7f, 0x00000000, 0x00000080, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x28000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x64b22427, 0x00766932, 0x00222222, 0x00000000, 0x37644302, 0x2f97d40c, 0x00080740, 0x00020403, 0x0000907f, 0x20010201, 0xa0633333, 0x3333bc43, 0x7a8f5b6b, 0xcc979975, 0x00000000, 0x80608000, 0x00000000, 0x00027293, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x6437140a, 0x00000000, 0x00000000, 0x30032064, 0x4653de68, 0x04518a3c, 0x00002101, 0x2a201c16, 0x1812362e, 0x322c2220, 0x000e3c24, 0x2a2a2a2a, 0x2a2a2a2a, 0x03902a2a, 0x2a2a2a2a, 0x2a2a2a2a, 0x2a2a2a2a, 0x2a2a2a2a, 0x00000000, 0x1000dc1f, 0x10008c1f, 0x02140102, 0x681604c2, 0x01007c00, 0x01004800, 0xfb000000, 0x000028d1, 0x1000dc1f, 0x10008c1f, 0x02140102, 0x28160d05, 0x00000010, 0x001b25a4, 0x63db25a4, 0x63db25a4, 0x0c1b25a4, 0x0c1b25a4, 0x0c1b25a4, 0x0c1b25a4, 0x63db25a4, 0x0c1b25a4, 0x63db25a4, 0x63db25a4, 0x63db25a4, 0x63db25a4, 0x001b25a4, 0x001b25a4, 0x6fdb25a4, 0x00000003, 0x00000000, 0x00000300 }; static const uint32_t rtl8192ce_agc_vals[] = { 0x7b000001, 0x7b010001, 0x7b020001, 0x7b030001, 0x7b040001, 0x7b050001, 0x7a060001, 0x79070001, 0x78080001, 0x77090001, 0x760a0001, 0x750b0001, 0x740c0001, 0x730d0001, 0x720e0001, 0x710f0001, 0x70100001, 0x6f110001, 0x6e120001, 0x6d130001, 0x6c140001, 0x6b150001, 0x6a160001, 0x69170001, 0x68180001, 0x67190001, 0x661a0001, 0x651b0001, 0x641c0001, 0x631d0001, 0x621e0001, 0x611f0001, 0x60200001, 0x49210001, 0x48220001, 0x47230001, 0x46240001, 0x45250001, 0x44260001, 0x43270001, 0x42280001, 0x41290001, 0x402a0001, 0x262b0001, 0x252c0001, 0x242d0001, 0x232e0001, 0x222f0001, 0x21300001, 0x20310001, 0x06320001, 0x05330001, 0x04340001, 0x03350001, 0x02360001, 0x01370001, 0x00380001, 0x00390001, 0x003a0001, 0x003b0001, 0x003c0001, 0x003d0001, 0x003e0001, 0x003f0001, 0x7b400001, 0x7b410001, 0x7b420001, 0x7b430001, 0x7b440001, 0x7b450001, 0x7a460001, 0x79470001, 0x78480001, 0x77490001, 0x764a0001, 0x754b0001, 0x744c0001, 0x734d0001, 0x724e0001, 0x714f0001, 0x70500001, 0x6f510001, 0x6e520001, 0x6d530001, 0x6c540001, 0x6b550001, 0x6a560001, 0x69570001, 0x68580001, 0x67590001, 0x665a0001, 0x655b0001, 0x645c0001, 0x635d0001, 0x625e0001, 0x615f0001, 0x60600001, 0x49610001, 0x48620001, 0x47630001, 0x46640001, 0x45650001, 0x44660001, 0x43670001, 0x42680001, 0x41690001, 0x406a0001, 0x266b0001, 0x256c0001, 0x246d0001, 0x236e0001, 0x226f0001, 0x21700001, 0x20710001, 0x06720001, 0x05730001, 0x04740001, 0x03750001, 0x02760001, 0x01770001, 0x00780001, 0x00790001, 0x007a0001, 0x007b0001, 0x007c0001, 0x007d0001, 0x007e0001, 0x007f0001, 0x3800001e, 0x3801001e, 0x3802001e, 0x3803001e, 0x3804001e, 0x3805001e, 0x3806001e, 0x3807001e, 0x3808001e, 0x3c09001e, 0x3e0a001e, 0x400b001e, 0x440c001e, 0x480d001e, 0x4c0e001e, 0x500f001e, 0x5210001e, 0x5611001e, 0x5a12001e, 0x5e13001e, 0x6014001e, 0x6015001e, 0x6016001e, 0x6217001e, 0x6218001e, 0x6219001e, 0x621a001e, 0x621b001e, 0x621c001e, 0x621d001e, 0x621e001e, 0x621f001e }; static const struct urtwn_bb_prog rtl8192ce_bb_prog = { nitems(rtl8192ce_bb_regs), rtl8192ce_bb_regs, rtl8192ce_bb_vals, nitems(rtl8192ce_agc_vals), rtl8192ce_agc_vals }; /* * RTL8188CU. */ static const uint32_t rtl8192cu_bb_vals[] = { 0x0011800d, 0x00ffdb83, 0x80040002, 0x00000003, 0x0000fc00, 0x0000000a, 0x10005388, 0x020c3d10, 0x02200385, 0x00000000, 0x01000100, 0x00390004, 0x01000100, 0x00390004, 0x27272727, 0x27272727, 0x27272727, 0x27272727, 0x00010000, 0x00010000, 0x27272727, 0x27272727, 0x00000000, 0x00000000, 0x569a569a, 0x0c1b25a4, 0x66e60230, 0x061f0130, 0x27272727, 0x2b2b2b27, 0x07000700, 0x22184000, 0x08080808, 0x00000000, 0xc0083070, 0x000004d5, 0x00000000, 0xcc0000c0, 0x00000800, 0xfffffffe, 0x40302010, 0x00706050, 0x00000000, 0x00000023, 0x00000000, 0x81121313, 0x00d047c8, 0x80ff000c, 0x8c838300, 0x2e68120f, 0x9500bb78, 0x11144028, 0x00881117, 0x89140f00, 0x1a1b0000, 0x090e1317, 0x00000204, 0x00d30000, 0x101fbf00, 0x00000007, 0x48071d40, 0x03a05633, 0x000000e4, 0x6c6c6c6c, 0x08800000, 0x40000100, 0x08800000, 0x40000100, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x69e9ac44, 0x469652cf, 0x49795994, 0x0a97971c, 0x1f7c403f, 0x000100b7, 0xec020107, 0x007f037f, 0x6954341e, 0x43bc0094, 0x6954341e, 0x433c0094, 0x00000000, 0x5116848b, 0x47c00bff, 0x00000036, 0x2c7f000d, 0x0186115b, 0x0000001f, 0x00b99612, 0x40000100, 0x20f60000, 0x40000100, 0x20200000, 0x00121820, 0x00000000, 0x00121820, 0x00007f7f, 0x00000000, 0x00000080, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x28000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x64b22427, 0x00766932, 0x00222222, 0x00000000, 0x37644302, 0x2f97d40c, 0x00080740, 0x00020403, 0x0000907f, 0x20010201, 0xa0633333, 0x3333bc43, 0x7a8f5b6b, 0xcc979975, 0x00000000, 0x80608000, 0x00000000, 0x00027293, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x6437140a, 0x00000000, 0x00000000, 0x30032064, 0x4653de68, 0x04518a3c, 0x00002101, 0x2a201c16, 0x1812362e, 0x322c2220, 0x000e3c24, 0x2a2a2a2a, 0x2a2a2a2a, 0x03902a2a, 0x2a2a2a2a, 0x2a2a2a2a, 0x2a2a2a2a, 0x2a2a2a2a, 0x00000000, 0x1000dc1f, 0x10008c1f, 0x02140102, 0x681604c2, 0x01007c00, 0x01004800, 0xfb000000, 0x000028d1, 0x1000dc1f, 0x10008c1f, 0x02140102, 0x28160d05, 0x00000010, 0x001b25a4, 0x63db25a4, 0x63db25a4, 0x0c1b25a4, 0x0c1b25a4, 0x0c1b25a4, 0x0c1b25a4, 0x63db25a4, 0x0c1b25a4, 0x63db25a4, 0x63db25a4, 0x63db25a4, 0x63db25a4, 0x001b25a4, 0x001b25a4, 0x6fdb25a4, 0x00000003, 0x00000000, 0x00000300 }; static const struct urtwn_bb_prog rtl8192cu_bb_prog = { nitems(rtl8192ce_bb_regs), rtl8192ce_bb_regs, rtl8192cu_bb_vals, nitems(rtl8192ce_agc_vals), rtl8192ce_agc_vals }; /* * RTL8188CE-VAU. */ static const uint32_t rtl8188ce_bb_vals[] = { 0x0011800d, 0x00ffdb83, 0x80040000, 0x00000001, 0x0000fc00, 0x0000000a, 0x10005388, 0x020c3d10, 0x02200385, 0x00000000, 0x01000100, 0x00390004, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00010000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x569a569a, 0x001b25a4, 0x66e60230, 0x061f0130, 0x00000000, 0x32323200, 0x07000700, 0x22004000, 0x00000808, 0x00000000, 0xc0083070, 0x000004d5, 0x00000000, 0xccc000c0, 0x00000800, 0xfffffffe, 0x40302010, 0x00706050, 0x00000000, 0x00000023, 0x00000000, 0x81121111, 0x00d047c8, 0x80ff000c, 0x8c838300, 0x2e68120f, 0x9500bb78, 0x11144028, 0x00881117, 0x89140f00, 0x1a1b0000, 0x090e1317, 0x00000204, 0x00d30000, 0x101fbf00, 0x00000007, 0x48071d40, 0x03a05611, 0x000000e4, 0x6c6c6c6c, 0x08800000, 0x40000100, 0x08800000, 0x40000100, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x69e9ac44, 0x469652cf, 0x49795994, 0x0a97971c, 0x1f7c403f, 0x000100b7, 0xec020107, 0x007f037f, 0x6954341e, 0x43bc0094, 0x6954341e, 0x433c0094, 0x00000000, 0x5116848b, 0x47c00bff, 0x00000036, 0x2c7f000d, 0x018610db, 0x0000001f, 0x00b91612, 0x40000100, 0x20f60000, 0x40000100, 0x20200000, 0x00121820, 0x00000000, 0x00121820, 0x00007f7f, 0x00000000, 0x00000080, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x28000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x64b22427, 0x00766932, 0x00222222, 0x00000000, 0x37644302, 0x2f97d40c, 0x00080740, 0x00020401, 0x0000907f, 0x20010201, 0xa0633333, 0x3333bc43, 0x7a8f5b6b, 0xcc979975, 0x00000000, 0x80608000, 0x00000000, 0x00027293, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x6437140a, 0x00000000, 0x00000000, 0x30032064, 0x4653de68, 0x04518a3c, 0x00002101, 0x2a201c16, 0x1812362e, 0x322c2220, 0x000e3c24, 0x2a2a2a2a, 0x2a2a2a2a, 0x03902a2a, 0x2a2a2a2a, 0x2a2a2a2a, 0x2a2a2a2a, 0x2a2a2a2a, 0x00000000, 0x1000dc1f, 0x10008c1f, 0x02140102, 0x681604c2, 0x01007c00, 0x01004800, 0xfb000000, 0x000028d1, 0x1000dc1f, 0x10008c1f, 0x02140102, 0x28160d05, 0x00000008, 0x001b25a4, 0x631b25a0, 0x631b25a0, 0x081b25a0, 0x081b25a0, 0x081b25a0, 0x081b25a0, 0x631b25a0, 0x081b25a0, 0x631b25a0, 0x631b25a0, 0x631b25a0, 0x631b25a0, 0x001b25a0, 0x001b25a0, 0x6b1b25a0, 0x00000003, 0x00000000, 0x00000300 }; static const uint32_t rtl8188ce_agc_vals[] = { 0x7b000001, 0x7b010001, 0x7b020001, 0x7b030001, 0x7b040001, 0x7b050001, 0x7a060001, 0x79070001, 0x78080001, 0x77090001, 0x760a0001, 0x750b0001, 0x740c0001, 0x730d0001, 0x720e0001, 0x710f0001, 0x70100001, 0x6f110001, 0x6e120001, 0x6d130001, 0x6c140001, 0x6b150001, 0x6a160001, 0x69170001, 0x68180001, 0x67190001, 0x661a0001, 0x651b0001, 0x641c0001, 0x631d0001, 0x621e0001, 0x611f0001, 0x60200001, 0x49210001, 0x48220001, 0x47230001, 0x46240001, 0x45250001, 0x44260001, 0x43270001, 0x42280001, 0x41290001, 0x402a0001, 0x262b0001, 0x252c0001, 0x242d0001, 0x232e0001, 0x222f0001, 0x21300001, 0x20310001, 0x06320001, 0x05330001, 0x04340001, 0x03350001, 0x02360001, 0x01370001, 0x00380001, 0x00390001, 0x003a0001, 0x003b0001, 0x003c0001, 0x003d0001, 0x003e0001, 0x003f0001, 0x7b400001, 0x7b410001, 0x7b420001, 0x7b430001, 0x7b440001, 0x7b450001, 0x7a460001, 0x79470001, 0x78480001, 0x77490001, 0x764a0001, 0x754b0001, 0x744c0001, 0x734d0001, 0x724e0001, 0x714f0001, 0x70500001, 0x6f510001, 0x6e520001, 0x6d530001, 0x6c540001, 0x6b550001, 0x6a560001, 0x69570001, 0x68580001, 0x67590001, 0x665a0001, 0x655b0001, 0x645c0001, 0x635d0001, 0x625e0001, 0x615f0001, 0x60600001, 0x49610001, 0x48620001, 0x47630001, 0x46640001, 0x45650001, 0x44660001, 0x43670001, 0x42680001, 0x41690001, 0x406a0001, 0x266b0001, 0x256c0001, 0x246d0001, 0x236e0001, 0x226f0001, 0x21700001, 0x20710001, 0x06720001, 0x05730001, 0x04740001, 0x03750001, 0x02760001, 0x01770001, 0x00780001, 0x00790001, 0x007a0001, 0x007b0001, 0x007c0001, 0x007d0001, 0x007e0001, 0x007f0001, 0x3800001e, 0x3801001e, 0x3802001e, 0x3803001e, 0x3804001e, 0x3805001e, 0x3806001e, 0x3807001e, 0x3808001e, 0x3c09001e, 0x3e0a001e, 0x400b001e, 0x440c001e, 0x480d001e, 0x4c0e001e, 0x500f001e, 0x5210001e, 0x5611001e, 0x5a12001e, 0x5e13001e, 0x6014001e, 0x6015001e, 0x6016001e, 0x6217001e, 0x6218001e, 0x6219001e, 0x621a001e, 0x621b001e, 0x621c001e, 0x621d001e, 0x621e001e, 0x621f001e }; static const struct urtwn_bb_prog rtl8188ce_bb_prog = { nitems(rtl8192ce_bb_regs), rtl8192ce_bb_regs, rtl8188ce_bb_vals, nitems(rtl8188ce_agc_vals), rtl8188ce_agc_vals }; static const uint32_t rtl8188cu_bb_vals[] = { 0x0011800d, 0x00ffdb83, 0x80040000, 0x00000001, 0x0000fc00, 0x0000000a, 0x10005388, 0x020c3d10, 0x02200385, 0x00000000, 0x01000100, 0x00390004, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00010000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x569a569a, 0x001b25a4, 0x66e60230, 0x061f0130, 0x00000000, 0x32323200, 0x07000700, 0x22004000, 0x00000808, 0x00000000, 0xc0083070, 0x000004d5, 0x00000000, 0xccc000c0, 0x00000800, 0xfffffffe, 0x40302010, 0x00706050, 0x00000000, 0x00000023, 0x00000000, 0x81121111, 0x00d047c8, 0x80ff000c, 0x8c838300, 0x2e68120f, 0x9500bb78, 0x11144028, 0x00881117, 0x89140f00, 0x1a1b0000, 0x090e1317, 0x00000204, 0x00d30000, 0x101fbf00, 0x00000007, 0x48071d40, 0x03a05611, 0x000000e4, 0x6c6c6c6c, 0x08800000, 0x40000100, 0x08800000, 0x40000100, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x69e9ac44, 0x469652cf, 0x49795994, 0x0a97971c, 0x1f7c403f, 0x000100b7, 0xec020107, 0x007f037f, 0x6954341e, 0x43bc0094, 0x6954341e, 0x433c0094, 0x00000000, 0x5116848b, 0x47c00bff, 0x00000036, 0x2c7f000d, 0x018610db, 0x0000001f, 0x00b91612, 0x40000100, 0x20f60000, 0x40000100, 0x20200000, 0x00121820, 0x00000000, 0x00121820, 0x00007f7f, 0x00000000, 0x00000080, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x28000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x64b22427, 0x00766932, 0x00222222, 0x00000000, 0x37644302, 0x2f97d40c, 0x00080740, 0x00020401, 0x0000907f, 0x20010201, 0xa0633333, 0x3333bc43, 0x7a8f5b6b, 0xcc979975, 0x00000000, 0x80608000, 0x00000000, 0x00027293, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x6437140a, 0x00000000, 0x00000000, 0x30032064, 0x4653de68, 0x04518a3c, 0x00002101, 0x2a201c16, 0x1812362e, 0x322c2220, 0x000e3c24, 0x2a2a2a2a, 0x2a2a2a2a, 0x03902a2a, 0x2a2a2a2a, 0x2a2a2a2a, 0x2a2a2a2a, 0x2a2a2a2a, 0x00000000, 0x1000dc1f, 0x10008c1f, 0x02140102, 0x681604c2, 0x01007c00, 0x01004800, 0xfb000000, 0x000028d1, 0x1000dc1f, 0x10008c1f, 0x02140102, 0x28160d05, 0x00000008, 0x001b25a4, 0x631b25a0, 0x631b25a0, 0x081b25a0, 0x081b25a0, 0x081b25a0, 0x081b25a0, 0x631b25a0, 0x081b25a0, 0x631b25a0, 0x631b25a0, 0x631b25a0, 0x631b25a0, 0x001b25a0, 0x001b25a0, 0x6b1b25a0, 0x00000003, 0x00000000, 0x00000300 }; static const struct urtwn_bb_prog rtl8188cu_bb_prog = { nitems(rtl8192ce_bb_regs), rtl8192ce_bb_regs, rtl8188cu_bb_vals, nitems(rtl8188ce_agc_vals), rtl8188ce_agc_vals }; /* * RTL8188EU. */ static const uint16_t rtl8188eu_bb_regs[] = { 0x800, 0x804, 0x808, 0x80c, 0x810, 0x814, 0x818, 0x81c, 0x820, 0x824, 0x828, 0x82c, 0x830, 0x834, 0x838, 0x83c, 0x840, 0x844, 0x848, 0x84c, 0x850, 0x854, 0x858, 0x85c, 0x860, 0x864, 0x868, 0x86c, 0x870, 0x874, 0x878, 0x87c, 0x880, 0x884, 0x888, 0x88c, 0x890, 0x894, 0x898, 0x89c, 0x900, 0x904, 0x908, 0x90c, 0x910, 0x914, 0xa00, 0xa04, 0xa08, 0xa0c, 0xa10, 0xa14, 0xa18, 0xa1c, 0xa20, 0xa24, 0xa28, 0xa2c, 0xa70, 0xa74, 0xa78, 0xa7c, 0xa80, 0xb2c, 0xc00, 0xc04, 0xc08, 0xc0c, 0xc10, 0xc14, 0xc18, 0xc1c, 0xc20, 0xc24, 0xc28, 0xc2c, 0xc30, 0xc34, 0xc38, 0xc3c, 0xc40, 0xc44, 0xc48, 0xc4c, 0xc50, 0xc54, 0xc58, 0xc5c, 0xc60, 0xc64, 0xc68, 0xc6c, 0xc70, 0xc74, 0xc78, 0xc7c, 0xc80, 0xc84, 0xc88, 0xc8c, 0xc90, 0xc94, 0xc98, 0xc9c, 0xca0, 0xca4, 0xca8, 0xcac, 0xcb0, 0xcb4, 0xcb8, 0xcbc, 0xcc0, 0xcc4, 0xcc8, 0xccc, 0xcd0, 0xcd4, 0xcd8, 0xcdc, 0xce0, 0xce4, 0xce8, 0xcec, 0xd00, 0xd04, 0xd08, 0xd0c, 0xd10, 0xd14, 0xd18, 0xd2c, 0xd30, 0xd34, 0xd38, 0xd3c, 0xd40, 0xd44, 0xd48, 0xd4c, 0xd50, 0xd54, 0xd58, 0xd5c, 0xd60, 0xd64, 0xd68, 0xd6c, 0xd70, 0xd74, 0xd78, 0xe00, 0xe04, 0xe08, 0xe10, 0xe14, 0xe18, 0xe1c, 0xe28, 0xe30, 0xe34, 0xe38, 0xe3c, 0xe40, 0xe44, 0xe48, 0xe4c, 0xe50, 0xe54, 0xe58, 0xe5c, 0xe60, 0xe68, 0xe6c, 0xe70, 0xe74, 0xe78, 0xe7c, 0xe80, 0xe84, 0xe88, 0xe8c, 0xed0, 0xed4, 0xed8, 0xedc, 0xee0, 0xee8, 0xeec, 0xf14, 0xf4c, 0xf00 }; static const uint32_t rtl8188eu_bb_vals[] = { 0x80040000, 0x00000003, 0x0000fc00, 0x0000000a, 0x10001331, 0x020c3d10, 0x02200385, 0x00000000, 0x01000100, 0x00390204, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00010000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x569a11a9, 0x01000014, 0x66f60110, 0x061f0649, 0x00000000, 0x27272700, 0x07000760, 0x25004000, 0x00000808, 0x00000000, 0xb0000c1c, 0x00000001, 0x00000000, 0xccc000c0, 0x00000800, 0xfffffffe, 0x40302010, 0x00706050, 0x00000000, 0x00000023, 0x00000000, 0x81121111, 0x00000002, 0x00000201, 0x00d047c8, 0x80ff000c, 0x8c838300, 0x2e7f120f, 0x9500bb78, 0x1114d028, 0x00881117, 0x89140f00, 0x1a1b0000, 0x090e1317, 0x00000204, 0x00d30000, 0x101fbf00, 0x00000007, 0x00000900, 0x225b0606, 0x218075b1, 0x80000000, 0x48071d40, 0x03a05611, 0x000000e4, 0x6c6c6c6c, 0x08800000, 0x40000100, 0x08800000, 0x40000100, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x69e9ac47, 0x469652af, 0x49795994, 0x0a97971c, 0x1f7c403f, 0x000100b7, 0xec020107, 0x007f037f, 0x69553420, 0x43bc0094, 0x00013169, 0x00250492, 0x00000000, 0x7112848b, 0x47c00bff, 0x00000036, 0x2c7f000d, 0x020610db, 0x0000001f, 0x00b91612, 0x390000e4, 0x20f60000, 0x40000100, 0x20200000, 0x00091521, 0x00000000, 0x00121820, 0x00007f7f, 0x00000000, 0x000300a0, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x28000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x64b22427, 0x00766932, 0x00222222, 0x00000000, 0x37644302, 0x2f97d40c, 0x00000740, 0x00020401, 0x0000907f, 0x20010201, 0xa0633333, 0x3333bc43, 0x7a8f5b6f, 0xcc979975, 0x00000000, 0x80608000, 0x00000000, 0x00127353, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x6437140a, 0x00000000, 0x00000282, 0x30032064, 0x4653de68, 0x04518a3c, 0x00002101, 0x2a201c16, 0x1812362e, 0x322c2220, 0x000e3c24, 0x2d2d2d2d, 0x2d2d2d2d, 0x0390272d, 0x2d2d2d2d, 0x2d2d2d2d, 0x2d2d2d2d, 0x2d2d2d2d, 0x00000000, 0x1000dc1f, 0x10008c1f, 0x02140102, 0x681604c2, 0x01007c00, 0x01004800, 0xfb000000, 0x000028d1, 0x1000dc1f, 0x10008c1f, 0x02140102, 0x28160d05, 0x00000008, 0x001b25a4, 0x00c00014, 0x00c00014, 0x01000014, 0x01000014, 0x01000014, 0x01000014, 0x00c00014, 0x01000014, 0x00c00014, 0x00c00014, 0x00c00014, 0x00c00014, 0x00000014, 0x00000014, 0x21555448, 0x01c00014, 0x00000003, 0x00000000, 0x00000300 }; static const uint32_t rtl8188eu_agc_vals[] = { 0xfb000001, 0xfb010001, 0xfb020001, 0xfb030001, 0xfb040001, 0xfb050001, 0xfa060001, 0xf9070001, 0xf8080001, 0xf7090001, 0xf60a0001, 0xf50b0001, 0xf40c0001, 0xf30d0001, 0xf20e0001, 0xf10f0001, 0xf0100001, 0xef110001, 0xee120001, 0xed130001, 0xec140001, 0xeb150001, 0xea160001, 0xe9170001, 0xe8180001, 0xe7190001, 0xe61a0001, 0xe51b0001, 0xe41c0001, 0xe31d0001, 0xe21e0001, 0xe11f0001, 0x8a200001, 0x89210001, 0x88220001, 0x87230001, 0x86240001, 0x85250001, 0x84260001, 0x83270001, 0x82280001, 0x6b290001, 0x6a2a0001, 0x692b0001, 0x682c0001, 0x672d0001, 0x662e0001, 0x652f0001, 0x64300001, 0x63310001, 0x62320001, 0x61330001, 0x46340001, 0x45350001, 0x44360001, 0x43370001, 0x42380001, 0x41390001, 0x403a0001, 0x403b0001, 0x403c0001, 0x403d0001, 0x403e0001, 0x403f0001, 0xfb400001, 0xfb410001, 0xfb420001, 0xfb430001, 0xfb440001, 0xfb450001, 0xfb460001, 0xfb470001, 0xfb480001, 0xfa490001, 0xf94a0001, 0xf84B0001, 0xf74c0001, 0xf64d0001, 0xf54e0001, 0xf44f0001, 0xf3500001, 0xf2510001, 0xf1520001, 0xf0530001, 0xef540001, 0xee550001, 0xed560001, 0xec570001, 0xeb580001, 0xea590001, 0xe95a0001, 0xe85b0001, 0xe75c0001, 0xe65d0001, 0xe55e0001, 0xe45f0001, 0xe3600001, 0xe2610001, 0xc3620001, 0xc2630001, 0xc1640001, 0x8b650001, 0x8a660001, 0x89670001, 0x88680001, 0x87690001, 0x866a0001, 0x856b0001, 0x846c0001, 0x676d0001, 0x666e0001, 0x656f0001, 0x64700001, 0x63710001, 0x62720001, 0x61730001, 0x60740001, 0x46750001, 0x45760001, 0x44770001, 0x43780001, 0x42790001, 0x417a0001, 0x407b0001, 0x407c0001, 0x407d0001, 0x407e0001, 0x407f0001 }; static const struct urtwn_bb_prog rtl8188eu_bb_prog = { nitems(rtl8188eu_bb_regs), rtl8188eu_bb_regs, rtl8188eu_bb_vals, nitems(rtl8188eu_agc_vals), rtl8188eu_agc_vals }; /* * RTL8188RU. */ static const uint16_t rtl8188ru_bb_regs[] = { 0x024, 0x028, 0x040, 0x800, 0x804, 0x808, 0x80c, 0x810, 0x814, 0x818, 0x81c, 0x820, 0x824, 0x828, 0x82c, 0x830, 0x834, 0x838, 0x83c, 0x840, 0x844, 0x848, 0x84c, 0x850, 0x854, 0x858, 0x85c, 0x860, 0x864, 0x868, 0x86c, 0x870, 0x874, 0x878, 0x87c, 0x880, 0x884, 0x888, 0x88c, 0x890, 0x894, 0x898, 0x89c, 0x900, 0x904, 0x908, 0x90c, 0xa00, 0xa04, 0xa08, 0xa0c, 0xa10, 0xa14, 0xa18, 0xa1c, 0xa20, 0xa24, 0xa28, 0xa2c, 0xa70, 0xa74, 0xc00, 0xc04, 0xc08, 0xc0c, 0xc10, 0xc14, 0xc18, 0xc1c, 0xc20, 0xc24, 0xc28, 0xc2c, 0xc30, 0xc34, 0xc38, 0xc3c, 0xc40, 0xc44, 0xc48, 0xc4c, 0xc50, 0xc54, 0xc58, 0xc5c, 0xc60, 0xc64, 0xc68, 0xc6c, 0xc70, 0xc74, 0xc78, 0xc7c, 0xc80, 0xc84, 0xc88, 0xc8c, 0xc90, 0xc94, 0xc98, 0xc9c, 0xca0, 0xca4, 0xca8, 0xcac, 0xcb0, 0xcb4, 0xcb8, 0xcbc, 0xcc0, 0xcc4, 0xcc8, 0xccc, 0xcd0, 0xcd4, 0xcd8, 0xcdc, 0xce0, 0xce4, 0xce8, 0xcec, 0xd00, 0xd04, 0xd08, 0xd0c, 0xd10, 0xd14, 0xd18, 0xd2c, 0xd30, 0xd34, 0xd38, 0xd3c, 0xd40, 0xd44, 0xd48, 0xd4c, 0xd50, 0xd54, 0xd58, 0xd5c, 0xd60, 0xd64, 0xd68, 0xd6c, 0xd70, 0xd74, 0xd78, 0xe00, 0xe04, 0xe08, 0xe10, 0xe14, 0xe18, 0xe1c, 0xe28, 0xe30, 0xe34, 0xe38, 0xe3c, 0xe40, 0xe44, 0xe48, 0xe4c, 0xe50, 0xe54, 0xe58, 0xe5c, 0xe60, 0xe68, 0xe6c, 0xe70, 0xe74, 0xe78, 0xe7c, 0xe80, 0xe84, 0xe88, 0xe8c, 0xed0, 0xed4, 0xed8, 0xedc, 0xee0, 0xeec, 0xee8, 0xf14, 0xf4c, 0xf00 }; static const uint32_t rtl8188ru_bb_vals[] = { 0x0011800d, 0x00ffdb83, 0x000c0004, 0x80040000, 0x00000001, 0x0000fc00, 0x0000000a, 0x10005388, 0x020c3d10, 0x02200385, 0x00000000, 0x01000100, 0x00390204, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00010000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x569a569a, 0x001b25a4, 0x66e60230, 0x061f0130, 0x00000000, 0x32323200, 0x03000300, 0x22004000, 0x00000808, 0x00ffc3f1, 0xc0083070, 0x000004d5, 0x00000000, 0xccc000c0, 0x00000800, 0xfffffffe, 0x40302010, 0x00706050, 0x00000000, 0x00000023, 0x00000000, 0x81121111, 0x00d047c8, 0x80ff000c, 0x8c838300, 0x2e68120f, 0x9500bb78, 0x11144028, 0x00881117, 0x89140f00, 0x15160000, 0x070b0f12, 0x00000104, 0x00d30000, 0x101fbf00, 0x00000007, 0x48071d40, 0x03a05611, 0x000000e4, 0x6c6c6c6c, 0x08800000, 0x40000100, 0x08800000, 0x40000100, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x69e9ac44, 0x469652cf, 0x49795994, 0x0a97971c, 0x1f7c403f, 0x000100b7, 0xec020107, 0x007f037f, 0x6954342e, 0x43bc0094, 0x6954342f, 0x433c0094, 0x00000000, 0x5116848b, 0x47c00bff, 0x00000036, 0x2c56000d, 0x018610db, 0x0000001f, 0x00b91612, 0x24000090, 0x20f60000, 0x24000090, 0x20200000, 0x00121820, 0x00000000, 0x00121820, 0x00007f7f, 0x00000000, 0x00000080, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x28000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x64b22427, 0x00766932, 0x00222222, 0x00000000, 0x37644302, 0x2f97d40c, 0x00080740, 0x00020401, 0x0000907f, 0x20010201, 0xa0633333, 0x3333bc43, 0x7a8f5b6b, 0xcc979975, 0x00000000, 0x80608000, 0x00000000, 0x00027293, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x6437140a, 0x00000000, 0x00000000, 0x30032064, 0x4653de68, 0x04518a3c, 0x00002101, 0x2a201c16, 0x1812362e, 0x322c2220, 0x000e3c24, 0x2a2a2a2a, 0x2a2a2a2a, 0x03902a2a, 0x2a2a2a2a, 0x2a2a2a2a, 0x2a2a2a2a, 0x2a2a2a2a, 0x00000000, 0x1000dc1f, 0x10008c1f, 0x02140102, 0x681604c2, 0x01007c00, 0x01004800, 0xfb000000, 0x000028d1, 0x1000dc1f, 0x10008c1f, 0x02140102, 0x28160d05, 0x00000010, 0x001b25a4, 0x631b25a0, 0x631b25a0, 0x081b25a0, 0x081b25a0, 0x081b25a0, 0x081b25a0, 0x631b25a0, 0x081b25a0, 0x631b25a0, 0x631b25a0, 0x631b25a0, 0x631b25a0, 0x001b25a0, 0x001b25a0, 0x6b1b25a0, 0x31555448, 0x00000003, 0x00000000, 0x00000300 }; static const uint32_t rtl8188ru_agc_vals[] = { 0x7b000001, 0x7b010001, 0x7b020001, 0x7b030001, 0x7b040001, 0x7b050001, 0x7b060001, 0x7b070001, 0x7b080001, 0x7a090001, 0x790a0001, 0x780b0001, 0x770c0001, 0x760d0001, 0x750e0001, 0x740f0001, 0x73100001, 0x72110001, 0x71120001, 0x70130001, 0x6f140001, 0x6e150001, 0x6d160001, 0x6c170001, 0x6b180001, 0x6a190001, 0x691a0001, 0x681b0001, 0x671c0001, 0x661d0001, 0x651e0001, 0x641f0001, 0x63200001, 0x62210001, 0x61220001, 0x60230001, 0x46240001, 0x45250001, 0x44260001, 0x43270001, 0x42280001, 0x41290001, 0x402a0001, 0x262b0001, 0x252c0001, 0x242d0001, 0x232e0001, 0x222f0001, 0x21300001, 0x20310001, 0x06320001, 0x05330001, 0x04340001, 0x03350001, 0x02360001, 0x01370001, 0x00380001, 0x00390001, 0x003a0001, 0x003b0001, 0x003c0001, 0x003d0001, 0x003e0001, 0x003f0001, 0x7b400001, 0x7b410001, 0x7b420001, 0x7b430001, 0x7b440001, 0x7b450001, 0x7b460001, 0x7b470001, 0x7b480001, 0x7a490001, 0x794a0001, 0x784b0001, 0x774c0001, 0x764d0001, 0x754e0001, 0x744f0001, 0x73500001, 0x72510001, 0x71520001, 0x70530001, 0x6f540001, 0x6e550001, 0x6d560001, 0x6c570001, 0x6b580001, 0x6a590001, 0x695a0001, 0x685b0001, 0x675c0001, 0x665d0001, 0x655e0001, 0x645f0001, 0x63600001, 0x62610001, 0x61620001, 0x60630001, 0x46640001, 0x45650001, 0x44660001, 0x43670001, 0x42680001, 0x41690001, 0x406a0001, 0x266b0001, 0x256c0001, 0x246d0001, 0x236e0001, 0x226f0001, 0x21700001, 0x20710001, 0x06720001, 0x05730001, 0x04740001, 0x03750001, 0x02760001, 0x01770001, 0x00780001, 0x00790001, 0x007a0001, 0x007b0001, 0x007c0001, 0x007d0001, 0x007e0001, 0x007f0001, 0x3800001e, 0x3801001e, 0x3802001e, 0x3803001e, 0x3804001e, 0x3805001e, 0x3806001e, 0x3807001e, 0x3808001e, 0x3c09001e, 0x3e0a001e, 0x400b001e, 0x440c001e, 0x480d001e, 0x4c0e001e, 0x500f001e, 0x5210001e, 0x5611001e, 0x5a12001e, 0x5e13001e, 0x6014001e, 0x6015001e, 0x6016001e, 0x6217001e, 0x6218001e, 0x6219001e, 0x621a001e, 0x621b001e, 0x621c001e, 0x621d001e, 0x621e001e, 0x621f001e }; static const struct urtwn_bb_prog rtl8188ru_bb_prog = { nitems(rtl8188ru_bb_regs), rtl8188ru_bb_regs, rtl8188ru_bb_vals, nitems(rtl8188ru_agc_vals), rtl8188ru_agc_vals }; /* * RF initialization values. */ struct urtwn_rf_prog { int count; const uint8_t *regs; const uint32_t *vals; }; /* * RTL8192CU and RTL8192CE-VAU. */ static const uint8_t rtl8192ce_rf1_regs[] = { 0x00, 0x01, 0x02, 0x03, 0x04, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f, 0x20, 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28, 0x29, 0x2a, 0x2b, 0x2a, 0x2b, 0x2a, 0x2b, 0x2b, 0x2c, 0x2a, 0x2b, 0x2b, 0x2c, 0x2a, 0x2b, 0x2b, 0x2c, 0x2a, 0x2b, 0x2b, 0x2c, 0x2a, 0x2b, 0x2b, 0x2c, 0x2a, 0x2b, 0x2b, 0x2c, 0x2a, 0x2b, 0x2b, 0x2c, 0x2a, 0x2b, 0x2b, 0x2c, 0x2a, 0x2b, 0x2b, 0x2c, 0x2a, 0x2b, 0x2b, 0x2c, 0x2a, 0x2b, 0x2b, 0x2c, 0x2a, 0x2b, 0x2b, 0x2c, 0x2a, 0x2b, 0x2b, 0x2c, 0x2a, 0x2b, 0x2b, 0x2c, 0x2a, 0x10, 0x11, 0x10, 0x11, 0x10, 0x11, 0x10, 0x11, 0x10, 0x11, 0x10, 0x11, 0x10, 0x11, 0x12, 0x12, 0x12, 0x12, 0x13, 0x13, 0x13, 0x13, 0x13, 0x13, 0x13, 0x13, 0x13, 0x13, 0x13, 0x14, 0x14, 0x14, 0x14, 0x15, 0x15, 0x15, 0x15, 0x16, 0x16, 0x16, 0x16, 0x00, 0x18, 0xfe, 0xfe, 0x1f, 0xfe, 0xfe, 0x1e, 0x1f, 0x00 }; static const uint32_t rtl8192ce_rf1_vals[] = { 0x30159, 0x31284, 0x98000, 0x18c63, 0x210e7, 0x2044f, 0x1adb1, 0x54867, 0x8992e, 0x0e52c, 0x39ce7, 0x00451, 0x00000, 0x10255, 0x60a00, 0xfc378, 0xa1250, 0x4445f, 0x80001, 0x0b614, 0x6c000, 0x00000, 0x01558, 0x00060, 0x00483, 0x4f000, 0xec7d9, 0x577c0, 0x04783, 0x00001, 0x21334, 0x00000, 0x00054, 0x00001, 0x00808, 0x53333, 0x0000c, 0x00002, 0x00808, 0x5b333, 0x0000d, 0x00003, 0x00808, 0x63333, 0x0000d, 0x00004, 0x00808, 0x6b333, 0x0000d, 0x00005, 0x00808, 0x73333, 0x0000d, 0x00006, 0x00709, 0x5b333, 0x0000d, 0x00007, 0x00709, 0x63333, 0x0000d, 0x00008, 0x0060a, 0x4b333, 0x0000d, 0x00009, 0x0060a, 0x53333, 0x0000d, 0x0000a, 0x0060a, 0x5b333, 0x0000d, 0x0000b, 0x0060a, 0x63333, 0x0000d, 0x0000c, 0x0060a, 0x6b333, 0x0000d, 0x0000d, 0x0060a, 0x73333, 0x0000d, 0x0000e, 0x0050b, 0x66666, 0x0001a, 0xe0000, 0x4000f, 0xe31fc, 0x6000f, 0xff9f8, 0x2000f, 0x203f9, 0x3000f, 0xff500, 0x00000, 0x00000, 0x8000f, 0x3f100, 0x9000f, 0x23100, 0x32000, 0x71000, 0xb0000, 0xfc000, 0x287af, 0x244b7, 0x204ab, 0x1c49f, 0x18493, 0x14297, 0x10295, 0x0c298, 0x0819c, 0x040a8, 0x0001c, 0x1944c, 0x59444, 0x9944c, 0xd9444, 0x0f424, 0x4f424, 0x8f424, 0xcf424, 0xe0330, 0xa0330, 0x60330, 0x20330, 0x10159, 0x0f401, 0x00000, 0x00000, 0x80003, 0x00000, 0x00000, 0x44457, 0x80000, 0x30159 }; static const uint8_t rtl8192ce_rf2_regs[] = { 0x00, 0x01, 0x02, 0x03, 0x04, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, 0x12, 0x12, 0x12, 0x12, 0x13, 0x13, 0x13, 0x13, 0x13, 0x13, 0x13, 0x13, 0x13, 0x13, 0x13, 0x14, 0x14, 0x14, 0x14, 0x15, 0x15, 0x15, 0x15, 0x16, 0x16, 0x16, 0x16 }; static const uint32_t rtl8192ce_rf2_vals[] = { 0x30159, 0x31284, 0x98000, 0x18c63, 0x210e7, 0x2044f, 0x1adb1, 0x54867, 0x8992e, 0x0e52c, 0x39ce7, 0x00451, 0x32000, 0x71000, 0xb0000, 0xfc000, 0x287af, 0x244b7, 0x204ab, 0x1c49f, 0x18493, 0x14297, 0x10295, 0x0c298, 0x0819c, 0x040a8, 0x0001c, 0x1944c, 0x59444, 0x9944c, 0xd9444, 0x0f424, 0x4f424, 0x8f424, 0xcf424, 0xe0330, 0xa0330, 0x60330, 0x20330 }; static const struct urtwn_rf_prog rtl8192ce_rf_prog[] = { { nitems(rtl8192ce_rf1_regs), rtl8192ce_rf1_regs, rtl8192ce_rf1_vals }, { nitems(rtl8192ce_rf2_regs), rtl8192ce_rf2_regs, rtl8192ce_rf2_vals } }; /* * RTL8188CE-VAU. */ static const uint32_t rtl8188ce_rf_vals[] = { 0x30159, 0x31284, 0x98000, 0x18c63, 0x210e7, 0x2044f, 0x1adb1, 0x54867, 0x8992e, 0x0e52c, 0x39ce7, 0x00451, 0x00000, 0x10255, 0x60a00, 0xfc378, 0xa1250, 0x4445f, 0x80001, 0x0b614, 0x6c000, 0x00000, 0x01558, 0x00060, 0x00483, 0x4f200, 0xec7d9, 0x577c0, 0x04783, 0x00001, 0x21334, 0x00000, 0x00054, 0x00001, 0x00808, 0x53333, 0x0000c, 0x00002, 0x00808, 0x5b333, 0x0000d, 0x00003, 0x00808, 0x63333, 0x0000d, 0x00004, 0x00808, 0x6b333, 0x0000d, 0x00005, 0x00808, 0x73333, 0x0000d, 0x00006, 0x00709, 0x5b333, 0x0000d, 0x00007, 0x00709, 0x63333, 0x0000d, 0x00008, 0x0060a, 0x4b333, 0x0000d, 0x00009, 0x0060a, 0x53333, 0x0000d, 0x0000a, 0x0060a, 0x5b333, 0x0000d, 0x0000b, 0x0060a, 0x63333, 0x0000d, 0x0000c, 0x0060a, 0x6b333, 0x0000d, 0x0000d, 0x0060a, 0x73333, 0x0000d, 0x0000e, 0x0050b, 0x66666, 0x0001a, 0xe0000, 0x4000f, 0xe31fc, 0x6000f, 0xff9f8, 0x2000f, 0x203f9, 0x3000f, 0xff500, 0x00000, 0x00000, 0x8000f, 0x3f100, 0x9000f, 0x23100, 0x32000, 0x71000, 0xb0000, 0xfc000, 0x287b3, 0x244b7, 0x204ab, 0x1c49f, 0x18493, 0x1429b, 0x10299, 0x0c29c, 0x081a0, 0x040ac, 0x00020, 0x1944c, 0x59444, 0x9944c, 0xd9444, 0x0f424, 0x4f424, 0x8f424, 0xcf424, 0xe0330, 0xa0330, 0x60330, 0x20330, 0x10159, 0x0f401, 0x00000, 0x00000, 0x80003, 0x00000, 0x00000, 0x44457, 0x80000, 0x30159 }; static const struct urtwn_rf_prog rtl8188ce_rf_prog[] = { { nitems(rtl8192ce_rf1_regs), rtl8192ce_rf1_regs, rtl8188ce_rf_vals } }; /* * RTL8188CU. */ static const uint32_t rtl8188cu_rf_vals[] = { 0x30159, 0x31284, 0x98000, 0x18c63, 0x210e7, 0x2044f, 0x1adb1, 0x54867, 0x8992e, 0x0e52c, 0x39ce7, 0x00451, 0x00000, 0x10255, 0x60a00, 0xfc378, 0xa1250, 0x4445f, 0x80001, 0x0b614, 0x6c000, 0x00000, 0x01558, 0x00060, 0x00483, 0x4f000, 0xec7d9, 0x577c0, 0x04783, 0x00001, 0x21334, 0x00000, 0x00054, 0x00001, 0x00808, 0x53333, 0x0000c, 0x00002, 0x00808, 0x5b333, 0x0000d, 0x00003, 0x00808, 0x63333, 0x0000d, 0x00004, 0x00808, 0x6b333, 0x0000d, 0x00005, 0x00808, 0x73333, 0x0000d, 0x00006, 0x00709, 0x5b333, 0x0000d, 0x00007, 0x00709, 0x63333, 0x0000d, 0x00008, 0x0060a, 0x4b333, 0x0000d, 0x00009, 0x0060a, 0x53333, 0x0000d, 0x0000a, 0x0060a, 0x5b333, 0x0000d, 0x0000b, 0x0060a, 0x63333, 0x0000d, 0x0000c, 0x0060a, 0x6b333, 0x0000d, 0x0000d, 0x0060a, 0x73333, 0x0000d, 0x0000e, 0x0050b, 0x66666, 0x0001a, 0xe0000, 0x4000f, 0xe31fc, 0x6000f, 0xff9f8, 0x2000f, 0x203f9, 0x3000f, 0xff500, 0x00000, 0x00000, 0x8000f, 0x3f100, 0x9000f, 0x23100, 0x32000, 0x71000, 0xb0000, 0xfc000, 0x287b3, 0x244b7, 0x204ab, 0x1c49f, 0x18493, 0x1429b, 0x10299, 0x0c29c, 0x081a0, 0x040ac, 0x00020, 0x1944c, 0x59444, 0x9944c, 0xd9444, 0x0f405, 0x4f405, 0x8f405, 0xcf405, 0xe0330, 0xa0330, 0x60330, 0x20330, 0x10159, 0x0f401, 0x00000, 0x00000, 0x80003, 0x00000, 0x00000, 0x44457, 0x80000, 0x30159 }; static const struct urtwn_rf_prog rtl8188cu_rf_prog[] = { { nitems(rtl8192ce_rf1_regs), rtl8192ce_rf1_regs, rtl8188cu_rf_vals } }; /* * RTL8188EU. */ static const uint8_t rtl8188eu_rf_regs[] = { 0x00, 0x08, 0x18, 0x19, 0x1e, 0x1f, 0x2f, 0x3f, 0x42, 0x57, 0x58, 0x67, 0x83, 0xb0, 0xb1, 0xb2, 0xb4, 0xb6, 0xb7, 0xb8, 0xb9, 0xba, 0xbb, 0xbf, 0xc2, 0xc3, 0xc4, 0xc5, 0xc6, 0xc7, 0xc8, 0xc9, 0xca, 0xdf, 0xef, 0x51, 0x52, 0x53, 0x56, 0x35, 0x35, 0x35, 0x36, 0x36, 0x36, 0x36, 0xb6, 0x18, 0x5a, 0x19, 0x34, 0x34, 0x34, 0x34, 0x34, 0x34, 0x34, 0x34, 0x34, 0x34, 0x34, 0x00, 0x84, 0x86, 0x87, 0x8e, 0x8f, 0xef, 0x3b, 0x3b, 0x3b, 0x3b, 0x3b, 0x3b, 0x3b, 0x3b, 0x3b, 0x3b, 0x3b, 0x3b, 0x3b, 0x3b, 0x3b, 0x3b, 0xef, 0x00, 0x18, 0xfe, 0xfe, 0x1f, 0xfe, 0xfe, 0x1e, 0x1f, 0x00 }; static const uint32_t rtl8188eu_rf_vals[] = { 0x30000, 0x84000, 0x00407, 0x00012, 0x80009, 0x00880, 0x1a060, 0x00000, 0x060c0, 0xd0000, 0xbe180, 0x01552, 0x00000, 0xff8fc, 0x54400, 0xccc19, 0x43003, 0x4953e, 0x1c718, 0x060ff, 0x80001, 0x40000, 0x00400, 0xc0000, 0x02400, 0x00009, 0x40c91, 0x99999, 0x000a3, 0x88820, 0x76c06, 0x00000, 0x80000, 0x00180, 0x001a0, 0x6b27d, 0x7e49d, 0x00073, 0x51ff3, 0x00086, 0x00186, 0x00286, 0x01c25, 0x09c25, 0x11c25, 0x19c25, 0x48538, 0x00c07, 0x4bd00, 0x739d0, 0x0adf3, 0x09df0, 0x08ded, 0x07dea, 0x06de7, 0x054ee, 0x044eb, 0x034e8, 0x0246b, 0x01468, 0x0006d, 0x30159, 0x68200, 0x000ce, 0x48a00, 0x65540, 0x88000, 0x020a0, 0xf02b0, 0xef7b0, 0xd4fb0, 0xcf060, 0xb0090, 0xa0080, 0x90080, 0x8f780, 0x722b0, 0x6f7b0, 0x54fb0, 0x4f060, 0x30090, 0x20080, 0x10080, 0x0f780, 0x000a0, 0x10159, 0x0f407, 0x00000, 0x00000, 0x80003, 0x00000, 0x00000, 0x00001, 0x80000, 0x33e60 }; static const struct urtwn_rf_prog rtl8188eu_rf_prog[] = { { nitems(rtl8188eu_rf_regs), rtl8188eu_rf_regs, rtl8188eu_rf_vals } }; /* * RTL8188RU. */ static const uint32_t rtl8188ru_rf_vals[] = { 0x30159, 0x31284, 0x98000, 0x18c63, 0x210e7, 0x2044f, 0x1adb0, 0x54867, 0x8992e, 0x0e529, 0x39ce7, 0x00451, 0x00000, 0x00255, 0x60a00, 0xfc378, 0xa1250, 0x4445f, 0x80001, 0x0b614, 0x6c000, 0x0083c, 0x01558, 0x00060, 0x00483, 0x4f000, 0xec7d9, 0x977c0, 0x04783, 0x00001, 0x21334, 0x00000, 0x00054, 0x00001, 0x00808, 0x53333, 0x0000c, 0x00002, 0x00808, 0x5b333, 0x0000d, 0x00003, 0x00808, 0x63333, 0x0000d, 0x00004, 0x00808, 0x6b333, 0x0000d, 0x00005, 0x00808, 0x73333, 0x0000d, 0x00006, 0x00709, 0x5b333, 0x0000d, 0x00007, 0x00709, 0x63333, 0x0000d, 0x00008, 0x0060a, 0x4b333, 0x0000d, 0x00009, 0x0060a, 0x53333, 0x0000d, 0x0000a, 0x0060a, 0x5b333, 0x0000d, 0x0000b, 0x0060a, 0x63333, 0x0000d, 0x0000c, 0x0060a, 0x6b333, 0x0000d, 0x0000d, 0x0060a, 0x73333, 0x0000d, 0x0000e, 0x0050b, 0x66666, 0x0001a, 0xe0000, 0x4000f, 0xe31fc, 0x6000f, 0xff9f8, 0x2000f, 0x203f9, 0x3000f, 0xff500, 0x00000, 0x00000, 0x8000f, 0x3f100, 0x9000f, 0x23100, 0xd8000, 0x90000, 0x51000, 0x12000, 0x28fb4, 0x24fa8, 0x207a4, 0x1c798, 0x183a4, 0x14398, 0x101a4, 0x0c198, 0x080a4, 0x04098, 0x00014, 0x1944c, 0x59444, 0x9944c, 0xd9444, 0x0f405, 0x4f405, 0x8f405, 0xcf405, 0xe0330, 0xa0330, 0x60330, 0x20330, 0x10159, 0x0f401, 0x00000, 0x00000, 0x80003, 0x00000, 0x00000, 0x44457, 0x80000, 0x30159 }; static const struct urtwn_rf_prog rtl8188ru_rf_prog[] = { { nitems(rtl8192ce_rf1_regs), rtl8192ce_rf1_regs, rtl8188ru_rf_vals } }; struct urtwn_txpwr { uint8_t pwr[3][28]; }; struct urtwn_r88e_txpwr { uint8_t pwr[6][28]; }; /* * Per RF chain/group/rate Tx gain values. */ static const struct urtwn_txpwr rtl8192cu_txagc[] = { { { /* Chain 0. */ { /* Group 0. */ 0x00, 0x00, 0x00, 0x00, /* CCK1~11. */ 0x0c, 0x0c, 0x0c, 0x0a, 0x08, 0x06, 0x04, 0x02, /* OFDM6~54. */ 0x0e, 0x0d, 0x0c, 0x0a, 0x08, 0x06, 0x04, 0x02, /* MCS0~7. */ 0x0e, 0x0d, 0x0c, 0x0a, 0x08, 0x06, 0x04, 0x02 /* MCS8~15. */ }, { /* Group 1. */ 0x00, 0x00, 0x00, 0x00, /* CCK1~11. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* OFDM6~54. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* MCS0~7. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 /* MCS8~15. */ }, { /* Group 2. */ 0x00, 0x00, 0x00, 0x00, /* CCK1~11. */ 0x04, 0x04, 0x04, 0x04, 0x04, 0x02, 0x02, 0x00, /* OFDM6~54. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* MCS0~7. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 /* MCS8~15. */ } } }, { { /* Chain 1. */ { /* Group 0. */ 0x00, 0x00, 0x00, 0x00, /* CCK1~11. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* OFDM6~54. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* MCS0~7. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 /* MCS8~15. */ }, { /* Group 1. */ 0x00, 0x00, 0x00, 0x00, /* CCK1~11. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* OFDM6~54. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* MCS0~7. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 /* MCS8~15. */ }, { /* Group 2. */ 0x00, 0x00, 0x00, 0x00, /* CCK1~11. */ 0x04, 0x04, 0x04, 0x04, 0x04, 0x02, 0x02, 0x00, /* OFDM6~54. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* MCS0~7. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 /* MCS8~15. */ } } } }; static const struct urtwn_txpwr rtl8188ru_txagc[] = { { { /* Chain 0. */ { /* Group 0. */ 0x00, 0x00, 0x00, 0x00, /* CCK1~11. */ 0x08, 0x08, 0x08, 0x06, 0x06, 0x04, 0x04, 0x00, /* OFDM6~54. */ 0x08, 0x06, 0x06, 0x04, 0x04, 0x02, 0x02, 0x00, /* MCS0~7. */ 0x08, 0x06, 0x06, 0x04, 0x04, 0x02, 0x02, 0x00 /* MCS8~15. */ }, { /* Group 1. */ 0x00, 0x00, 0x00, 0x00, /* CCK1~11. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* OFDM6~54. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* MCS0~7. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 /* MCS8~15. */ }, { /* Group 2. */ 0x00, 0x00, 0x00, 0x00, /* CCK1~11. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* OFDM6~54. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* MCS0~7. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 /* MCS8~15. */ } } } }; static const struct urtwn_r88e_txpwr rtl8188eu_txagc[] = { { { /* Chain 0. */ { /* Group 0. */ 0x00, 0x00, 0x00, 0x00, /* CCK1~11. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* OFDM6~54. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* MCS0~7. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 /* MCS8~15. */ }, { /* Group 1. */ 0x00, 0x00, 0x00, 0x00, /* CCK1~11. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* OFDM6~54. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* MCS0~7. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 /* MCS8~15. */ }, { /* Group 2. */ 0x00, 0x00, 0x00, 0x00, /* CCK1~11. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* OFDM6~54. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* MCS0~7. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 /* MCS8~15. */ }, { /* Group 3. */ 0x00, 0x00, 0x00, 0x00, /* CCK1~11. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* OFDM6~54. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* MCS0~7. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 /* MCS8~15. */ }, { /* Group 4. */ 0x00, 0x00, 0x00, 0x00, /* CCK1~11. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* OFDM6~54. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* MCS0~7. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 /* MCS8~15. */ }, { /* Group 5. */ 0x00, 0x00, 0x00, 0x00, /* CCK1~11. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* OFDM6~54. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* MCS0~7. */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 /* MCS8~15. */ } } } }; Index: user/ngie/more-tests2/sys/dev/usb/wlan/if_urtwnvar.h =================================================================== --- user/ngie/more-tests2/sys/dev/usb/wlan/if_urtwnvar.h (revision 292053) +++ user/ngie/more-tests2/sys/dev/usb/wlan/if_urtwnvar.h (revision 292054) @@ -1,216 +1,215 @@ /*- * Copyright (c) 2010 Damien Bergamini * * Permission to use, copy, modify, and distribute this software for any * purpose with or without fee is hereby granted, provided that the above * copyright notice and this permission notice appear in all copies. * * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. * * $OpenBSD: if_urtwnreg.h,v 1.3 2010/11/16 18:02:59 damien Exp $ * $FreeBSD$ */ #define URTWN_RX_LIST_COUNT 1 #define URTWN_TX_LIST_COUNT 8 #define URTWN_HOST_CMD_RING_COUNT 32 #define URTWN_RXBUFSZ (16 * 1024) #define URTWN_TXBUFSZ (sizeof(struct r92c_tx_desc) + IEEE80211_MAX_LEN) #define URTWN_RX_DESC_SIZE (sizeof(struct r92c_rx_stat)) #define URTWN_TX_DESC_SIZE (sizeof(struct r92c_tx_desc)) #define URTWN_TX_TIMEOUT 5000 /* ms */ #define URTWN_LED_LINK 0 #define URTWN_LED_DATA 1 struct urtwn_rx_radiotap_header { struct ieee80211_radiotap_header wr_ihdr; uint8_t wr_flags; uint8_t wr_rate; uint16_t wr_chan_freq; uint16_t wr_chan_flags; int8_t wr_dbm_antsignal; int8_t wr_dbm_antnoise; } __packed __aligned(8); #define URTWN_RX_RADIOTAP_PRESENT \ (1 << IEEE80211_RADIOTAP_FLAGS | \ 1 << IEEE80211_RADIOTAP_RATE | \ 1 << IEEE80211_RADIOTAP_CHANNEL | \ 1 << IEEE80211_RADIOTAP_DBM_ANTSIGNAL | \ 1 << IEEE80211_RADIOTAP_DBM_ANTNOISE) struct urtwn_tx_radiotap_header { struct ieee80211_radiotap_header wt_ihdr; uint8_t wt_flags; uint16_t wt_chan_freq; uint16_t wt_chan_flags; } __packed __aligned(8); #define URTWN_TX_RADIOTAP_PRESENT \ (1 << IEEE80211_RADIOTAP_FLAGS | \ 1 << IEEE80211_RADIOTAP_CHANNEL) struct urtwn_softc; struct urtwn_data { struct urtwn_softc *sc; uint8_t *buf; uint16_t buflen; struct mbuf *m; struct ieee80211_node *ni; STAILQ_ENTRY(urtwn_data) next; }; typedef STAILQ_HEAD(, urtwn_data) urtwn_datahead; struct urtwn_cmdq { void *arg0; void *arg1; void (*func)(void *); struct ieee80211_key *k; struct ieee80211_key key; uint8_t mac[IEEE80211_ADDR_LEN]; uint8_t wcid; }; struct urtwn_fw_info { const uint8_t *data; size_t size; }; struct urtwn_vap { struct ieee80211vap vap; struct r92c_tx_desc bcn_desc; struct mbuf *bcn_mbuf; struct task tsf_task_adhoc; int (*newstate)(struct ieee80211vap *, enum ieee80211_state, int); void (*recv_mgmt)(struct ieee80211_node *, struct mbuf *, int, const struct ieee80211_rx_stats *, int, int); }; #define URTWN_VAP(vap) ((struct urtwn_vap *)(vap)) struct urtwn_host_cmd { void (*cb)(struct urtwn_softc *, void *); uint8_t data[256]; }; struct urtwn_cmd_newstate { enum ieee80211_state state; int arg; }; struct urtwn_cmd_key { struct ieee80211_key key; uint16_t associd; }; enum { URTWN_BULK_RX, URTWN_BULK_TX_BE, /* = WME_AC_BE */ URTWN_BULK_TX_BK, /* = WME_AC_BK */ URTWN_BULK_TX_VI, /* = WME_AC_VI */ URTWN_BULK_TX_VO, /* = WME_AC_VI */ URTWN_N_TRANSFER = 5, }; #define URTWN_EP_QUEUES URTWN_BULK_RX union urtwn_rom { struct r92c_rom r92c_rom; uint8_t r88e_rom[URTWN_EFUSE_MAX_LEN]; }; struct urtwn_softc { struct ieee80211com sc_ic; struct mbufq sc_snd; device_t sc_dev; struct usb_device *sc_udev; uint8_t sc_iface_index; - int ac2idx[WME_NUM_AC]; u_int sc_flags; #define URTWN_FLAG_CCK_HIPWR 0x01 #define URTWN_DETACHED 0x02 #define URTWN_RUNNING 0x04 u_int chip; #define URTWN_CHIP_92C 0x01 #define URTWN_CHIP_92C_1T2R 0x02 #define URTWN_CHIP_UMC 0x04 #define URTWN_CHIP_UMC_A_CUT 0x08 #define URTWN_CHIP_88E 0x10 void (*sc_rf_write)(struct urtwn_softc *, int, uint8_t, uint32_t); int (*sc_power_on)(struct urtwn_softc *); uint8_t board_type; uint8_t regulatory; uint8_t pa_setting; int avg_pwdb; int thcal_state; int thcal_lctemp; int ntxchains; int nrxchains; int ledlink; int sc_txtimer; int fwcur; struct urtwn_data sc_rx[URTWN_RX_LIST_COUNT]; urtwn_datahead sc_rx_active; urtwn_datahead sc_rx_inactive; struct urtwn_data sc_tx[URTWN_TX_LIST_COUNT]; urtwn_datahead sc_tx_active; urtwn_datahead sc_tx_inactive; urtwn_datahead sc_tx_pending; const char *fwname; const struct firmware *fw_fp; struct urtwn_fw_info fw; void *fw_virtaddr; union urtwn_rom rom; uint8_t cck_tx_pwr[6]; uint8_t ht40_tx_pwr[5]; int8_t bw20_tx_pwr_diff; int8_t ofdm_tx_pwr_diff; uint16_t last_rom_addr; struct callout sc_watchdog_ch; struct mtx sc_mtx; /* need to be power of 2, otherwise URTWN_CMDQ_GET fails */ #define URTWN_CMDQ_MAX 16 #define URTWN_CMDQ_MASQ (URTWN_CMDQ_MAX - 1) struct urtwn_cmdq cmdq[URTWN_CMDQ_MAX]; struct task cmdq_task; uint32_t cmdq_store; uint8_t cmdq_exec; uint8_t cmdq_run; uint8_t cmdq_key_set; #define URTWN_CMDQ_ABORT 0 #define URTWN_CMDQ_GO 1 uint32_t rf_chnlbw[R92C_MAX_CHAINS]; struct usb_xfer *sc_xfer[URTWN_N_TRANSFER]; struct urtwn_rx_radiotap_header sc_rxtap; struct urtwn_tx_radiotap_header sc_txtap; }; #define URTWN_LOCK(sc) mtx_lock(&(sc)->sc_mtx) #define URTWN_UNLOCK(sc) mtx_unlock(&(sc)->sc_mtx) #define URTWN_ASSERT_LOCKED(sc) mtx_assert(&(sc)->sc_mtx, MA_OWNED) Index: user/ngie/more-tests2/sys/netinet/cc/cc_cubic.c =================================================================== --- user/ngie/more-tests2/sys/netinet/cc/cc_cubic.c (revision 292053) +++ user/ngie/more-tests2/sys/netinet/cc/cc_cubic.c (revision 292054) @@ -1,406 +1,411 @@ /*- * Copyright (c) 2008-2010 Lawrence Stewart * Copyright (c) 2010 The FreeBSD Foundation * All rights reserved. * * This software was developed by Lawrence Stewart while studying at the Centre * for Advanced Internet Architectures, Swinburne University of Technology, made * possible in part by a grant from the Cisco University Research Program Fund * at Community Foundation Silicon Valley. * * Portions of this software were developed at the Centre for Advanced * Internet Architectures, Swinburne University of Technology, Melbourne, * Australia by David Hayes under sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * An implementation of the CUBIC congestion control algorithm for FreeBSD, * based on the Internet Draft "draft-rhee-tcpm-cubic-02" by Rhee, Xu and Ha. * Originally released as part of the NewTCP research project at Swinburne * University of Technology's Centre for Advanced Internet Architectures, * Melbourne, Australia, which was made possible in part by a grant from the * Cisco University Research Program Fund at Community Foundation Silicon * Valley. More details are available at: * http://caia.swin.edu.au/urp/newtcp/ */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include static void cubic_ack_received(struct cc_var *ccv, uint16_t type); static void cubic_cb_destroy(struct cc_var *ccv); static int cubic_cb_init(struct cc_var *ccv); static void cubic_cong_signal(struct cc_var *ccv, uint32_t type); static void cubic_conn_init(struct cc_var *ccv); static int cubic_mod_init(void); static void cubic_post_recovery(struct cc_var *ccv); static void cubic_record_rtt(struct cc_var *ccv); static void cubic_ssthresh_update(struct cc_var *ccv); struct cubic { /* Cubic K in fixed point form with CUBIC_SHIFT worth of precision. */ int64_t K; /* Sum of RTT samples across an epoch in ticks. */ int64_t sum_rtt_ticks; /* cwnd at the most recent congestion event. */ unsigned long max_cwnd; /* cwnd at the previous congestion event. */ unsigned long prev_max_cwnd; /* Number of congestion events. */ uint32_t num_cong_events; /* Minimum observed rtt in ticks. */ int min_rtt_ticks; /* Mean observed rtt between congestion epochs. */ int mean_rtt_ticks; /* ACKs since last congestion event. */ int epoch_ack_count; /* Time of last congestion event in ticks. */ int t_last_cong; }; static MALLOC_DEFINE(M_CUBIC, "cubic data", "Per connection data required for the CUBIC congestion control algorithm"); struct cc_algo cubic_cc_algo = { .name = "cubic", .ack_received = cubic_ack_received, .cb_destroy = cubic_cb_destroy, .cb_init = cubic_cb_init, .cong_signal = cubic_cong_signal, .conn_init = cubic_conn_init, .mod_init = cubic_mod_init, .post_recovery = cubic_post_recovery, }; static void cubic_ack_received(struct cc_var *ccv, uint16_t type) { struct cubic *cubic_data; unsigned long w_tf, w_cubic_next; int ticks_since_cong; cubic_data = ccv->cc_data; cubic_record_rtt(ccv); /* * Regular ACK and we're not in cong/fast recovery and we're cwnd * limited and we're either not doing ABC or are slow starting or are * doing ABC and we've sent a cwnd's worth of bytes. */ if (type == CC_ACK && !IN_RECOVERY(CCV(ccv, t_flags)) && (ccv->flags & CCF_CWND_LIMITED) && (!V_tcp_do_rfc3465 || CCV(ccv, snd_cwnd) <= CCV(ccv, snd_ssthresh) || (V_tcp_do_rfc3465 && ccv->flags & CCF_ABC_SENTAWND))) { /* Use the logic in NewReno ack_received() for slow start. */ if (CCV(ccv, snd_cwnd) <= CCV(ccv, snd_ssthresh) || cubic_data->min_rtt_ticks == TCPTV_SRTTBASE) newreno_cc_algo.ack_received(ccv, type); else { ticks_since_cong = ticks - cubic_data->t_last_cong; /* * The mean RTT is used to best reflect the equations in * the I-D. Using min_rtt in the tf_cwnd calculation * causes w_tf to grow much faster than it should if the * RTT is dominated by network buffering rather than * propogation delay. */ w_tf = tf_cwnd(ticks_since_cong, cubic_data->mean_rtt_ticks, cubic_data->max_cwnd, CCV(ccv, t_maxseg)); w_cubic_next = cubic_cwnd(ticks_since_cong + cubic_data->mean_rtt_ticks, cubic_data->max_cwnd, CCV(ccv, t_maxseg), cubic_data->K); ccv->flags &= ~CCF_ABC_SENTAWND; if (w_cubic_next < w_tf) /* * TCP-friendly region, follow tf * cwnd growth. */ CCV(ccv, snd_cwnd) = w_tf; else if (CCV(ccv, snd_cwnd) < w_cubic_next) { /* * Concave or convex region, follow CUBIC * cwnd growth. */ if (V_tcp_do_rfc3465) CCV(ccv, snd_cwnd) = w_cubic_next; else CCV(ccv, snd_cwnd) += ((w_cubic_next - CCV(ccv, snd_cwnd)) * CCV(ccv, t_maxseg)) / CCV(ccv, snd_cwnd); } /* * If we're not in slow start and we're probing for a * new cwnd limit at the start of a connection * (happens when hostcache has a relevant entry), * keep updating our current estimate of the * max_cwnd. */ if (cubic_data->num_cong_events == 0 && cubic_data->max_cwnd < CCV(ccv, snd_cwnd)) cubic_data->max_cwnd = CCV(ccv, snd_cwnd); } } } static void cubic_cb_destroy(struct cc_var *ccv) { if (ccv->cc_data != NULL) free(ccv->cc_data, M_CUBIC); } static int cubic_cb_init(struct cc_var *ccv) { struct cubic *cubic_data; cubic_data = malloc(sizeof(struct cubic), M_CUBIC, M_NOWAIT|M_ZERO); if (cubic_data == NULL) return (ENOMEM); /* Init some key variables with sensible defaults. */ cubic_data->t_last_cong = ticks; cubic_data->min_rtt_ticks = TCPTV_SRTTBASE; cubic_data->mean_rtt_ticks = 1; ccv->cc_data = cubic_data; return (0); } /* * Perform any necessary tasks before we enter congestion recovery. */ static void cubic_cong_signal(struct cc_var *ccv, uint32_t type) { struct cubic *cubic_data; cubic_data = ccv->cc_data; switch (type) { case CC_NDUPACK: if (!IN_FASTRECOVERY(CCV(ccv, t_flags))) { if (!IN_CONGRECOVERY(CCV(ccv, t_flags))) { cubic_ssthresh_update(ccv); cubic_data->num_cong_events++; cubic_data->prev_max_cwnd = cubic_data->max_cwnd; cubic_data->max_cwnd = CCV(ccv, snd_cwnd); } ENTER_RECOVERY(CCV(ccv, t_flags)); } break; case CC_ECN: if (!IN_CONGRECOVERY(CCV(ccv, t_flags))) { cubic_ssthresh_update(ccv); cubic_data->num_cong_events++; cubic_data->prev_max_cwnd = cubic_data->max_cwnd; cubic_data->max_cwnd = CCV(ccv, snd_cwnd); cubic_data->t_last_cong = ticks; CCV(ccv, snd_cwnd) = CCV(ccv, snd_ssthresh); ENTER_CONGRECOVERY(CCV(ccv, t_flags)); } break; case CC_RTO: /* * Grab the current time and record it so we know when the * most recent congestion event was. Only record it when the * timeout has fired more than once, as there is a reasonable * chance the first one is a false alarm and may not indicate * congestion. */ if (CCV(ccv, t_rxtshift) >= 2) cubic_data->num_cong_events++; cubic_data->t_last_cong = ticks; break; } } static void cubic_conn_init(struct cc_var *ccv) { struct cubic *cubic_data; cubic_data = ccv->cc_data; /* * Ensure we have a sane initial value for max_cwnd recorded. Without * this here bad things happen when entries from the TCP hostcache * get used. */ cubic_data->max_cwnd = CCV(ccv, snd_cwnd); } static int cubic_mod_init(void) { cubic_cc_algo.after_idle = newreno_cc_algo.after_idle; return (0); } /* * Perform any necessary tasks before we exit congestion recovery. */ static void cubic_post_recovery(struct cc_var *ccv) { struct cubic *cubic_data; + int pipe; cubic_data = ccv->cc_data; + pipe = 0; /* Fast convergence heuristic. */ if (cubic_data->max_cwnd < cubic_data->prev_max_cwnd) cubic_data->max_cwnd = (cubic_data->max_cwnd * CUBIC_FC_FACTOR) >> CUBIC_SHIFT; if (IN_FASTRECOVERY(CCV(ccv, t_flags))) { /* * If inflight data is less than ssthresh, set cwnd * conservatively to avoid a burst of data, as suggested in * the NewReno RFC. Otherwise, use the CUBIC method. * * XXXLAS: Find a way to do this without needing curack */ - if (SEQ_GT(ccv->curack + CCV(ccv, snd_ssthresh), - CCV(ccv, snd_max))) - CCV(ccv, snd_cwnd) = CCV(ccv, snd_max) - ccv->curack + - CCV(ccv, t_maxseg); + if (V_tcp_do_rfc6675_pipe) + pipe = tcp_compute_pipe(ccv->ccvc.tcp); + else + pipe = CCV(ccv, snd_max) - ccv->curack; + + if (pipe < CCV(ccv, snd_ssthresh)) + CCV(ccv, snd_cwnd) = pipe + CCV(ccv, t_maxseg); else /* Update cwnd based on beta and adjusted max_cwnd. */ CCV(ccv, snd_cwnd) = max(1, ((CUBIC_BETA * cubic_data->max_cwnd) >> CUBIC_SHIFT)); } cubic_data->t_last_cong = ticks; /* Calculate the average RTT between congestion epochs. */ if (cubic_data->epoch_ack_count > 0 && cubic_data->sum_rtt_ticks >= cubic_data->epoch_ack_count) { cubic_data->mean_rtt_ticks = (int)(cubic_data->sum_rtt_ticks / cubic_data->epoch_ack_count); } cubic_data->epoch_ack_count = 0; cubic_data->sum_rtt_ticks = 0; cubic_data->K = cubic_k(cubic_data->max_cwnd / CCV(ccv, t_maxseg)); } /* * Record the min RTT and sum samples for the epoch average RTT calculation. */ static void cubic_record_rtt(struct cc_var *ccv) { struct cubic *cubic_data; int t_srtt_ticks; /* Ignore srtt until a min number of samples have been taken. */ if (CCV(ccv, t_rttupdated) >= CUBIC_MIN_RTT_SAMPLES) { cubic_data = ccv->cc_data; t_srtt_ticks = CCV(ccv, t_srtt) / TCP_RTT_SCALE; /* * Record the current SRTT as our minrtt if it's the smallest * we've seen or minrtt is currently equal to its initialised * value. * * XXXLAS: Should there be some hysteresis for minrtt? */ if ((t_srtt_ticks < cubic_data->min_rtt_ticks || cubic_data->min_rtt_ticks == TCPTV_SRTTBASE)) { cubic_data->min_rtt_ticks = max(1, t_srtt_ticks); /* * If the connection is within its first congestion * epoch, ensure we prime mean_rtt_ticks with a * reasonable value until the epoch average RTT is * calculated in cubic_post_recovery(). */ if (cubic_data->min_rtt_ticks > cubic_data->mean_rtt_ticks) cubic_data->mean_rtt_ticks = cubic_data->min_rtt_ticks; } /* Sum samples for epoch average RTT calculation. */ cubic_data->sum_rtt_ticks += t_srtt_ticks; cubic_data->epoch_ack_count++; } } /* * Update the ssthresh in the event of congestion. */ static void cubic_ssthresh_update(struct cc_var *ccv) { struct cubic *cubic_data; cubic_data = ccv->cc_data; /* * On the first congestion event, set ssthresh to cwnd * 0.5, on * subsequent congestion events, set it to cwnd * beta. */ if (cubic_data->num_cong_events == 0) CCV(ccv, snd_ssthresh) = CCV(ccv, snd_cwnd) >> 1; else CCV(ccv, snd_ssthresh) = (CCV(ccv, snd_cwnd) * CUBIC_BETA) >> CUBIC_SHIFT; } DECLARE_CC_MODULE(cubic, &cubic_cc_algo); Index: user/ngie/more-tests2/sys/netinet/cc/cc_newreno.c =================================================================== --- user/ngie/more-tests2/sys/netinet/cc/cc_newreno.c (revision 292053) +++ user/ngie/more-tests2/sys/netinet/cc/cc_newreno.c (revision 292054) @@ -1,237 +1,243 @@ /*- * Copyright (c) 1982, 1986, 1988, 1990, 1993, 1994, 1995 * The Regents of the University of California. * Copyright (c) 2007-2008,2010 * Swinburne University of Technology, Melbourne, Australia. * Copyright (c) 2009-2010 Lawrence Stewart * Copyright (c) 2010 The FreeBSD Foundation * All rights reserved. * * This software was developed at the Centre for Advanced Internet * Architectures, Swinburne University of Technology, by Lawrence Stewart, James * Healy and David Hayes, made possible in part by a grant from the Cisco * University Research Program Fund at Community Foundation Silicon Valley. * * Portions of this software were developed at the Centre for Advanced * Internet Architectures, Swinburne University of Technology, Melbourne, * Australia by David Hayes under sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * This software was first released in 2007 by James Healy and Lawrence Stewart * whilst working on the NewTCP research project at Swinburne University of * Technology's Centre for Advanced Internet Architectures, Melbourne, * Australia, which was made possible in part by a grant from the Cisco * University Research Program Fund at Community Foundation Silicon Valley. * More details are available at: * http://caia.swin.edu.au/urp/newtcp/ */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include static void newreno_ack_received(struct cc_var *ccv, uint16_t type); static void newreno_after_idle(struct cc_var *ccv); static void newreno_cong_signal(struct cc_var *ccv, uint32_t type); static void newreno_post_recovery(struct cc_var *ccv); struct cc_algo newreno_cc_algo = { .name = "newreno", .ack_received = newreno_ack_received, .after_idle = newreno_after_idle, .cong_signal = newreno_cong_signal, .post_recovery = newreno_post_recovery, }; static void newreno_ack_received(struct cc_var *ccv, uint16_t type) { if (type == CC_ACK && !IN_RECOVERY(CCV(ccv, t_flags)) && (ccv->flags & CCF_CWND_LIMITED)) { u_int cw = CCV(ccv, snd_cwnd); u_int incr = CCV(ccv, t_maxseg); /* * Regular in-order ACK, open the congestion window. * Method depends on which congestion control state we're * in (slow start or cong avoid) and if ABC (RFC 3465) is * enabled. * * slow start: cwnd <= ssthresh * cong avoid: cwnd > ssthresh * * slow start and ABC (RFC 3465): * Grow cwnd exponentially by the amount of data * ACKed capping the max increment per ACK to * (abc_l_var * maxseg) bytes. * * slow start without ABC (RFC 5681): * Grow cwnd exponentially by maxseg per ACK. * * cong avoid and ABC (RFC 3465): * Grow cwnd linearly by maxseg per RTT for each * cwnd worth of ACKed data. * * cong avoid without ABC (RFC 5681): * Grow cwnd linearly by approximately maxseg per RTT using * maxseg^2 / cwnd per ACK as the increment. * If cwnd > maxseg^2, fix the cwnd increment at 1 byte to * avoid capping cwnd. */ if (cw > CCV(ccv, snd_ssthresh)) { if (V_tcp_do_rfc3465) { if (ccv->flags & CCF_ABC_SENTAWND) ccv->flags &= ~CCF_ABC_SENTAWND; else incr = 0; } else incr = max((incr * incr / cw), 1); } else if (V_tcp_do_rfc3465) { /* * In slow-start with ABC enabled and no RTO in sight? * (Must not use abc_l_var > 1 if slow starting after * an RTO. On RTO, snd_nxt = snd_una, so the * snd_nxt == snd_max check is sufficient to * handle this). * * XXXLAS: Find a way to signal SS after RTO that * doesn't rely on tcpcb vars. */ if (CCV(ccv, snd_nxt) == CCV(ccv, snd_max)) incr = min(ccv->bytes_this_ack, V_tcp_abc_l_var * CCV(ccv, t_maxseg)); else incr = min(ccv->bytes_this_ack, CCV(ccv, t_maxseg)); } /* ABC is on by default, so incr equals 0 frequently. */ if (incr > 0) CCV(ccv, snd_cwnd) = min(cw + incr, TCP_MAXWIN << CCV(ccv, snd_scale)); } } static void newreno_after_idle(struct cc_var *ccv) { int rw; /* * If we've been idle for more than one retransmit timeout the old * congestion window is no longer current and we have to reduce it to * the restart window before we can transmit again. * * The restart window is the initial window or the last CWND, whichever * is smaller. * * This is done to prevent us from flooding the path with a full CWND at * wirespeed, overloading router and switch buffers along the way. * * See RFC5681 Section 4.1. "Restarting Idle Connections". */ if (V_tcp_do_rfc3390) rw = min(4 * CCV(ccv, t_maxseg), max(2 * CCV(ccv, t_maxseg), 4380)); else rw = CCV(ccv, t_maxseg) * 2; CCV(ccv, snd_cwnd) = min(rw, CCV(ccv, snd_cwnd)); } /* * Perform any necessary tasks before we enter congestion recovery. */ static void newreno_cong_signal(struct cc_var *ccv, uint32_t type) { u_int win; /* Catch algos which mistakenly leak private signal types. */ KASSERT((type & CC_SIGPRIVMASK) == 0, ("%s: congestion signal type 0x%08x is private\n", __func__, type)); win = max(CCV(ccv, snd_cwnd) / 2 / CCV(ccv, t_maxseg), 2) * CCV(ccv, t_maxseg); switch (type) { case CC_NDUPACK: if (!IN_FASTRECOVERY(CCV(ccv, t_flags))) { if (!IN_CONGRECOVERY(CCV(ccv, t_flags))) CCV(ccv, snd_ssthresh) = win; ENTER_RECOVERY(CCV(ccv, t_flags)); } break; case CC_ECN: if (!IN_CONGRECOVERY(CCV(ccv, t_flags))) { CCV(ccv, snd_ssthresh) = win; CCV(ccv, snd_cwnd) = win; ENTER_CONGRECOVERY(CCV(ccv, t_flags)); } break; } } /* * Perform any necessary tasks before we exit congestion recovery. */ static void newreno_post_recovery(struct cc_var *ccv) { + int pipe; + pipe = 0; + if (IN_FASTRECOVERY(CCV(ccv, t_flags))) { /* * Fast recovery will conclude after returning from this * function. Window inflation should have left us with * approximately snd_ssthresh outstanding data. But in case we * would be inclined to send a burst, better to do it via the * slow start mechanism. * * XXXLAS: Find a way to do this without needing curack */ - if (SEQ_GT(ccv->curack + CCV(ccv, snd_ssthresh), - CCV(ccv, snd_max))) - CCV(ccv, snd_cwnd) = CCV(ccv, snd_max) - - ccv->curack + CCV(ccv, t_maxseg); + if (V_tcp_do_rfc6675_pipe) + pipe = tcp_compute_pipe(ccv->ccvc.tcp); + else + pipe = CCV(ccv, snd_max) - ccv->curack; + + if (pipe < CCV(ccv, snd_ssthresh)) + CCV(ccv, snd_cwnd) = pipe + CCV(ccv, t_maxseg); else CCV(ccv, snd_cwnd) = CCV(ccv, snd_ssthresh); } } DECLARE_CC_MODULE(newreno, &newreno_cc_algo); Index: user/ngie/more-tests2/sys/netinet/if_ether.c =================================================================== --- user/ngie/more-tests2/sys/netinet/if_ether.c (revision 292053) +++ user/ngie/more-tests2/sys/netinet/if_ether.c (revision 292054) @@ -1,1235 +1,1227 @@ /*- * Copyright (c) 1982, 1986, 1988, 1993 * The Regents of the University of California. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * @(#)if_ether.c 8.1 (Berkeley) 6/10/93 */ /* * Ethernet address resolution protocol. * TODO: * add "inuse/lock" bit (or ref. count) along with valid bit */ #include __FBSDID("$FreeBSD$"); #include "opt_inet.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include +#include #include #include #include #ifdef INET #include #endif #include #define SIN(s) ((const struct sockaddr_in *)(s)) static struct timeval arp_lastlog; static int arp_curpps; static int arp_maxpps = 1; /* Simple ARP state machine */ enum arp_llinfo_state { ARP_LLINFO_INCOMPLETE = 0, /* No LLE data */ ARP_LLINFO_REACHABLE, /* LLE is valid */ ARP_LLINFO_VERIFY, /* LLE is valid, need refresh */ ARP_LLINFO_DELETED, /* LLE is deleted */ }; SYSCTL_DECL(_net_link_ether); static SYSCTL_NODE(_net_link_ether, PF_INET, inet, CTLFLAG_RW, 0, ""); static SYSCTL_NODE(_net_link_ether, PF_ARP, arp, CTLFLAG_RW, 0, ""); /* timer values */ static VNET_DEFINE(int, arpt_keep) = (20*60); /* once resolved, good for 20 * minutes */ static VNET_DEFINE(int, arp_maxtries) = 5; static VNET_DEFINE(int, arp_proxyall) = 0; static VNET_DEFINE(int, arpt_down) = 20; /* keep incomplete entries for * 20 seconds */ static VNET_DEFINE(int, arpt_rexmit) = 1; /* retransmit arp entries, sec*/ VNET_PCPUSTAT_DEFINE(struct arpstat, arpstat); /* ARP statistics, see if_arp.h */ VNET_PCPUSTAT_SYSINIT(arpstat); #ifdef VIMAGE VNET_PCPUSTAT_SYSUNINIT(arpstat); #endif /* VIMAGE */ static VNET_DEFINE(int, arp_maxhold) = 1; #define V_arpt_keep VNET(arpt_keep) #define V_arpt_down VNET(arpt_down) #define V_arpt_rexmit VNET(arpt_rexmit) #define V_arp_maxtries VNET(arp_maxtries) #define V_arp_proxyall VNET(arp_proxyall) #define V_arp_maxhold VNET(arp_maxhold) SYSCTL_INT(_net_link_ether_inet, OID_AUTO, max_age, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(arpt_keep), 0, "ARP entry lifetime in seconds"); SYSCTL_INT(_net_link_ether_inet, OID_AUTO, maxtries, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(arp_maxtries), 0, "ARP resolution attempts before returning error"); SYSCTL_INT(_net_link_ether_inet, OID_AUTO, proxyall, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(arp_proxyall), 0, "Enable proxy ARP for all suitable requests"); SYSCTL_INT(_net_link_ether_inet, OID_AUTO, wait, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(arpt_down), 0, "Incomplete ARP entry lifetime in seconds"); SYSCTL_VNET_PCPUSTAT(_net_link_ether_arp, OID_AUTO, stats, struct arpstat, arpstat, "ARP statistics (struct arpstat, net/if_arp.h)"); SYSCTL_INT(_net_link_ether_inet, OID_AUTO, maxhold, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(arp_maxhold), 0, "Number of packets to hold per ARP entry"); SYSCTL_INT(_net_link_ether_inet, OID_AUTO, max_log_per_second, CTLFLAG_RW, &arp_maxpps, 0, "Maximum number of remotely triggered ARP messages that can be " "logged per second"); #define ARP_LOG(pri, ...) do { \ if (ppsratecheck(&arp_lastlog, &arp_curpps, arp_maxpps)) \ log((pri), "arp: " __VA_ARGS__); \ } while (0) static void arp_init(void); static void arpintr(struct mbuf *); static void arptimer(void *); #ifdef INET static void in_arpinput(struct mbuf *); #endif static void arp_check_update_lle(struct arphdr *ah, struct in_addr isaddr, struct ifnet *ifp, int bridged, struct llentry *la); static void arp_mark_lle_reachable(struct llentry *la); static void arp_iflladdr(void *arg __unused, struct ifnet *ifp); static eventhandler_tag iflladdr_tag; static const struct netisr_handler arp_nh = { .nh_name = "arp", .nh_handler = arpintr, .nh_proto = NETISR_ARP, .nh_policy = NETISR_POLICY_SOURCE, }; /* * Timeout routine. Age arp_tab entries periodically. */ static void arptimer(void *arg) { struct llentry *lle = (struct llentry *)arg; struct ifnet *ifp; int r_skip_req; if (lle->la_flags & LLE_STATIC) { return; } LLE_WLOCK(lle); if (callout_pending(&lle->lle_timer)) { /* * Here we are a bit odd here in the treatment of * active/pending. If the pending bit is set, it got * rescheduled before I ran. The active * bit we ignore, since if it was stopped * in ll_tablefree() and was currently running * it would have return 0 so the code would * not have deleted it since the callout could * not be stopped so we want to go through * with the delete here now. If the callout * was restarted, the pending bit will be back on and * we just want to bail since the callout_reset would * return 1 and our reference would have been removed * by arpresolve() below. */ LLE_WUNLOCK(lle); return; } ifp = lle->lle_tbl->llt_ifp; CURVNET_SET(ifp->if_vnet); switch (lle->ln_state) { case ARP_LLINFO_REACHABLE: /* * Expiration time is approaching. * Let's try to refresh entry if it is still * in use. * * Set r_skip_req to get feedback from * fast path. Change state and re-schedule * ourselves. */ LLE_REQ_LOCK(lle); lle->r_skip_req = 1; LLE_REQ_UNLOCK(lle); lle->ln_state = ARP_LLINFO_VERIFY; callout_schedule(&lle->lle_timer, hz * V_arpt_rexmit); LLE_WUNLOCK(lle); CURVNET_RESTORE(); return; case ARP_LLINFO_VERIFY: LLE_REQ_LOCK(lle); r_skip_req = lle->r_skip_req; LLE_REQ_UNLOCK(lle); if (r_skip_req == 0 && lle->la_preempt > 0) { /* Entry was used, issue refresh request */ struct in_addr dst; dst = lle->r_l3addr.addr4; lle->la_preempt--; callout_schedule(&lle->lle_timer, hz * V_arpt_rexmit); LLE_WUNLOCK(lle); arprequest(ifp, NULL, &dst, NULL); CURVNET_RESTORE(); return; } /* Nothing happened. Reschedule if not too late */ if (lle->la_expire > time_uptime) { callout_schedule(&lle->lle_timer, hz * V_arpt_rexmit); LLE_WUNLOCK(lle); CURVNET_RESTORE(); return; } break; case ARP_LLINFO_INCOMPLETE: case ARP_LLINFO_DELETED: break; } if ((lle->la_flags & LLE_DELETED) == 0) { int evt; if (lle->la_flags & LLE_VALID) evt = LLENTRY_EXPIRED; else evt = LLENTRY_TIMEDOUT; EVENTHANDLER_INVOKE(lle_event, lle, evt); } callout_stop(&lle->lle_timer); /* XXX: LOR avoidance. We still have ref on lle. */ LLE_WUNLOCK(lle); IF_AFDATA_LOCK(ifp); LLE_WLOCK(lle); /* Guard against race with other llentry_free(). */ if (lle->la_flags & LLE_LINKED) { LLE_REMREF(lle); lltable_unlink_entry(lle->lle_tbl, lle); } IF_AFDATA_UNLOCK(ifp); size_t pkts_dropped = llentry_free(lle); ARPSTAT_ADD(dropped, pkts_dropped); ARPSTAT_INC(timeouts); CURVNET_RESTORE(); } /* * Broadcast an ARP request. Caller specifies: * - arp header source ip address * - arp header target ip address * - arp header source ethernet address */ void arprequest(struct ifnet *ifp, const struct in_addr *sip, const struct in_addr *tip, u_char *enaddr) { struct mbuf *m; struct arphdr *ah; struct sockaddr sa; u_char *carpaddr = NULL; if (sip == NULL) { /* * The caller did not supply a source address, try to find * a compatible one among those assigned to this interface. */ struct ifaddr *ifa; IF_ADDR_RLOCK(ifp); TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) { if (ifa->ifa_addr->sa_family != AF_INET) continue; if (ifa->ifa_carp) { if ((*carp_iamatch_p)(ifa, &carpaddr) == 0) continue; sip = &IA_SIN(ifa)->sin_addr; } else { carpaddr = NULL; sip = &IA_SIN(ifa)->sin_addr; } if (0 == ((sip->s_addr ^ tip->s_addr) & IA_MASKSIN(ifa)->sin_addr.s_addr)) break; /* found it. */ } IF_ADDR_RUNLOCK(ifp); if (sip == NULL) { printf("%s: cannot find matching address\n", __func__); return; } } if (enaddr == NULL) enaddr = carpaddr ? carpaddr : (u_char *)IF_LLADDR(ifp); if ((m = m_gethdr(M_NOWAIT, MT_DATA)) == NULL) return; m->m_len = sizeof(*ah) + 2 * sizeof(struct in_addr) + 2 * ifp->if_addrlen; m->m_pkthdr.len = m->m_len; M_ALIGN(m, m->m_len); ah = mtod(m, struct arphdr *); bzero((caddr_t)ah, m->m_len); #ifdef MAC mac_netinet_arp_send(ifp, m); #endif ah->ar_pro = htons(ETHERTYPE_IP); ah->ar_hln = ifp->if_addrlen; /* hardware address length */ ah->ar_pln = sizeof(struct in_addr); /* protocol address length */ ah->ar_op = htons(ARPOP_REQUEST); bcopy(enaddr, ar_sha(ah), ah->ar_hln); bcopy(sip, ar_spa(ah), ah->ar_pln); bcopy(tip, ar_tpa(ah), ah->ar_pln); sa.sa_family = AF_ARP; sa.sa_len = 2; m->m_flags |= M_BCAST; m_clrprotoflags(m); /* Avoid confusing lower layers. */ (*ifp->if_output)(ifp, m, &sa, NULL); ARPSTAT_INC(txrequests); } /* * Resolve an IP address into an ethernet address - heavy version. * Used internally by arpresolve(). * We have already checked than we can't use existing lle without * modification so we have to acquire LLE_EXCLUSIVE lle lock. * * On success, desten and flags are filled in and the function returns 0; * If the packet must be held pending resolution, we return EWOULDBLOCK * On other errors, we return the corresponding error code. * Note that m_freem() handles NULL. */ static int arpresolve_full(struct ifnet *ifp, int is_gw, int create, struct mbuf *m, const struct sockaddr *dst, u_char *desten, uint32_t *pflags) { struct llentry *la = NULL, *la_tmp; struct mbuf *curr = NULL; struct mbuf *next = NULL; int error, renew; if (pflags != NULL) *pflags = 0; if (create == 0) { IF_AFDATA_RLOCK(ifp); la = lla_lookup(LLTABLE(ifp), LLE_EXCLUSIVE, dst); IF_AFDATA_RUNLOCK(ifp); } if (la == NULL && (ifp->if_flags & (IFF_NOARP | IFF_STATICARP)) == 0) { la = lltable_alloc_entry(LLTABLE(ifp), 0, dst); if (la == NULL) { log(LOG_DEBUG, "arpresolve: can't allocate llinfo for %s on %s\n", inet_ntoa(SIN(dst)->sin_addr), if_name(ifp)); m_freem(m); return (EINVAL); } IF_AFDATA_WLOCK(ifp); LLE_WLOCK(la); la_tmp = lla_lookup(LLTABLE(ifp), LLE_EXCLUSIVE, dst); /* Prefer ANY existing lle over newly-created one */ if (la_tmp == NULL) lltable_link_entry(LLTABLE(ifp), la); IF_AFDATA_WUNLOCK(ifp); if (la_tmp != NULL) { lltable_free_entry(LLTABLE(ifp), la); la = la_tmp; } } if (la == NULL) { m_freem(m); return (EINVAL); } if ((la->la_flags & LLE_VALID) && ((la->la_flags & LLE_STATIC) || la->la_expire > time_uptime)) { bcopy(&la->ll_addr, desten, ifp->if_addrlen); /* Check if we have feedback request from arptimer() */ if (la->r_skip_req != 0) { LLE_REQ_LOCK(la); la->r_skip_req = 0; /* Notify that entry was used */ LLE_REQ_UNLOCK(la); } if (pflags != NULL) *pflags = la->la_flags & (LLE_VALID|LLE_IFADDR); LLE_WUNLOCK(la); return (0); } renew = (la->la_asked == 0 || la->la_expire != time_uptime); /* * There is an arptab entry, but no ethernet address * response yet. Add the mbuf to the list, dropping * the oldest packet if we have exceeded the system * setting. */ if (m != NULL) { if (la->la_numheld >= V_arp_maxhold) { if (la->la_hold != NULL) { next = la->la_hold->m_nextpkt; m_freem(la->la_hold); la->la_hold = next; la->la_numheld--; ARPSTAT_INC(dropped); } } if (la->la_hold != NULL) { curr = la->la_hold; while (curr->m_nextpkt != NULL) curr = curr->m_nextpkt; curr->m_nextpkt = m; } else la->la_hold = m; la->la_numheld++; } /* * Return EWOULDBLOCK if we have tried less than arp_maxtries. It * will be masked by ether_output(). Return EHOSTDOWN/EHOSTUNREACH * if we have already sent arp_maxtries ARP requests. Retransmit the * ARP request, but not faster than one request per second. */ if (la->la_asked < V_arp_maxtries) error = EWOULDBLOCK; /* First request. */ else error = is_gw != 0 ? EHOSTUNREACH : EHOSTDOWN; if (renew) { int canceled; LLE_ADDREF(la); la->la_expire = time_uptime; canceled = callout_reset(&la->lle_timer, hz * V_arpt_down, arptimer, la); if (canceled) LLE_REMREF(la); la->la_asked++; LLE_WUNLOCK(la); arprequest(ifp, NULL, &SIN(dst)->sin_addr, NULL); return (error); } LLE_WUNLOCK(la); return (error); } /* * Resolve an IP address into an ethernet address. * On input: * ifp is the interface we use * is_gw != 0 if @dst represents gateway to some destination * m is the mbuf. May be NULL if we don't have a packet. * dst is the next hop, * desten is the storage to put LL address. * flags returns subset of lle flags: LLE_VALID | LLE_IFADDR * * On success, desten and flags are filled in and the function returns 0; * If the packet must be held pending resolution, we return EWOULDBLOCK * On other errors, we return the corresponding error code. * Note that m_freem() handles NULL. */ int arpresolve(struct ifnet *ifp, int is_gw, struct mbuf *m, const struct sockaddr *dst, u_char *desten, uint32_t *pflags) { struct llentry *la = 0; if (pflags != NULL) *pflags = 0; if (m != NULL) { if (m->m_flags & M_BCAST) { /* broadcast */ (void)memcpy(desten, ifp->if_broadcastaddr, ifp->if_addrlen); return (0); } if (m->m_flags & M_MCAST) { /* multicast */ ETHER_MAP_IP_MULTICAST(&SIN(dst)->sin_addr, desten); return (0); } } IF_AFDATA_RLOCK(ifp); la = lla_lookup(LLTABLE(ifp), LLE_UNLOCKED, dst); if (la != NULL && (la->r_flags & RLLE_VALID) != 0) { /* Entry found, let's copy lle info */ bcopy(&la->ll_addr, desten, ifp->if_addrlen); if (pflags != NULL) *pflags = LLE_VALID | (la->r_flags & RLLE_IFADDR); /* Check if we have feedback request from arptimer() */ if (la->r_skip_req != 0) { LLE_REQ_LOCK(la); la->r_skip_req = 0; /* Notify that entry was used */ LLE_REQ_UNLOCK(la); } IF_AFDATA_RUNLOCK(ifp); return (0); } IF_AFDATA_RUNLOCK(ifp); return (arpresolve_full(ifp, is_gw, 1, m, dst, desten, pflags)); } /* * Common length and type checks are done here, * then the protocol-specific routine is called. */ static void arpintr(struct mbuf *m) { struct arphdr *ar; struct ifnet *ifp; char *layer; int hlen; ifp = m->m_pkthdr.rcvif; if (m->m_len < sizeof(struct arphdr) && ((m = m_pullup(m, sizeof(struct arphdr))) == NULL)) { ARP_LOG(LOG_NOTICE, "packet with short header received on %s\n", if_name(ifp)); return; } ar = mtod(m, struct arphdr *); /* Check if length is sufficient */ if (m->m_len < arphdr_len(ar)) { m = m_pullup(m, arphdr_len(ar)); if (m == NULL) { ARP_LOG(LOG_NOTICE, "short packet received on %s\n", if_name(ifp)); return; } ar = mtod(m, struct arphdr *); } hlen = 0; layer = ""; switch (ntohs(ar->ar_hrd)) { case ARPHRD_ETHER: hlen = ETHER_ADDR_LEN; /* RFC 826 */ layer = "ethernet"; break; case ARPHRD_IEEE802: hlen = 6; /* RFC 1390, FDDI_ADDR_LEN */ layer = "fddi"; break; case ARPHRD_ARCNET: hlen = 1; /* RFC 1201, ARC_ADDR_LEN */ layer = "arcnet"; break; case ARPHRD_INFINIBAND: hlen = 20; /* RFC 4391, INFINIBAND_ALEN */ layer = "infiniband"; break; case ARPHRD_IEEE1394: hlen = 0; /* SHALL be 16 */ /* RFC 2734 */ layer = "firewire"; /* * Restrict too long harware addresses. * Currently we are capable of handling 20-byte * addresses ( sizeof(lle->ll_addr) ) */ if (ar->ar_hln >= 20) hlen = 16; break; default: ARP_LOG(LOG_NOTICE, "packet with unknown harware format 0x%02d received on %s\n", ntohs(ar->ar_hrd), if_name(ifp)); m_freem(m); return; } if (hlen != 0 && hlen != ar->ar_hln) { ARP_LOG(LOG_NOTICE, "packet with invalid %s address length %d received on %s\n", layer, ar->ar_hln, if_name(ifp)); m_freem(m); return; } ARPSTAT_INC(received); switch (ntohs(ar->ar_pro)) { #ifdef INET case ETHERTYPE_IP: in_arpinput(m); return; #endif } m_freem(m); } #ifdef INET /* * ARP for Internet protocols on 10 Mb/s Ethernet. * Algorithm is that given in RFC 826. * In addition, a sanity check is performed on the sender * protocol address, to catch impersonators. * We no longer handle negotiations for use of trailer protocol: * Formerly, ARP replied for protocol type ETHERTYPE_TRAIL sent * along with IP replies if we wanted trailers sent to us, * and also sent them in response to IP replies. * This allowed either end to announce the desire to receive * trailer packets. * We no longer reply to requests for ETHERTYPE_TRAIL protocol either, * but formerly didn't normally send requests. */ static int log_arp_wrong_iface = 1; static int log_arp_movements = 1; static int log_arp_permanent_modify = 1; static int allow_multicast = 0; SYSCTL_INT(_net_link_ether_inet, OID_AUTO, log_arp_wrong_iface, CTLFLAG_RW, &log_arp_wrong_iface, 0, "log arp packets arriving on the wrong interface"); SYSCTL_INT(_net_link_ether_inet, OID_AUTO, log_arp_movements, CTLFLAG_RW, &log_arp_movements, 0, "log arp replies from MACs different than the one in the cache"); SYSCTL_INT(_net_link_ether_inet, OID_AUTO, log_arp_permanent_modify, CTLFLAG_RW, &log_arp_permanent_modify, 0, "log arp replies from MACs different than the one in the permanent arp entry"); SYSCTL_INT(_net_link_ether_inet, OID_AUTO, allow_multicast, CTLFLAG_RW, &allow_multicast, 0, "accept multicast addresses"); static void in_arpinput(struct mbuf *m) { struct rm_priotracker in_ifa_tracker; struct arphdr *ah; struct ifnet *ifp = m->m_pkthdr.rcvif; struct llentry *la = NULL, *la_tmp; - struct rtentry *rt; struct ifaddr *ifa; struct in_ifaddr *ia; struct sockaddr sa; struct in_addr isaddr, itaddr, myaddr; u_int8_t *enaddr = NULL; int op; int bridged = 0, is_bridge = 0; int carped; struct sockaddr_in sin; struct sockaddr *dst; + struct nhop4_basic nh4; + sin.sin_len = sizeof(struct sockaddr_in); sin.sin_family = AF_INET; sin.sin_addr.s_addr = 0; if (ifp->if_bridge) bridged = 1; if (ifp->if_type == IFT_BRIDGE) is_bridge = 1; /* * We already have checked that mbuf contains enough contiguous data * to hold entire arp message according to the arp header. */ ah = mtod(m, struct arphdr *); /* * ARP is only for IPv4 so we can reject packets with * a protocol length not equal to an IPv4 address. */ if (ah->ar_pln != sizeof(struct in_addr)) { ARP_LOG(LOG_NOTICE, "requested protocol length != %zu\n", sizeof(struct in_addr)); goto drop; } if (allow_multicast == 0 && ETHER_IS_MULTICAST(ar_sha(ah))) { ARP_LOG(LOG_NOTICE, "%*D is multicast\n", ifp->if_addrlen, (u_char *)ar_sha(ah), ":"); goto drop; } op = ntohs(ah->ar_op); (void)memcpy(&isaddr, ar_spa(ah), sizeof (isaddr)); (void)memcpy(&itaddr, ar_tpa(ah), sizeof (itaddr)); if (op == ARPOP_REPLY) ARPSTAT_INC(rxreplies); /* * For a bridge, we want to check the address irrespective * of the receive interface. (This will change slightly * when we have clusters of interfaces). */ IN_IFADDR_RLOCK(&in_ifa_tracker); LIST_FOREACH(ia, INADDR_HASH(itaddr.s_addr), ia_hash) { if (((bridged && ia->ia_ifp->if_bridge == ifp->if_bridge) || ia->ia_ifp == ifp) && itaddr.s_addr == ia->ia_addr.sin_addr.s_addr && (ia->ia_ifa.ifa_carp == NULL || (*carp_iamatch_p)(&ia->ia_ifa, &enaddr))) { ifa_ref(&ia->ia_ifa); IN_IFADDR_RUNLOCK(&in_ifa_tracker); goto match; } } LIST_FOREACH(ia, INADDR_HASH(isaddr.s_addr), ia_hash) if (((bridged && ia->ia_ifp->if_bridge == ifp->if_bridge) || ia->ia_ifp == ifp) && isaddr.s_addr == ia->ia_addr.sin_addr.s_addr) { ifa_ref(&ia->ia_ifa); IN_IFADDR_RUNLOCK(&in_ifa_tracker); goto match; } #define BDG_MEMBER_MATCHES_ARP(addr, ifp, ia) \ (ia->ia_ifp->if_bridge == ifp->if_softc && \ !bcmp(IF_LLADDR(ia->ia_ifp), IF_LLADDR(ifp), ifp->if_addrlen) && \ addr == ia->ia_addr.sin_addr.s_addr) /* * Check the case when bridge shares its MAC address with * some of its children, so packets are claimed by bridge * itself (bridge_input() does it first), but they are really * meant to be destined to the bridge member. */ if (is_bridge) { LIST_FOREACH(ia, INADDR_HASH(itaddr.s_addr), ia_hash) { if (BDG_MEMBER_MATCHES_ARP(itaddr.s_addr, ifp, ia)) { ifa_ref(&ia->ia_ifa); ifp = ia->ia_ifp; IN_IFADDR_RUNLOCK(&in_ifa_tracker); goto match; } } } #undef BDG_MEMBER_MATCHES_ARP IN_IFADDR_RUNLOCK(&in_ifa_tracker); /* * No match, use the first inet address on the receive interface * as a dummy address for the rest of the function. */ IF_ADDR_RLOCK(ifp); TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) if (ifa->ifa_addr->sa_family == AF_INET && (ifa->ifa_carp == NULL || (*carp_iamatch_p)(ifa, &enaddr))) { ia = ifatoia(ifa); ifa_ref(ifa); IF_ADDR_RUNLOCK(ifp); goto match; } IF_ADDR_RUNLOCK(ifp); /* * If bridging, fall back to using any inet address. */ IN_IFADDR_RLOCK(&in_ifa_tracker); if (!bridged || (ia = TAILQ_FIRST(&V_in_ifaddrhead)) == NULL) { IN_IFADDR_RUNLOCK(&in_ifa_tracker); goto drop; } ifa_ref(&ia->ia_ifa); IN_IFADDR_RUNLOCK(&in_ifa_tracker); match: if (!enaddr) enaddr = (u_int8_t *)IF_LLADDR(ifp); carped = (ia->ia_ifa.ifa_carp != NULL); myaddr = ia->ia_addr.sin_addr; ifa_free(&ia->ia_ifa); if (!bcmp(ar_sha(ah), enaddr, ifp->if_addrlen)) goto drop; /* it's from me, ignore it. */ if (!bcmp(ar_sha(ah), ifp->if_broadcastaddr, ifp->if_addrlen)) { ARP_LOG(LOG_NOTICE, "link address is broadcast for IP address " "%s!\n", inet_ntoa(isaddr)); goto drop; } if (ifp->if_addrlen != ah->ar_hln) { ARP_LOG(LOG_WARNING, "from %*D: addr len: new %d, " "i/f %d (ignored)\n", ifp->if_addrlen, (u_char *) ar_sha(ah), ":", ah->ar_hln, ifp->if_addrlen); goto drop; } /* * Warn if another host is using the same IP address, but only if the * IP address isn't 0.0.0.0, which is used for DHCP only, in which * case we suppress the warning to avoid false positive complaints of * potential misconfiguration. */ if (!bridged && !carped && isaddr.s_addr == myaddr.s_addr && myaddr.s_addr != 0) { ARP_LOG(LOG_ERR, "%*D is using my IP address %s on %s!\n", ifp->if_addrlen, (u_char *)ar_sha(ah), ":", inet_ntoa(isaddr), ifp->if_xname); itaddr = myaddr; ARPSTAT_INC(dupips); goto reply; } if (ifp->if_flags & IFF_STATICARP) goto reply; bzero(&sin, sizeof(sin)); sin.sin_len = sizeof(struct sockaddr_in); sin.sin_family = AF_INET; sin.sin_addr = isaddr; dst = (struct sockaddr *)&sin; IF_AFDATA_RLOCK(ifp); la = lla_lookup(LLTABLE(ifp), LLE_EXCLUSIVE, dst); IF_AFDATA_RUNLOCK(ifp); if (la != NULL) arp_check_update_lle(ah, isaddr, ifp, bridged, la); else if (itaddr.s_addr == myaddr.s_addr) { /* * Reply to our address, but no lle exists yet. * do we really have to create an entry? */ la = lltable_alloc_entry(LLTABLE(ifp), 0, dst); if (la == NULL) goto drop; lltable_set_entry_addr(ifp, la, ar_sha(ah)); IF_AFDATA_WLOCK(ifp); LLE_WLOCK(la); la_tmp = lla_lookup(LLTABLE(ifp), LLE_EXCLUSIVE, dst); /* * Check if lle still does not exists. * If it does, that means that we either * 1) have configured it explicitly, via * 1a) 'arp -s' static entry or * 1b) interface address static record * or * 2) it was the result of sending first packet to-host * or * 3) it was another arp reply packet we handled in * different thread. * * In all cases except 3) we definitely need to prefer * existing lle. For the sake of simplicity, prefer any * existing lle over newly-create one. */ if (la_tmp == NULL) lltable_link_entry(LLTABLE(ifp), la); IF_AFDATA_WUNLOCK(ifp); if (la_tmp == NULL) { arp_mark_lle_reachable(la); LLE_WUNLOCK(la); } else { /* Free newly-create entry and handle packet */ lltable_free_entry(LLTABLE(ifp), la); la = la_tmp; la_tmp = NULL; arp_check_update_lle(ah, isaddr, ifp, bridged, la); /* arp_check_update_lle() returns @la unlocked */ } la = NULL; } reply: if (op != ARPOP_REQUEST) goto drop; ARPSTAT_INC(rxrequests); if (itaddr.s_addr == myaddr.s_addr) { /* Shortcut.. the receiving interface is the target. */ (void)memcpy(ar_tha(ah), ar_sha(ah), ah->ar_hln); (void)memcpy(ar_sha(ah), enaddr, ah->ar_hln); } else { struct llentry *lle = NULL; sin.sin_addr = itaddr; IF_AFDATA_RLOCK(ifp); lle = lla_lookup(LLTABLE(ifp), 0, (struct sockaddr *)&sin); IF_AFDATA_RUNLOCK(ifp); if ((lle != NULL) && (lle->la_flags & LLE_PUB)) { (void)memcpy(ar_tha(ah), ar_sha(ah), ah->ar_hln); (void)memcpy(ar_sha(ah), &lle->ll_addr, ah->ar_hln); LLE_RUNLOCK(lle); } else { if (lle != NULL) LLE_RUNLOCK(lle); if (!V_arp_proxyall) goto drop; - sin.sin_addr = itaddr; /* XXX MRT use table 0 for arp reply */ - rt = in_rtalloc1((struct sockaddr *)&sin, 0, 0UL, 0); - if (!rt) + if (fib4_lookup_nh_basic(0, itaddr, 0, 0, &nh4) != 0) goto drop; /* * Don't send proxies for nodes on the same interface * as this one came out of, or we'll get into a fight * over who claims what Ether address. */ - if (!rt->rt_ifp || rt->rt_ifp == ifp) { - RTFREE_LOCKED(rt); + if (nh4.nh_ifp == ifp) goto drop; - } - RTFREE_LOCKED(rt); (void)memcpy(ar_tha(ah), ar_sha(ah), ah->ar_hln); (void)memcpy(ar_sha(ah), enaddr, ah->ar_hln); /* * Also check that the node which sent the ARP packet * is on the interface we expect it to be on. This * avoids ARP chaos if an interface is connected to the * wrong network. */ - sin.sin_addr = isaddr; /* XXX MRT use table 0 for arp checks */ - rt = in_rtalloc1((struct sockaddr *)&sin, 0, 0UL, 0); - if (!rt) + if (fib4_lookup_nh_basic(0, isaddr, 0, 0, &nh4) != 0) goto drop; - if (rt->rt_ifp != ifp) { + if (nh4.nh_ifp != ifp) { ARP_LOG(LOG_INFO, "proxy: ignoring request" - " from %s via %s, expecting %s\n", - inet_ntoa(isaddr), ifp->if_xname, - rt->rt_ifp->if_xname); - RTFREE_LOCKED(rt); + " from %s via %s\n", + inet_ntoa(isaddr), ifp->if_xname); goto drop; } - RTFREE_LOCKED(rt); #ifdef DEBUG_PROXY printf("arp: proxying for %s\n", inet_ntoa(itaddr)); #endif } } if (itaddr.s_addr == myaddr.s_addr && IN_LINKLOCAL(ntohl(itaddr.s_addr))) { /* RFC 3927 link-local IPv4; always reply by broadcast. */ #ifdef DEBUG_LINKLOCAL printf("arp: sending reply for link-local addr %s\n", inet_ntoa(itaddr)); #endif m->m_flags |= M_BCAST; m->m_flags &= ~M_MCAST; } else { /* default behaviour; never reply by broadcast. */ m->m_flags &= ~(M_BCAST|M_MCAST); } (void)memcpy(ar_tpa(ah), ar_spa(ah), ah->ar_pln); (void)memcpy(ar_spa(ah), &itaddr, ah->ar_pln); ah->ar_op = htons(ARPOP_REPLY); ah->ar_pro = htons(ETHERTYPE_IP); /* let's be sure! */ m->m_len = sizeof(*ah) + (2 * ah->ar_pln) + (2 * ah->ar_hln); m->m_pkthdr.len = m->m_len; m->m_pkthdr.rcvif = NULL; sa.sa_family = AF_ARP; sa.sa_len = 2; m_clrprotoflags(m); /* Avoid confusing lower layers. */ (*ifp->if_output)(ifp, m, &sa, NULL); ARPSTAT_INC(txreplies); return; drop: m_freem(m); } #endif /* * Checks received arp data against existing @la. * Updates lle state/performs notification if necessary. */ static void arp_check_update_lle(struct arphdr *ah, struct in_addr isaddr, struct ifnet *ifp, int bridged, struct llentry *la) { struct sockaddr sa; struct mbuf *m_hold, *m_hold_next; LLE_WLOCK_ASSERT(la); /* the following is not an error when doing bridging */ if (!bridged && la->lle_tbl->llt_ifp != ifp) { if (log_arp_wrong_iface) ARP_LOG(LOG_WARNING, "%s is on %s " "but got reply from %*D on %s\n", inet_ntoa(isaddr), la->lle_tbl->llt_ifp->if_xname, ifp->if_addrlen, (u_char *)ar_sha(ah), ":", ifp->if_xname); LLE_WUNLOCK(la); return; } if ((la->la_flags & LLE_VALID) && bcmp(ar_sha(ah), &la->ll_addr, ifp->if_addrlen)) { if (la->la_flags & LLE_STATIC) { LLE_WUNLOCK(la); if (log_arp_permanent_modify) ARP_LOG(LOG_ERR, "%*D attempts to modify " "permanent entry for %s on %s\n", ifp->if_addrlen, (u_char *)ar_sha(ah), ":", inet_ntoa(isaddr), ifp->if_xname); return; } if (log_arp_movements) { ARP_LOG(LOG_INFO, "%s moved from %*D " "to %*D on %s\n", inet_ntoa(isaddr), ifp->if_addrlen, (u_char *)&la->ll_addr, ":", ifp->if_addrlen, (u_char *)ar_sha(ah), ":", ifp->if_xname); } } /* Check if something has changed */ if (memcmp(&la->ll_addr, ar_sha(ah), ifp->if_addrlen) != 0 || (la->la_flags & LLE_VALID) == 0) { /* Perform real LLE update */ /* use afdata WLOCK to update fields */ LLE_ADDREF(la); LLE_WUNLOCK(la); IF_AFDATA_WLOCK(ifp); LLE_WLOCK(la); /* * Since we droppped LLE lock, other thread might have deleted * this lle. Check and return */ if ((la->la_flags & LLE_DELETED) != 0) { IF_AFDATA_WUNLOCK(ifp); LLE_FREE_LOCKED(la); return; } /* Update data */ lltable_set_entry_addr(ifp, la, ar_sha(ah)); IF_AFDATA_WUNLOCK(ifp); LLE_REMREF(la); /* Clear fast path feedback request if set */ la->r_skip_req = 0; } arp_mark_lle_reachable(la); /* * The packets are all freed within the call to the output * routine. * * NB: The lock MUST be released before the call to the * output routine. */ if (la->la_hold != NULL) { m_hold = la->la_hold; la->la_hold = NULL; la->la_numheld = 0; lltable_fill_sa_entry(la, &sa); LLE_WUNLOCK(la); for (; m_hold != NULL; m_hold = m_hold_next) { m_hold_next = m_hold->m_nextpkt; m_hold->m_nextpkt = NULL; /* Avoid confusing lower layers. */ m_clrprotoflags(m_hold); (*ifp->if_output)(ifp, m_hold, &sa, NULL); } } else LLE_WUNLOCK(la); } static void arp_mark_lle_reachable(struct llentry *la) { int canceled, wtime; LLE_WLOCK_ASSERT(la); la->ln_state = ARP_LLINFO_REACHABLE; EVENTHANDLER_INVOKE(lle_event, la, LLENTRY_RESOLVED); if (!(la->la_flags & LLE_STATIC)) { LLE_ADDREF(la); la->la_expire = time_uptime + V_arpt_keep; wtime = V_arpt_keep - V_arp_maxtries * V_arpt_rexmit; if (wtime < 0) wtime = V_arpt_keep; canceled = callout_reset(&la->lle_timer, hz * wtime, arptimer, la); if (canceled) LLE_REMREF(la); } la->la_asked = 0; la->la_preempt = V_arp_maxtries; } /* * Add pernament link-layer record for given interface address. */ static __noinline void arp_add_ifa_lle(struct ifnet *ifp, const struct sockaddr *dst) { struct llentry *lle, *lle_tmp; /* * Interface address LLE record is considered static * because kernel code relies on LLE_STATIC flag to check * if these entries can be rewriten by arp updates. */ lle = lltable_alloc_entry(LLTABLE(ifp), LLE_IFADDR | LLE_STATIC, dst); if (lle == NULL) { log(LOG_INFO, "arp_ifinit: cannot create arp " "entry for interface address\n"); return; } IF_AFDATA_WLOCK(ifp); LLE_WLOCK(lle); /* Unlink any entry if exists */ lle_tmp = lla_lookup(LLTABLE(ifp), LLE_EXCLUSIVE, dst); if (lle_tmp != NULL) lltable_unlink_entry(LLTABLE(ifp), lle_tmp); lltable_link_entry(LLTABLE(ifp), lle); IF_AFDATA_WUNLOCK(ifp); if (lle_tmp != NULL) EVENTHANDLER_INVOKE(lle_event, lle_tmp, LLENTRY_EXPIRED); EVENTHANDLER_INVOKE(lle_event, lle, LLENTRY_RESOLVED); LLE_WUNLOCK(lle); if (lle_tmp != NULL) lltable_free_entry(LLTABLE(ifp), lle_tmp); } void arp_ifinit(struct ifnet *ifp, struct ifaddr *ifa) { const struct sockaddr_in *dst_in; const struct sockaddr *dst; if (ifa->ifa_carp != NULL) return; dst = ifa->ifa_addr; dst_in = (const struct sockaddr_in *)dst; if (ntohl(dst_in->sin_addr.s_addr) == INADDR_ANY) return; arp_announce_ifaddr(ifp, dst_in->sin_addr, IF_LLADDR(ifp)); arp_add_ifa_lle(ifp, dst); } void arp_announce_ifaddr(struct ifnet *ifp, struct in_addr addr, u_char *enaddr) { if (ntohl(addr.s_addr) != INADDR_ANY) arprequest(ifp, &addr, &addr, enaddr); } /* * Sends gratuitous ARPs for each ifaddr to notify other * nodes about the address change. */ static __noinline void arp_handle_ifllchange(struct ifnet *ifp) { struct ifaddr *ifa; TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) { if (ifa->ifa_addr->sa_family == AF_INET) arp_ifinit(ifp, ifa); } } /* * A handler for interface link layer address change event. */ static __noinline void arp_iflladdr(void *arg __unused, struct ifnet *ifp) { if ((ifp->if_flags & IFF_UP) != 0) arp_handle_ifllchange(ifp); } static void arp_init(void) { netisr_register(&arp_nh); if (IS_DEFAULT_VNET(curvnet)) iflladdr_tag = EVENTHANDLER_REGISTER(iflladdr_event, arp_iflladdr, NULL, EVENTHANDLER_PRI_ANY); } SYSINIT(arp, SI_SUB_PROTO_DOMAIN, SI_ORDER_ANY, arp_init, 0); Index: user/ngie/more-tests2/sys/netinet/in_mcast.c =================================================================== --- user/ngie/more-tests2/sys/netinet/in_mcast.c (revision 292053) +++ user/ngie/more-tests2/sys/netinet/in_mcast.c (revision 292054) @@ -1,3012 +1,3009 @@ /*- * Copyright (c) 2007-2009 Bruce Simpson. * Copyright (c) 2005 Robert N. M. Watson. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. The name of the author may not be used to endorse or promote * products derived from this software without specific prior written * permission. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * IPv4 multicast socket, group, and socket option processing module. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include +#include #include #include #include #include #ifndef KTR_IGMPV3 #define KTR_IGMPV3 KTR_INET #endif #ifndef __SOCKUNION_DECLARED union sockunion { struct sockaddr_storage ss; struct sockaddr sa; struct sockaddr_dl sdl; struct sockaddr_in sin; }; typedef union sockunion sockunion_t; #define __SOCKUNION_DECLARED #endif /* __SOCKUNION_DECLARED */ static MALLOC_DEFINE(M_INMFILTER, "in_mfilter", "IPv4 multicast PCB-layer source filter"); static MALLOC_DEFINE(M_IPMADDR, "in_multi", "IPv4 multicast group"); static MALLOC_DEFINE(M_IPMOPTS, "ip_moptions", "IPv4 multicast options"); static MALLOC_DEFINE(M_IPMSOURCE, "ip_msource", "IPv4 multicast IGMP-layer source filter"); /* * Locking: * - Lock order is: Giant, INP_WLOCK, IN_MULTI_LOCK, IGMP_LOCK, IF_ADDR_LOCK. * - The IF_ADDR_LOCK is implicitly taken by inm_lookup() earlier, however * it can be taken by code in net/if.c also. * - ip_moptions and in_mfilter are covered by the INP_WLOCK. * * struct in_multi is covered by IN_MULTI_LOCK. There isn't strictly * any need for in_multi itself to be virtualized -- it is bound to an ifp * anyway no matter what happens. */ struct mtx in_multi_mtx; MTX_SYSINIT(in_multi_mtx, &in_multi_mtx, "in_multi_mtx", MTX_DEF); /* * Functions with non-static linkage defined in this file should be * declared in in_var.h: * imo_multi_filter() * in_addmulti() * in_delmulti() * in_joingroup() * in_joingroup_locked() * in_leavegroup() * in_leavegroup_locked() * and ip_var.h: * inp_freemoptions() * inp_getmoptions() * inp_setmoptions() * * XXX: Both carp and pf need to use the legacy (*,G) KPIs in_addmulti() * and in_delmulti(). */ static void imf_commit(struct in_mfilter *); static int imf_get_source(struct in_mfilter *imf, const struct sockaddr_in *psin, struct in_msource **); static struct in_msource * imf_graft(struct in_mfilter *, const uint8_t, const struct sockaddr_in *); static void imf_leave(struct in_mfilter *); static int imf_prune(struct in_mfilter *, const struct sockaddr_in *); static void imf_purge(struct in_mfilter *); static void imf_rollback(struct in_mfilter *); static void imf_reap(struct in_mfilter *); static int imo_grow(struct ip_moptions *); static size_t imo_match_group(const struct ip_moptions *, const struct ifnet *, const struct sockaddr *); static struct in_msource * imo_match_source(const struct ip_moptions *, const size_t, const struct sockaddr *); static void ims_merge(struct ip_msource *ims, const struct in_msource *lims, const int rollback); static int in_getmulti(struct ifnet *, const struct in_addr *, struct in_multi **); static int inm_get_source(struct in_multi *inm, const in_addr_t haddr, const int noalloc, struct ip_msource **pims); #ifdef KTR static int inm_is_ifp_detached(const struct in_multi *); #endif static int inm_merge(struct in_multi *, /*const*/ struct in_mfilter *); static void inm_purge(struct in_multi *); static void inm_reap(struct in_multi *); static struct ip_moptions * inp_findmoptions(struct inpcb *); static void inp_freemoptions_internal(struct ip_moptions *); static void inp_gcmoptions(void *, int); static int inp_get_source_filters(struct inpcb *, struct sockopt *); static int inp_join_group(struct inpcb *, struct sockopt *); static int inp_leave_group(struct inpcb *, struct sockopt *); static struct ifnet * inp_lookup_mcast_ifp(const struct inpcb *, const struct sockaddr_in *, const struct in_addr); static int inp_block_unblock_source(struct inpcb *, struct sockopt *); static int inp_set_multicast_if(struct inpcb *, struct sockopt *); static int inp_set_source_filters(struct inpcb *, struct sockopt *); static int sysctl_ip_mcast_filters(SYSCTL_HANDLER_ARGS); static SYSCTL_NODE(_net_inet_ip, OID_AUTO, mcast, CTLFLAG_RW, 0, "IPv4 multicast"); static u_long in_mcast_maxgrpsrc = IP_MAX_GROUP_SRC_FILTER; SYSCTL_ULONG(_net_inet_ip_mcast, OID_AUTO, maxgrpsrc, CTLFLAG_RWTUN, &in_mcast_maxgrpsrc, 0, "Max source filters per group"); static u_long in_mcast_maxsocksrc = IP_MAX_SOCK_SRC_FILTER; SYSCTL_ULONG(_net_inet_ip_mcast, OID_AUTO, maxsocksrc, CTLFLAG_RWTUN, &in_mcast_maxsocksrc, 0, "Max source filters per socket"); int in_mcast_loop = IP_DEFAULT_MULTICAST_LOOP; SYSCTL_INT(_net_inet_ip_mcast, OID_AUTO, loop, CTLFLAG_RWTUN, &in_mcast_loop, 0, "Loopback multicast datagrams by default"); static SYSCTL_NODE(_net_inet_ip_mcast, OID_AUTO, filters, CTLFLAG_RD | CTLFLAG_MPSAFE, sysctl_ip_mcast_filters, "Per-interface stack-wide source filters"); static STAILQ_HEAD(, ip_moptions) imo_gc_list = STAILQ_HEAD_INITIALIZER(imo_gc_list); static struct task imo_gc_task = TASK_INITIALIZER(0, inp_gcmoptions, NULL); #ifdef KTR /* * Inline function which wraps assertions for a valid ifp. * The ifnet layer will set the ifma's ifp pointer to NULL if the ifp * is detached. */ static int __inline inm_is_ifp_detached(const struct in_multi *inm) { struct ifnet *ifp; KASSERT(inm->inm_ifma != NULL, ("%s: no ifma", __func__)); ifp = inm->inm_ifma->ifma_ifp; if (ifp != NULL) { /* * Sanity check that netinet's notion of ifp is the * same as net's. */ KASSERT(inm->inm_ifp == ifp, ("%s: bad ifp", __func__)); } return (ifp == NULL); } #endif /* * Initialize an in_mfilter structure to a known state at t0, t1 * with an empty source filter list. */ static __inline void imf_init(struct in_mfilter *imf, const int st0, const int st1) { memset(imf, 0, sizeof(struct in_mfilter)); RB_INIT(&imf->imf_sources); imf->imf_st[0] = st0; imf->imf_st[1] = st1; } /* * Function for looking up an in_multi record for an IPv4 multicast address * on a given interface. ifp must be valid. If no record found, return NULL. * The IN_MULTI_LOCK and IF_ADDR_LOCK on ifp must be held. */ struct in_multi * inm_lookup_locked(struct ifnet *ifp, const struct in_addr ina) { struct ifmultiaddr *ifma; struct in_multi *inm; IN_MULTI_LOCK_ASSERT(); IF_ADDR_LOCK_ASSERT(ifp); inm = NULL; TAILQ_FOREACH(ifma, &((ifp)->if_multiaddrs), ifma_link) { if (ifma->ifma_addr->sa_family == AF_INET) { inm = (struct in_multi *)ifma->ifma_protospec; if (inm->inm_addr.s_addr == ina.s_addr) break; inm = NULL; } } return (inm); } /* * Wrapper for inm_lookup_locked(). * The IF_ADDR_LOCK will be taken on ifp and released on return. */ struct in_multi * inm_lookup(struct ifnet *ifp, const struct in_addr ina) { struct in_multi *inm; IN_MULTI_LOCK_ASSERT(); IF_ADDR_RLOCK(ifp); inm = inm_lookup_locked(ifp, ina); IF_ADDR_RUNLOCK(ifp); return (inm); } /* * Resize the ip_moptions vector to the next power-of-two minus 1. * May be called with locks held; do not sleep. */ static int imo_grow(struct ip_moptions *imo) { struct in_multi **nmships; struct in_multi **omships; struct in_mfilter *nmfilters; struct in_mfilter *omfilters; size_t idx; size_t newmax; size_t oldmax; nmships = NULL; nmfilters = NULL; omships = imo->imo_membership; omfilters = imo->imo_mfilters; oldmax = imo->imo_max_memberships; newmax = ((oldmax + 1) * 2) - 1; if (newmax <= IP_MAX_MEMBERSHIPS) { nmships = (struct in_multi **)realloc(omships, sizeof(struct in_multi *) * newmax, M_IPMOPTS, M_NOWAIT); nmfilters = (struct in_mfilter *)realloc(omfilters, sizeof(struct in_mfilter) * newmax, M_INMFILTER, M_NOWAIT); if (nmships != NULL && nmfilters != NULL) { /* Initialize newly allocated source filter heads. */ for (idx = oldmax; idx < newmax; idx++) { imf_init(&nmfilters[idx], MCAST_UNDEFINED, MCAST_EXCLUDE); } imo->imo_max_memberships = newmax; imo->imo_membership = nmships; imo->imo_mfilters = nmfilters; } } if (nmships == NULL || nmfilters == NULL) { if (nmships != NULL) free(nmships, M_IPMOPTS); if (nmfilters != NULL) free(nmfilters, M_INMFILTER); return (ETOOMANYREFS); } return (0); } /* * Find an IPv4 multicast group entry for this ip_moptions instance * which matches the specified group, and optionally an interface. * Return its index into the array, or -1 if not found. */ static size_t imo_match_group(const struct ip_moptions *imo, const struct ifnet *ifp, const struct sockaddr *group) { const struct sockaddr_in *gsin; struct in_multi **pinm; int idx; int nmships; gsin = (const struct sockaddr_in *)group; /* The imo_membership array may be lazy allocated. */ if (imo->imo_membership == NULL || imo->imo_num_memberships == 0) return (-1); nmships = imo->imo_num_memberships; pinm = &imo->imo_membership[0]; for (idx = 0; idx < nmships; idx++, pinm++) { if (*pinm == NULL) continue; if ((ifp == NULL || ((*pinm)->inm_ifp == ifp)) && in_hosteq((*pinm)->inm_addr, gsin->sin_addr)) { break; } } if (idx >= nmships) idx = -1; return (idx); } /* * Find an IPv4 multicast source entry for this imo which matches * the given group index for this socket, and source address. * * NOTE: This does not check if the entry is in-mode, merely if * it exists, which may not be the desired behaviour. */ static struct in_msource * imo_match_source(const struct ip_moptions *imo, const size_t gidx, const struct sockaddr *src) { struct ip_msource find; struct in_mfilter *imf; struct ip_msource *ims; const sockunion_t *psa; KASSERT(src->sa_family == AF_INET, ("%s: !AF_INET", __func__)); KASSERT(gidx != -1 && gidx < imo->imo_num_memberships, ("%s: invalid index %d\n", __func__, (int)gidx)); /* The imo_mfilters array may be lazy allocated. */ if (imo->imo_mfilters == NULL) return (NULL); imf = &imo->imo_mfilters[gidx]; /* Source trees are keyed in host byte order. */ psa = (const sockunion_t *)src; find.ims_haddr = ntohl(psa->sin.sin_addr.s_addr); ims = RB_FIND(ip_msource_tree, &imf->imf_sources, &find); return ((struct in_msource *)ims); } /* * Perform filtering for multicast datagrams on a socket by group and source. * * Returns 0 if a datagram should be allowed through, or various error codes * if the socket was not a member of the group, or the source was muted, etc. */ int imo_multi_filter(const struct ip_moptions *imo, const struct ifnet *ifp, const struct sockaddr *group, const struct sockaddr *src) { size_t gidx; struct in_msource *ims; int mode; KASSERT(ifp != NULL, ("%s: null ifp", __func__)); gidx = imo_match_group(imo, ifp, group); if (gidx == -1) return (MCAST_NOTGMEMBER); /* * Check if the source was included in an (S,G) join. * Allow reception on exclusive memberships by default, * reject reception on inclusive memberships by default. * Exclude source only if an in-mode exclude filter exists. * Include source only if an in-mode include filter exists. * NOTE: We are comparing group state here at IGMP t1 (now) * with socket-layer t0 (since last downcall). */ mode = imo->imo_mfilters[gidx].imf_st[1]; ims = imo_match_source(imo, gidx, src); if ((ims == NULL && mode == MCAST_INCLUDE) || (ims != NULL && ims->imsl_st[0] != mode)) return (MCAST_NOTSMEMBER); return (MCAST_PASS); } /* * Find and return a reference to an in_multi record for (ifp, group), * and bump its reference count. * If one does not exist, try to allocate it, and update link-layer multicast * filters on ifp to listen for group. * Assumes the IN_MULTI lock is held across the call. * Return 0 if successful, otherwise return an appropriate error code. */ static int in_getmulti(struct ifnet *ifp, const struct in_addr *group, struct in_multi **pinm) { struct sockaddr_in gsin; struct ifmultiaddr *ifma; struct in_ifinfo *ii; struct in_multi *inm; int error; IN_MULTI_LOCK_ASSERT(); ii = (struct in_ifinfo *)ifp->if_afdata[AF_INET]; inm = inm_lookup(ifp, *group); if (inm != NULL) { /* * If we already joined this group, just bump the * refcount and return it. */ KASSERT(inm->inm_refcount >= 1, ("%s: bad refcount %d", __func__, inm->inm_refcount)); ++inm->inm_refcount; *pinm = inm; return (0); } memset(&gsin, 0, sizeof(gsin)); gsin.sin_family = AF_INET; gsin.sin_len = sizeof(struct sockaddr_in); gsin.sin_addr = *group; /* * Check if a link-layer group is already associated * with this network-layer group on the given ifnet. */ error = if_addmulti(ifp, (struct sockaddr *)&gsin, &ifma); if (error != 0) return (error); /* XXX ifma_protospec must be covered by IF_ADDR_LOCK */ IF_ADDR_WLOCK(ifp); /* * If something other than netinet is occupying the link-layer * group, print a meaningful error message and back out of * the allocation. * Otherwise, bump the refcount on the existing network-layer * group association and return it. */ if (ifma->ifma_protospec != NULL) { inm = (struct in_multi *)ifma->ifma_protospec; #ifdef INVARIANTS KASSERT(ifma->ifma_addr != NULL, ("%s: no ifma_addr", __func__)); KASSERT(ifma->ifma_addr->sa_family == AF_INET, ("%s: ifma not AF_INET", __func__)); KASSERT(inm != NULL, ("%s: no ifma_protospec", __func__)); if (inm->inm_ifma != ifma || inm->inm_ifp != ifp || !in_hosteq(inm->inm_addr, *group)) panic("%s: ifma %p is inconsistent with %p (%s)", __func__, ifma, inm, inet_ntoa(*group)); #endif ++inm->inm_refcount; *pinm = inm; IF_ADDR_WUNLOCK(ifp); return (0); } IF_ADDR_WLOCK_ASSERT(ifp); /* * A new in_multi record is needed; allocate and initialize it. * We DO NOT perform an IGMP join as the in_ layer may need to * push an initial source list down to IGMP to support SSM. * * The initial source filter state is INCLUDE, {} as per the RFC. */ inm = malloc(sizeof(*inm), M_IPMADDR, M_NOWAIT | M_ZERO); if (inm == NULL) { IF_ADDR_WUNLOCK(ifp); if_delmulti_ifma(ifma); return (ENOMEM); } inm->inm_addr = *group; inm->inm_ifp = ifp; inm->inm_igi = ii->ii_igmp; inm->inm_ifma = ifma; inm->inm_refcount = 1; inm->inm_state = IGMP_NOT_MEMBER; mbufq_init(&inm->inm_scq, IGMP_MAX_STATE_CHANGES); inm->inm_st[0].iss_fmode = MCAST_UNDEFINED; inm->inm_st[1].iss_fmode = MCAST_UNDEFINED; RB_INIT(&inm->inm_srcs); ifma->ifma_protospec = inm; *pinm = inm; IF_ADDR_WUNLOCK(ifp); return (0); } /* * Drop a reference to an in_multi record. * * If the refcount drops to 0, free the in_multi record and * delete the underlying link-layer membership. */ void inm_release_locked(struct in_multi *inm) { struct ifmultiaddr *ifma; IN_MULTI_LOCK_ASSERT(); CTR2(KTR_IGMPV3, "%s: refcount is %d", __func__, inm->inm_refcount); if (--inm->inm_refcount > 0) { CTR2(KTR_IGMPV3, "%s: refcount is now %d", __func__, inm->inm_refcount); return; } CTR2(KTR_IGMPV3, "%s: freeing inm %p", __func__, inm); ifma = inm->inm_ifma; /* XXX this access is not covered by IF_ADDR_LOCK */ CTR2(KTR_IGMPV3, "%s: purging ifma %p", __func__, ifma); KASSERT(ifma->ifma_protospec == inm, ("%s: ifma_protospec != inm", __func__)); ifma->ifma_protospec = NULL; inm_purge(inm); free(inm, M_IPMADDR); if_delmulti_ifma(ifma); } /* * Clear recorded source entries for a group. * Used by the IGMP code. Caller must hold the IN_MULTI lock. * FIXME: Should reap. */ void inm_clear_recorded(struct in_multi *inm) { struct ip_msource *ims; IN_MULTI_LOCK_ASSERT(); RB_FOREACH(ims, ip_msource_tree, &inm->inm_srcs) { if (ims->ims_stp) { ims->ims_stp = 0; --inm->inm_st[1].iss_rec; } } KASSERT(inm->inm_st[1].iss_rec == 0, ("%s: iss_rec %d not 0", __func__, inm->inm_st[1].iss_rec)); } /* * Record a source as pending for a Source-Group IGMPv3 query. * This lives here as it modifies the shared tree. * * inm is the group descriptor. * naddr is the address of the source to record in network-byte order. * * If the net.inet.igmp.sgalloc sysctl is non-zero, we will * lazy-allocate a source node in response to an SG query. * Otherwise, no allocation is performed. This saves some memory * with the trade-off that the source will not be reported to the * router if joined in the window between the query response and * the group actually being joined on the local host. * * VIMAGE: XXX: Currently the igmp_sgalloc feature has been removed. * This turns off the allocation of a recorded source entry if * the group has not been joined. * * Return 0 if the source didn't exist or was already marked as recorded. * Return 1 if the source was marked as recorded by this function. * Return <0 if any error occured (negated errno code). */ int inm_record_source(struct in_multi *inm, const in_addr_t naddr) { struct ip_msource find; struct ip_msource *ims, *nims; IN_MULTI_LOCK_ASSERT(); find.ims_haddr = ntohl(naddr); ims = RB_FIND(ip_msource_tree, &inm->inm_srcs, &find); if (ims && ims->ims_stp) return (0); if (ims == NULL) { if (inm->inm_nsrc == in_mcast_maxgrpsrc) return (-ENOSPC); nims = malloc(sizeof(struct ip_msource), M_IPMSOURCE, M_NOWAIT | M_ZERO); if (nims == NULL) return (-ENOMEM); nims->ims_haddr = find.ims_haddr; RB_INSERT(ip_msource_tree, &inm->inm_srcs, nims); ++inm->inm_nsrc; ims = nims; } /* * Mark the source as recorded and update the recorded * source count. */ ++ims->ims_stp; ++inm->inm_st[1].iss_rec; return (1); } /* * Return a pointer to an in_msource owned by an in_mfilter, * given its source address. * Lazy-allocate if needed. If this is a new entry its filter state is * undefined at t0. * * imf is the filter set being modified. * haddr is the source address in *host* byte-order. * * SMPng: May be called with locks held; malloc must not block. */ static int imf_get_source(struct in_mfilter *imf, const struct sockaddr_in *psin, struct in_msource **plims) { struct ip_msource find; struct ip_msource *ims, *nims; struct in_msource *lims; int error; error = 0; ims = NULL; lims = NULL; /* key is host byte order */ find.ims_haddr = ntohl(psin->sin_addr.s_addr); ims = RB_FIND(ip_msource_tree, &imf->imf_sources, &find); lims = (struct in_msource *)ims; if (lims == NULL) { if (imf->imf_nsrc == in_mcast_maxsocksrc) return (ENOSPC); nims = malloc(sizeof(struct in_msource), M_INMFILTER, M_NOWAIT | M_ZERO); if (nims == NULL) return (ENOMEM); lims = (struct in_msource *)nims; lims->ims_haddr = find.ims_haddr; lims->imsl_st[0] = MCAST_UNDEFINED; RB_INSERT(ip_msource_tree, &imf->imf_sources, nims); ++imf->imf_nsrc; } *plims = lims; return (error); } /* * Graft a source entry into an existing socket-layer filter set, * maintaining any required invariants and checking allocations. * * The source is marked as being in the new filter mode at t1. * * Return the pointer to the new node, otherwise return NULL. */ static struct in_msource * imf_graft(struct in_mfilter *imf, const uint8_t st1, const struct sockaddr_in *psin) { struct ip_msource *nims; struct in_msource *lims; nims = malloc(sizeof(struct in_msource), M_INMFILTER, M_NOWAIT | M_ZERO); if (nims == NULL) return (NULL); lims = (struct in_msource *)nims; lims->ims_haddr = ntohl(psin->sin_addr.s_addr); lims->imsl_st[0] = MCAST_UNDEFINED; lims->imsl_st[1] = st1; RB_INSERT(ip_msource_tree, &imf->imf_sources, nims); ++imf->imf_nsrc; return (lims); } /* * Prune a source entry from an existing socket-layer filter set, * maintaining any required invariants and checking allocations. * * The source is marked as being left at t1, it is not freed. * * Return 0 if no error occurred, otherwise return an errno value. */ static int imf_prune(struct in_mfilter *imf, const struct sockaddr_in *psin) { struct ip_msource find; struct ip_msource *ims; struct in_msource *lims; /* key is host byte order */ find.ims_haddr = ntohl(psin->sin_addr.s_addr); ims = RB_FIND(ip_msource_tree, &imf->imf_sources, &find); if (ims == NULL) return (ENOENT); lims = (struct in_msource *)ims; lims->imsl_st[1] = MCAST_UNDEFINED; return (0); } /* * Revert socket-layer filter set deltas at t1 to t0 state. */ static void imf_rollback(struct in_mfilter *imf) { struct ip_msource *ims, *tims; struct in_msource *lims; RB_FOREACH_SAFE(ims, ip_msource_tree, &imf->imf_sources, tims) { lims = (struct in_msource *)ims; if (lims->imsl_st[0] == lims->imsl_st[1]) { /* no change at t1 */ continue; } else if (lims->imsl_st[0] != MCAST_UNDEFINED) { /* revert change to existing source at t1 */ lims->imsl_st[1] = lims->imsl_st[0]; } else { /* revert source added t1 */ CTR2(KTR_IGMPV3, "%s: free ims %p", __func__, ims); RB_REMOVE(ip_msource_tree, &imf->imf_sources, ims); free(ims, M_INMFILTER); imf->imf_nsrc--; } } imf->imf_st[1] = imf->imf_st[0]; } /* * Mark socket-layer filter set as INCLUDE {} at t1. */ static void imf_leave(struct in_mfilter *imf) { struct ip_msource *ims; struct in_msource *lims; RB_FOREACH(ims, ip_msource_tree, &imf->imf_sources) { lims = (struct in_msource *)ims; lims->imsl_st[1] = MCAST_UNDEFINED; } imf->imf_st[1] = MCAST_INCLUDE; } /* * Mark socket-layer filter set deltas as committed. */ static void imf_commit(struct in_mfilter *imf) { struct ip_msource *ims; struct in_msource *lims; RB_FOREACH(ims, ip_msource_tree, &imf->imf_sources) { lims = (struct in_msource *)ims; lims->imsl_st[0] = lims->imsl_st[1]; } imf->imf_st[0] = imf->imf_st[1]; } /* * Reap unreferenced sources from socket-layer filter set. */ static void imf_reap(struct in_mfilter *imf) { struct ip_msource *ims, *tims; struct in_msource *lims; RB_FOREACH_SAFE(ims, ip_msource_tree, &imf->imf_sources, tims) { lims = (struct in_msource *)ims; if ((lims->imsl_st[0] == MCAST_UNDEFINED) && (lims->imsl_st[1] == MCAST_UNDEFINED)) { CTR2(KTR_IGMPV3, "%s: free lims %p", __func__, ims); RB_REMOVE(ip_msource_tree, &imf->imf_sources, ims); free(ims, M_INMFILTER); imf->imf_nsrc--; } } } /* * Purge socket-layer filter set. */ static void imf_purge(struct in_mfilter *imf) { struct ip_msource *ims, *tims; RB_FOREACH_SAFE(ims, ip_msource_tree, &imf->imf_sources, tims) { CTR2(KTR_IGMPV3, "%s: free ims %p", __func__, ims); RB_REMOVE(ip_msource_tree, &imf->imf_sources, ims); free(ims, M_INMFILTER); imf->imf_nsrc--; } imf->imf_st[0] = imf->imf_st[1] = MCAST_UNDEFINED; KASSERT(RB_EMPTY(&imf->imf_sources), ("%s: imf_sources not empty", __func__)); } /* * Look up a source filter entry for a multicast group. * * inm is the group descriptor to work with. * haddr is the host-byte-order IPv4 address to look up. * noalloc may be non-zero to suppress allocation of sources. * *pims will be set to the address of the retrieved or allocated source. * * SMPng: NOTE: may be called with locks held. * Return 0 if successful, otherwise return a non-zero error code. */ static int inm_get_source(struct in_multi *inm, const in_addr_t haddr, const int noalloc, struct ip_msource **pims) { struct ip_msource find; struct ip_msource *ims, *nims; #ifdef KTR struct in_addr ia; #endif find.ims_haddr = haddr; ims = RB_FIND(ip_msource_tree, &inm->inm_srcs, &find); if (ims == NULL && !noalloc) { if (inm->inm_nsrc == in_mcast_maxgrpsrc) return (ENOSPC); nims = malloc(sizeof(struct ip_msource), M_IPMSOURCE, M_NOWAIT | M_ZERO); if (nims == NULL) return (ENOMEM); nims->ims_haddr = haddr; RB_INSERT(ip_msource_tree, &inm->inm_srcs, nims); ++inm->inm_nsrc; ims = nims; #ifdef KTR ia.s_addr = htonl(haddr); CTR3(KTR_IGMPV3, "%s: allocated %s as %p", __func__, inet_ntoa(ia), ims); #endif } *pims = ims; return (0); } /* * Merge socket-layer source into IGMP-layer source. * If rollback is non-zero, perform the inverse of the merge. */ static void ims_merge(struct ip_msource *ims, const struct in_msource *lims, const int rollback) { int n = rollback ? -1 : 1; #ifdef KTR struct in_addr ia; ia.s_addr = htonl(ims->ims_haddr); #endif if (lims->imsl_st[0] == MCAST_EXCLUDE) { CTR3(KTR_IGMPV3, "%s: t1 ex -= %d on %s", __func__, n, inet_ntoa(ia)); ims->ims_st[1].ex -= n; } else if (lims->imsl_st[0] == MCAST_INCLUDE) { CTR3(KTR_IGMPV3, "%s: t1 in -= %d on %s", __func__, n, inet_ntoa(ia)); ims->ims_st[1].in -= n; } if (lims->imsl_st[1] == MCAST_EXCLUDE) { CTR3(KTR_IGMPV3, "%s: t1 ex += %d on %s", __func__, n, inet_ntoa(ia)); ims->ims_st[1].ex += n; } else if (lims->imsl_st[1] == MCAST_INCLUDE) { CTR3(KTR_IGMPV3, "%s: t1 in += %d on %s", __func__, n, inet_ntoa(ia)); ims->ims_st[1].in += n; } } /* * Atomically update the global in_multi state, when a membership's * filter list is being updated in any way. * * imf is the per-inpcb-membership group filter pointer. * A fake imf may be passed for in-kernel consumers. * * XXX This is a candidate for a set-symmetric-difference style loop * which would eliminate the repeated lookup from root of ims nodes, * as they share the same key space. * * If any error occurred this function will back out of refcounts * and return a non-zero value. */ static int inm_merge(struct in_multi *inm, /*const*/ struct in_mfilter *imf) { struct ip_msource *ims, *nims; struct in_msource *lims; int schanged, error; int nsrc0, nsrc1; schanged = 0; error = 0; nsrc1 = nsrc0 = 0; /* * Update the source filters first, as this may fail. * Maintain count of in-mode filters at t0, t1. These are * used to work out if we transition into ASM mode or not. * Maintain a count of source filters whose state was * actually modified by this operation. */ RB_FOREACH(ims, ip_msource_tree, &imf->imf_sources) { lims = (struct in_msource *)ims; if (lims->imsl_st[0] == imf->imf_st[0]) nsrc0++; if (lims->imsl_st[1] == imf->imf_st[1]) nsrc1++; if (lims->imsl_st[0] == lims->imsl_st[1]) continue; error = inm_get_source(inm, lims->ims_haddr, 0, &nims); ++schanged; if (error) break; ims_merge(nims, lims, 0); } if (error) { struct ip_msource *bims; RB_FOREACH_REVERSE_FROM(ims, ip_msource_tree, nims) { lims = (struct in_msource *)ims; if (lims->imsl_st[0] == lims->imsl_st[1]) continue; (void)inm_get_source(inm, lims->ims_haddr, 1, &bims); if (bims == NULL) continue; ims_merge(bims, lims, 1); } goto out_reap; } CTR3(KTR_IGMPV3, "%s: imf filters in-mode: %d at t0, %d at t1", __func__, nsrc0, nsrc1); /* Handle transition between INCLUDE {n} and INCLUDE {} on socket. */ if (imf->imf_st[0] == imf->imf_st[1] && imf->imf_st[1] == MCAST_INCLUDE) { if (nsrc1 == 0) { CTR1(KTR_IGMPV3, "%s: --in on inm at t1", __func__); --inm->inm_st[1].iss_in; } } /* Handle filter mode transition on socket. */ if (imf->imf_st[0] != imf->imf_st[1]) { CTR3(KTR_IGMPV3, "%s: imf transition %d to %d", __func__, imf->imf_st[0], imf->imf_st[1]); if (imf->imf_st[0] == MCAST_EXCLUDE) { CTR1(KTR_IGMPV3, "%s: --ex on inm at t1", __func__); --inm->inm_st[1].iss_ex; } else if (imf->imf_st[0] == MCAST_INCLUDE) { CTR1(KTR_IGMPV3, "%s: --in on inm at t1", __func__); --inm->inm_st[1].iss_in; } if (imf->imf_st[1] == MCAST_EXCLUDE) { CTR1(KTR_IGMPV3, "%s: ex++ on inm at t1", __func__); inm->inm_st[1].iss_ex++; } else if (imf->imf_st[1] == MCAST_INCLUDE && nsrc1 > 0) { CTR1(KTR_IGMPV3, "%s: in++ on inm at t1", __func__); inm->inm_st[1].iss_in++; } } /* * Track inm filter state in terms of listener counts. * If there are any exclusive listeners, stack-wide * membership is exclusive. * Otherwise, if only inclusive listeners, stack-wide is inclusive. * If no listeners remain, state is undefined at t1, * and the IGMP lifecycle for this group should finish. */ if (inm->inm_st[1].iss_ex > 0) { CTR1(KTR_IGMPV3, "%s: transition to EX", __func__); inm->inm_st[1].iss_fmode = MCAST_EXCLUDE; } else if (inm->inm_st[1].iss_in > 0) { CTR1(KTR_IGMPV3, "%s: transition to IN", __func__); inm->inm_st[1].iss_fmode = MCAST_INCLUDE; } else { CTR1(KTR_IGMPV3, "%s: transition to UNDEF", __func__); inm->inm_st[1].iss_fmode = MCAST_UNDEFINED; } /* Decrement ASM listener count on transition out of ASM mode. */ if (imf->imf_st[0] == MCAST_EXCLUDE && nsrc0 == 0) { if ((imf->imf_st[1] != MCAST_EXCLUDE) || (imf->imf_st[1] == MCAST_EXCLUDE && nsrc1 > 0)) CTR1(KTR_IGMPV3, "%s: --asm on inm at t1", __func__); --inm->inm_st[1].iss_asm; } /* Increment ASM listener count on transition to ASM mode. */ if (imf->imf_st[1] == MCAST_EXCLUDE && nsrc1 == 0) { CTR1(KTR_IGMPV3, "%s: asm++ on inm at t1", __func__); inm->inm_st[1].iss_asm++; } CTR3(KTR_IGMPV3, "%s: merged imf %p to inm %p", __func__, imf, inm); inm_print(inm); out_reap: if (schanged > 0) { CTR1(KTR_IGMPV3, "%s: sources changed; reaping", __func__); inm_reap(inm); } return (error); } /* * Mark an in_multi's filter set deltas as committed. * Called by IGMP after a state change has been enqueued. */ void inm_commit(struct in_multi *inm) { struct ip_msource *ims; CTR2(KTR_IGMPV3, "%s: commit inm %p", __func__, inm); CTR1(KTR_IGMPV3, "%s: pre commit:", __func__); inm_print(inm); RB_FOREACH(ims, ip_msource_tree, &inm->inm_srcs) { ims->ims_st[0] = ims->ims_st[1]; } inm->inm_st[0] = inm->inm_st[1]; } /* * Reap unreferenced nodes from an in_multi's filter set. */ static void inm_reap(struct in_multi *inm) { struct ip_msource *ims, *tims; RB_FOREACH_SAFE(ims, ip_msource_tree, &inm->inm_srcs, tims) { if (ims->ims_st[0].ex > 0 || ims->ims_st[0].in > 0 || ims->ims_st[1].ex > 0 || ims->ims_st[1].in > 0 || ims->ims_stp != 0) continue; CTR2(KTR_IGMPV3, "%s: free ims %p", __func__, ims); RB_REMOVE(ip_msource_tree, &inm->inm_srcs, ims); free(ims, M_IPMSOURCE); inm->inm_nsrc--; } } /* * Purge all source nodes from an in_multi's filter set. */ static void inm_purge(struct in_multi *inm) { struct ip_msource *ims, *tims; RB_FOREACH_SAFE(ims, ip_msource_tree, &inm->inm_srcs, tims) { CTR2(KTR_IGMPV3, "%s: free ims %p", __func__, ims); RB_REMOVE(ip_msource_tree, &inm->inm_srcs, ims); free(ims, M_IPMSOURCE); inm->inm_nsrc--; } } /* * Join a multicast group; unlocked entry point. * * SMPng: XXX: in_joingroup() is called from in_control() when Giant * is not held. Fortunately, ifp is unlikely to have been detached * at this point, so we assume it's OK to recurse. */ int in_joingroup(struct ifnet *ifp, const struct in_addr *gina, /*const*/ struct in_mfilter *imf, struct in_multi **pinm) { int error; IN_MULTI_LOCK(); error = in_joingroup_locked(ifp, gina, imf, pinm); IN_MULTI_UNLOCK(); return (error); } /* * Join a multicast group; real entry point. * * Only preserves atomicity at inm level. * NOTE: imf argument cannot be const due to sys/tree.h limitations. * * If the IGMP downcall fails, the group is not joined, and an error * code is returned. */ int in_joingroup_locked(struct ifnet *ifp, const struct in_addr *gina, /*const*/ struct in_mfilter *imf, struct in_multi **pinm) { struct in_mfilter timf; struct in_multi *inm; int error; IN_MULTI_LOCK_ASSERT(); CTR4(KTR_IGMPV3, "%s: join %s on %p(%s))", __func__, inet_ntoa(*gina), ifp, ifp->if_xname); error = 0; inm = NULL; /* * If no imf was specified (i.e. kernel consumer), * fake one up and assume it is an ASM join. */ if (imf == NULL) { imf_init(&timf, MCAST_UNDEFINED, MCAST_EXCLUDE); imf = &timf; } error = in_getmulti(ifp, gina, &inm); if (error) { CTR1(KTR_IGMPV3, "%s: in_getmulti() failure", __func__); return (error); } CTR1(KTR_IGMPV3, "%s: merge inm state", __func__); error = inm_merge(inm, imf); if (error) { CTR1(KTR_IGMPV3, "%s: failed to merge inm state", __func__); goto out_inm_release; } CTR1(KTR_IGMPV3, "%s: doing igmp downcall", __func__); error = igmp_change_state(inm); if (error) { CTR1(KTR_IGMPV3, "%s: failed to update source", __func__); goto out_inm_release; } out_inm_release: if (error) { CTR2(KTR_IGMPV3, "%s: dropping ref on %p", __func__, inm); inm_release_locked(inm); } else { *pinm = inm; } return (error); } /* * Leave a multicast group; unlocked entry point. */ int in_leavegroup(struct in_multi *inm, /*const*/ struct in_mfilter *imf) { int error; IN_MULTI_LOCK(); error = in_leavegroup_locked(inm, imf); IN_MULTI_UNLOCK(); return (error); } /* * Leave a multicast group; real entry point. * All source filters will be expunged. * * Only preserves atomicity at inm level. * * Holding the write lock for the INP which contains imf * is highly advisable. We can't assert for it as imf does not * contain a back-pointer to the owning inp. * * Note: This is not the same as inm_release(*) as this function also * makes a state change downcall into IGMP. */ int in_leavegroup_locked(struct in_multi *inm, /*const*/ struct in_mfilter *imf) { struct in_mfilter timf; int error; error = 0; IN_MULTI_LOCK_ASSERT(); CTR5(KTR_IGMPV3, "%s: leave inm %p, %s/%s, imf %p", __func__, inm, inet_ntoa(inm->inm_addr), (inm_is_ifp_detached(inm) ? "null" : inm->inm_ifp->if_xname), imf); /* * If no imf was specified (i.e. kernel consumer), * fake one up and assume it is an ASM join. */ if (imf == NULL) { imf_init(&timf, MCAST_EXCLUDE, MCAST_UNDEFINED); imf = &timf; } /* * Begin state merge transaction at IGMP layer. * * As this particular invocation should not cause any memory * to be allocated, and there is no opportunity to roll back * the transaction, it MUST NOT fail. */ CTR1(KTR_IGMPV3, "%s: merge inm state", __func__); error = inm_merge(inm, imf); KASSERT(error == 0, ("%s: failed to merge inm state", __func__)); CTR1(KTR_IGMPV3, "%s: doing igmp downcall", __func__); CURVNET_SET(inm->inm_ifp->if_vnet); error = igmp_change_state(inm); CURVNET_RESTORE(); if (error) CTR1(KTR_IGMPV3, "%s: failed igmp downcall", __func__); CTR2(KTR_IGMPV3, "%s: dropping ref on %p", __func__, inm); inm_release_locked(inm); return (error); } /*#ifndef BURN_BRIDGES*/ /* * Join an IPv4 multicast group in (*,G) exclusive mode. * The group must be a 224.0.0.0/24 link-scope group. * This KPI is for legacy kernel consumers only. */ struct in_multi * in_addmulti(struct in_addr *ap, struct ifnet *ifp) { struct in_multi *pinm; int error; KASSERT(IN_LOCAL_GROUP(ntohl(ap->s_addr)), ("%s: %s not in 224.0.0.0/24", __func__, inet_ntoa(*ap))); error = in_joingroup(ifp, ap, NULL, &pinm); if (error != 0) pinm = NULL; return (pinm); } /* * Leave an IPv4 multicast group, assumed to be in exclusive (*,G) mode. * This KPI is for legacy kernel consumers only. */ void in_delmulti(struct in_multi *inm) { (void)in_leavegroup(inm, NULL); } /*#endif*/ /* * Block or unblock an ASM multicast source on an inpcb. * This implements the delta-based API described in RFC 3678. * * The delta-based API applies only to exclusive-mode memberships. * An IGMP downcall will be performed. * * SMPng: NOTE: Must take Giant as a join may create a new ifma. * * Return 0 if successful, otherwise return an appropriate error code. */ static int inp_block_unblock_source(struct inpcb *inp, struct sockopt *sopt) { struct group_source_req gsr; sockunion_t *gsa, *ssa; struct ifnet *ifp; struct in_mfilter *imf; struct ip_moptions *imo; struct in_msource *ims; struct in_multi *inm; size_t idx; uint16_t fmode; int error, doblock; ifp = NULL; error = 0; doblock = 0; memset(&gsr, 0, sizeof(struct group_source_req)); gsa = (sockunion_t *)&gsr.gsr_group; ssa = (sockunion_t *)&gsr.gsr_source; switch (sopt->sopt_name) { case IP_BLOCK_SOURCE: case IP_UNBLOCK_SOURCE: { struct ip_mreq_source mreqs; error = sooptcopyin(sopt, &mreqs, sizeof(struct ip_mreq_source), sizeof(struct ip_mreq_source)); if (error) return (error); gsa->sin.sin_family = AF_INET; gsa->sin.sin_len = sizeof(struct sockaddr_in); gsa->sin.sin_addr = mreqs.imr_multiaddr; ssa->sin.sin_family = AF_INET; ssa->sin.sin_len = sizeof(struct sockaddr_in); ssa->sin.sin_addr = mreqs.imr_sourceaddr; if (!in_nullhost(mreqs.imr_interface)) INADDR_TO_IFP(mreqs.imr_interface, ifp); if (sopt->sopt_name == IP_BLOCK_SOURCE) doblock = 1; CTR3(KTR_IGMPV3, "%s: imr_interface = %s, ifp = %p", __func__, inet_ntoa(mreqs.imr_interface), ifp); break; } case MCAST_BLOCK_SOURCE: case MCAST_UNBLOCK_SOURCE: error = sooptcopyin(sopt, &gsr, sizeof(struct group_source_req), sizeof(struct group_source_req)); if (error) return (error); if (gsa->sin.sin_family != AF_INET || gsa->sin.sin_len != sizeof(struct sockaddr_in)) return (EINVAL); if (ssa->sin.sin_family != AF_INET || ssa->sin.sin_len != sizeof(struct sockaddr_in)) return (EINVAL); if (gsr.gsr_interface == 0 || V_if_index < gsr.gsr_interface) return (EADDRNOTAVAIL); ifp = ifnet_byindex(gsr.gsr_interface); if (sopt->sopt_name == MCAST_BLOCK_SOURCE) doblock = 1; break; default: CTR2(KTR_IGMPV3, "%s: unknown sopt_name %d", __func__, sopt->sopt_name); return (EOPNOTSUPP); break; } if (!IN_MULTICAST(ntohl(gsa->sin.sin_addr.s_addr))) return (EINVAL); /* * Check if we are actually a member of this group. */ imo = inp_findmoptions(inp); idx = imo_match_group(imo, ifp, &gsa->sa); if (idx == -1 || imo->imo_mfilters == NULL) { error = EADDRNOTAVAIL; goto out_inp_locked; } KASSERT(imo->imo_mfilters != NULL, ("%s: imo_mfilters not allocated", __func__)); imf = &imo->imo_mfilters[idx]; inm = imo->imo_membership[idx]; /* * Attempting to use the delta-based API on an * non exclusive-mode membership is an error. */ fmode = imf->imf_st[0]; if (fmode != MCAST_EXCLUDE) { error = EINVAL; goto out_inp_locked; } /* * Deal with error cases up-front: * Asked to block, but already blocked; or * Asked to unblock, but nothing to unblock. * If adding a new block entry, allocate it. */ ims = imo_match_source(imo, idx, &ssa->sa); if ((ims != NULL && doblock) || (ims == NULL && !doblock)) { CTR3(KTR_IGMPV3, "%s: source %s %spresent", __func__, inet_ntoa(ssa->sin.sin_addr), doblock ? "" : "not "); error = EADDRNOTAVAIL; goto out_inp_locked; } INP_WLOCK_ASSERT(inp); /* * Begin state merge transaction at socket layer. */ if (doblock) { CTR2(KTR_IGMPV3, "%s: %s source", __func__, "block"); ims = imf_graft(imf, fmode, &ssa->sin); if (ims == NULL) error = ENOMEM; } else { CTR2(KTR_IGMPV3, "%s: %s source", __func__, "allow"); error = imf_prune(imf, &ssa->sin); } if (error) { CTR1(KTR_IGMPV3, "%s: merge imf state failed", __func__); goto out_imf_rollback; } /* * Begin state merge transaction at IGMP layer. */ IN_MULTI_LOCK(); CTR1(KTR_IGMPV3, "%s: merge inm state", __func__); error = inm_merge(inm, imf); if (error) { CTR1(KTR_IGMPV3, "%s: failed to merge inm state", __func__); goto out_in_multi_locked; } CTR1(KTR_IGMPV3, "%s: doing igmp downcall", __func__); error = igmp_change_state(inm); if (error) CTR1(KTR_IGMPV3, "%s: failed igmp downcall", __func__); out_in_multi_locked: IN_MULTI_UNLOCK(); out_imf_rollback: if (error) imf_rollback(imf); else imf_commit(imf); imf_reap(imf); out_inp_locked: INP_WUNLOCK(inp); return (error); } /* * Given an inpcb, return its multicast options structure pointer. Accepts * an unlocked inpcb pointer, but will return it locked. May sleep. * * SMPng: NOTE: Potentially calls malloc(M_WAITOK) with Giant held. * SMPng: NOTE: Returns with the INP write lock held. */ static struct ip_moptions * inp_findmoptions(struct inpcb *inp) { struct ip_moptions *imo; struct in_multi **immp; struct in_mfilter *imfp; size_t idx; INP_WLOCK(inp); if (inp->inp_moptions != NULL) return (inp->inp_moptions); INP_WUNLOCK(inp); imo = malloc(sizeof(*imo), M_IPMOPTS, M_WAITOK); immp = malloc(sizeof(*immp) * IP_MIN_MEMBERSHIPS, M_IPMOPTS, M_WAITOK | M_ZERO); imfp = malloc(sizeof(struct in_mfilter) * IP_MIN_MEMBERSHIPS, M_INMFILTER, M_WAITOK); imo->imo_multicast_ifp = NULL; imo->imo_multicast_addr.s_addr = INADDR_ANY; imo->imo_multicast_vif = -1; imo->imo_multicast_ttl = IP_DEFAULT_MULTICAST_TTL; imo->imo_multicast_loop = in_mcast_loop; imo->imo_num_memberships = 0; imo->imo_max_memberships = IP_MIN_MEMBERSHIPS; imo->imo_membership = immp; /* Initialize per-group source filters. */ for (idx = 0; idx < IP_MIN_MEMBERSHIPS; idx++) imf_init(&imfp[idx], MCAST_UNDEFINED, MCAST_EXCLUDE); imo->imo_mfilters = imfp; INP_WLOCK(inp); if (inp->inp_moptions != NULL) { free(imfp, M_INMFILTER); free(immp, M_IPMOPTS); free(imo, M_IPMOPTS); return (inp->inp_moptions); } inp->inp_moptions = imo; return (imo); } /* * Discard the IP multicast options (and source filters). To minimize * the amount of work done while holding locks such as the INP's * pcbinfo lock (which is used in the receive path), the free * operation is performed asynchronously in a separate task. * * SMPng: NOTE: assumes INP write lock is held. */ void inp_freemoptions(struct ip_moptions *imo) { KASSERT(imo != NULL, ("%s: ip_moptions is NULL", __func__)); IN_MULTI_LOCK(); STAILQ_INSERT_TAIL(&imo_gc_list, imo, imo_link); IN_MULTI_UNLOCK(); taskqueue_enqueue(taskqueue_thread, &imo_gc_task); } static void inp_freemoptions_internal(struct ip_moptions *imo) { struct in_mfilter *imf; size_t idx, nmships; nmships = imo->imo_num_memberships; for (idx = 0; idx < nmships; ++idx) { imf = imo->imo_mfilters ? &imo->imo_mfilters[idx] : NULL; if (imf) imf_leave(imf); (void)in_leavegroup(imo->imo_membership[idx], imf); if (imf) imf_purge(imf); } if (imo->imo_mfilters) free(imo->imo_mfilters, M_INMFILTER); free(imo->imo_membership, M_IPMOPTS); free(imo, M_IPMOPTS); } static void inp_gcmoptions(void *context, int pending) { struct ip_moptions *imo; IN_MULTI_LOCK(); while (!STAILQ_EMPTY(&imo_gc_list)) { imo = STAILQ_FIRST(&imo_gc_list); STAILQ_REMOVE_HEAD(&imo_gc_list, imo_link); IN_MULTI_UNLOCK(); inp_freemoptions_internal(imo); IN_MULTI_LOCK(); } IN_MULTI_UNLOCK(); } /* * Atomically get source filters on a socket for an IPv4 multicast group. * Called with INP lock held; returns with lock released. */ static int inp_get_source_filters(struct inpcb *inp, struct sockopt *sopt) { struct __msfilterreq msfr; sockunion_t *gsa; struct ifnet *ifp; struct ip_moptions *imo; struct in_mfilter *imf; struct ip_msource *ims; struct in_msource *lims; struct sockaddr_in *psin; struct sockaddr_storage *ptss; struct sockaddr_storage *tss; int error; size_t idx, nsrcs, ncsrcs; INP_WLOCK_ASSERT(inp); imo = inp->inp_moptions; KASSERT(imo != NULL, ("%s: null ip_moptions", __func__)); INP_WUNLOCK(inp); error = sooptcopyin(sopt, &msfr, sizeof(struct __msfilterreq), sizeof(struct __msfilterreq)); if (error) return (error); if (msfr.msfr_ifindex == 0 || V_if_index < msfr.msfr_ifindex) return (EINVAL); ifp = ifnet_byindex(msfr.msfr_ifindex); if (ifp == NULL) return (EINVAL); INP_WLOCK(inp); /* * Lookup group on the socket. */ gsa = (sockunion_t *)&msfr.msfr_group; idx = imo_match_group(imo, ifp, &gsa->sa); if (idx == -1 || imo->imo_mfilters == NULL) { INP_WUNLOCK(inp); return (EADDRNOTAVAIL); } imf = &imo->imo_mfilters[idx]; /* * Ignore memberships which are in limbo. */ if (imf->imf_st[1] == MCAST_UNDEFINED) { INP_WUNLOCK(inp); return (EAGAIN); } msfr.msfr_fmode = imf->imf_st[1]; /* * If the user specified a buffer, copy out the source filter * entries to userland gracefully. * We only copy out the number of entries which userland * has asked for, but we always tell userland how big the * buffer really needs to be. */ if (msfr.msfr_nsrcs > in_mcast_maxsocksrc) msfr.msfr_nsrcs = in_mcast_maxsocksrc; tss = NULL; if (msfr.msfr_srcs != NULL && msfr.msfr_nsrcs > 0) { tss = malloc(sizeof(struct sockaddr_storage) * msfr.msfr_nsrcs, M_TEMP, M_NOWAIT | M_ZERO); if (tss == NULL) { INP_WUNLOCK(inp); return (ENOBUFS); } } /* * Count number of sources in-mode at t0. * If buffer space exists and remains, copy out source entries. */ nsrcs = msfr.msfr_nsrcs; ncsrcs = 0; ptss = tss; RB_FOREACH(ims, ip_msource_tree, &imf->imf_sources) { lims = (struct in_msource *)ims; if (lims->imsl_st[0] == MCAST_UNDEFINED || lims->imsl_st[0] != imf->imf_st[0]) continue; ++ncsrcs; if (tss != NULL && nsrcs > 0) { psin = (struct sockaddr_in *)ptss; psin->sin_family = AF_INET; psin->sin_len = sizeof(struct sockaddr_in); psin->sin_addr.s_addr = htonl(lims->ims_haddr); psin->sin_port = 0; ++ptss; --nsrcs; } } INP_WUNLOCK(inp); if (tss != NULL) { error = copyout(tss, msfr.msfr_srcs, sizeof(struct sockaddr_storage) * msfr.msfr_nsrcs); free(tss, M_TEMP); if (error) return (error); } msfr.msfr_nsrcs = ncsrcs; error = sooptcopyout(sopt, &msfr, sizeof(struct __msfilterreq)); return (error); } /* * Return the IP multicast options in response to user getsockopt(). */ int inp_getmoptions(struct inpcb *inp, struct sockopt *sopt) { struct rm_priotracker in_ifa_tracker; struct ip_mreqn mreqn; struct ip_moptions *imo; struct ifnet *ifp; struct in_ifaddr *ia; int error, optval; u_char coptval; INP_WLOCK(inp); imo = inp->inp_moptions; /* * If socket is neither of type SOCK_RAW or SOCK_DGRAM, * or is a divert socket, reject it. */ if (inp->inp_socket->so_proto->pr_protocol == IPPROTO_DIVERT || (inp->inp_socket->so_proto->pr_type != SOCK_RAW && inp->inp_socket->so_proto->pr_type != SOCK_DGRAM)) { INP_WUNLOCK(inp); return (EOPNOTSUPP); } error = 0; switch (sopt->sopt_name) { case IP_MULTICAST_VIF: if (imo != NULL) optval = imo->imo_multicast_vif; else optval = -1; INP_WUNLOCK(inp); error = sooptcopyout(sopt, &optval, sizeof(int)); break; case IP_MULTICAST_IF: memset(&mreqn, 0, sizeof(struct ip_mreqn)); if (imo != NULL) { ifp = imo->imo_multicast_ifp; if (!in_nullhost(imo->imo_multicast_addr)) { mreqn.imr_address = imo->imo_multicast_addr; } else if (ifp != NULL) { mreqn.imr_ifindex = ifp->if_index; IFP_TO_IA(ifp, ia, &in_ifa_tracker); if (ia != NULL) { mreqn.imr_address = IA_SIN(ia)->sin_addr; ifa_free(&ia->ia_ifa); } } } INP_WUNLOCK(inp); if (sopt->sopt_valsize == sizeof(struct ip_mreqn)) { error = sooptcopyout(sopt, &mreqn, sizeof(struct ip_mreqn)); } else { error = sooptcopyout(sopt, &mreqn.imr_address, sizeof(struct in_addr)); } break; case IP_MULTICAST_TTL: if (imo == 0) optval = coptval = IP_DEFAULT_MULTICAST_TTL; else optval = coptval = imo->imo_multicast_ttl; INP_WUNLOCK(inp); if (sopt->sopt_valsize == sizeof(u_char)) error = sooptcopyout(sopt, &coptval, sizeof(u_char)); else error = sooptcopyout(sopt, &optval, sizeof(int)); break; case IP_MULTICAST_LOOP: if (imo == 0) optval = coptval = IP_DEFAULT_MULTICAST_LOOP; else optval = coptval = imo->imo_multicast_loop; INP_WUNLOCK(inp); if (sopt->sopt_valsize == sizeof(u_char)) error = sooptcopyout(sopt, &coptval, sizeof(u_char)); else error = sooptcopyout(sopt, &optval, sizeof(int)); break; case IP_MSFILTER: if (imo == NULL) { error = EADDRNOTAVAIL; INP_WUNLOCK(inp); } else { error = inp_get_source_filters(inp, sopt); } break; default: INP_WUNLOCK(inp); error = ENOPROTOOPT; break; } INP_UNLOCK_ASSERT(inp); return (error); } /* * Look up the ifnet to use for a multicast group membership, * given the IPv4 address of an interface, and the IPv4 group address. * * This routine exists to support legacy multicast applications * which do not understand that multicast memberships are scoped to * specific physical links in the networking stack, or which need * to join link-scope groups before IPv4 addresses are configured. * * If inp is non-NULL, use this socket's current FIB number for any * required FIB lookup. * If ina is INADDR_ANY, look up the group address in the unicast FIB, * and use its ifp; usually, this points to the default next-hop. * * If the FIB lookup fails, attempt to use the first non-loopback * interface with multicast capability in the system as a * last resort. The legacy IPv4 ASM API requires that we do * this in order to allow groups to be joined when the routing * table has not yet been populated during boot. * * Returns NULL if no ifp could be found. * * SMPng: TODO: Acquire the appropriate locks for INADDR_TO_IFP. * FUTURE: Implement IPv4 source-address selection. */ static struct ifnet * inp_lookup_mcast_ifp(const struct inpcb *inp, const struct sockaddr_in *gsin, const struct in_addr ina) { struct rm_priotracker in_ifa_tracker; struct ifnet *ifp; + struct nhop4_basic nh4; + uint32_t fibnum; KASSERT(gsin->sin_family == AF_INET, ("%s: not AF_INET", __func__)); KASSERT(IN_MULTICAST(ntohl(gsin->sin_addr.s_addr)), ("%s: not multicast", __func__)); ifp = NULL; if (!in_nullhost(ina)) { INADDR_TO_IFP(ina, ifp); } else { - struct route ro; - - ro.ro_rt = NULL; - memcpy(&ro.ro_dst, gsin, sizeof(struct sockaddr_in)); - in_rtalloc_ign(&ro, 0, inp ? inp->inp_inc.inc_fibnum : 0); - if (ro.ro_rt != NULL) { - ifp = ro.ro_rt->rt_ifp; - KASSERT(ifp != NULL, ("%s: null ifp", __func__)); - RTFREE(ro.ro_rt); - } else { + fibnum = inp ? inp->inp_inc.inc_fibnum : 0; + if (fib4_lookup_nh_basic(fibnum, gsin->sin_addr, 0, 0, &nh4)==0) + ifp = nh4.nh_ifp; + else { struct in_ifaddr *ia; struct ifnet *mifp; mifp = NULL; IN_IFADDR_RLOCK(&in_ifa_tracker); TAILQ_FOREACH(ia, &V_in_ifaddrhead, ia_link) { mifp = ia->ia_ifp; if (!(mifp->if_flags & IFF_LOOPBACK) && (mifp->if_flags & IFF_MULTICAST)) { ifp = mifp; break; } } IN_IFADDR_RUNLOCK(&in_ifa_tracker); } } return (ifp); } /* * Join an IPv4 multicast group, possibly with a source. */ static int inp_join_group(struct inpcb *inp, struct sockopt *sopt) { struct group_source_req gsr; sockunion_t *gsa, *ssa; struct ifnet *ifp; struct in_mfilter *imf; struct ip_moptions *imo; struct in_multi *inm; struct in_msource *lims; size_t idx; int error, is_new; ifp = NULL; imf = NULL; lims = NULL; error = 0; is_new = 0; memset(&gsr, 0, sizeof(struct group_source_req)); gsa = (sockunion_t *)&gsr.gsr_group; gsa->ss.ss_family = AF_UNSPEC; ssa = (sockunion_t *)&gsr.gsr_source; ssa->ss.ss_family = AF_UNSPEC; switch (sopt->sopt_name) { case IP_ADD_MEMBERSHIP: case IP_ADD_SOURCE_MEMBERSHIP: { struct ip_mreq_source mreqs; if (sopt->sopt_name == IP_ADD_MEMBERSHIP) { error = sooptcopyin(sopt, &mreqs, sizeof(struct ip_mreq), sizeof(struct ip_mreq)); /* * Do argument switcharoo from ip_mreq into * ip_mreq_source to avoid using two instances. */ mreqs.imr_interface = mreqs.imr_sourceaddr; mreqs.imr_sourceaddr.s_addr = INADDR_ANY; } else if (sopt->sopt_name == IP_ADD_SOURCE_MEMBERSHIP) { error = sooptcopyin(sopt, &mreqs, sizeof(struct ip_mreq_source), sizeof(struct ip_mreq_source)); } if (error) return (error); gsa->sin.sin_family = AF_INET; gsa->sin.sin_len = sizeof(struct sockaddr_in); gsa->sin.sin_addr = mreqs.imr_multiaddr; if (sopt->sopt_name == IP_ADD_SOURCE_MEMBERSHIP) { ssa->sin.sin_family = AF_INET; ssa->sin.sin_len = sizeof(struct sockaddr_in); ssa->sin.sin_addr = mreqs.imr_sourceaddr; } if (!IN_MULTICAST(ntohl(gsa->sin.sin_addr.s_addr))) return (EINVAL); ifp = inp_lookup_mcast_ifp(inp, &gsa->sin, mreqs.imr_interface); CTR3(KTR_IGMPV3, "%s: imr_interface = %s, ifp = %p", __func__, inet_ntoa(mreqs.imr_interface), ifp); break; } case MCAST_JOIN_GROUP: case MCAST_JOIN_SOURCE_GROUP: if (sopt->sopt_name == MCAST_JOIN_GROUP) { error = sooptcopyin(sopt, &gsr, sizeof(struct group_req), sizeof(struct group_req)); } else if (sopt->sopt_name == MCAST_JOIN_SOURCE_GROUP) { error = sooptcopyin(sopt, &gsr, sizeof(struct group_source_req), sizeof(struct group_source_req)); } if (error) return (error); if (gsa->sin.sin_family != AF_INET || gsa->sin.sin_len != sizeof(struct sockaddr_in)) return (EINVAL); /* * Overwrite the port field if present, as the sockaddr * being copied in may be matched with a binary comparison. */ gsa->sin.sin_port = 0; if (sopt->sopt_name == MCAST_JOIN_SOURCE_GROUP) { if (ssa->sin.sin_family != AF_INET || ssa->sin.sin_len != sizeof(struct sockaddr_in)) return (EINVAL); ssa->sin.sin_port = 0; } if (!IN_MULTICAST(ntohl(gsa->sin.sin_addr.s_addr))) return (EINVAL); if (gsr.gsr_interface == 0 || V_if_index < gsr.gsr_interface) return (EADDRNOTAVAIL); ifp = ifnet_byindex(gsr.gsr_interface); break; default: CTR2(KTR_IGMPV3, "%s: unknown sopt_name %d", __func__, sopt->sopt_name); return (EOPNOTSUPP); break; } if (ifp == NULL || (ifp->if_flags & IFF_MULTICAST) == 0) return (EADDRNOTAVAIL); imo = inp_findmoptions(inp); idx = imo_match_group(imo, ifp, &gsa->sa); if (idx == -1) { is_new = 1; } else { inm = imo->imo_membership[idx]; imf = &imo->imo_mfilters[idx]; if (ssa->ss.ss_family != AF_UNSPEC) { /* * MCAST_JOIN_SOURCE_GROUP on an exclusive membership * is an error. On an existing inclusive membership, * it just adds the source to the filter list. */ if (imf->imf_st[1] != MCAST_INCLUDE) { error = EINVAL; goto out_inp_locked; } /* * Throw out duplicates. * * XXX FIXME: This makes a naive assumption that * even if entries exist for *ssa in this imf, * they will be rejected as dupes, even if they * are not valid in the current mode (in-mode). * * in_msource is transactioned just as for anything * else in SSM -- but note naive use of inm_graft() * below for allocating new filter entries. * * This is only an issue if someone mixes the * full-state SSM API with the delta-based API, * which is discouraged in the relevant RFCs. */ lims = imo_match_source(imo, idx, &ssa->sa); if (lims != NULL /*&& lims->imsl_st[1] == MCAST_INCLUDE*/) { error = EADDRNOTAVAIL; goto out_inp_locked; } } else { /* * MCAST_JOIN_GROUP on an existing exclusive * membership is an error; return EADDRINUSE * to preserve 4.4BSD API idempotence, and * avoid tedious detour to code below. * NOTE: This is bending RFC 3678 a bit. * * On an existing inclusive membership, this is also * an error; if you want to change filter mode, * you must use the userland API setsourcefilter(). * XXX We don't reject this for imf in UNDEFINED * state at t1, because allocation of a filter * is atomic with allocation of a membership. */ error = EINVAL; if (imf->imf_st[1] == MCAST_EXCLUDE) error = EADDRINUSE; goto out_inp_locked; } } /* * Begin state merge transaction at socket layer. */ INP_WLOCK_ASSERT(inp); if (is_new) { if (imo->imo_num_memberships == imo->imo_max_memberships) { error = imo_grow(imo); if (error) goto out_inp_locked; } /* * Allocate the new slot upfront so we can deal with * grafting the new source filter in same code path * as for join-source on existing membership. */ idx = imo->imo_num_memberships; imo->imo_membership[idx] = NULL; imo->imo_num_memberships++; KASSERT(imo->imo_mfilters != NULL, ("%s: imf_mfilters vector was not allocated", __func__)); imf = &imo->imo_mfilters[idx]; KASSERT(RB_EMPTY(&imf->imf_sources), ("%s: imf_sources not empty", __func__)); } /* * Graft new source into filter list for this inpcb's * membership of the group. The in_multi may not have * been allocated yet if this is a new membership, however, * the in_mfilter slot will be allocated and must be initialized. * * Note: Grafting of exclusive mode filters doesn't happen * in this path. * XXX: Should check for non-NULL lims (node exists but may * not be in-mode) for interop with full-state API. */ if (ssa->ss.ss_family != AF_UNSPEC) { /* Membership starts in IN mode */ if (is_new) { CTR1(KTR_IGMPV3, "%s: new join w/source", __func__); imf_init(imf, MCAST_UNDEFINED, MCAST_INCLUDE); } else { CTR2(KTR_IGMPV3, "%s: %s source", __func__, "allow"); } lims = imf_graft(imf, MCAST_INCLUDE, &ssa->sin); if (lims == NULL) { CTR1(KTR_IGMPV3, "%s: merge imf state failed", __func__); error = ENOMEM; goto out_imo_free; } } else { /* No address specified; Membership starts in EX mode */ if (is_new) { CTR1(KTR_IGMPV3, "%s: new join w/o source", __func__); imf_init(imf, MCAST_UNDEFINED, MCAST_EXCLUDE); } } /* * Begin state merge transaction at IGMP layer. */ IN_MULTI_LOCK(); if (is_new) { error = in_joingroup_locked(ifp, &gsa->sin.sin_addr, imf, &inm); if (error) { CTR1(KTR_IGMPV3, "%s: in_joingroup_locked failed", __func__); IN_MULTI_UNLOCK(); goto out_imo_free; } imo->imo_membership[idx] = inm; } else { CTR1(KTR_IGMPV3, "%s: merge inm state", __func__); error = inm_merge(inm, imf); if (error) { CTR1(KTR_IGMPV3, "%s: failed to merge inm state", __func__); goto out_in_multi_locked; } CTR1(KTR_IGMPV3, "%s: doing igmp downcall", __func__); error = igmp_change_state(inm); if (error) { CTR1(KTR_IGMPV3, "%s: failed igmp downcall", __func__); goto out_in_multi_locked; } } out_in_multi_locked: IN_MULTI_UNLOCK(); INP_WLOCK_ASSERT(inp); if (error) { imf_rollback(imf); if (is_new) imf_purge(imf); else imf_reap(imf); } else { imf_commit(imf); } out_imo_free: if (error && is_new) { imo->imo_membership[idx] = NULL; --imo->imo_num_memberships; } out_inp_locked: INP_WUNLOCK(inp); return (error); } /* * Leave an IPv4 multicast group on an inpcb, possibly with a source. */ static int inp_leave_group(struct inpcb *inp, struct sockopt *sopt) { struct group_source_req gsr; struct ip_mreq_source mreqs; sockunion_t *gsa, *ssa; struct ifnet *ifp; struct in_mfilter *imf; struct ip_moptions *imo; struct in_msource *ims; struct in_multi *inm; size_t idx; int error, is_final; ifp = NULL; error = 0; is_final = 1; memset(&gsr, 0, sizeof(struct group_source_req)); gsa = (sockunion_t *)&gsr.gsr_group; gsa->ss.ss_family = AF_UNSPEC; ssa = (sockunion_t *)&gsr.gsr_source; ssa->ss.ss_family = AF_UNSPEC; switch (sopt->sopt_name) { case IP_DROP_MEMBERSHIP: case IP_DROP_SOURCE_MEMBERSHIP: if (sopt->sopt_name == IP_DROP_MEMBERSHIP) { error = sooptcopyin(sopt, &mreqs, sizeof(struct ip_mreq), sizeof(struct ip_mreq)); /* * Swap interface and sourceaddr arguments, * as ip_mreq and ip_mreq_source are laid * out differently. */ mreqs.imr_interface = mreqs.imr_sourceaddr; mreqs.imr_sourceaddr.s_addr = INADDR_ANY; } else if (sopt->sopt_name == IP_DROP_SOURCE_MEMBERSHIP) { error = sooptcopyin(sopt, &mreqs, sizeof(struct ip_mreq_source), sizeof(struct ip_mreq_source)); } if (error) return (error); gsa->sin.sin_family = AF_INET; gsa->sin.sin_len = sizeof(struct sockaddr_in); gsa->sin.sin_addr = mreqs.imr_multiaddr; if (sopt->sopt_name == IP_DROP_SOURCE_MEMBERSHIP) { ssa->sin.sin_family = AF_INET; ssa->sin.sin_len = sizeof(struct sockaddr_in); ssa->sin.sin_addr = mreqs.imr_sourceaddr; } /* * Attempt to look up hinted ifp from interface address. * Fallthrough with null ifp iff lookup fails, to * preserve 4.4BSD mcast API idempotence. * XXX NOTE WELL: The RFC 3678 API is preferred because * using an IPv4 address as a key is racy. */ if (!in_nullhost(mreqs.imr_interface)) INADDR_TO_IFP(mreqs.imr_interface, ifp); CTR3(KTR_IGMPV3, "%s: imr_interface = %s, ifp = %p", __func__, inet_ntoa(mreqs.imr_interface), ifp); break; case MCAST_LEAVE_GROUP: case MCAST_LEAVE_SOURCE_GROUP: if (sopt->sopt_name == MCAST_LEAVE_GROUP) { error = sooptcopyin(sopt, &gsr, sizeof(struct group_req), sizeof(struct group_req)); } else if (sopt->sopt_name == MCAST_LEAVE_SOURCE_GROUP) { error = sooptcopyin(sopt, &gsr, sizeof(struct group_source_req), sizeof(struct group_source_req)); } if (error) return (error); if (gsa->sin.sin_family != AF_INET || gsa->sin.sin_len != sizeof(struct sockaddr_in)) return (EINVAL); if (sopt->sopt_name == MCAST_LEAVE_SOURCE_GROUP) { if (ssa->sin.sin_family != AF_INET || ssa->sin.sin_len != sizeof(struct sockaddr_in)) return (EINVAL); } if (gsr.gsr_interface == 0 || V_if_index < gsr.gsr_interface) return (EADDRNOTAVAIL); ifp = ifnet_byindex(gsr.gsr_interface); if (ifp == NULL) return (EADDRNOTAVAIL); break; default: CTR2(KTR_IGMPV3, "%s: unknown sopt_name %d", __func__, sopt->sopt_name); return (EOPNOTSUPP); break; } if (!IN_MULTICAST(ntohl(gsa->sin.sin_addr.s_addr))) return (EINVAL); /* * Find the membership in the membership array. */ imo = inp_findmoptions(inp); idx = imo_match_group(imo, ifp, &gsa->sa); if (idx == -1) { error = EADDRNOTAVAIL; goto out_inp_locked; } inm = imo->imo_membership[idx]; imf = &imo->imo_mfilters[idx]; if (ssa->ss.ss_family != AF_UNSPEC) is_final = 0; /* * Begin state merge transaction at socket layer. */ INP_WLOCK_ASSERT(inp); /* * If we were instructed only to leave a given source, do so. * MCAST_LEAVE_SOURCE_GROUP is only valid for inclusive memberships. */ if (is_final) { imf_leave(imf); } else { if (imf->imf_st[0] == MCAST_EXCLUDE) { error = EADDRNOTAVAIL; goto out_inp_locked; } ims = imo_match_source(imo, idx, &ssa->sa); if (ims == NULL) { CTR3(KTR_IGMPV3, "%s: source %s %spresent", __func__, inet_ntoa(ssa->sin.sin_addr), "not "); error = EADDRNOTAVAIL; goto out_inp_locked; } CTR2(KTR_IGMPV3, "%s: %s source", __func__, "block"); error = imf_prune(imf, &ssa->sin); if (error) { CTR1(KTR_IGMPV3, "%s: merge imf state failed", __func__); goto out_inp_locked; } } /* * Begin state merge transaction at IGMP layer. */ IN_MULTI_LOCK(); if (is_final) { /* * Give up the multicast address record to which * the membership points. */ (void)in_leavegroup_locked(inm, imf); } else { CTR1(KTR_IGMPV3, "%s: merge inm state", __func__); error = inm_merge(inm, imf); if (error) { CTR1(KTR_IGMPV3, "%s: failed to merge inm state", __func__); goto out_in_multi_locked; } CTR1(KTR_IGMPV3, "%s: doing igmp downcall", __func__); error = igmp_change_state(inm); if (error) { CTR1(KTR_IGMPV3, "%s: failed igmp downcall", __func__); } } out_in_multi_locked: IN_MULTI_UNLOCK(); if (error) imf_rollback(imf); else imf_commit(imf); imf_reap(imf); if (is_final) { /* Remove the gap in the membership and filter array. */ for (++idx; idx < imo->imo_num_memberships; ++idx) { imo->imo_membership[idx-1] = imo->imo_membership[idx]; imo->imo_mfilters[idx-1] = imo->imo_mfilters[idx]; } imo->imo_num_memberships--; } out_inp_locked: INP_WUNLOCK(inp); return (error); } /* * Select the interface for transmitting IPv4 multicast datagrams. * * Either an instance of struct in_addr or an instance of struct ip_mreqn * may be passed to this socket option. An address of INADDR_ANY or an * interface index of 0 is used to remove a previous selection. * When no interface is selected, one is chosen for every send. */ static int inp_set_multicast_if(struct inpcb *inp, struct sockopt *sopt) { struct in_addr addr; struct ip_mreqn mreqn; struct ifnet *ifp; struct ip_moptions *imo; int error; if (sopt->sopt_valsize == sizeof(struct ip_mreqn)) { /* * An interface index was specified using the * Linux-derived ip_mreqn structure. */ error = sooptcopyin(sopt, &mreqn, sizeof(struct ip_mreqn), sizeof(struct ip_mreqn)); if (error) return (error); if (mreqn.imr_ifindex < 0 || V_if_index < mreqn.imr_ifindex) return (EINVAL); if (mreqn.imr_ifindex == 0) { ifp = NULL; } else { ifp = ifnet_byindex(mreqn.imr_ifindex); if (ifp == NULL) return (EADDRNOTAVAIL); } } else { /* * An interface was specified by IPv4 address. * This is the traditional BSD usage. */ error = sooptcopyin(sopt, &addr, sizeof(struct in_addr), sizeof(struct in_addr)); if (error) return (error); if (in_nullhost(addr)) { ifp = NULL; } else { INADDR_TO_IFP(addr, ifp); if (ifp == NULL) return (EADDRNOTAVAIL); } CTR3(KTR_IGMPV3, "%s: ifp = %p, addr = %s", __func__, ifp, inet_ntoa(addr)); } /* Reject interfaces which do not support multicast. */ if (ifp != NULL && (ifp->if_flags & IFF_MULTICAST) == 0) return (EOPNOTSUPP); imo = inp_findmoptions(inp); imo->imo_multicast_ifp = ifp; imo->imo_multicast_addr.s_addr = INADDR_ANY; INP_WUNLOCK(inp); return (0); } /* * Atomically set source filters on a socket for an IPv4 multicast group. * * SMPng: NOTE: Potentially calls malloc(M_WAITOK) with Giant held. */ static int inp_set_source_filters(struct inpcb *inp, struct sockopt *sopt) { struct __msfilterreq msfr; sockunion_t *gsa; struct ifnet *ifp; struct in_mfilter *imf; struct ip_moptions *imo; struct in_multi *inm; size_t idx; int error; error = sooptcopyin(sopt, &msfr, sizeof(struct __msfilterreq), sizeof(struct __msfilterreq)); if (error) return (error); if (msfr.msfr_nsrcs > in_mcast_maxsocksrc) return (ENOBUFS); if ((msfr.msfr_fmode != MCAST_EXCLUDE && msfr.msfr_fmode != MCAST_INCLUDE)) return (EINVAL); if (msfr.msfr_group.ss_family != AF_INET || msfr.msfr_group.ss_len != sizeof(struct sockaddr_in)) return (EINVAL); gsa = (sockunion_t *)&msfr.msfr_group; if (!IN_MULTICAST(ntohl(gsa->sin.sin_addr.s_addr))) return (EINVAL); gsa->sin.sin_port = 0; /* ignore port */ if (msfr.msfr_ifindex == 0 || V_if_index < msfr.msfr_ifindex) return (EADDRNOTAVAIL); ifp = ifnet_byindex(msfr.msfr_ifindex); if (ifp == NULL) return (EADDRNOTAVAIL); /* * Take the INP write lock. * Check if this socket is a member of this group. */ imo = inp_findmoptions(inp); idx = imo_match_group(imo, ifp, &gsa->sa); if (idx == -1 || imo->imo_mfilters == NULL) { error = EADDRNOTAVAIL; goto out_inp_locked; } inm = imo->imo_membership[idx]; imf = &imo->imo_mfilters[idx]; /* * Begin state merge transaction at socket layer. */ INP_WLOCK_ASSERT(inp); imf->imf_st[1] = msfr.msfr_fmode; /* * Apply any new source filters, if present. * Make a copy of the user-space source vector so * that we may copy them with a single copyin. This * allows us to deal with page faults up-front. */ if (msfr.msfr_nsrcs > 0) { struct in_msource *lims; struct sockaddr_in *psin; struct sockaddr_storage *kss, *pkss; int i; INP_WUNLOCK(inp); CTR2(KTR_IGMPV3, "%s: loading %lu source list entries", __func__, (unsigned long)msfr.msfr_nsrcs); kss = malloc(sizeof(struct sockaddr_storage) * msfr.msfr_nsrcs, M_TEMP, M_WAITOK); error = copyin(msfr.msfr_srcs, kss, sizeof(struct sockaddr_storage) * msfr.msfr_nsrcs); if (error) { free(kss, M_TEMP); return (error); } INP_WLOCK(inp); /* * Mark all source filters as UNDEFINED at t1. * Restore new group filter mode, as imf_leave() * will set it to INCLUDE. */ imf_leave(imf); imf->imf_st[1] = msfr.msfr_fmode; /* * Update socket layer filters at t1, lazy-allocating * new entries. This saves a bunch of memory at the * cost of one RB_FIND() per source entry; duplicate * entries in the msfr_nsrcs vector are ignored. * If we encounter an error, rollback transaction. * * XXX This too could be replaced with a set-symmetric * difference like loop to avoid walking from root * every time, as the key space is common. */ for (i = 0, pkss = kss; i < msfr.msfr_nsrcs; i++, pkss++) { psin = (struct sockaddr_in *)pkss; if (psin->sin_family != AF_INET) { error = EAFNOSUPPORT; break; } if (psin->sin_len != sizeof(struct sockaddr_in)) { error = EINVAL; break; } error = imf_get_source(imf, psin, &lims); if (error) break; lims->imsl_st[1] = imf->imf_st[1]; } free(kss, M_TEMP); } if (error) goto out_imf_rollback; INP_WLOCK_ASSERT(inp); IN_MULTI_LOCK(); /* * Begin state merge transaction at IGMP layer. */ CTR1(KTR_IGMPV3, "%s: merge inm state", __func__); error = inm_merge(inm, imf); if (error) { CTR1(KTR_IGMPV3, "%s: failed to merge inm state", __func__); goto out_in_multi_locked; } CTR1(KTR_IGMPV3, "%s: doing igmp downcall", __func__); error = igmp_change_state(inm); if (error) CTR1(KTR_IGMPV3, "%s: failed igmp downcall", __func__); out_in_multi_locked: IN_MULTI_UNLOCK(); out_imf_rollback: if (error) imf_rollback(imf); else imf_commit(imf); imf_reap(imf); out_inp_locked: INP_WUNLOCK(inp); return (error); } /* * Set the IP multicast options in response to user setsockopt(). * * Many of the socket options handled in this function duplicate the * functionality of socket options in the regular unicast API. However, * it is not possible to merge the duplicate code, because the idempotence * of the IPv4 multicast part of the BSD Sockets API must be preserved; * the effects of these options must be treated as separate and distinct. * * SMPng: XXX: Unlocked read of inp_socket believed OK. * FUTURE: The IP_MULTICAST_VIF option may be eliminated if MROUTING * is refactored to no longer use vifs. */ int inp_setmoptions(struct inpcb *inp, struct sockopt *sopt) { struct ip_moptions *imo; int error; error = 0; /* * If socket is neither of type SOCK_RAW or SOCK_DGRAM, * or is a divert socket, reject it. */ if (inp->inp_socket->so_proto->pr_protocol == IPPROTO_DIVERT || (inp->inp_socket->so_proto->pr_type != SOCK_RAW && inp->inp_socket->so_proto->pr_type != SOCK_DGRAM)) return (EOPNOTSUPP); switch (sopt->sopt_name) { case IP_MULTICAST_VIF: { int vifi; /* * Select a multicast VIF for transmission. * Only useful if multicast forwarding is active. */ if (legal_vif_num == NULL) { error = EOPNOTSUPP; break; } error = sooptcopyin(sopt, &vifi, sizeof(int), sizeof(int)); if (error) break; if (!legal_vif_num(vifi) && (vifi != -1)) { error = EINVAL; break; } imo = inp_findmoptions(inp); imo->imo_multicast_vif = vifi; INP_WUNLOCK(inp); break; } case IP_MULTICAST_IF: error = inp_set_multicast_if(inp, sopt); break; case IP_MULTICAST_TTL: { u_char ttl; /* * Set the IP time-to-live for outgoing multicast packets. * The original multicast API required a char argument, * which is inconsistent with the rest of the socket API. * We allow either a char or an int. */ if (sopt->sopt_valsize == sizeof(u_char)) { error = sooptcopyin(sopt, &ttl, sizeof(u_char), sizeof(u_char)); if (error) break; } else { u_int ittl; error = sooptcopyin(sopt, &ittl, sizeof(u_int), sizeof(u_int)); if (error) break; if (ittl > 255) { error = EINVAL; break; } ttl = (u_char)ittl; } imo = inp_findmoptions(inp); imo->imo_multicast_ttl = ttl; INP_WUNLOCK(inp); break; } case IP_MULTICAST_LOOP: { u_char loop; /* * Set the loopback flag for outgoing multicast packets. * Must be zero or one. The original multicast API required a * char argument, which is inconsistent with the rest * of the socket API. We allow either a char or an int. */ if (sopt->sopt_valsize == sizeof(u_char)) { error = sooptcopyin(sopt, &loop, sizeof(u_char), sizeof(u_char)); if (error) break; } else { u_int iloop; error = sooptcopyin(sopt, &iloop, sizeof(u_int), sizeof(u_int)); if (error) break; loop = (u_char)iloop; } imo = inp_findmoptions(inp); imo->imo_multicast_loop = !!loop; INP_WUNLOCK(inp); break; } case IP_ADD_MEMBERSHIP: case IP_ADD_SOURCE_MEMBERSHIP: case MCAST_JOIN_GROUP: case MCAST_JOIN_SOURCE_GROUP: error = inp_join_group(inp, sopt); break; case IP_DROP_MEMBERSHIP: case IP_DROP_SOURCE_MEMBERSHIP: case MCAST_LEAVE_GROUP: case MCAST_LEAVE_SOURCE_GROUP: error = inp_leave_group(inp, sopt); break; case IP_BLOCK_SOURCE: case IP_UNBLOCK_SOURCE: case MCAST_BLOCK_SOURCE: case MCAST_UNBLOCK_SOURCE: error = inp_block_unblock_source(inp, sopt); break; case IP_MSFILTER: error = inp_set_source_filters(inp, sopt); break; default: error = EOPNOTSUPP; break; } INP_UNLOCK_ASSERT(inp); return (error); } /* * Expose IGMP's multicast filter mode and source list(s) to userland, * keyed by (ifindex, group). * The filter mode is written out as a uint32_t, followed by * 0..n of struct in_addr. * For use by ifmcstat(8). * SMPng: NOTE: unlocked read of ifindex space. */ static int sysctl_ip_mcast_filters(SYSCTL_HANDLER_ARGS) { struct in_addr src, group; struct ifnet *ifp; struct ifmultiaddr *ifma; struct in_multi *inm; struct ip_msource *ims; int *name; int retval; u_int namelen; uint32_t fmode, ifindex; name = (int *)arg1; namelen = arg2; if (req->newptr != NULL) return (EPERM); if (namelen != 2) return (EINVAL); ifindex = name[0]; if (ifindex <= 0 || ifindex > V_if_index) { CTR2(KTR_IGMPV3, "%s: ifindex %u out of range", __func__, ifindex); return (ENOENT); } group.s_addr = name[1]; if (!IN_MULTICAST(ntohl(group.s_addr))) { CTR2(KTR_IGMPV3, "%s: group %s is not multicast", __func__, inet_ntoa(group)); return (EINVAL); } ifp = ifnet_byindex(ifindex); if (ifp == NULL) { CTR2(KTR_IGMPV3, "%s: no ifp for ifindex %u", __func__, ifindex); return (ENOENT); } retval = sysctl_wire_old_buffer(req, sizeof(uint32_t) + (in_mcast_maxgrpsrc * sizeof(struct in_addr))); if (retval) return (retval); IN_MULTI_LOCK(); IF_ADDR_RLOCK(ifp); TAILQ_FOREACH(ifma, &ifp->if_multiaddrs, ifma_link) { if (ifma->ifma_addr->sa_family != AF_INET || ifma->ifma_protospec == NULL) continue; inm = (struct in_multi *)ifma->ifma_protospec; if (!in_hosteq(inm->inm_addr, group)) continue; fmode = inm->inm_st[1].iss_fmode; retval = SYSCTL_OUT(req, &fmode, sizeof(uint32_t)); if (retval != 0) break; RB_FOREACH(ims, ip_msource_tree, &inm->inm_srcs) { #ifdef KTR struct in_addr ina; ina.s_addr = htonl(ims->ims_haddr); CTR2(KTR_IGMPV3, "%s: visit node %s", __func__, inet_ntoa(ina)); #endif /* * Only copy-out sources which are in-mode. */ if (fmode != ims_get_mode(inm, ims, 1)) { CTR1(KTR_IGMPV3, "%s: skip non-in-mode", __func__); continue; } src.s_addr = htonl(ims->ims_haddr); retval = SYSCTL_OUT(req, &src, sizeof(struct in_addr)); if (retval != 0) break; } } IF_ADDR_RUNLOCK(ifp); IN_MULTI_UNLOCK(); return (retval); } #if defined(KTR) && (KTR_COMPILE & KTR_IGMPV3) static const char *inm_modestrs[] = { "un", "in", "ex" }; static const char * inm_mode_str(const int mode) { if (mode >= MCAST_UNDEFINED && mode <= MCAST_EXCLUDE) return (inm_modestrs[mode]); return ("??"); } static const char *inm_statestrs[] = { "not-member", "silent", "idle", "lazy", "sleeping", "awakening", "query-pending", "sg-query-pending", "leaving" }; static const char * inm_state_str(const int state) { if (state >= IGMP_NOT_MEMBER && state <= IGMP_LEAVING_MEMBER) return (inm_statestrs[state]); return ("??"); } /* * Dump an in_multi structure to the console. */ void inm_print(const struct in_multi *inm) { int t; if ((ktr_mask & KTR_IGMPV3) == 0) return; printf("%s: --- begin inm %p ---\n", __func__, inm); printf("addr %s ifp %p(%s) ifma %p\n", inet_ntoa(inm->inm_addr), inm->inm_ifp, inm->inm_ifp->if_xname, inm->inm_ifma); printf("timer %u state %s refcount %u scq.len %u\n", inm->inm_timer, inm_state_str(inm->inm_state), inm->inm_refcount, inm->inm_scq.mq_len); printf("igi %p nsrc %lu sctimer %u scrv %u\n", inm->inm_igi, inm->inm_nsrc, inm->inm_sctimer, inm->inm_scrv); for (t = 0; t < 2; t++) { printf("t%d: fmode %s asm %u ex %u in %u rec %u\n", t, inm_mode_str(inm->inm_st[t].iss_fmode), inm->inm_st[t].iss_asm, inm->inm_st[t].iss_ex, inm->inm_st[t].iss_in, inm->inm_st[t].iss_rec); } printf("%s: --- end inm %p ---\n", __func__, inm); } #else /* !KTR || !(KTR_COMPILE & KTR_IGMPV3) */ void inm_print(const struct in_multi *inm) { } #endif /* KTR && (KTR_COMPILE & KTR_IGMPV3) */ RB_GENERATE(ip_msource_tree, ip_msource, ims_link, ip_msource_cmp); Index: user/ngie/more-tests2/sys/netinet/ip_icmp.c =================================================================== --- user/ngie/more-tests2/sys/netinet/ip_icmp.c (revision 292053) +++ user/ngie/more-tests2/sys/netinet/ip_icmp.c (revision 292054) @@ -1,969 +1,969 @@ /*- * Copyright (c) 1982, 1986, 1988, 1993 * The Regents of the University of California. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * @(#)ip_icmp.c 8.2 (Berkeley) 1/4/94 */ #include __FBSDID("$FreeBSD$"); #include "opt_inet.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include +#include #include #include #include #include #include #include #include #include #include #include #include #ifdef INET #include #include #endif /* INET */ /* * ICMP routines: error generation, receive packet processing, and * routines to turnaround packets back to the originator, and * host table maintenance routines. */ static VNET_DEFINE(int, icmplim) = 200; #define V_icmplim VNET(icmplim) SYSCTL_INT(_net_inet_icmp, ICMPCTL_ICMPLIM, icmplim, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(icmplim), 0, "Maximum number of ICMP responses per second"); static VNET_DEFINE(int, icmplim_output) = 1; #define V_icmplim_output VNET(icmplim_output) SYSCTL_INT(_net_inet_icmp, OID_AUTO, icmplim_output, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(icmplim_output), 0, "Enable logging of ICMP response rate limiting"); #ifdef INET VNET_PCPUSTAT_DEFINE(struct icmpstat, icmpstat); VNET_PCPUSTAT_SYSINIT(icmpstat); SYSCTL_VNET_PCPUSTAT(_net_inet_icmp, ICMPCTL_STATS, stats, struct icmpstat, icmpstat, "ICMP statistics (struct icmpstat, netinet/icmp_var.h)"); #ifdef VIMAGE VNET_PCPUSTAT_SYSUNINIT(icmpstat); #endif /* VIMAGE */ static VNET_DEFINE(int, icmpmaskrepl) = 0; #define V_icmpmaskrepl VNET(icmpmaskrepl) SYSCTL_INT(_net_inet_icmp, ICMPCTL_MASKREPL, maskrepl, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(icmpmaskrepl), 0, "Reply to ICMP Address Mask Request packets."); static VNET_DEFINE(u_int, icmpmaskfake) = 0; #define V_icmpmaskfake VNET(icmpmaskfake) SYSCTL_UINT(_net_inet_icmp, OID_AUTO, maskfake, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(icmpmaskfake), 0, "Fake reply to ICMP Address Mask Request packets."); VNET_DEFINE(int, drop_redirect) = 0; #define V_drop_redirect VNET(drop_redirect) SYSCTL_INT(_net_inet_icmp, OID_AUTO, drop_redirect, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(drop_redirect), 0, "Ignore ICMP redirects"); static VNET_DEFINE(int, log_redirect) = 0; #define V_log_redirect VNET(log_redirect) SYSCTL_INT(_net_inet_icmp, OID_AUTO, log_redirect, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(log_redirect), 0, "Log ICMP redirects to the console"); static VNET_DEFINE(char, reply_src[IFNAMSIZ]); #define V_reply_src VNET(reply_src) SYSCTL_STRING(_net_inet_icmp, OID_AUTO, reply_src, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(reply_src), IFNAMSIZ, "icmp reply source for non-local packets."); static VNET_DEFINE(int, icmp_rfi) = 0; #define V_icmp_rfi VNET(icmp_rfi) SYSCTL_INT(_net_inet_icmp, OID_AUTO, reply_from_interface, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(icmp_rfi), 0, "ICMP reply from incoming interface for non-local packets"); static VNET_DEFINE(int, icmp_quotelen) = 8; #define V_icmp_quotelen VNET(icmp_quotelen) SYSCTL_INT(_net_inet_icmp, OID_AUTO, quotelen, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(icmp_quotelen), 0, "Number of bytes from original packet to quote in ICMP reply"); /* * ICMP broadcast echo sysctl */ static VNET_DEFINE(int, icmpbmcastecho) = 0; #define V_icmpbmcastecho VNET(icmpbmcastecho) SYSCTL_INT(_net_inet_icmp, OID_AUTO, bmcastecho, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(icmpbmcastecho), 0, ""); static VNET_DEFINE(int, icmptstamprepl) = 1; #define V_icmptstamprepl VNET(icmptstamprepl) SYSCTL_INT(_net_inet_icmp, OID_AUTO, tstamprepl, CTLFLAG_RW, &VNET_NAME(icmptstamprepl), 0, "Respond to ICMP Timestamp packets"); #ifdef ICMPPRINTFS int icmpprintfs = 0; #endif static void icmp_reflect(struct mbuf *); static void icmp_send(struct mbuf *, struct mbuf *); extern struct protosw inetsw[]; /* * Kernel module interface for updating icmpstat. The argument is an index * into icmpstat treated as an array of u_long. While this encodes the * general layout of icmpstat into the caller, it doesn't encode its * location, so that future changes to add, for example, per-CPU stats * support won't cause binary compatibility problems for kernel modules. */ void kmod_icmpstat_inc(int statnum) { counter_u64_add(VNET(icmpstat)[statnum], 1); } /* * Generate an error packet of type error * in response to bad packet ip. */ void icmp_error(struct mbuf *n, int type, int code, uint32_t dest, int mtu) { register struct ip *oip = mtod(n, struct ip *), *nip; register unsigned oiphlen = oip->ip_hl << 2; register struct icmp *icp; register struct mbuf *m; unsigned icmplen, icmpelen, nlen; KASSERT((u_int)type <= ICMP_MAXTYPE, ("%s: illegal ICMP type", __func__)); #ifdef ICMPPRINTFS if (icmpprintfs) printf("icmp_error(%p, %x, %d)\n", oip, type, code); #endif if (type != ICMP_REDIRECT) ICMPSTAT_INC(icps_error); /* * Don't send error: * if the original packet was encrypted. * if not the first fragment of message. * in response to a multicast or broadcast packet. * if the old packet protocol was an ICMP error message. */ if (n->m_flags & M_DECRYPTED) goto freeit; if (oip->ip_off & htons(~(IP_MF|IP_DF))) goto freeit; if (n->m_flags & (M_BCAST|M_MCAST)) goto freeit; if (oip->ip_p == IPPROTO_ICMP && type != ICMP_REDIRECT && n->m_len >= oiphlen + ICMP_MINLEN && !ICMP_INFOTYPE(((struct icmp *)((caddr_t)oip + oiphlen))->icmp_type)) { ICMPSTAT_INC(icps_oldicmp); goto freeit; } /* Drop if IP header plus 8 bytes is not contignous in first mbuf. */ if (oiphlen + 8 > n->m_len) goto freeit; /* * Calculate length to quote from original packet and * prevent the ICMP mbuf from overflowing. * Unfortunatly this is non-trivial since ip_forward() * sends us truncated packets. */ nlen = m_length(n, NULL); if (oip->ip_p == IPPROTO_TCP) { struct tcphdr *th; int tcphlen; if (oiphlen + sizeof(struct tcphdr) > n->m_len && n->m_next == NULL) goto stdreply; if (n->m_len < oiphlen + sizeof(struct tcphdr) && ((n = m_pullup(n, oiphlen + sizeof(struct tcphdr))) == NULL)) goto freeit; th = (struct tcphdr *)((caddr_t)oip + oiphlen); tcphlen = th->th_off << 2; if (tcphlen < sizeof(struct tcphdr)) goto freeit; if (ntohs(oip->ip_len) < oiphlen + tcphlen) goto freeit; if (oiphlen + tcphlen > n->m_len && n->m_next == NULL) goto stdreply; if (n->m_len < oiphlen + tcphlen && ((n = m_pullup(n, oiphlen + tcphlen)) == NULL)) goto freeit; icmpelen = max(tcphlen, min(V_icmp_quotelen, ntohs(oip->ip_len) - oiphlen)); } else stdreply: icmpelen = max(8, min(V_icmp_quotelen, ntohs(oip->ip_len) - oiphlen)); icmplen = min(oiphlen + icmpelen, nlen); if (icmplen < sizeof(struct ip)) goto freeit; if (MHLEN > sizeof(struct ip) + ICMP_MINLEN + icmplen) m = m_gethdr(M_NOWAIT, MT_DATA); else m = m_getcl(M_NOWAIT, MT_DATA, M_PKTHDR); if (m == NULL) goto freeit; #ifdef MAC mac_netinet_icmp_reply(n, m); #endif icmplen = min(icmplen, M_TRAILINGSPACE(m) - sizeof(struct ip) - ICMP_MINLEN); m_align(m, ICMP_MINLEN + icmplen); m->m_len = ICMP_MINLEN + icmplen; /* XXX MRT make the outgoing packet use the same FIB * that was associated with the incoming packet */ M_SETFIB(m, M_GETFIB(n)); icp = mtod(m, struct icmp *); ICMPSTAT_INC(icps_outhist[type]); icp->icmp_type = type; if (type == ICMP_REDIRECT) icp->icmp_gwaddr.s_addr = dest; else { icp->icmp_void = 0; /* * The following assignments assume an overlay with the * just zeroed icmp_void field. */ if (type == ICMP_PARAMPROB) { icp->icmp_pptr = code; code = 0; } else if (type == ICMP_UNREACH && code == ICMP_UNREACH_NEEDFRAG && mtu) { icp->icmp_nextmtu = htons(mtu); } } icp->icmp_code = code; /* * Copy the quotation into ICMP message and * convert quoted IP header back to network representation. */ m_copydata(n, 0, icmplen, (caddr_t)&icp->icmp_ip); nip = &icp->icmp_ip; /* * Set up ICMP message mbuf and copy old IP header (without options * in front of ICMP message. * If the original mbuf was meant to bypass the firewall, the error * reply should bypass as well. */ m->m_flags |= n->m_flags & M_SKIP_FIREWALL; m->m_data -= sizeof(struct ip); m->m_len += sizeof(struct ip); m->m_pkthdr.len = m->m_len; m->m_pkthdr.rcvif = n->m_pkthdr.rcvif; nip = mtod(m, struct ip *); bcopy((caddr_t)oip, (caddr_t)nip, sizeof(struct ip)); nip->ip_len = htons(m->m_len); nip->ip_v = IPVERSION; nip->ip_hl = 5; nip->ip_p = IPPROTO_ICMP; nip->ip_tos = 0; nip->ip_off = 0; icmp_reflect(m); freeit: m_freem(n); } /* * Process a received ICMP message. */ int icmp_input(struct mbuf **mp, int *offp, int proto) { struct icmp *icp; struct in_ifaddr *ia; struct mbuf *m = *mp; struct ip *ip = mtod(m, struct ip *); struct sockaddr_in icmpsrc, icmpdst, icmpgw; int hlen = *offp; int icmplen = ntohs(ip->ip_len) - *offp; int i, code; void (*ctlfunc)(int, struct sockaddr *, void *); int fibnum; *mp = NULL; /* * Locate icmp structure in mbuf, and check * that not corrupted and of at least minimum length. */ #ifdef ICMPPRINTFS if (icmpprintfs) { char buf[4 * sizeof "123"]; strcpy(buf, inet_ntoa(ip->ip_src)); printf("icmp_input from %s to %s, len %d\n", buf, inet_ntoa(ip->ip_dst), icmplen); } #endif if (icmplen < ICMP_MINLEN) { ICMPSTAT_INC(icps_tooshort); goto freeit; } i = hlen + min(icmplen, ICMP_ADVLENMIN); if (m->m_len < i && (m = m_pullup(m, i)) == NULL) { ICMPSTAT_INC(icps_tooshort); return (IPPROTO_DONE); } ip = mtod(m, struct ip *); m->m_len -= hlen; m->m_data += hlen; icp = mtod(m, struct icmp *); if (in_cksum(m, icmplen)) { ICMPSTAT_INC(icps_checksum); goto freeit; } m->m_len += hlen; m->m_data -= hlen; #ifdef ICMPPRINTFS if (icmpprintfs) printf("icmp_input, type %d code %d\n", icp->icmp_type, icp->icmp_code); #endif /* * Message type specific processing. */ if (icp->icmp_type > ICMP_MAXTYPE) goto raw; /* Initialize */ bzero(&icmpsrc, sizeof(icmpsrc)); icmpsrc.sin_len = sizeof(struct sockaddr_in); icmpsrc.sin_family = AF_INET; bzero(&icmpdst, sizeof(icmpdst)); icmpdst.sin_len = sizeof(struct sockaddr_in); icmpdst.sin_family = AF_INET; bzero(&icmpgw, sizeof(icmpgw)); icmpgw.sin_len = sizeof(struct sockaddr_in); icmpgw.sin_family = AF_INET; ICMPSTAT_INC(icps_inhist[icp->icmp_type]); code = icp->icmp_code; switch (icp->icmp_type) { case ICMP_UNREACH: switch (code) { case ICMP_UNREACH_NET: case ICMP_UNREACH_HOST: case ICMP_UNREACH_SRCFAIL: case ICMP_UNREACH_NET_UNKNOWN: case ICMP_UNREACH_HOST_UNKNOWN: case ICMP_UNREACH_ISOLATED: case ICMP_UNREACH_TOSNET: case ICMP_UNREACH_TOSHOST: case ICMP_UNREACH_HOST_PRECEDENCE: case ICMP_UNREACH_PRECEDENCE_CUTOFF: code = PRC_UNREACH_NET; break; case ICMP_UNREACH_NEEDFRAG: code = PRC_MSGSIZE; break; /* * RFC 1122, Sections 3.2.2.1 and 4.2.3.9. * Treat subcodes 2,3 as immediate RST */ case ICMP_UNREACH_PROTOCOL: case ICMP_UNREACH_PORT: code = PRC_UNREACH_PORT; break; case ICMP_UNREACH_NET_PROHIB: case ICMP_UNREACH_HOST_PROHIB: case ICMP_UNREACH_FILTER_PROHIB: code = PRC_UNREACH_ADMIN_PROHIB; break; default: goto badcode; } goto deliver; case ICMP_TIMXCEED: if (code > 1) goto badcode; code += PRC_TIMXCEED_INTRANS; goto deliver; case ICMP_PARAMPROB: if (code > 1) goto badcode; code = PRC_PARAMPROB; deliver: /* * Problem with datagram; advise higher level routines. */ if (icmplen < ICMP_ADVLENMIN || icmplen < ICMP_ADVLEN(icp) || icp->icmp_ip.ip_hl < (sizeof(struct ip) >> 2)) { ICMPSTAT_INC(icps_badlen); goto freeit; } /* Discard ICMP's in response to multicast packets */ if (IN_MULTICAST(ntohl(icp->icmp_ip.ip_dst.s_addr))) goto badcode; #ifdef ICMPPRINTFS if (icmpprintfs) printf("deliver to protocol %d\n", icp->icmp_ip.ip_p); #endif icmpsrc.sin_addr = icp->icmp_ip.ip_dst; /* * XXX if the packet contains [IPv4 AH TCP], we can't make a * notification to TCP layer. */ ctlfunc = inetsw[ip_protox[icp->icmp_ip.ip_p]].pr_ctlinput; if (ctlfunc) (*ctlfunc)(code, (struct sockaddr *)&icmpsrc, (void *)&icp->icmp_ip); break; badcode: ICMPSTAT_INC(icps_badcode); break; case ICMP_ECHO: if (!V_icmpbmcastecho && (m->m_flags & (M_MCAST | M_BCAST)) != 0) { ICMPSTAT_INC(icps_bmcastecho); break; } icp->icmp_type = ICMP_ECHOREPLY; if (badport_bandlim(BANDLIM_ICMP_ECHO) < 0) goto freeit; else goto reflect; case ICMP_TSTAMP: if (V_icmptstamprepl == 0) break; if (!V_icmpbmcastecho && (m->m_flags & (M_MCAST | M_BCAST)) != 0) { ICMPSTAT_INC(icps_bmcasttstamp); break; } if (icmplen < ICMP_TSLEN) { ICMPSTAT_INC(icps_badlen); break; } icp->icmp_type = ICMP_TSTAMPREPLY; icp->icmp_rtime = iptime(); icp->icmp_ttime = icp->icmp_rtime; /* bogus, do later! */ if (badport_bandlim(BANDLIM_ICMP_TSTAMP) < 0) goto freeit; else goto reflect; case ICMP_MASKREQ: if (V_icmpmaskrepl == 0) break; /* * We are not able to respond with all ones broadcast * unless we receive it over a point-to-point interface. */ if (icmplen < ICMP_MASKLEN) break; switch (ip->ip_dst.s_addr) { case INADDR_BROADCAST: case INADDR_ANY: icmpdst.sin_addr = ip->ip_src; break; default: icmpdst.sin_addr = ip->ip_dst; } ia = (struct in_ifaddr *)ifaof_ifpforaddr( (struct sockaddr *)&icmpdst, m->m_pkthdr.rcvif); if (ia == NULL) break; if (ia->ia_ifp == NULL) { ifa_free(&ia->ia_ifa); break; } icp->icmp_type = ICMP_MASKREPLY; if (V_icmpmaskfake == 0) icp->icmp_mask = ia->ia_sockmask.sin_addr.s_addr; else icp->icmp_mask = V_icmpmaskfake; if (ip->ip_src.s_addr == 0) { if (ia->ia_ifp->if_flags & IFF_BROADCAST) ip->ip_src = satosin(&ia->ia_broadaddr)->sin_addr; else if (ia->ia_ifp->if_flags & IFF_POINTOPOINT) ip->ip_src = satosin(&ia->ia_dstaddr)->sin_addr; } ifa_free(&ia->ia_ifa); reflect: ICMPSTAT_INC(icps_reflect); ICMPSTAT_INC(icps_outhist[icp->icmp_type]); icmp_reflect(m); return (IPPROTO_DONE); case ICMP_REDIRECT: if (V_log_redirect) { u_long src, dst, gw; src = ntohl(ip->ip_src.s_addr); dst = ntohl(icp->icmp_ip.ip_dst.s_addr); gw = ntohl(icp->icmp_gwaddr.s_addr); printf("icmp redirect from %d.%d.%d.%d: " "%d.%d.%d.%d => %d.%d.%d.%d\n", (int)(src >> 24), (int)((src >> 16) & 0xff), (int)((src >> 8) & 0xff), (int)(src & 0xff), (int)(dst >> 24), (int)((dst >> 16) & 0xff), (int)((dst >> 8) & 0xff), (int)(dst & 0xff), (int)(gw >> 24), (int)((gw >> 16) & 0xff), (int)((gw >> 8) & 0xff), (int)(gw & 0xff)); } /* * RFC1812 says we must ignore ICMP redirects if we * are acting as router. */ if (V_drop_redirect || V_ipforwarding) break; if (code > 3) goto badcode; if (icmplen < ICMP_ADVLENMIN || icmplen < ICMP_ADVLEN(icp) || icp->icmp_ip.ip_hl < (sizeof(struct ip) >> 2)) { ICMPSTAT_INC(icps_badlen); break; } /* * Short circuit routing redirects to force * immediate change in the kernel's routing * tables. The message is also handed to anyone * listening on a raw socket (e.g. the routing * daemon for use in updating its tables). */ icmpgw.sin_addr = ip->ip_src; icmpdst.sin_addr = icp->icmp_gwaddr; #ifdef ICMPPRINTFS if (icmpprintfs) { char buf[4 * sizeof "123"]; strcpy(buf, inet_ntoa(icp->icmp_ip.ip_dst)); printf("redirect dst %s to %s\n", buf, inet_ntoa(icp->icmp_gwaddr)); } #endif icmpsrc.sin_addr = icp->icmp_ip.ip_dst; for ( fibnum = 0; fibnum < rt_numfibs; fibnum++) { in_rtredirect((struct sockaddr *)&icmpsrc, (struct sockaddr *)&icmpdst, (struct sockaddr *)0, RTF_GATEWAY | RTF_HOST, (struct sockaddr *)&icmpgw, fibnum); } pfctlinput(PRC_REDIRECT_HOST, (struct sockaddr *)&icmpsrc); break; /* * No kernel processing for the following; * just fall through to send to raw listener. */ case ICMP_ECHOREPLY: case ICMP_ROUTERADVERT: case ICMP_ROUTERSOLICIT: case ICMP_TSTAMPREPLY: case ICMP_IREQREPLY: case ICMP_MASKREPLY: case ICMP_SOURCEQUENCH: default: break; } raw: *mp = m; rip_input(mp, offp, proto); return (IPPROTO_DONE); freeit: m_freem(m); return (IPPROTO_DONE); } /* * Reflect the ip packet back to the source */ static void icmp_reflect(struct mbuf *m) { struct rm_priotracker in_ifa_tracker; struct ip *ip = mtod(m, struct ip *); struct ifaddr *ifa; struct ifnet *ifp; struct in_ifaddr *ia; struct in_addr t; + struct nhop4_extended nh_ext; struct mbuf *opts = 0; int optlen = (ip->ip_hl << 2) - sizeof(struct ip); if (IN_MULTICAST(ntohl(ip->ip_src.s_addr)) || IN_EXPERIMENTAL(ntohl(ip->ip_src.s_addr)) || IN_ZERONET(ntohl(ip->ip_src.s_addr)) ) { m_freem(m); /* Bad return address */ ICMPSTAT_INC(icps_badaddr); goto done; /* Ip_output() will check for broadcast */ } t = ip->ip_dst; ip->ip_dst = ip->ip_src; /* * Source selection for ICMP replies: * * If the incoming packet was addressed directly to one of our * own addresses, use dst as the src for the reply. */ IN_IFADDR_RLOCK(&in_ifa_tracker); LIST_FOREACH(ia, INADDR_HASH(t.s_addr), ia_hash) { if (t.s_addr == IA_SIN(ia)->sin_addr.s_addr) { t = IA_SIN(ia)->sin_addr; IN_IFADDR_RUNLOCK(&in_ifa_tracker); goto match; } } IN_IFADDR_RUNLOCK(&in_ifa_tracker); /* * If the incoming packet was addressed to one of our broadcast * addresses, use the first non-broadcast address which corresponds * to the incoming interface. */ ifp = m->m_pkthdr.rcvif; if (ifp != NULL && ifp->if_flags & IFF_BROADCAST) { IF_ADDR_RLOCK(ifp); TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) { if (ifa->ifa_addr->sa_family != AF_INET) continue; ia = ifatoia(ifa); if (satosin(&ia->ia_broadaddr)->sin_addr.s_addr == t.s_addr) { t = IA_SIN(ia)->sin_addr; IF_ADDR_RUNLOCK(ifp); goto match; } } IF_ADDR_RUNLOCK(ifp); } /* * If the packet was transiting through us, use the address of * the interface the packet came through in. If that interface * doesn't have a suitable IP address, the normal selection * criteria apply. */ if (V_icmp_rfi && ifp != NULL) { IF_ADDR_RLOCK(ifp); TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) { if (ifa->ifa_addr->sa_family != AF_INET) continue; ia = ifatoia(ifa); t = IA_SIN(ia)->sin_addr; IF_ADDR_RUNLOCK(ifp); goto match; } IF_ADDR_RUNLOCK(ifp); } /* * If the incoming packet was not addressed directly to us, use * designated interface for icmp replies specified by sysctl * net.inet.icmp.reply_src (default not set). Otherwise continue * with normal source selection. */ if (V_reply_src[0] != '\0' && (ifp = ifunit(V_reply_src))) { IF_ADDR_RLOCK(ifp); TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) { if (ifa->ifa_addr->sa_family != AF_INET) continue; ia = ifatoia(ifa); t = IA_SIN(ia)->sin_addr; IF_ADDR_RUNLOCK(ifp); goto match; } IF_ADDR_RUNLOCK(ifp); } /* * If the packet was transiting through us, use the address of * the interface that is the closest to the packet source. * When we don't have a route back to the packet source, stop here * and drop the packet. */ - ia = ip_rtaddr(ip->ip_dst, M_GETFIB(m)); - if (ia == NULL) { + if (fib4_lookup_nh_ext(M_GETFIB(m), ip->ip_dst, 0, 0, &nh_ext) != 0) { m_freem(m); ICMPSTAT_INC(icps_noroute); goto done; } - t = IA_SIN(ia)->sin_addr; - ifa_free(&ia->ia_ifa); + t = nh_ext.nh_src; match: #ifdef MAC mac_netinet_icmp_replyinplace(m); #endif ip->ip_src = t; ip->ip_ttl = V_ip_defttl; if (optlen > 0) { register u_char *cp; int opt, cnt; u_int len; /* * Retrieve any source routing from the incoming packet; * add on any record-route or timestamp options. */ cp = (u_char *) (ip + 1); if ((opts = ip_srcroute(m)) == 0 && (opts = m_gethdr(M_NOWAIT, MT_DATA))) { opts->m_len = sizeof(struct in_addr); mtod(opts, struct in_addr *)->s_addr = 0; } if (opts) { #ifdef ICMPPRINTFS if (icmpprintfs) printf("icmp_reflect optlen %d rt %d => ", optlen, opts->m_len); #endif for (cnt = optlen; cnt > 0; cnt -= len, cp += len) { opt = cp[IPOPT_OPTVAL]; if (opt == IPOPT_EOL) break; if (opt == IPOPT_NOP) len = 1; else { if (cnt < IPOPT_OLEN + sizeof(*cp)) break; len = cp[IPOPT_OLEN]; if (len < IPOPT_OLEN + sizeof(*cp) || len > cnt) break; } /* * Should check for overflow, but it "can't happen" */ if (opt == IPOPT_RR || opt == IPOPT_TS || opt == IPOPT_SECURITY) { bcopy((caddr_t)cp, mtod(opts, caddr_t) + opts->m_len, len); opts->m_len += len; } } /* Terminate & pad, if necessary */ cnt = opts->m_len % 4; if (cnt) { for (; cnt < 4; cnt++) { *(mtod(opts, caddr_t) + opts->m_len) = IPOPT_EOL; opts->m_len++; } } #ifdef ICMPPRINTFS if (icmpprintfs) printf("%d\n", opts->m_len); #endif } ip_stripoptions(m); } m_tag_delete_nonpersistent(m); m->m_flags &= ~(M_BCAST|M_MCAST); icmp_send(m, opts); done: if (opts) (void)m_free(opts); } /* * Send an icmp packet back to the ip level, * after supplying a checksum. */ static void icmp_send(struct mbuf *m, struct mbuf *opts) { register struct ip *ip = mtod(m, struct ip *); register int hlen; register struct icmp *icp; hlen = ip->ip_hl << 2; m->m_data += hlen; m->m_len -= hlen; icp = mtod(m, struct icmp *); icp->icmp_cksum = 0; icp->icmp_cksum = in_cksum(m, ntohs(ip->ip_len) - hlen); m->m_data -= hlen; m->m_len += hlen; m->m_pkthdr.rcvif = (struct ifnet *)0; #ifdef ICMPPRINTFS if (icmpprintfs) { char buf[4 * sizeof "123"]; strcpy(buf, inet_ntoa(ip->ip_dst)); printf("icmp_send dst %s src %s\n", buf, inet_ntoa(ip->ip_src)); } #endif (void) ip_output(m, opts, NULL, 0, NULL, NULL); } /* * Return milliseconds since 00:00 UTC in network format. */ uint32_t iptime(void) { struct timeval atv; u_long t; getmicrotime(&atv); t = (atv.tv_sec % (24*60*60)) * 1000 + atv.tv_usec / 1000; return (htonl(t)); } /* * Return the next larger or smaller MTU plateau (table from RFC 1191) * given current value MTU. If DIR is less than zero, a larger plateau * is returned; otherwise, a smaller value is returned. */ int ip_next_mtu(int mtu, int dir) { static int mtutab[] = { 65535, 32000, 17914, 8166, 4352, 2002, 1492, 1280, 1006, 508, 296, 68, 0 }; int i, size; size = (sizeof mtutab) / (sizeof mtutab[0]); if (dir >= 0) { for (i = 0; i < size; i++) if (mtu > mtutab[i]) return mtutab[i]; } else { for (i = size - 1; i >= 0; i--) if (mtu < mtutab[i]) return mtutab[i]; if (mtu == mtutab[0]) return mtutab[0]; } return 0; } #endif /* INET */ /* * badport_bandlim() - check for ICMP bandwidth limit * * Return 0 if it is ok to send an ICMP error response, -1 if we have * hit our bandwidth limit and it is not ok. * * If icmplim is <= 0, the feature is disabled and 0 is returned. * * For now we separate the TCP and UDP subsystems w/ different 'which' * values. We may eventually remove this separation (and simplify the * code further). * * Note that the printing of the error message is delayed so we can * properly print the icmp error rate that the system was trying to do * (i.e. 22000/100 pps, etc...). This can cause long delays in printing * the 'final' error, but it doesn't make sense to solve the printing * delay with more complex code. */ int badport_bandlim(int which) { #define N(a) (sizeof (a) / sizeof (a[0])) static struct rate { const char *type; struct timeval lasttime; int curpps; } rates[BANDLIM_MAX+1] = { { "icmp unreach response" }, { "icmp ping response" }, { "icmp tstamp response" }, { "closed port RST response" }, { "open port RST response" }, { "icmp6 unreach response" }, { "sctp ootb response" } }; /* * Return ok status if feature disabled or argument out of range. */ if (V_icmplim > 0 && (u_int) which < N(rates)) { struct rate *r = &rates[which]; int opps = r->curpps; if (!ppsratecheck(&r->lasttime, &r->curpps, V_icmplim)) return -1; /* discard packet */ /* * If we've dropped below the threshold after having * rate-limited traffic print the message. This preserves * the previous behaviour at the expense of added complexity. */ if (V_icmplim_output && opps > V_icmplim) log(LOG_NOTICE, "Limiting %s from %d to %d packets/sec\n", r->type, opps, V_icmplim); } return 0; /* okay to send packet */ #undef N } Index: user/ngie/more-tests2/sys/netinet/ip_input.c =================================================================== --- user/ngie/more-tests2/sys/netinet/ip_input.c (revision 292053) +++ user/ngie/more-tests2/sys/netinet/ip_input.c (revision 292054) @@ -1,1371 +1,1344 @@ /*- * Copyright (c) 1982, 1986, 1988, 1993 * The Regents of the University of California. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * @(#)ip_input.c 8.2 (Berkeley) 1/4/94 */ #include __FBSDID("$FreeBSD$"); #include "opt_bootp.h" #include "opt_ipfw.h" #include "opt_ipstealth.h" #include "opt_ipsec.h" #include "opt_route.h" #include "opt_rss.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef IPSEC #include #include #include #endif /* IPSEC */ #include #include #include #ifdef CTASSERT CTASSERT(sizeof(struct ip) == 20); #endif /* IP reassembly functions are defined in ip_reass.c. */ extern void ipreass_init(void); extern void ipreass_drain(void); extern void ipreass_slowtimo(void); #ifdef VIMAGE extern void ipreass_destroy(void); #endif struct rmlock in_ifaddr_lock; RM_SYSINIT(in_ifaddr_lock, &in_ifaddr_lock, "in_ifaddr_lock"); VNET_DEFINE(int, rsvp_on); VNET_DEFINE(int, ipforwarding); SYSCTL_INT(_net_inet_ip, IPCTL_FORWARDING, forwarding, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(ipforwarding), 0, "Enable IP forwarding between interfaces"); static VNET_DEFINE(int, ipsendredirects) = 1; /* XXX */ #define V_ipsendredirects VNET(ipsendredirects) SYSCTL_INT(_net_inet_ip, IPCTL_SENDREDIRECTS, redirect, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(ipsendredirects), 0, "Enable sending IP redirects"); /* * XXX - Setting ip_checkinterface mostly implements the receive side of * the Strong ES model described in RFC 1122, but since the routing table * and transmit implementation do not implement the Strong ES model, * setting this to 1 results in an odd hybrid. * * XXX - ip_checkinterface currently must be disabled if you use ipnat * to translate the destination address to another local interface. * * XXX - ip_checkinterface must be disabled if you add IP aliases * to the loopback interface instead of the interface where the * packets for those addresses are received. */ static VNET_DEFINE(int, ip_checkinterface); #define V_ip_checkinterface VNET(ip_checkinterface) SYSCTL_INT(_net_inet_ip, OID_AUTO, check_interface, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(ip_checkinterface), 0, "Verify packet arrives on correct interface"); VNET_DEFINE(struct pfil_head, inet_pfil_hook); /* Packet filter hooks */ static struct netisr_handler ip_nh = { .nh_name = "ip", .nh_handler = ip_input, .nh_proto = NETISR_IP, #ifdef RSS .nh_m2cpuid = rss_soft_m2cpuid_v4, .nh_policy = NETISR_POLICY_CPU, .nh_dispatch = NETISR_DISPATCH_HYBRID, #else .nh_policy = NETISR_POLICY_FLOW, #endif }; #ifdef RSS /* * Directly dispatched frames are currently assumed * to have a flowid already calculated. * * It should likely have something that assert it * actually has valid flow details. */ static struct netisr_handler ip_direct_nh = { .nh_name = "ip_direct", .nh_handler = ip_direct_input, .nh_proto = NETISR_IP_DIRECT, .nh_m2cpuid = rss_soft_m2cpuid_v4, .nh_policy = NETISR_POLICY_CPU, .nh_dispatch = NETISR_DISPATCH_HYBRID, }; #endif extern struct domain inetdomain; extern struct protosw inetsw[]; u_char ip_protox[IPPROTO_MAX]; VNET_DEFINE(struct in_ifaddrhead, in_ifaddrhead); /* first inet address */ VNET_DEFINE(struct in_ifaddrhashhead *, in_ifaddrhashtbl); /* inet addr hash table */ VNET_DEFINE(u_long, in_ifaddrhmask); /* mask for hash table */ #ifdef IPCTL_DEFMTU SYSCTL_INT(_net_inet_ip, IPCTL_DEFMTU, mtu, CTLFLAG_RW, &ip_mtu, 0, "Default MTU"); #endif #ifdef IPSTEALTH VNET_DEFINE(int, ipstealth); SYSCTL_INT(_net_inet_ip, OID_AUTO, stealth, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(ipstealth), 0, "IP stealth mode, no TTL decrementation on forwarding"); #endif /* * IP statistics are stored in the "array" of counter(9)s. */ VNET_PCPUSTAT_DEFINE(struct ipstat, ipstat); VNET_PCPUSTAT_SYSINIT(ipstat); SYSCTL_VNET_PCPUSTAT(_net_inet_ip, IPCTL_STATS, stats, struct ipstat, ipstat, "IP statistics (struct ipstat, netinet/ip_var.h)"); #ifdef VIMAGE VNET_PCPUSTAT_SYSUNINIT(ipstat); #endif /* VIMAGE */ /* * Kernel module interface for updating ipstat. The argument is an index * into ipstat treated as an array. */ void kmod_ipstat_inc(int statnum) { counter_u64_add(VNET(ipstat)[statnum], 1); } void kmod_ipstat_dec(int statnum) { counter_u64_add(VNET(ipstat)[statnum], -1); } static int sysctl_netinet_intr_queue_maxlen(SYSCTL_HANDLER_ARGS) { int error, qlimit; netisr_getqlimit(&ip_nh, &qlimit); error = sysctl_handle_int(oidp, &qlimit, 0, req); if (error || !req->newptr) return (error); if (qlimit < 1) return (EINVAL); return (netisr_setqlimit(&ip_nh, qlimit)); } SYSCTL_PROC(_net_inet_ip, IPCTL_INTRQMAXLEN, intr_queue_maxlen, CTLTYPE_INT|CTLFLAG_RW, 0, 0, sysctl_netinet_intr_queue_maxlen, "I", "Maximum size of the IP input queue"); static int sysctl_netinet_intr_queue_drops(SYSCTL_HANDLER_ARGS) { u_int64_t qdrops_long; int error, qdrops; netisr_getqdrops(&ip_nh, &qdrops_long); qdrops = qdrops_long; error = sysctl_handle_int(oidp, &qdrops, 0, req); if (error || !req->newptr) return (error); if (qdrops != 0) return (EINVAL); netisr_clearqdrops(&ip_nh); return (0); } SYSCTL_PROC(_net_inet_ip, IPCTL_INTRQDROPS, intr_queue_drops, CTLTYPE_INT|CTLFLAG_RD, 0, 0, sysctl_netinet_intr_queue_drops, "I", "Number of packets dropped from the IP input queue"); #ifdef RSS static int sysctl_netinet_intr_direct_queue_maxlen(SYSCTL_HANDLER_ARGS) { int error, qlimit; netisr_getqlimit(&ip_direct_nh, &qlimit); error = sysctl_handle_int(oidp, &qlimit, 0, req); if (error || !req->newptr) return (error); if (qlimit < 1) return (EINVAL); return (netisr_setqlimit(&ip_direct_nh, qlimit)); } SYSCTL_PROC(_net_inet_ip, IPCTL_INTRQMAXLEN, intr_direct_queue_maxlen, CTLTYPE_INT|CTLFLAG_RW, 0, 0, sysctl_netinet_intr_direct_queue_maxlen, "I", "Maximum size of the IP direct input queue"); static int sysctl_netinet_intr_direct_queue_drops(SYSCTL_HANDLER_ARGS) { u_int64_t qdrops_long; int error, qdrops; netisr_getqdrops(&ip_direct_nh, &qdrops_long); qdrops = qdrops_long; error = sysctl_handle_int(oidp, &qdrops, 0, req); if (error || !req->newptr) return (error); if (qdrops != 0) return (EINVAL); netisr_clearqdrops(&ip_direct_nh); return (0); } SYSCTL_PROC(_net_inet_ip, IPCTL_INTRQDROPS, intr_direct_queue_drops, CTLTYPE_INT|CTLFLAG_RD, 0, 0, sysctl_netinet_intr_direct_queue_drops, "I", "Number of packets dropped from the IP direct input queue"); #endif /* RSS */ /* * IP initialization: fill in IP protocol switch table. * All protocols not implemented in kernel go to raw IP protocol handler. */ void ip_init(void) { struct protosw *pr; int i; TAILQ_INIT(&V_in_ifaddrhead); V_in_ifaddrhashtbl = hashinit(INADDR_NHASH, M_IFADDR, &V_in_ifaddrhmask); /* Initialize IP reassembly queue. */ ipreass_init(); /* Initialize packet filter hooks. */ V_inet_pfil_hook.ph_type = PFIL_TYPE_AF; V_inet_pfil_hook.ph_af = AF_INET; if ((i = pfil_head_register(&V_inet_pfil_hook)) != 0) printf("%s: WARNING: unable to register pfil hook, " "error %d\n", __func__, i); if (hhook_head_register(HHOOK_TYPE_IPSEC_IN, AF_INET, &V_ipsec_hhh_in[HHOOK_IPSEC_INET], HHOOK_WAITOK | HHOOK_HEADISINVNET) != 0) printf("%s: WARNING: unable to register input helper hook\n", __func__); if (hhook_head_register(HHOOK_TYPE_IPSEC_OUT, AF_INET, &V_ipsec_hhh_out[HHOOK_IPSEC_INET], HHOOK_WAITOK | HHOOK_HEADISINVNET) != 0) printf("%s: WARNING: unable to register output helper hook\n", __func__); /* Skip initialization of globals for non-default instances. */ if (!IS_DEFAULT_VNET(curvnet)) return; pr = pffindproto(PF_INET, IPPROTO_RAW, SOCK_RAW); if (pr == NULL) panic("ip_init: PF_INET not found"); /* Initialize the entire ip_protox[] array to IPPROTO_RAW. */ for (i = 0; i < IPPROTO_MAX; i++) ip_protox[i] = pr - inetsw; /* * Cycle through IP protocols and put them into the appropriate place * in ip_protox[]. */ for (pr = inetdomain.dom_protosw; pr < inetdomain.dom_protoswNPROTOSW; pr++) if (pr->pr_domain->dom_family == PF_INET && pr->pr_protocol && pr->pr_protocol != IPPROTO_RAW) { /* Be careful to only index valid IP protocols. */ if (pr->pr_protocol < IPPROTO_MAX) ip_protox[pr->pr_protocol] = pr - inetsw; } netisr_register(&ip_nh); #ifdef RSS netisr_register(&ip_direct_nh); #endif } #ifdef VIMAGE void ip_destroy(void) { int error; if ((error = pfil_head_unregister(&V_inet_pfil_hook)) != 0) printf("%s: WARNING: unable to unregister pfil hook, " "error %d\n", __func__, error); error = hhook_head_deregister(V_ipsec_hhh_in[HHOOK_IPSEC_INET]); if (error != 0) { printf("%s: WARNING: unable to deregister input helper hook " "type HHOOK_TYPE_IPSEC_IN, id HHOOK_IPSEC_INET: " "error %d returned\n", __func__, error); } error = hhook_head_deregister(V_ipsec_hhh_out[HHOOK_IPSEC_INET]); if (error != 0) { printf("%s: WARNING: unable to deregister output helper hook " "type HHOOK_TYPE_IPSEC_OUT, id HHOOK_IPSEC_INET: " "error %d returned\n", __func__, error); } /* Cleanup in_ifaddr hash table; should be empty. */ hashdestroy(V_in_ifaddrhashtbl, M_IFADDR, V_in_ifaddrhmask); /* Destroy IP reassembly queue. */ ipreass_destroy(); } #endif #ifdef RSS /* * IP direct input routine. * * This is called when reinjecting completed fragments where * all of the previous checking and book-keeping has been done. */ void ip_direct_input(struct mbuf *m) { struct ip *ip; int hlen; ip = mtod(m, struct ip *); hlen = ip->ip_hl << 2; IPSTAT_INC(ips_delivered); (*inetsw[ip_protox[ip->ip_p]].pr_input)(&m, &hlen, ip->ip_p); return; } #endif /* * Ip input routine. Checksum and byte swap header. If fragmented * try to reassemble. Process options. Pass to next level. */ void ip_input(struct mbuf *m) { struct ip *ip = NULL; struct in_ifaddr *ia = NULL; struct ifaddr *ifa; struct ifnet *ifp; int checkif, hlen = 0; uint16_t sum, ip_len; int dchg = 0; /* dest changed after fw */ struct in_addr odst; /* original dst address */ M_ASSERTPKTHDR(m); if (m->m_flags & M_FASTFWD_OURS) { m->m_flags &= ~M_FASTFWD_OURS; /* Set up some basics that will be used later. */ ip = mtod(m, struct ip *); hlen = ip->ip_hl << 2; ip_len = ntohs(ip->ip_len); goto ours; } IPSTAT_INC(ips_total); if (m->m_pkthdr.len < sizeof(struct ip)) goto tooshort; if (m->m_len < sizeof (struct ip) && (m = m_pullup(m, sizeof (struct ip))) == NULL) { IPSTAT_INC(ips_toosmall); return; } ip = mtod(m, struct ip *); if (ip->ip_v != IPVERSION) { IPSTAT_INC(ips_badvers); goto bad; } hlen = ip->ip_hl << 2; if (hlen < sizeof(struct ip)) { /* minimum header length */ IPSTAT_INC(ips_badhlen); goto bad; } if (hlen > m->m_len) { if ((m = m_pullup(m, hlen)) == NULL) { IPSTAT_INC(ips_badhlen); return; } ip = mtod(m, struct ip *); } IP_PROBE(receive, NULL, NULL, ip, m->m_pkthdr.rcvif, ip, NULL); /* 127/8 must not appear on wire - RFC1122 */ ifp = m->m_pkthdr.rcvif; if ((ntohl(ip->ip_dst.s_addr) >> IN_CLASSA_NSHIFT) == IN_LOOPBACKNET || (ntohl(ip->ip_src.s_addr) >> IN_CLASSA_NSHIFT) == IN_LOOPBACKNET) { if ((ifp->if_flags & IFF_LOOPBACK) == 0) { IPSTAT_INC(ips_badaddr); goto bad; } } if (m->m_pkthdr.csum_flags & CSUM_IP_CHECKED) { sum = !(m->m_pkthdr.csum_flags & CSUM_IP_VALID); } else { if (hlen == sizeof(struct ip)) { sum = in_cksum_hdr(ip); } else { sum = in_cksum(m, hlen); } } if (sum) { IPSTAT_INC(ips_badsum); goto bad; } #ifdef ALTQ if (altq_input != NULL && (*altq_input)(m, AF_INET) == 0) /* packet is dropped by traffic conditioner */ return; #endif ip_len = ntohs(ip->ip_len); if (ip_len < hlen) { IPSTAT_INC(ips_badlen); goto bad; } /* * Check that the amount of data in the buffers * is as at least much as the IP header would have us expect. * Trim mbufs if longer than we expect. * Drop packet if shorter than we expect. */ if (m->m_pkthdr.len < ip_len) { tooshort: IPSTAT_INC(ips_tooshort); goto bad; } if (m->m_pkthdr.len > ip_len) { if (m->m_len == m->m_pkthdr.len) { m->m_len = ip_len; m->m_pkthdr.len = ip_len; } else m_adj(m, ip_len - m->m_pkthdr.len); } /* Try to forward the packet, but if we fail continue */ #ifdef IPSEC /* For now we do not handle IPSEC in tryforward. */ if (!key_havesp(IPSEC_DIR_INBOUND) && !key_havesp(IPSEC_DIR_OUTBOUND) && (V_ipforwarding == 1)) if (ip_tryforward(m) == NULL) return; /* * Bypass packet filtering for packets previously handled by IPsec. */ if (ip_ipsec_filtertunnel(m)) goto passin; #else if (V_ipforwarding == 1) if (ip_tryforward(m) == NULL) return; #endif /* IPSEC */ /* * Run through list of hooks for input packets. * * NB: Beware of the destination address changing (e.g. * by NAT rewriting). When this happens, tell * ip_forward to do the right thing. */ /* Jump over all PFIL processing if hooks are not active. */ if (!PFIL_HOOKED(&V_inet_pfil_hook)) goto passin; odst = ip->ip_dst; if (pfil_run_hooks(&V_inet_pfil_hook, &m, ifp, PFIL_IN, NULL) != 0) return; if (m == NULL) /* consumed by filter */ return; ip = mtod(m, struct ip *); dchg = (odst.s_addr != ip->ip_dst.s_addr); ifp = m->m_pkthdr.rcvif; if (m->m_flags & M_FASTFWD_OURS) { m->m_flags &= ~M_FASTFWD_OURS; goto ours; } if (m->m_flags & M_IP_NEXTHOP) { dchg = (m_tag_find(m, PACKET_TAG_IPFORWARD, NULL) != NULL); if (dchg != 0) { /* * Directly ship the packet on. This allows * forwarding packets originally destined to us * to some other directly connected host. */ ip_forward(m, 1); return; } } passin: /* * Process options and, if not destined for us, * ship it on. ip_dooptions returns 1 when an * error was detected (causing an icmp message * to be sent and the original packet to be freed). */ if (hlen > sizeof (struct ip) && ip_dooptions(m, 0)) return; /* greedy RSVP, snatches any PATH packet of the RSVP protocol and no * matter if it is destined to another node, or whether it is * a multicast one, RSVP wants it! and prevents it from being forwarded * anywhere else. Also checks if the rsvp daemon is running before * grabbing the packet. */ if (V_rsvp_on && ip->ip_p==IPPROTO_RSVP) goto ours; /* * Check our list of addresses, to see if the packet is for us. * If we don't have any addresses, assume any unicast packet * we receive might be for us (and let the upper layers deal * with it). */ if (TAILQ_EMPTY(&V_in_ifaddrhead) && (m->m_flags & (M_MCAST|M_BCAST)) == 0) goto ours; /* * Enable a consistency check between the destination address * and the arrival interface for a unicast packet (the RFC 1122 * strong ES model) if IP forwarding is disabled and the packet * is not locally generated and the packet is not subject to * 'ipfw fwd'. * * XXX - Checking also should be disabled if the destination * address is ipnat'ed to a different interface. * * XXX - Checking is incompatible with IP aliases added * to the loopback interface instead of the interface where * the packets are received. * * XXX - This is the case for carp vhost IPs as well so we * insert a workaround. If the packet got here, we already * checked with carp_iamatch() and carp_forus(). */ checkif = V_ip_checkinterface && (V_ipforwarding == 0) && ifp != NULL && ((ifp->if_flags & IFF_LOOPBACK) == 0) && ifp->if_carp == NULL && (dchg == 0); /* * Check for exact addresses in the hash bucket. */ /* IN_IFADDR_RLOCK(); */ LIST_FOREACH(ia, INADDR_HASH(ip->ip_dst.s_addr), ia_hash) { /* * If the address matches, verify that the packet * arrived via the correct interface if checking is * enabled. */ if (IA_SIN(ia)->sin_addr.s_addr == ip->ip_dst.s_addr && (!checkif || ia->ia_ifp == ifp)) { counter_u64_add(ia->ia_ifa.ifa_ipackets, 1); counter_u64_add(ia->ia_ifa.ifa_ibytes, m->m_pkthdr.len); /* IN_IFADDR_RUNLOCK(); */ goto ours; } } /* IN_IFADDR_RUNLOCK(); */ /* * Check for broadcast addresses. * * Only accept broadcast packets that arrive via the matching * interface. Reception of forwarded directed broadcasts would * be handled via ip_forward() and ether_output() with the loopback * into the stack for SIMPLEX interfaces handled by ether_output(). */ if (ifp != NULL && ifp->if_flags & IFF_BROADCAST) { IF_ADDR_RLOCK(ifp); TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) { if (ifa->ifa_addr->sa_family != AF_INET) continue; ia = ifatoia(ifa); if (satosin(&ia->ia_broadaddr)->sin_addr.s_addr == ip->ip_dst.s_addr) { counter_u64_add(ia->ia_ifa.ifa_ipackets, 1); counter_u64_add(ia->ia_ifa.ifa_ibytes, m->m_pkthdr.len); IF_ADDR_RUNLOCK(ifp); goto ours; } #ifdef BOOTP_COMPAT if (IA_SIN(ia)->sin_addr.s_addr == INADDR_ANY) { counter_u64_add(ia->ia_ifa.ifa_ipackets, 1); counter_u64_add(ia->ia_ifa.ifa_ibytes, m->m_pkthdr.len); IF_ADDR_RUNLOCK(ifp); goto ours; } #endif } IF_ADDR_RUNLOCK(ifp); ia = NULL; } /* RFC 3927 2.7: Do not forward datagrams for 169.254.0.0/16. */ if (IN_LINKLOCAL(ntohl(ip->ip_dst.s_addr))) { IPSTAT_INC(ips_cantforward); m_freem(m); return; } if (IN_MULTICAST(ntohl(ip->ip_dst.s_addr))) { if (V_ip_mrouter) { /* * If we are acting as a multicast router, all * incoming multicast packets are passed to the * kernel-level multicast forwarding function. * The packet is returned (relatively) intact; if * ip_mforward() returns a non-zero value, the packet * must be discarded, else it may be accepted below. */ if (ip_mforward && ip_mforward(ip, ifp, m, 0) != 0) { IPSTAT_INC(ips_cantforward); m_freem(m); return; } /* * The process-level routing daemon needs to receive * all multicast IGMP packets, whether or not this * host belongs to their destination groups. */ if (ip->ip_p == IPPROTO_IGMP) goto ours; IPSTAT_INC(ips_forward); } /* * Assume the packet is for us, to avoid prematurely taking * a lock on the in_multi hash. Protocols must perform * their own filtering and update statistics accordingly. */ goto ours; } if (ip->ip_dst.s_addr == (u_long)INADDR_BROADCAST) goto ours; if (ip->ip_dst.s_addr == INADDR_ANY) goto ours; /* * Not for us; forward if possible and desirable. */ if (V_ipforwarding == 0) { IPSTAT_INC(ips_cantforward); m_freem(m); } else { ip_forward(m, dchg); } return; ours: #ifdef IPSTEALTH /* * IPSTEALTH: Process non-routing options only * if the packet is destined for us. */ if (V_ipstealth && hlen > sizeof (struct ip) && ip_dooptions(m, 1)) return; #endif /* IPSTEALTH */ /* * Attempt reassembly; if it succeeds, proceed. * ip_reass() will return a different mbuf. */ if (ip->ip_off & htons(IP_MF | IP_OFFMASK)) { /* XXXGL: shouldn't we save & set m_flags? */ m = ip_reass(m); if (m == NULL) return; ip = mtod(m, struct ip *); /* Get the header length of the reassembled packet */ hlen = ip->ip_hl << 2; } #ifdef IPSEC /* * enforce IPsec policy checking if we are seeing last header. * note that we do not visit this with protocols with pcb layer * code - like udp/tcp/raw ip. */ if (ip_ipsec_input(m, ip->ip_p) != 0) goto bad; #endif /* IPSEC */ /* * Switch out to protocol's input routine. */ IPSTAT_INC(ips_delivered); (*inetsw[ip_protox[ip->ip_p]].pr_input)(&m, &hlen, ip->ip_p); return; bad: m_freem(m); } /* * IP timer processing; * if a timer expires on a reassembly * queue, discard it. */ void ip_slowtimo(void) { VNET_ITERATOR_DECL(vnet_iter); VNET_LIST_RLOCK_NOSLEEP(); VNET_FOREACH(vnet_iter) { CURVNET_SET(vnet_iter); ipreass_slowtimo(); CURVNET_RESTORE(); } VNET_LIST_RUNLOCK_NOSLEEP(); } void ip_drain(void) { VNET_ITERATOR_DECL(vnet_iter); VNET_LIST_RLOCK_NOSLEEP(); VNET_FOREACH(vnet_iter) { CURVNET_SET(vnet_iter); ipreass_drain(); CURVNET_RESTORE(); } VNET_LIST_RUNLOCK_NOSLEEP(); } /* * The protocol to be inserted into ip_protox[] must be already registered * in inetsw[], either statically or through pf_proto_register(). */ int ipproto_register(short ipproto) { struct protosw *pr; /* Sanity checks. */ if (ipproto <= 0 || ipproto >= IPPROTO_MAX) return (EPROTONOSUPPORT); /* * The protocol slot must not be occupied by another protocol * already. An index pointing to IPPROTO_RAW is unused. */ pr = pffindproto(PF_INET, IPPROTO_RAW, SOCK_RAW); if (pr == NULL) return (EPFNOSUPPORT); if (ip_protox[ipproto] != pr - inetsw) /* IPPROTO_RAW */ return (EEXIST); /* Find the protocol position in inetsw[] and set the index. */ for (pr = inetdomain.dom_protosw; pr < inetdomain.dom_protoswNPROTOSW; pr++) { if (pr->pr_domain->dom_family == PF_INET && pr->pr_protocol && pr->pr_protocol == ipproto) { ip_protox[pr->pr_protocol] = pr - inetsw; return (0); } } return (EPROTONOSUPPORT); } int ipproto_unregister(short ipproto) { struct protosw *pr; /* Sanity checks. */ if (ipproto <= 0 || ipproto >= IPPROTO_MAX) return (EPROTONOSUPPORT); /* Check if the protocol was indeed registered. */ pr = pffindproto(PF_INET, IPPROTO_RAW, SOCK_RAW); if (pr == NULL) return (EPFNOSUPPORT); if (ip_protox[ipproto] == pr - inetsw) /* IPPROTO_RAW */ return (ENOENT); /* Reset the protocol slot to IPPROTO_RAW. */ ip_protox[ipproto] = pr - inetsw; return (0); } -/* - * Given address of next destination (final or next hop), return (referenced) - * internet address info of interface to be used to get there. - */ -struct in_ifaddr * -ip_rtaddr(struct in_addr dst, u_int fibnum) -{ - struct route sro; - struct sockaddr_in *sin; - struct in_ifaddr *ia; - - bzero(&sro, sizeof(sro)); - sin = (struct sockaddr_in *)&sro.ro_dst; - sin->sin_family = AF_INET; - sin->sin_len = sizeof(*sin); - sin->sin_addr = dst; - in_rtalloc_ign(&sro, 0, fibnum); - - if (sro.ro_rt == NULL) - return (NULL); - - ia = ifatoia(sro.ro_rt->rt_ifa); - ifa_ref(&ia->ia_ifa); - RTFREE(sro.ro_rt); - return (ia); -} - u_char inetctlerrmap[PRC_NCMDS] = { 0, 0, 0, 0, 0, EMSGSIZE, EHOSTDOWN, EHOSTUNREACH, EHOSTUNREACH, EHOSTUNREACH, ECONNREFUSED, ECONNREFUSED, EMSGSIZE, EHOSTUNREACH, 0, 0, 0, 0, EHOSTUNREACH, 0, ENOPROTOOPT, ECONNREFUSED }; /* * Forward a packet. If some error occurs return the sender * an icmp packet. Note we can't always generate a meaningful * icmp message because icmp doesn't have a large enough repertoire * of codes and types. * * If not forwarding, just drop the packet. This could be confusing * if ipforwarding was zero but some routing protocol was advancing * us as a gateway to somewhere. However, we must let the routing * protocol deal with that. * * The srcrt parameter indicates whether the packet is being forwarded * via a source route. */ void ip_forward(struct mbuf *m, int srcrt) { struct ip *ip = mtod(m, struct ip *); struct in_ifaddr *ia; struct mbuf *mcopy; struct sockaddr_in *sin; struct in_addr dest; struct route ro; int error, type = 0, code = 0, mtu = 0; if (m->m_flags & (M_BCAST|M_MCAST) || in_canforward(ip->ip_dst) == 0) { IPSTAT_INC(ips_cantforward); m_freem(m); return; } #ifdef IPSEC if (ip_ipsec_fwd(m) != 0) { IPSTAT_INC(ips_cantforward); m_freem(m); return; } #endif /* IPSEC */ #ifdef IPSTEALTH if (!V_ipstealth) { #endif if (ip->ip_ttl <= IPTTLDEC) { icmp_error(m, ICMP_TIMXCEED, ICMP_TIMXCEED_INTRANS, 0, 0); return; } #ifdef IPSTEALTH } #endif bzero(&ro, sizeof(ro)); sin = (struct sockaddr_in *)&ro.ro_dst; sin->sin_family = AF_INET; sin->sin_len = sizeof(*sin); sin->sin_addr = ip->ip_dst; #ifdef RADIX_MPATH rtalloc_mpath_fib(&ro, ntohl(ip->ip_src.s_addr ^ ip->ip_dst.s_addr), M_GETFIB(m)); #else in_rtalloc_ign(&ro, 0, M_GETFIB(m)); #endif if (ro.ro_rt != NULL) { ia = ifatoia(ro.ro_rt->rt_ifa); ifa_ref(&ia->ia_ifa); } else ia = NULL; #ifndef IPSEC /* * 'ia' may be NULL if there is no route for this destination. * In case of IPsec, Don't discard it just yet, but pass it to * ip_output in case of outgoing IPsec policy. */ if (!srcrt && ia == NULL) { icmp_error(m, ICMP_UNREACH, ICMP_UNREACH_HOST, 0, 0); RO_RTFREE(&ro); return; } #endif /* * Save the IP header and at most 8 bytes of the payload, * in case we need to generate an ICMP message to the src. * * XXX this can be optimized a lot by saving the data in a local * buffer on the stack (72 bytes at most), and only allocating the * mbuf if really necessary. The vast majority of the packets * are forwarded without having to send an ICMP back (either * because unnecessary, or because rate limited), so we are * really we are wasting a lot of work here. * * We don't use m_copy() because it might return a reference * to a shared cluster. Both this function and ip_output() * assume exclusive access to the IP header in `m', so any * data in a cluster may change before we reach icmp_error(). */ mcopy = m_gethdr(M_NOWAIT, m->m_type); if (mcopy != NULL && !m_dup_pkthdr(mcopy, m, M_NOWAIT)) { /* * It's probably ok if the pkthdr dup fails (because * the deep copy of the tag chain failed), but for now * be conservative and just discard the copy since * code below may some day want the tags. */ m_free(mcopy); mcopy = NULL; } if (mcopy != NULL) { mcopy->m_len = min(ntohs(ip->ip_len), M_TRAILINGSPACE(mcopy)); mcopy->m_pkthdr.len = mcopy->m_len; m_copydata(m, 0, mcopy->m_len, mtod(mcopy, caddr_t)); } #ifdef IPSTEALTH if (!V_ipstealth) { #endif ip->ip_ttl -= IPTTLDEC; #ifdef IPSTEALTH } #endif /* * If forwarding packet using same interface that it came in on, * perhaps should send a redirect to sender to shortcut a hop. * Only send redirect if source is sending directly to us, * and if packet was not source routed (or has any options). * Also, don't send redirect if forwarding using a default route * or a route modified by a redirect. */ dest.s_addr = 0; if (!srcrt && V_ipsendredirects && ia != NULL && ia->ia_ifp == m->m_pkthdr.rcvif) { struct rtentry *rt; rt = ro.ro_rt; if (rt && (rt->rt_flags & (RTF_DYNAMIC|RTF_MODIFIED)) == 0 && satosin(rt_key(rt))->sin_addr.s_addr != 0) { #define RTA(rt) ((struct in_ifaddr *)(rt->rt_ifa)) u_long src = ntohl(ip->ip_src.s_addr); if (RTA(rt) && (src & RTA(rt)->ia_subnetmask) == RTA(rt)->ia_subnet) { if (rt->rt_flags & RTF_GATEWAY) dest.s_addr = satosin(rt->rt_gateway)->sin_addr.s_addr; else dest.s_addr = ip->ip_dst.s_addr; /* Router requirements says to only send host redirects */ type = ICMP_REDIRECT; code = ICMP_REDIRECT_HOST; } } } error = ip_output(m, NULL, &ro, IP_FORWARDING, NULL, NULL); if (error == EMSGSIZE && ro.ro_rt) mtu = ro.ro_rt->rt_mtu; RO_RTFREE(&ro); if (error) IPSTAT_INC(ips_cantforward); else { IPSTAT_INC(ips_forward); if (type) IPSTAT_INC(ips_redirectsent); else { if (mcopy) m_freem(mcopy); if (ia != NULL) ifa_free(&ia->ia_ifa); return; } } if (mcopy == NULL) { if (ia != NULL) ifa_free(&ia->ia_ifa); return; } switch (error) { case 0: /* forwarded, but need redirect */ /* type, code set above */ break; case ENETUNREACH: case EHOSTUNREACH: case ENETDOWN: case EHOSTDOWN: default: type = ICMP_UNREACH; code = ICMP_UNREACH_HOST; break; case EMSGSIZE: type = ICMP_UNREACH; code = ICMP_UNREACH_NEEDFRAG; #ifdef IPSEC /* * If IPsec is configured for this path, * override any possibly mtu value set by ip_output. */ mtu = ip_ipsec_mtu(mcopy, mtu); #endif /* IPSEC */ /* * If the MTU was set before make sure we are below the * interface MTU. * If the MTU wasn't set before use the interface mtu or * fall back to the next smaller mtu step compared to the * current packet size. */ if (mtu != 0) { if (ia != NULL) mtu = min(mtu, ia->ia_ifp->if_mtu); } else { if (ia != NULL) mtu = ia->ia_ifp->if_mtu; else mtu = ip_next_mtu(ntohs(ip->ip_len), 0); } IPSTAT_INC(ips_cantfrag); break; case ENOBUFS: case EACCES: /* ipfw denied packet */ m_freem(mcopy); if (ia != NULL) ifa_free(&ia->ia_ifa); return; } if (ia != NULL) ifa_free(&ia->ia_ifa); icmp_error(mcopy, type, code, dest.s_addr, mtu); } void ip_savecontrol(struct inpcb *inp, struct mbuf **mp, struct ip *ip, struct mbuf *m) { if (inp->inp_socket->so_options & (SO_BINTIME | SO_TIMESTAMP)) { struct bintime bt; bintime(&bt); if (inp->inp_socket->so_options & SO_BINTIME) { *mp = sbcreatecontrol((caddr_t)&bt, sizeof(bt), SCM_BINTIME, SOL_SOCKET); if (*mp) mp = &(*mp)->m_next; } if (inp->inp_socket->so_options & SO_TIMESTAMP) { struct timeval tv; bintime2timeval(&bt, &tv); *mp = sbcreatecontrol((caddr_t)&tv, sizeof(tv), SCM_TIMESTAMP, SOL_SOCKET); if (*mp) mp = &(*mp)->m_next; } } if (inp->inp_flags & INP_RECVDSTADDR) { *mp = sbcreatecontrol((caddr_t)&ip->ip_dst, sizeof(struct in_addr), IP_RECVDSTADDR, IPPROTO_IP); if (*mp) mp = &(*mp)->m_next; } if (inp->inp_flags & INP_RECVTTL) { *mp = sbcreatecontrol((caddr_t)&ip->ip_ttl, sizeof(u_char), IP_RECVTTL, IPPROTO_IP); if (*mp) mp = &(*mp)->m_next; } #ifdef notyet /* XXX * Moving these out of udp_input() made them even more broken * than they already were. */ /* options were tossed already */ if (inp->inp_flags & INP_RECVOPTS) { *mp = sbcreatecontrol((caddr_t)opts_deleted_above, sizeof(struct in_addr), IP_RECVOPTS, IPPROTO_IP); if (*mp) mp = &(*mp)->m_next; } /* ip_srcroute doesn't do what we want here, need to fix */ if (inp->inp_flags & INP_RECVRETOPTS) { *mp = sbcreatecontrol((caddr_t)ip_srcroute(m), sizeof(struct in_addr), IP_RECVRETOPTS, IPPROTO_IP); if (*mp) mp = &(*mp)->m_next; } #endif if (inp->inp_flags & INP_RECVIF) { struct ifnet *ifp; struct sdlbuf { struct sockaddr_dl sdl; u_char pad[32]; } sdlbuf; struct sockaddr_dl *sdp; struct sockaddr_dl *sdl2 = &sdlbuf.sdl; if ((ifp = m->m_pkthdr.rcvif) && ifp->if_index && ifp->if_index <= V_if_index) { sdp = (struct sockaddr_dl *)ifp->if_addr->ifa_addr; /* * Change our mind and don't try copy. */ if (sdp->sdl_family != AF_LINK || sdp->sdl_len > sizeof(sdlbuf)) { goto makedummy; } bcopy(sdp, sdl2, sdp->sdl_len); } else { makedummy: sdl2->sdl_len = offsetof(struct sockaddr_dl, sdl_data[0]); sdl2->sdl_family = AF_LINK; sdl2->sdl_index = 0; sdl2->sdl_nlen = sdl2->sdl_alen = sdl2->sdl_slen = 0; } *mp = sbcreatecontrol((caddr_t)sdl2, sdl2->sdl_len, IP_RECVIF, IPPROTO_IP); if (*mp) mp = &(*mp)->m_next; } if (inp->inp_flags & INP_RECVTOS) { *mp = sbcreatecontrol((caddr_t)&ip->ip_tos, sizeof(u_char), IP_RECVTOS, IPPROTO_IP); if (*mp) mp = &(*mp)->m_next; } if (inp->inp_flags2 & INP_RECVFLOWID) { uint32_t flowid, flow_type; flowid = m->m_pkthdr.flowid; flow_type = M_HASHTYPE_GET(m); /* * XXX should handle the failure of one or the * other - don't populate both? */ *mp = sbcreatecontrol((caddr_t) &flowid, sizeof(uint32_t), IP_FLOWID, IPPROTO_IP); if (*mp) mp = &(*mp)->m_next; *mp = sbcreatecontrol((caddr_t) &flow_type, sizeof(uint32_t), IP_FLOWTYPE, IPPROTO_IP); if (*mp) mp = &(*mp)->m_next; } #ifdef RSS if (inp->inp_flags2 & INP_RECVRSSBUCKETID) { uint32_t flowid, flow_type; uint32_t rss_bucketid; flowid = m->m_pkthdr.flowid; flow_type = M_HASHTYPE_GET(m); if (rss_hash2bucket(flowid, flow_type, &rss_bucketid) == 0) { *mp = sbcreatecontrol((caddr_t) &rss_bucketid, sizeof(uint32_t), IP_RSSBUCKETID, IPPROTO_IP); if (*mp) mp = &(*mp)->m_next; } } #endif } /* * XXXRW: Multicast routing code in ip_mroute.c is generally MPSAFE, but the * ip_rsvp and ip_rsvp_on variables need to be interlocked with rsvp_on * locking. This code remains in ip_input.c as ip_mroute.c is optionally * compiled. */ static VNET_DEFINE(int, ip_rsvp_on); VNET_DEFINE(struct socket *, ip_rsvpd); #define V_ip_rsvp_on VNET(ip_rsvp_on) int ip_rsvp_init(struct socket *so) { if (so->so_type != SOCK_RAW || so->so_proto->pr_protocol != IPPROTO_RSVP) return EOPNOTSUPP; if (V_ip_rsvpd != NULL) return EADDRINUSE; V_ip_rsvpd = so; /* * This may seem silly, but we need to be sure we don't over-increment * the RSVP counter, in case something slips up. */ if (!V_ip_rsvp_on) { V_ip_rsvp_on = 1; V_rsvp_on++; } return 0; } int ip_rsvp_done(void) { V_ip_rsvpd = NULL; /* * This may seem silly, but we need to be sure we don't over-decrement * the RSVP counter, in case something slips up. */ if (V_ip_rsvp_on) { V_ip_rsvp_on = 0; V_rsvp_on--; } return 0; } int rsvp_input(struct mbuf **mp, int *offp, int proto) { struct mbuf *m; m = *mp; *mp = NULL; if (rsvp_input_p) { /* call the real one if loaded */ *mp = m; rsvp_input_p(mp, offp, proto); return (IPPROTO_DONE); } /* Can still get packets with rsvp_on = 0 if there is a local member * of the group to which the RSVP packet is addressed. But in this * case we want to throw the packet away. */ if (!V_rsvp_on) { m_freem(m); return (IPPROTO_DONE); } if (V_ip_rsvpd != NULL) { *mp = m; rip_input(mp, offp, proto); return (IPPROTO_DONE); } /* Drop the packet */ m_freem(m); return (IPPROTO_DONE); } Index: user/ngie/more-tests2/sys/netinet/ip_options.c =================================================================== --- user/ngie/more-tests2/sys/netinet/ip_options.c (revision 292053) +++ user/ngie/more-tests2/sys/netinet/ip_options.c (revision 292054) @@ -1,753 +1,757 @@ /* * Copyright (c) 1982, 1986, 1988, 1993 * The Regents of the University of California. * Copyright (c) 2005 Andre Oppermann, Internet Business Solutions AG. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include "opt_ipstealth.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include static VNET_DEFINE(int, ip_dosourceroute); SYSCTL_INT(_net_inet_ip, IPCTL_SOURCEROUTE, sourceroute, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(ip_dosourceroute), 0, "Enable forwarding source routed IP packets"); #define V_ip_dosourceroute VNET(ip_dosourceroute) static VNET_DEFINE(int, ip_acceptsourceroute); SYSCTL_INT(_net_inet_ip, IPCTL_ACCEPTSOURCEROUTE, accept_sourceroute, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(ip_acceptsourceroute), 0, "Enable accepting source routed IP packets"); #define V_ip_acceptsourceroute VNET(ip_acceptsourceroute) VNET_DEFINE(int, ip_doopts) = 1; /* 0 = ignore, 1 = process, 2 = reject */ SYSCTL_INT(_net_inet_ip, OID_AUTO, process_options, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(ip_doopts), 0, "Enable IP options processing ([LS]SRR, RR, TS)"); static void save_rte(struct mbuf *m, u_char *, struct in_addr); /* * Do option processing on a datagram, possibly discarding it if bad options * are encountered, or forwarding it if source-routed. * * The pass argument is used when operating in the IPSTEALTH mode to tell * what options to process: [LS]SRR (pass 0) or the others (pass 1). The * reason for as many as two passes is that when doing IPSTEALTH, non-routing * options should be processed only if the packet is for us. * * Returns 1 if packet has been forwarded/freed, 0 if the packet should be * processed further. */ int ip_dooptions(struct mbuf *m, int pass) { struct ip *ip = mtod(m, struct ip *); u_char *cp; struct in_ifaddr *ia; int opt, optlen, cnt, off, code, type = ICMP_PARAMPROB, forward = 0; struct in_addr *sin, dst; uint32_t ntime; struct nhop4_extended nh_ext; struct sockaddr_in ipaddr = { sizeof(ipaddr), AF_INET }; /* Ignore or reject packets with IP options. */ if (V_ip_doopts == 0) return 0; else if (V_ip_doopts == 2) { type = ICMP_UNREACH; code = ICMP_UNREACH_FILTER_PROHIB; goto bad; } dst = ip->ip_dst; cp = (u_char *)(ip + 1); cnt = (ip->ip_hl << 2) - sizeof (struct ip); for (; cnt > 0; cnt -= optlen, cp += optlen) { opt = cp[IPOPT_OPTVAL]; if (opt == IPOPT_EOL) break; if (opt == IPOPT_NOP) optlen = 1; else { if (cnt < IPOPT_OLEN + sizeof(*cp)) { code = &cp[IPOPT_OLEN] - (u_char *)ip; goto bad; } optlen = cp[IPOPT_OLEN]; if (optlen < IPOPT_OLEN + sizeof(*cp) || optlen > cnt) { code = &cp[IPOPT_OLEN] - (u_char *)ip; goto bad; } } switch (opt) { default: break; /* * Source routing with record. Find interface with current * destination address. If none on this machine then drop if * strictly routed, or do nothing if loosely routed. Record * interface address and bring up next address component. If * strictly routed make sure next address is on directly * accessible net. */ case IPOPT_LSRR: case IPOPT_SSRR: #ifdef IPSTEALTH if (V_ipstealth && pass > 0) break; #endif if (optlen < IPOPT_OFFSET + sizeof(*cp)) { code = &cp[IPOPT_OLEN] - (u_char *)ip; goto bad; } if ((off = cp[IPOPT_OFFSET]) < IPOPT_MINOFF) { code = &cp[IPOPT_OFFSET] - (u_char *)ip; goto bad; } ipaddr.sin_addr = ip->ip_dst; if (ifa_ifwithaddr_check((struct sockaddr *)&ipaddr) == 0) { if (opt == IPOPT_SSRR) { type = ICMP_UNREACH; code = ICMP_UNREACH_SRCFAIL; goto bad; } if (!V_ip_dosourceroute) goto nosourcerouting; /* * Loose routing, and not at next destination * yet; nothing to do except forward. */ break; } off--; /* 0 origin */ if (off > optlen - (int)sizeof(struct in_addr)) { /* * End of source route. Should be for us. */ if (!V_ip_acceptsourceroute) goto nosourcerouting; save_rte(m, cp, ip->ip_src); break; } #ifdef IPSTEALTH if (V_ipstealth) goto dropit; #endif if (!V_ip_dosourceroute) { if (V_ipforwarding) { char buf[16]; /* aaa.bbb.ccc.ddd\0 */ /* * Acting as a router, so generate * ICMP */ nosourcerouting: strcpy(buf, inet_ntoa(ip->ip_dst)); log(LOG_WARNING, "attempted source route from %s to %s\n", inet_ntoa(ip->ip_src), buf); type = ICMP_UNREACH; code = ICMP_UNREACH_SRCFAIL; goto bad; } else { /* * Not acting as a router, so * silently drop. */ #ifdef IPSTEALTH dropit: #endif IPSTAT_INC(ips_cantforward); m_freem(m); return (1); } } /* * locate outgoing interface */ (void)memcpy(&ipaddr.sin_addr, cp + off, sizeof(ipaddr.sin_addr)); type = ICMP_UNREACH; code = ICMP_UNREACH_SRCFAIL; if (opt == IPOPT_SSRR) { #define INA struct in_ifaddr * #define SA struct sockaddr * ia = (INA)ifa_ifwithdstaddr((SA)&ipaddr, RT_ALL_FIBS); if (ia == NULL) ia = (INA)ifa_ifwithnet((SA)&ipaddr, 0, RT_ALL_FIBS); if (ia == NULL) goto bad; memcpy(cp + off, &(IA_SIN(ia)->sin_addr), sizeof(struct in_addr)); ifa_free(&ia->ia_ifa); } else { /* XXX MRT 0 for routing */ if (fib4_lookup_nh_ext(M_GETFIB(m), ipaddr.sin_addr, 0, 0, &nh_ext) != 0) goto bad; memcpy(cp + off, &nh_ext.nh_src, sizeof(struct in_addr)); } ip->ip_dst = ipaddr.sin_addr; cp[IPOPT_OFFSET] += sizeof(struct in_addr); /* * Let ip_intr's mcast routing check handle mcast pkts */ forward = !IN_MULTICAST(ntohl(ip->ip_dst.s_addr)); break; case IPOPT_RR: #ifdef IPSTEALTH if (V_ipstealth && pass == 0) break; #endif if (optlen < IPOPT_OFFSET + sizeof(*cp)) { code = &cp[IPOPT_OFFSET] - (u_char *)ip; goto bad; } if ((off = cp[IPOPT_OFFSET]) < IPOPT_MINOFF) { code = &cp[IPOPT_OFFSET] - (u_char *)ip; goto bad; } /* * If no space remains, ignore. */ off--; /* 0 origin */ if (off > optlen - (int)sizeof(struct in_addr)) break; (void)memcpy(&ipaddr.sin_addr, &ip->ip_dst, sizeof(ipaddr.sin_addr)); /* * Locate outgoing interface; if we're the * destination, use the incoming interface (should be * same). */ - if ((ia = (INA)ifa_ifwithaddr((SA)&ipaddr)) == NULL && - (ia = ip_rtaddr(ipaddr.sin_addr, M_GETFIB(m))) == NULL) { + if ((ia = (INA)ifa_ifwithaddr((SA)&ipaddr)) != NULL) { + memcpy(cp + off, &(IA_SIN(ia)->sin_addr), + sizeof(struct in_addr)); + ifa_free(&ia->ia_ifa); + } else if (fib4_lookup_nh_ext(M_GETFIB(m), + ipaddr.sin_addr, 0, 0, &nh_ext) == 0) { + memcpy(cp + off, &nh_ext.nh_src, + sizeof(struct in_addr)); + } else { type = ICMP_UNREACH; code = ICMP_UNREACH_HOST; goto bad; } - (void)memcpy(cp + off, &(IA_SIN(ia)->sin_addr), - sizeof(struct in_addr)); - ifa_free(&ia->ia_ifa); cp[IPOPT_OFFSET] += sizeof(struct in_addr); break; case IPOPT_TS: #ifdef IPSTEALTH if (V_ipstealth && pass == 0) break; #endif code = cp - (u_char *)ip; if (optlen < 4 || optlen > 40) { code = &cp[IPOPT_OLEN] - (u_char *)ip; goto bad; } if ((off = cp[IPOPT_OFFSET]) < 5) { code = &cp[IPOPT_OLEN] - (u_char *)ip; goto bad; } if (off > optlen - (int)sizeof(int32_t)) { cp[IPOPT_OFFSET + 1] += (1 << 4); if ((cp[IPOPT_OFFSET + 1] & 0xf0) == 0) { code = &cp[IPOPT_OFFSET] - (u_char *)ip; goto bad; } break; } off--; /* 0 origin */ sin = (struct in_addr *)(cp + off); switch (cp[IPOPT_OFFSET + 1] & 0x0f) { case IPOPT_TS_TSONLY: break; case IPOPT_TS_TSANDADDR: if (off + sizeof(uint32_t) + sizeof(struct in_addr) > optlen) { code = &cp[IPOPT_OFFSET] - (u_char *)ip; goto bad; } ipaddr.sin_addr = dst; ia = (INA)ifaof_ifpforaddr((SA)&ipaddr, m->m_pkthdr.rcvif); if (ia == NULL) continue; (void)memcpy(sin, &IA_SIN(ia)->sin_addr, sizeof(struct in_addr)); ifa_free(&ia->ia_ifa); cp[IPOPT_OFFSET] += sizeof(struct in_addr); off += sizeof(struct in_addr); break; case IPOPT_TS_PRESPEC: if (off + sizeof(uint32_t) + sizeof(struct in_addr) > optlen) { code = &cp[IPOPT_OFFSET] - (u_char *)ip; goto bad; } (void)memcpy(&ipaddr.sin_addr, sin, sizeof(struct in_addr)); if (ifa_ifwithaddr_check((SA)&ipaddr) == 0) continue; cp[IPOPT_OFFSET] += sizeof(struct in_addr); off += sizeof(struct in_addr); break; default: code = &cp[IPOPT_OFFSET + 1] - (u_char *)ip; goto bad; } ntime = iptime(); (void)memcpy(cp + off, &ntime, sizeof(uint32_t)); cp[IPOPT_OFFSET] += sizeof(uint32_t); } } if (forward && V_ipforwarding) { ip_forward(m, 1); return (1); } return (0); bad: icmp_error(m, type, code, 0, 0); IPSTAT_INC(ips_badoptions); return (1); } /* * Save incoming source route for use in replies, to be picked up later by * ip_srcroute if the receiver is interested. */ static void save_rte(struct mbuf *m, u_char *option, struct in_addr dst) { unsigned olen; struct ipopt_tag *opts; opts = (struct ipopt_tag *)m_tag_get(PACKET_TAG_IPOPTIONS, sizeof(struct ipopt_tag), M_NOWAIT); if (opts == NULL) return; olen = option[IPOPT_OLEN]; if (olen > sizeof(opts->ip_srcrt) - (1 + sizeof(dst))) { m_tag_free((struct m_tag *)opts); return; } bcopy(option, opts->ip_srcrt.srcopt, olen); opts->ip_nhops = (olen - IPOPT_OFFSET - 1) / sizeof(struct in_addr); opts->ip_srcrt.dst = dst; m_tag_prepend(m, (struct m_tag *)opts); } /* * Retrieve incoming source route for use in replies, in the same form used * by setsockopt. The first hop is placed before the options, will be * removed later. */ struct mbuf * ip_srcroute(struct mbuf *m0) { struct in_addr *p, *q; struct mbuf *m; struct ipopt_tag *opts; opts = (struct ipopt_tag *)m_tag_find(m0, PACKET_TAG_IPOPTIONS, NULL); if (opts == NULL) return (NULL); if (opts->ip_nhops == 0) return (NULL); m = m_get(M_NOWAIT, MT_DATA); if (m == NULL) return (NULL); #define OPTSIZ (sizeof(opts->ip_srcrt.nop) + sizeof(opts->ip_srcrt.srcopt)) /* length is (nhops+1)*sizeof(addr) + sizeof(nop + srcrt header) */ m->m_len = opts->ip_nhops * sizeof(struct in_addr) + sizeof(struct in_addr) + OPTSIZ; /* * First, save first hop for return route. */ p = &(opts->ip_srcrt.route[opts->ip_nhops - 1]); *(mtod(m, struct in_addr *)) = *p--; /* * Copy option fields and padding (nop) to mbuf. */ opts->ip_srcrt.nop = IPOPT_NOP; opts->ip_srcrt.srcopt[IPOPT_OFFSET] = IPOPT_MINOFF; (void)memcpy(mtod(m, caddr_t) + sizeof(struct in_addr), &(opts->ip_srcrt.nop), OPTSIZ); q = (struct in_addr *)(mtod(m, caddr_t) + sizeof(struct in_addr) + OPTSIZ); #undef OPTSIZ /* * Record return path as an IP source route, reversing the path * (pointers are now aligned). */ while (p >= opts->ip_srcrt.route) { *q++ = *p--; } /* * Last hop goes to final destination. */ *q = opts->ip_srcrt.dst; m_tag_delete(m0, (struct m_tag *)opts); return (m); } /* * Strip out IP options, at higher level protocol in the kernel. */ void ip_stripoptions(struct mbuf *m) { struct ip *ip = mtod(m, struct ip *); int olen; olen = (ip->ip_hl << 2) - sizeof(struct ip); m->m_len -= olen; if (m->m_flags & M_PKTHDR) m->m_pkthdr.len -= olen; ip->ip_len = htons(ntohs(ip->ip_len) - olen); ip->ip_hl = sizeof(struct ip) >> 2; bcopy((char *)ip + sizeof(struct ip) + olen, (ip + 1), (size_t )(m->m_len - sizeof(struct ip))); } /* * Insert IP options into preformed packet. Adjust IP destination as * required for IP source routing, as indicated by a non-zero in_addr at the * start of the options. * * XXX This routine assumes that the packet has no options in place. */ struct mbuf * ip_insertoptions(struct mbuf *m, struct mbuf *opt, int *phlen) { struct ipoption *p = mtod(opt, struct ipoption *); struct mbuf *n; struct ip *ip = mtod(m, struct ip *); unsigned optlen; optlen = opt->m_len - sizeof(p->ipopt_dst); if (optlen + ntohs(ip->ip_len) > IP_MAXPACKET) { *phlen = 0; return (m); /* XXX should fail */ } if (p->ipopt_dst.s_addr) ip->ip_dst = p->ipopt_dst; if (!M_WRITABLE(m) || M_LEADINGSPACE(m) < optlen) { n = m_gethdr(M_NOWAIT, MT_DATA); if (n == NULL) { *phlen = 0; return (m); } m_move_pkthdr(n, m); n->m_pkthdr.rcvif = NULL; n->m_pkthdr.len += optlen; m->m_len -= sizeof(struct ip); m->m_data += sizeof(struct ip); n->m_next = m; m = n; m->m_len = optlen + sizeof(struct ip); m->m_data += max_linkhdr; bcopy(ip, mtod(m, void *), sizeof(struct ip)); } else { m->m_data -= optlen; m->m_len += optlen; m->m_pkthdr.len += optlen; bcopy(ip, mtod(m, void *), sizeof(struct ip)); } ip = mtod(m, struct ip *); bcopy(p->ipopt_list, ip + 1, optlen); *phlen = sizeof(struct ip) + optlen; ip->ip_v = IPVERSION; ip->ip_hl = *phlen >> 2; ip->ip_len = htons(ntohs(ip->ip_len) + optlen); return (m); } /* * Copy options from ip to jp, omitting those not copied during * fragmentation. */ int ip_optcopy(struct ip *ip, struct ip *jp) { u_char *cp, *dp; int opt, optlen, cnt; cp = (u_char *)(ip + 1); dp = (u_char *)(jp + 1); cnt = (ip->ip_hl << 2) - sizeof (struct ip); for (; cnt > 0; cnt -= optlen, cp += optlen) { opt = cp[0]; if (opt == IPOPT_EOL) break; if (opt == IPOPT_NOP) { /* Preserve for IP mcast tunnel's LSRR alignment. */ *dp++ = IPOPT_NOP; optlen = 1; continue; } KASSERT(cnt >= IPOPT_OLEN + sizeof(*cp), ("ip_optcopy: malformed ipv4 option")); optlen = cp[IPOPT_OLEN]; KASSERT(optlen >= IPOPT_OLEN + sizeof(*cp) && optlen <= cnt, ("ip_optcopy: malformed ipv4 option")); /* Bogus lengths should have been caught by ip_dooptions. */ if (optlen > cnt) optlen = cnt; if (IPOPT_COPIED(opt)) { bcopy(cp, dp, optlen); dp += optlen; } } for (optlen = dp - (u_char *)(jp+1); optlen & 0x3; optlen++) *dp++ = IPOPT_EOL; return (optlen); } /* * Set up IP options in pcb for insertion in output packets. Store in mbuf * with pointer in pcbopt, adding pseudo-option with destination address if * source routed. */ int ip_pcbopts(struct inpcb *inp, int optname, struct mbuf *m) { int cnt, optlen; u_char *cp; struct mbuf **pcbopt; u_char opt; INP_WLOCK_ASSERT(inp); pcbopt = &inp->inp_options; /* turn off any old options */ if (*pcbopt) (void)m_free(*pcbopt); *pcbopt = 0; if (m == NULL || m->m_len == 0) { /* * Only turning off any previous options. */ if (m != NULL) (void)m_free(m); return (0); } if (m->m_len % sizeof(int32_t)) goto bad; /* * IP first-hop destination address will be stored before actual * options; move other options back and clear it when none present. */ if (m->m_data + m->m_len + sizeof(struct in_addr) >= &m->m_dat[MLEN]) goto bad; cnt = m->m_len; m->m_len += sizeof(struct in_addr); cp = mtod(m, u_char *) + sizeof(struct in_addr); bcopy(mtod(m, void *), cp, (unsigned)cnt); bzero(mtod(m, void *), sizeof(struct in_addr)); for (; cnt > 0; cnt -= optlen, cp += optlen) { opt = cp[IPOPT_OPTVAL]; if (opt == IPOPT_EOL) break; if (opt == IPOPT_NOP) optlen = 1; else { if (cnt < IPOPT_OLEN + sizeof(*cp)) goto bad; optlen = cp[IPOPT_OLEN]; if (optlen < IPOPT_OLEN + sizeof(*cp) || optlen > cnt) goto bad; } switch (opt) { default: break; case IPOPT_LSRR: case IPOPT_SSRR: /* * User process specifies route as: * * ->A->B->C->D * * D must be our final destination (but we can't * check that since we may not have connected yet). * A is first hop destination, which doesn't appear * in actual IP option, but is stored before the * options. */ /* XXX-BZ PRIV_NETINET_SETHDROPTS? */ if (optlen < IPOPT_MINOFF - 1 + sizeof(struct in_addr)) goto bad; m->m_len -= sizeof(struct in_addr); cnt -= sizeof(struct in_addr); optlen -= sizeof(struct in_addr); cp[IPOPT_OLEN] = optlen; /* * Move first hop before start of options. */ bcopy((caddr_t)&cp[IPOPT_OFFSET+1], mtod(m, caddr_t), sizeof(struct in_addr)); /* * Then copy rest of options back * to close up the deleted entry. */ bcopy((&cp[IPOPT_OFFSET+1] + sizeof(struct in_addr)), &cp[IPOPT_OFFSET+1], (unsigned)cnt - (IPOPT_MINOFF - 1)); break; } } if (m->m_len > MAX_IPOPTLEN + sizeof(struct in_addr)) goto bad; *pcbopt = m; return (0); bad: (void)m_free(m); return (EINVAL); } /* * Check for the presence of the IP Router Alert option [RFC2113] * in the header of an IPv4 datagram. * * This call is not intended for use from the forwarding path; it is here * so that protocol domains may check for the presence of the option. * Given how FreeBSD's IPv4 stack is currently structured, the Router Alert * option does not have much relevance to the implementation, though this * may change in future. * Router alert options SHOULD be passed if running in IPSTEALTH mode and * we are not the endpoint. * Length checks on individual options should already have been peformed * by ip_dooptions() therefore they are folded under INVARIANTS here. * * Return zero if not present or options are invalid, non-zero if present. */ int ip_checkrouteralert(struct mbuf *m) { struct ip *ip = mtod(m, struct ip *); u_char *cp; int opt, optlen, cnt, found_ra; found_ra = 0; cp = (u_char *)(ip + 1); cnt = (ip->ip_hl << 2) - sizeof (struct ip); for (; cnt > 0; cnt -= optlen, cp += optlen) { opt = cp[IPOPT_OPTVAL]; if (opt == IPOPT_EOL) break; if (opt == IPOPT_NOP) optlen = 1; else { #ifdef INVARIANTS if (cnt < IPOPT_OLEN + sizeof(*cp)) break; #endif optlen = cp[IPOPT_OLEN]; #ifdef INVARIANTS if (optlen < IPOPT_OLEN + sizeof(*cp) || optlen > cnt) break; #endif } switch (opt) { case IPOPT_RA: #ifdef INVARIANTS if (optlen != IPOPT_OFFSET + sizeof(uint16_t) || (*((uint16_t *)&cp[IPOPT_OFFSET]) != 0)) break; else #endif found_ra = 1; break; default: break; } } return (found_ra); } Index: user/ngie/more-tests2/sys/netinet/ip_var.h =================================================================== --- user/ngie/more-tests2/sys/netinet/ip_var.h (revision 292053) +++ user/ngie/more-tests2/sys/netinet/ip_var.h (revision 292054) @@ -1,306 +1,304 @@ /*- * Copyright (c) 1982, 1986, 1993 * The Regents of the University of California. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * @(#)ip_var.h 8.2 (Berkeley) 1/9/95 * $FreeBSD$ */ #ifndef _NETINET_IP_VAR_H_ #define _NETINET_IP_VAR_H_ #include /* * Overlay for ip header used by other protocols (tcp, udp). */ struct ipovly { u_char ih_x1[9]; /* (unused) */ u_char ih_pr; /* protocol */ u_short ih_len; /* protocol length */ struct in_addr ih_src; /* source internet address */ struct in_addr ih_dst; /* destination internet address */ }; #ifdef _KERNEL /* * Ip reassembly queue structure. Each fragment * being reassembled is attached to one of these structures. * They are timed out after ipq_ttl drops to 0, and may also * be reclaimed if memory becomes tight. */ struct ipq { TAILQ_ENTRY(ipq) ipq_list; /* to other reass headers */ u_char ipq_ttl; /* time for reass q to live */ u_char ipq_p; /* protocol of this fragment */ u_short ipq_id; /* sequence id for reassembly */ struct mbuf *ipq_frags; /* to ip headers of fragments */ struct in_addr ipq_src,ipq_dst; u_char ipq_nfrags; /* # frags in this packet */ struct label *ipq_label; /* MAC label */ }; #endif /* _KERNEL */ /* * Structure stored in mbuf in inpcb.ip_options * and passed to ip_output when ip options are in use. * The actual length of the options (including ipopt_dst) * is in m_len. */ #define MAX_IPOPTLEN 40 struct ipoption { struct in_addr ipopt_dst; /* first-hop dst if source routed */ char ipopt_list[MAX_IPOPTLEN]; /* options proper */ }; /* * Structure attached to inpcb.ip_moptions and * passed to ip_output when IP multicast options are in use. * This structure is lazy-allocated. */ struct ip_moptions { struct ifnet *imo_multicast_ifp; /* ifp for outgoing multicasts */ struct in_addr imo_multicast_addr; /* ifindex/addr on MULTICAST_IF */ u_long imo_multicast_vif; /* vif num outgoing multicasts */ u_char imo_multicast_ttl; /* TTL for outgoing multicasts */ u_char imo_multicast_loop; /* 1 => hear sends if a member */ u_short imo_num_memberships; /* no. memberships this socket */ u_short imo_max_memberships; /* max memberships this socket */ struct in_multi **imo_membership; /* group memberships */ struct in_mfilter *imo_mfilters; /* source filters */ STAILQ_ENTRY(ip_moptions) imo_link; }; struct ipstat { uint64_t ips_total; /* total packets received */ uint64_t ips_badsum; /* checksum bad */ uint64_t ips_tooshort; /* packet too short */ uint64_t ips_toosmall; /* not enough data */ uint64_t ips_badhlen; /* ip header length < data size */ uint64_t ips_badlen; /* ip length < ip header length */ uint64_t ips_fragments; /* fragments received */ uint64_t ips_fragdropped; /* frags dropped (dups, out of space) */ uint64_t ips_fragtimeout; /* fragments timed out */ uint64_t ips_forward; /* packets forwarded */ uint64_t ips_fastforward; /* packets fast forwarded */ uint64_t ips_cantforward; /* packets rcvd for unreachable dest */ uint64_t ips_redirectsent; /* packets forwarded on same net */ uint64_t ips_noproto; /* unknown or unsupported protocol */ uint64_t ips_delivered; /* datagrams delivered to upper level*/ uint64_t ips_localout; /* total ip packets generated here */ uint64_t ips_odropped; /* lost packets due to nobufs, etc. */ uint64_t ips_reassembled; /* total packets reassembled ok */ uint64_t ips_fragmented; /* datagrams successfully fragmented */ uint64_t ips_ofragments; /* output fragments created */ uint64_t ips_cantfrag; /* don't fragment flag was set, etc. */ uint64_t ips_badoptions; /* error in option processing */ uint64_t ips_noroute; /* packets discarded due to no route */ uint64_t ips_badvers; /* ip version != 4 */ uint64_t ips_rawout; /* total raw ip packets generated */ uint64_t ips_toolong; /* ip length > max ip packet size */ uint64_t ips_notmember; /* multicasts for unregistered grps */ uint64_t ips_nogif; /* no match gif found */ uint64_t ips_badaddr; /* invalid address on header */ }; #ifdef _KERNEL #include #include VNET_PCPUSTAT_DECLARE(struct ipstat, ipstat); /* * In-kernel consumers can use these accessor macros directly to update * stats. */ #define IPSTAT_ADD(name, val) \ VNET_PCPUSTAT_ADD(struct ipstat, ipstat, name, (val)) #define IPSTAT_SUB(name, val) IPSTAT_ADD(name, -(val)) #define IPSTAT_INC(name) IPSTAT_ADD(name, 1) #define IPSTAT_DEC(name) IPSTAT_SUB(name, 1) /* * Kernel module consumers must use this accessor macro. */ void kmod_ipstat_inc(int statnum); #define KMOD_IPSTAT_INC(name) \ kmod_ipstat_inc(offsetof(struct ipstat, name) / sizeof(uint64_t)) void kmod_ipstat_dec(int statnum); #define KMOD_IPSTAT_DEC(name) \ kmod_ipstat_dec(offsetof(struct ipstat, name) / sizeof(uint64_t)) /* flags passed to ip_output as last parameter */ #define IP_FORWARDING 0x1 /* most of ip header exists */ #define IP_RAWOUTPUT 0x2 /* raw ip header exists */ #define IP_SENDONES 0x4 /* send all-ones broadcast */ #define IP_SENDTOIF 0x8 /* send on specific ifnet */ #define IP_ROUTETOIF SO_DONTROUTE /* 0x10 bypass routing tables */ #define IP_ALLOWBROADCAST SO_BROADCAST /* 0x20 can send broadcast packets */ #define IP_NODEFAULTFLOWID 0x40 /* Don't set the flowid from inp */ #ifdef __NO_STRICT_ALIGNMENT #define IP_HDR_ALIGNED_P(ip) 1 #else #define IP_HDR_ALIGNED_P(ip) ((((intptr_t) (ip)) & 3) == 0) #endif struct ip; struct inpcb; struct route; struct sockopt; VNET_DECLARE(int, ip_defttl); /* default IP ttl */ VNET_DECLARE(int, ipforwarding); /* ip forwarding */ #ifdef IPSTEALTH VNET_DECLARE(int, ipstealth); /* stealth forwarding */ #endif extern u_char ip_protox[]; VNET_DECLARE(struct socket *, ip_rsvpd); /* reservation protocol daemon*/ VNET_DECLARE(struct socket *, ip_mrouter); /* multicast routing daemon */ extern int (*legal_vif_num)(int); extern u_long (*ip_mcast_src)(int); VNET_DECLARE(int, rsvp_on); VNET_DECLARE(int, drop_redirect); extern struct pr_usrreqs rip_usrreqs; #define V_ip_id VNET(ip_id) #define V_ip_defttl VNET(ip_defttl) #define V_ipforwarding VNET(ipforwarding) #ifdef IPSTEALTH #define V_ipstealth VNET(ipstealth) #endif #define V_ip_rsvpd VNET(ip_rsvpd) #define V_ip_mrouter VNET(ip_mrouter) #define V_rsvp_on VNET(rsvp_on) #define V_drop_redirect VNET(drop_redirect) void inp_freemoptions(struct ip_moptions *); int inp_getmoptions(struct inpcb *, struct sockopt *); int inp_setmoptions(struct inpcb *, struct sockopt *); int ip_ctloutput(struct socket *, struct sockopt *sopt); void ip_drain(void); int ip_fragment(struct ip *ip, struct mbuf **m_frag, int mtu, u_long if_hwassist_flags); void ip_forward(struct mbuf *m, int srcrt); void ip_init(void); #ifdef VIMAGE void ip_destroy(void); #endif extern int (*ip_mforward)(struct ip *, struct ifnet *, struct mbuf *, struct ip_moptions *); int ip_output(struct mbuf *, struct mbuf *, struct route *, int, struct ip_moptions *, struct inpcb *); int ipproto_register(short); int ipproto_unregister(short); struct mbuf * ip_reass(struct mbuf *); -struct in_ifaddr * - ip_rtaddr(struct in_addr, u_int fibnum); void ip_savecontrol(struct inpcb *, struct mbuf **, struct ip *, struct mbuf *); void ip_slowtimo(void); void ip_fillid(struct ip *); int rip_ctloutput(struct socket *, struct sockopt *); void rip_ctlinput(int, struct sockaddr *, void *); void rip_init(void); #ifdef VIMAGE void rip_destroy(void); #endif int rip_input(struct mbuf **, int *, int); int rip_output(struct mbuf *, struct socket *, ...); int ipip_input(struct mbuf **, int *, int); int rsvp_input(struct mbuf **, int *, int); int ip_rsvp_init(struct socket *); int ip_rsvp_done(void); extern int (*ip_rsvp_vif)(struct socket *, struct sockopt *); extern void (*ip_rsvp_force_done)(struct socket *); extern int (*rsvp_input_p)(struct mbuf **, int *, int); VNET_DECLARE(struct pfil_head, inet_pfil_hook); /* packet filter hooks */ #define V_inet_pfil_hook VNET(inet_pfil_hook) void in_delayed_cksum(struct mbuf *m); /* Hooks for ipfw, dummynet, divert etc. Most are declared in raw_ip.c */ /* * Reference to an ipfw or packet filter rule that can be carried * outside critical sections. * A rule is identified by rulenum:rule_id which is ordered. * In version chain_id the rule can be found in slot 'slot', so * we don't need a lookup if chain_id == chain->id. * * On exit from the firewall this structure refers to the rule after * the matching one (slot points to the new rule; rulenum:rule_id-1 * is the matching rule), and additional info (e.g. info often contains * the insn argument or tablearg in the low 16 bits, in host format). * On entry, the structure is valid if slot>0, and refers to the starting * rules. 'info' contains the reason for reinject, e.g. divert port, * divert direction, and so on. */ struct ipfw_rule_ref { uint32_t slot; /* slot for matching rule */ uint32_t rulenum; /* matching rule number */ uint32_t rule_id; /* matching rule id */ uint32_t chain_id; /* ruleset id */ uint32_t info; /* see below */ }; enum { IPFW_INFO_MASK = 0x0000ffff, IPFW_INFO_OUT = 0x00000000, /* outgoing, just for convenience */ IPFW_INFO_IN = 0x80000000, /* incoming, overloads dir */ IPFW_ONEPASS = 0x40000000, /* One-pass, do not reinject */ IPFW_IS_MASK = 0x30000000, /* which source ? */ IPFW_IS_DIVERT = 0x20000000, IPFW_IS_DUMMYNET =0x10000000, IPFW_IS_PIPE = 0x08000000, /* pipe=1, queue = 0 */ }; #define MTAG_IPFW 1148380143 /* IPFW-tagged cookie */ #define MTAG_IPFW_RULE 1262273568 /* rule reference */ #define MTAG_IPFW_CALL 1308397630 /* call stack */ struct ip_fw_args; typedef int (*ip_fw_chk_ptr_t)(struct ip_fw_args *args); typedef int (*ip_fw_ctl_ptr_t)(struct sockopt *); VNET_DECLARE(ip_fw_ctl_ptr_t, ip_fw_ctl_ptr); #define V_ip_fw_ctl_ptr VNET(ip_fw_ctl_ptr) /* Divert hooks. */ extern void (*ip_divert_ptr)(struct mbuf *m, int incoming); /* ng_ipfw hooks -- XXX make it the same as divert and dummynet */ extern int (*ng_ipfw_input_p)(struct mbuf **, int, struct ip_fw_args *, int); extern int (*ip_dn_ctl_ptr)(struct sockopt *); extern int (*ip_dn_io_ptr)(struct mbuf **, int, struct ip_fw_args *); #endif /* _KERNEL */ #endif /* !_NETINET_IP_VAR_H_ */ Index: user/ngie/more-tests2/sys/netinet/tcp_var.h =================================================================== --- user/ngie/more-tests2/sys/netinet/tcp_var.h (revision 292053) +++ user/ngie/more-tests2/sys/netinet/tcp_var.h (revision 292054) @@ -1,780 +1,781 @@ /*- * Copyright (c) 1982, 1986, 1993, 1994, 1995 * The Regents of the University of California. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * @(#)tcp_var.h 8.4 (Berkeley) 5/24/95 * $FreeBSD$ */ #ifndef _NETINET_TCP_VAR_H_ #define _NETINET_TCP_VAR_H_ #include #ifdef _KERNEL #include #include /* * Kernel variables for tcp. */ VNET_DECLARE(int, tcp_do_rfc1323); #define V_tcp_do_rfc1323 VNET(tcp_do_rfc1323) #endif /* _KERNEL */ /* TCP segment queue entry */ struct tseg_qent { LIST_ENTRY(tseg_qent) tqe_q; int tqe_len; /* TCP segment data length */ struct tcphdr *tqe_th; /* a pointer to tcp header */ struct mbuf *tqe_m; /* mbuf contains packet */ }; LIST_HEAD(tsegqe_head, tseg_qent); struct sackblk { tcp_seq start; /* start seq no. of sack block */ tcp_seq end; /* end seq no. */ }; struct sackhole { tcp_seq start; /* start seq no. of hole */ tcp_seq end; /* end seq no. */ tcp_seq rxmit; /* next seq. no in hole to be retransmitted */ TAILQ_ENTRY(sackhole) scblink; /* scoreboard linkage */ }; struct sackhint { struct sackhole *nexthole; int sack_bytes_rexmit; tcp_seq last_sack_ack; /* Most recent/largest sacked ack */ int ispare; /* explicit pad for 64bit alignment */ int sacked_bytes; /* * Total sacked bytes reported by the * receiver via sack option */ + uint32_t _pad1[1]; /* TBD */ uint64_t _pad[1]; /* TBD */ }; struct tcptemp { u_char tt_ipgen[40]; /* the size must be of max ip header, now IPv6 */ struct tcphdr tt_t; }; #define tcp6cb tcpcb /* for KAME src sync over BSD*'s */ /* * Tcp control block, one per tcp; fields: * Organized for 16 byte cacheline efficiency. */ struct tcpcb { struct tsegqe_head t_segq; /* segment reassembly queue */ void *t_pspare[2]; /* new reassembly queue */ int t_segqlen; /* segment reassembly queue length */ int t_dupacks; /* consecutive dup acks recd */ struct tcp_timer *t_timers; /* All the TCP timers in one struct */ struct inpcb *t_inpcb; /* back pointer to internet pcb */ int t_state; /* state of this connection */ u_int t_flags; struct vnet *t_vnet; /* back pointer to parent vnet */ tcp_seq snd_una; /* sent but unacknowledged */ tcp_seq snd_max; /* highest sequence number sent; * used to recognize retransmits */ tcp_seq snd_nxt; /* send next */ tcp_seq snd_up; /* send urgent pointer */ tcp_seq snd_wl1; /* window update seg seq number */ tcp_seq snd_wl2; /* window update seg ack number */ tcp_seq iss; /* initial send sequence number */ tcp_seq irs; /* initial receive sequence number */ tcp_seq rcv_nxt; /* receive next */ tcp_seq rcv_adv; /* advertised window */ u_long rcv_wnd; /* receive window */ tcp_seq rcv_up; /* receive urgent pointer */ u_long snd_wnd; /* send window */ u_long snd_cwnd; /* congestion-controlled window */ u_long snd_spare1; /* unused */ u_long snd_ssthresh; /* snd_cwnd size threshold for * for slow start exponential to * linear switch */ u_long snd_spare2; /* unused */ tcp_seq snd_recover; /* for use in NewReno Fast Recovery */ u_int t_maxopd; /* mss plus options */ u_int t_rcvtime; /* inactivity time */ u_int t_starttime; /* time connection was established */ u_int t_rtttime; /* RTT measurement start time */ tcp_seq t_rtseq; /* sequence number being timed */ u_int t_bw_spare1; /* unused */ tcp_seq t_bw_spare2; /* unused */ int t_rxtcur; /* current retransmit value (ticks) */ u_int t_maxseg; /* maximum segment size */ int t_srtt; /* smoothed round-trip time */ int t_rttvar; /* variance in round-trip time */ int t_rxtshift; /* log(2) of rexmt exp. backoff */ u_int t_rttmin; /* minimum rtt allowed */ u_int t_rttbest; /* best rtt we've seen */ u_long t_rttupdated; /* number of times rtt sampled */ u_long max_sndwnd; /* largest window peer has offered */ int t_softerror; /* possible error not yet reported */ /* out-of-band data */ char t_oobflags; /* have some */ char t_iobc; /* input character */ /* RFC 1323 variables */ u_char snd_scale; /* window scaling for send window */ u_char rcv_scale; /* window scaling for recv window */ u_char request_r_scale; /* pending window scaling */ u_int32_t ts_recent; /* timestamp echo data */ u_int ts_recent_age; /* when last updated */ u_int32_t ts_offset; /* our timestamp offset */ tcp_seq last_ack_sent; /* experimental */ u_long snd_cwnd_prev; /* cwnd prior to retransmit */ u_long snd_ssthresh_prev; /* ssthresh prior to retransmit */ tcp_seq snd_recover_prev; /* snd_recover prior to retransmit */ int t_sndzerowin; /* zero-window updates sent */ u_int t_badrxtwin; /* window for retransmit recovery */ u_char snd_limited; /* segments limited transmitted */ /* SACK related state */ int snd_numholes; /* number of holes seen by sender */ TAILQ_HEAD(sackhole_head, sackhole) snd_holes; /* SACK scoreboard (sorted) */ tcp_seq snd_fack; /* last seq number(+1) sack'd by rcv'r*/ int rcv_numsacks; /* # distinct sack blks present */ struct sackblk sackblks[MAX_SACK_BLKS]; /* seq nos. of sack blocks */ tcp_seq sack_newdata; /* New data xmitted in this recovery episode starts at this seq number */ struct sackhint sackhint; /* SACK scoreboard hint */ int t_rttlow; /* smallest observerved RTT */ u_int32_t rfbuf_ts; /* recv buffer autoscaling timestamp */ int rfbuf_cnt; /* recv buffer autoscaling byte count */ struct toedev *tod; /* toedev handling this connection */ int t_sndrexmitpack; /* retransmit packets sent */ int t_rcvoopack; /* out-of-order packets received */ void *t_toe; /* TOE pcb pointer */ int t_bytes_acked; /* # bytes acked during current RTT */ struct cc_algo *cc_algo; /* congestion control algorithm */ struct cc_var *ccv; /* congestion control specific vars */ struct osd *osd; /* storage for Khelp module data */ u_int t_keepinit; /* time to establish connection */ u_int t_keepidle; /* time before keepalive probes begin */ u_int t_keepintvl; /* interval between keepalives */ u_int t_keepcnt; /* number of keepalives before close */ u_int t_tsomax; /* TSO total burst length limit in bytes */ u_int t_tsomaxsegcount; /* TSO maximum segment count */ u_int t_tsomaxsegsize; /* TSO maximum segment size in bytes */ u_int t_pmtud_saved_maxopd; /* pre-blackhole MSS */ u_int t_flags2; /* More tcpcb flags storage */ uint32_t t_ispare[8]; /* 5 UTO, 3 TBD */ void *t_pspare2[4]; /* 1 TCP_SIGNATURE, 3 TBD */ #if defined(_KERNEL) && defined(TCPPCAP) struct mbufq t_inpkts; /* List of saved input packets. */ struct mbufq t_outpkts; /* List of saved output packets. */ #ifdef _LP64 uint64_t _pad[0]; /* all used! */ #else uint64_t _pad[2]; /* 2 are available */ #endif /* _LP64 */ #else uint64_t _pad[6]; #endif /* defined(_KERNEL) && defined(TCPPCAP) */ }; /* * Flags and utility macros for the t_flags field. */ #define TF_ACKNOW 0x000001 /* ack peer immediately */ #define TF_DELACK 0x000002 /* ack, but try to delay it */ #define TF_NODELAY 0x000004 /* don't delay packets to coalesce */ #define TF_NOOPT 0x000008 /* don't use tcp options */ #define TF_SENTFIN 0x000010 /* have sent FIN */ #define TF_REQ_SCALE 0x000020 /* have/will request window scaling */ #define TF_RCVD_SCALE 0x000040 /* other side has requested scaling */ #define TF_REQ_TSTMP 0x000080 /* have/will request timestamps */ #define TF_RCVD_TSTMP 0x000100 /* a timestamp was received in SYN */ #define TF_SACK_PERMIT 0x000200 /* other side said I could SACK */ #define TF_NEEDSYN 0x000400 /* send SYN (implicit state) */ #define TF_NEEDFIN 0x000800 /* send FIN (implicit state) */ #define TF_NOPUSH 0x001000 /* don't push */ #define TF_PREVVALID 0x002000 /* saved values for bad rxmit valid */ #define TF_MORETOCOME 0x010000 /* More data to be appended to sock */ #define TF_LQ_OVERFLOW 0x020000 /* listen queue overflow */ #define TF_LASTIDLE 0x040000 /* connection was previously idle */ #define TF_RXWIN0SENT 0x080000 /* sent a receiver win 0 in response */ #define TF_FASTRECOVERY 0x100000 /* in NewReno Fast Recovery */ #define TF_WASFRECOVERY 0x200000 /* was in NewReno Fast Recovery */ #define TF_SIGNATURE 0x400000 /* require MD5 digests (RFC2385) */ #define TF_FORCEDATA 0x800000 /* force out a byte */ #define TF_TSO 0x1000000 /* TSO enabled on this connection */ #define TF_TOE 0x2000000 /* this connection is offloaded */ #define TF_ECN_PERMIT 0x4000000 /* connection ECN-ready */ #define TF_ECN_SND_CWR 0x8000000 /* ECN CWR in queue */ #define TF_ECN_SND_ECE 0x10000000 /* ECN ECE in queue */ #define TF_CONGRECOVERY 0x20000000 /* congestion recovery mode */ #define TF_WASCRECOVERY 0x40000000 /* was in congestion recovery */ #define IN_FASTRECOVERY(t_flags) (t_flags & TF_FASTRECOVERY) #define ENTER_FASTRECOVERY(t_flags) t_flags |= TF_FASTRECOVERY #define EXIT_FASTRECOVERY(t_flags) t_flags &= ~TF_FASTRECOVERY #define IN_CONGRECOVERY(t_flags) (t_flags & TF_CONGRECOVERY) #define ENTER_CONGRECOVERY(t_flags) t_flags |= TF_CONGRECOVERY #define EXIT_CONGRECOVERY(t_flags) t_flags &= ~TF_CONGRECOVERY #define IN_RECOVERY(t_flags) (t_flags & (TF_CONGRECOVERY | TF_FASTRECOVERY)) #define ENTER_RECOVERY(t_flags) t_flags |= (TF_CONGRECOVERY | TF_FASTRECOVERY) #define EXIT_RECOVERY(t_flags) t_flags &= ~(TF_CONGRECOVERY | TF_FASTRECOVERY) #define BYTES_THIS_ACK(tp, th) (th->th_ack - tp->snd_una) /* * Flags for the t_oobflags field. */ #define TCPOOB_HAVEDATA 0x01 #define TCPOOB_HADDATA 0x02 #ifdef TCP_SIGNATURE /* * Defines which are needed by the xform_tcp module and tcp_[in|out]put * for SADB verification and lookup. */ #define TCP_SIGLEN 16 /* length of computed digest in bytes */ #define TCP_KEYLEN_MIN 1 /* minimum length of TCP-MD5 key */ #define TCP_KEYLEN_MAX 80 /* maximum length of TCP-MD5 key */ /* * Only a single SA per host may be specified at this time. An SPI is * needed in order for the KEY_ALLOCSA() lookup to work. */ #define TCP_SIG_SPI 0x1000 #endif /* TCP_SIGNATURE */ /* * Flags for PLPMTU handling, t_flags2 */ #define TF2_PLPMTU_BLACKHOLE 0x00000001 /* Possible PLPMTUD Black Hole. */ #define TF2_PLPMTU_PMTUD 0x00000002 /* Allowed to attempt PLPMTUD. */ #define TF2_PLPMTU_MAXSEGSNT 0x00000004 /* Last seg sent was full seg. */ /* * Structure to hold TCP options that are only used during segment * processing (in tcp_input), but not held in the tcpcb. * It's basically used to reduce the number of parameters * to tcp_dooptions and tcp_addoptions. * The binary order of the to_flags is relevant for packing of the * options in tcp_addoptions. */ struct tcpopt { u_int64_t to_flags; /* which options are present */ #define TOF_MSS 0x0001 /* maximum segment size */ #define TOF_SCALE 0x0002 /* window scaling */ #define TOF_SACKPERM 0x0004 /* SACK permitted */ #define TOF_TS 0x0010 /* timestamp */ #define TOF_SIGNATURE 0x0040 /* TCP-MD5 signature option (RFC2385) */ #define TOF_SACK 0x0080 /* Peer sent SACK option */ #define TOF_MAXOPT 0x0100 u_int32_t to_tsval; /* new timestamp */ u_int32_t to_tsecr; /* reflected timestamp */ u_char *to_sacks; /* pointer to the first SACK blocks */ u_char *to_signature; /* pointer to the TCP-MD5 signature */ u_int16_t to_mss; /* maximum segment size */ u_int8_t to_wscale; /* window scaling */ u_int8_t to_nsacks; /* number of SACK blocks */ u_int32_t to_spare; /* UTO */ }; /* * Flags for tcp_dooptions. */ #define TO_SYN 0x01 /* parse SYN-only options */ struct hc_metrics_lite { /* must stay in sync with hc_metrics */ u_long rmx_mtu; /* MTU for this path */ u_long rmx_ssthresh; /* outbound gateway buffer limit */ u_long rmx_rtt; /* estimated round trip time */ u_long rmx_rttvar; /* estimated rtt variance */ u_long rmx_bandwidth; /* estimated bandwidth */ u_long rmx_cwnd; /* congestion window */ u_long rmx_sendpipe; /* outbound delay-bandwidth product */ u_long rmx_recvpipe; /* inbound delay-bandwidth product */ }; /* * Used by tcp_maxmtu() to communicate interface specific features * and limits at the time of connection setup. */ struct tcp_ifcap { int ifcap; u_int tsomax; u_int tsomaxsegcount; u_int tsomaxsegsize; }; #ifndef _NETINET_IN_PCB_H_ struct in_conninfo; #endif /* _NETINET_IN_PCB_H_ */ struct tcptw { struct inpcb *tw_inpcb; /* XXX back pointer to internet pcb */ tcp_seq snd_nxt; tcp_seq rcv_nxt; tcp_seq iss; tcp_seq irs; u_short last_win; /* cached window value */ u_short tw_so_options; /* copy of so_options */ struct ucred *tw_cred; /* user credentials */ u_int32_t t_recent; u_int32_t ts_offset; /* our timestamp offset */ u_int t_starttime; int tw_time; TAILQ_ENTRY(tcptw) tw_2msl; void *tw_pspare; /* TCP_SIGNATURE */ u_int *tw_spare; /* TCP_SIGNATURE */ }; #define intotcpcb(ip) ((struct tcpcb *)(ip)->inp_ppcb) #define intotw(ip) ((struct tcptw *)(ip)->inp_ppcb) #define sototcpcb(so) (intotcpcb(sotoinpcb(so))) /* * The smoothed round-trip time and estimated variance * are stored as fixed point numbers scaled by the values below. * For convenience, these scales are also used in smoothing the average * (smoothed = (1/scale)sample + ((scale-1)/scale)smoothed). * With these scales, srtt has 3 bits to the right of the binary point, * and thus an "ALPHA" of 0.875. rttvar has 2 bits to the right of the * binary point, and is smoothed with an ALPHA of 0.75. */ #define TCP_RTT_SCALE 32 /* multiplier for srtt; 3 bits frac. */ #define TCP_RTT_SHIFT 5 /* shift for srtt; 3 bits frac. */ #define TCP_RTTVAR_SCALE 16 /* multiplier for rttvar; 2 bits */ #define TCP_RTTVAR_SHIFT 4 /* shift for rttvar; 2 bits */ #define TCP_DELTA_SHIFT 2 /* see tcp_input.c */ /* * The initial retransmission should happen at rtt + 4 * rttvar. * Because of the way we do the smoothing, srtt and rttvar * will each average +1/2 tick of bias. When we compute * the retransmit timer, we want 1/2 tick of rounding and * 1 extra tick because of +-1/2 tick uncertainty in the * firing of the timer. The bias will give us exactly the * 1.5 tick we need. But, because the bias is * statistical, we have to test that we don't drop below * the minimum feasible timer (which is 2 ticks). * This version of the macro adapted from a paper by Lawrence * Brakmo and Larry Peterson which outlines a problem caused * by insufficient precision in the original implementation, * which results in inappropriately large RTO values for very * fast networks. */ #define TCP_REXMTVAL(tp) \ max((tp)->t_rttmin, (((tp)->t_srtt >> (TCP_RTT_SHIFT - TCP_DELTA_SHIFT)) \ + (tp)->t_rttvar) >> TCP_DELTA_SHIFT) /* * TCP statistics. * Many of these should be kept per connection, * but that's inconvenient at the moment. */ struct tcpstat { uint64_t tcps_connattempt; /* connections initiated */ uint64_t tcps_accepts; /* connections accepted */ uint64_t tcps_connects; /* connections established */ uint64_t tcps_drops; /* connections dropped */ uint64_t tcps_conndrops; /* embryonic connections dropped */ uint64_t tcps_minmssdrops; /* average minmss too low drops */ uint64_t tcps_closed; /* conn. closed (includes drops) */ uint64_t tcps_segstimed; /* segs where we tried to get rtt */ uint64_t tcps_rttupdated; /* times we succeeded */ uint64_t tcps_delack; /* delayed acks sent */ uint64_t tcps_timeoutdrop; /* conn. dropped in rxmt timeout */ uint64_t tcps_rexmttimeo; /* retransmit timeouts */ uint64_t tcps_persisttimeo; /* persist timeouts */ uint64_t tcps_keeptimeo; /* keepalive timeouts */ uint64_t tcps_keepprobe; /* keepalive probes sent */ uint64_t tcps_keepdrops; /* connections dropped in keepalive */ uint64_t tcps_sndtotal; /* total packets sent */ uint64_t tcps_sndpack; /* data packets sent */ uint64_t tcps_sndbyte; /* data bytes sent */ uint64_t tcps_sndrexmitpack; /* data packets retransmitted */ uint64_t tcps_sndrexmitbyte; /* data bytes retransmitted */ uint64_t tcps_sndrexmitbad; /* unnecessary packet retransmissions */ uint64_t tcps_sndacks; /* ack-only packets sent */ uint64_t tcps_sndprobe; /* window probes sent */ uint64_t tcps_sndurg; /* packets sent with URG only */ uint64_t tcps_sndwinup; /* window update-only packets sent */ uint64_t tcps_sndctrl; /* control (SYN|FIN|RST) packets sent */ uint64_t tcps_rcvtotal; /* total packets received */ uint64_t tcps_rcvpack; /* packets received in sequence */ uint64_t tcps_rcvbyte; /* bytes received in sequence */ uint64_t tcps_rcvbadsum; /* packets received with ccksum errs */ uint64_t tcps_rcvbadoff; /* packets received with bad offset */ uint64_t tcps_rcvreassfull; /* packets dropped for no reass space */ uint64_t tcps_rcvshort; /* packets received too short */ uint64_t tcps_rcvduppack; /* duplicate-only packets received */ uint64_t tcps_rcvdupbyte; /* duplicate-only bytes received */ uint64_t tcps_rcvpartduppack; /* packets with some duplicate data */ uint64_t tcps_rcvpartdupbyte; /* dup. bytes in part-dup. packets */ uint64_t tcps_rcvoopack; /* out-of-order packets received */ uint64_t tcps_rcvoobyte; /* out-of-order bytes received */ uint64_t tcps_rcvpackafterwin; /* packets with data after window */ uint64_t tcps_rcvbyteafterwin; /* bytes rcvd after window */ uint64_t tcps_rcvafterclose; /* packets rcvd after "close" */ uint64_t tcps_rcvwinprobe; /* rcvd window probe packets */ uint64_t tcps_rcvdupack; /* rcvd duplicate acks */ uint64_t tcps_rcvacktoomuch; /* rcvd acks for unsent data */ uint64_t tcps_rcvackpack; /* rcvd ack packets */ uint64_t tcps_rcvackbyte; /* bytes acked by rcvd acks */ uint64_t tcps_rcvwinupd; /* rcvd window update packets */ uint64_t tcps_pawsdrop; /* segments dropped due to PAWS */ uint64_t tcps_predack; /* times hdr predict ok for acks */ uint64_t tcps_preddat; /* times hdr predict ok for data pkts */ uint64_t tcps_pcbcachemiss; uint64_t tcps_cachedrtt; /* times cached RTT in route updated */ uint64_t tcps_cachedrttvar; /* times cached rttvar updated */ uint64_t tcps_cachedssthresh; /* times cached ssthresh updated */ uint64_t tcps_usedrtt; /* times RTT initialized from route */ uint64_t tcps_usedrttvar; /* times RTTVAR initialized from rt */ uint64_t tcps_usedssthresh; /* times ssthresh initialized from rt*/ uint64_t tcps_persistdrop; /* timeout in persist state */ uint64_t tcps_badsyn; /* bogus SYN, e.g. premature ACK */ uint64_t tcps_mturesent; /* resends due to MTU discovery */ uint64_t tcps_listendrop; /* listen queue overflows */ uint64_t tcps_badrst; /* ignored RSTs in the window */ uint64_t tcps_sc_added; /* entry added to syncache */ uint64_t tcps_sc_retransmitted; /* syncache entry was retransmitted */ uint64_t tcps_sc_dupsyn; /* duplicate SYN packet */ uint64_t tcps_sc_dropped; /* could not reply to packet */ uint64_t tcps_sc_completed; /* successful extraction of entry */ uint64_t tcps_sc_bucketoverflow;/* syncache per-bucket limit hit */ uint64_t tcps_sc_cacheoverflow; /* syncache cache limit hit */ uint64_t tcps_sc_reset; /* RST removed entry from syncache */ uint64_t tcps_sc_stale; /* timed out or listen socket gone */ uint64_t tcps_sc_aborted; /* syncache entry aborted */ uint64_t tcps_sc_badack; /* removed due to bad ACK */ uint64_t tcps_sc_unreach; /* ICMP unreachable received */ uint64_t tcps_sc_zonefail; /* zalloc() failed */ uint64_t tcps_sc_sendcookie; /* SYN cookie sent */ uint64_t tcps_sc_recvcookie; /* SYN cookie received */ uint64_t tcps_hc_added; /* entry added to hostcache */ uint64_t tcps_hc_bucketoverflow;/* hostcache per bucket limit hit */ uint64_t tcps_finwait2_drops; /* Drop FIN_WAIT_2 connection after time limit */ /* SACK related stats */ uint64_t tcps_sack_recovery_episode; /* SACK recovery episodes */ uint64_t tcps_sack_rexmits; /* SACK rexmit segments */ uint64_t tcps_sack_rexmit_bytes; /* SACK rexmit bytes */ uint64_t tcps_sack_rcv_blocks; /* SACK blocks (options) received */ uint64_t tcps_sack_send_blocks; /* SACK blocks (options) sent */ uint64_t tcps_sack_sboverflow; /* times scoreboard overflowed */ /* ECN related stats */ uint64_t tcps_ecn_ce; /* ECN Congestion Experienced */ uint64_t tcps_ecn_ect0; /* ECN Capable Transport */ uint64_t tcps_ecn_ect1; /* ECN Capable Transport */ uint64_t tcps_ecn_shs; /* ECN successful handshakes */ uint64_t tcps_ecn_rcwnd; /* # times ECN reduced the cwnd */ /* TCP_SIGNATURE related stats */ uint64_t tcps_sig_rcvgoodsig; /* Total matching signature received */ uint64_t tcps_sig_rcvbadsig; /* Total bad signature received */ uint64_t tcps_sig_err_buildsig; /* Mismatching signature received */ uint64_t tcps_sig_err_sigopt; /* No signature expected by socket */ uint64_t tcps_sig_err_nosigopt; /* No signature provided by segment */ uint64_t _pad[12]; /* 6 UTO, 6 TBD */ }; #define tcps_rcvmemdrop tcps_rcvreassfull /* compat */ #ifdef _KERNEL #include VNET_PCPUSTAT_DECLARE(struct tcpstat, tcpstat); /* tcp statistics */ /* * In-kernel consumers can use these accessor macros directly to update * stats. */ #define TCPSTAT_ADD(name, val) \ VNET_PCPUSTAT_ADD(struct tcpstat, tcpstat, name, (val)) #define TCPSTAT_INC(name) TCPSTAT_ADD(name, 1) /* * Kernel module consumers must use this accessor macro. */ void kmod_tcpstat_inc(int statnum); #define KMOD_TCPSTAT_INC(name) \ kmod_tcpstat_inc(offsetof(struct tcpstat, name) / sizeof(uint64_t)) /* * TCP specific helper hook point identifiers. */ #define HHOOK_TCP_EST_IN 0 #define HHOOK_TCP_EST_OUT 1 #define HHOOK_TCP_LAST HHOOK_TCP_EST_OUT struct tcp_hhook_data { struct tcpcb *tp; struct tcphdr *th; struct tcpopt *to; long len; int tso; tcp_seq curack; }; #endif /* * TCB structure exported to user-land via sysctl(3). * Evil hack: declare only if in_pcb.h and sys/socketvar.h have been * included. Not all of our clients do. */ #if defined(_NETINET_IN_PCB_H_) && defined(_SYS_SOCKETVAR_H_) struct xtcp_timer { int tt_rexmt; /* retransmit timer */ int tt_persist; /* retransmit persistence */ int tt_keep; /* keepalive */ int tt_2msl; /* 2*msl TIME_WAIT timer */ int tt_delack; /* delayed ACK timer */ int t_rcvtime; /* Time since last packet received */ }; struct xtcpcb { size_t xt_len; struct inpcb xt_inp; struct tcpcb xt_tp; struct xsocket xt_socket; struct xtcp_timer xt_timer; u_quad_t xt_alignment_hack; }; #endif /* * Identifiers for TCP sysctl nodes */ #define TCPCTL_DO_RFC1323 1 /* use RFC-1323 extensions */ #define TCPCTL_MSSDFLT 3 /* MSS default */ #define TCPCTL_STATS 4 /* statistics (read-only) */ #define TCPCTL_RTTDFLT 5 /* default RTT estimate */ #define TCPCTL_KEEPIDLE 6 /* keepalive idle timer */ #define TCPCTL_KEEPINTVL 7 /* interval to send keepalives */ #define TCPCTL_SENDSPACE 8 /* send buffer space */ #define TCPCTL_RECVSPACE 9 /* receive buffer space */ #define TCPCTL_KEEPINIT 10 /* timeout for establishing syn */ #define TCPCTL_PCBLIST 11 /* list of all outstanding PCBs */ #define TCPCTL_DELACKTIME 12 /* time before sending delayed ACK */ #define TCPCTL_V6MSSDFLT 13 /* MSS default for IPv6 */ #define TCPCTL_SACK 14 /* Selective Acknowledgement,rfc 2018 */ #define TCPCTL_DROP 15 /* drop tcp connection */ #ifdef _KERNEL #ifdef SYSCTL_DECL SYSCTL_DECL(_net_inet_tcp); SYSCTL_DECL(_net_inet_tcp_sack); MALLOC_DECLARE(M_TCPLOG); #endif VNET_DECLARE(struct inpcbhead, tcb); /* queue of active tcpcb's */ VNET_DECLARE(struct inpcbinfo, tcbinfo); extern int tcp_log_in_vain; VNET_DECLARE(int, tcp_mssdflt); /* XXX */ VNET_DECLARE(int, tcp_minmss); VNET_DECLARE(int, tcp_delack_enabled); VNET_DECLARE(int, tcp_do_rfc3390); VNET_DECLARE(int, tcp_initcwnd_segments); VNET_DECLARE(int, tcp_sendspace); VNET_DECLARE(int, tcp_recvspace); VNET_DECLARE(int, path_mtu_discovery); VNET_DECLARE(int, tcp_do_rfc3465); VNET_DECLARE(int, tcp_abc_l_var); #define V_tcb VNET(tcb) #define V_tcbinfo VNET(tcbinfo) #define V_tcp_mssdflt VNET(tcp_mssdflt) #define V_tcp_minmss VNET(tcp_minmss) #define V_tcp_delack_enabled VNET(tcp_delack_enabled) #define V_tcp_do_rfc3390 VNET(tcp_do_rfc3390) #define V_tcp_initcwnd_segments VNET(tcp_initcwnd_segments) #define V_tcp_sendspace VNET(tcp_sendspace) #define V_tcp_recvspace VNET(tcp_recvspace) #define V_path_mtu_discovery VNET(path_mtu_discovery) #define V_tcp_do_rfc3465 VNET(tcp_do_rfc3465) #define V_tcp_abc_l_var VNET(tcp_abc_l_var) VNET_DECLARE(int, tcp_do_sack); /* SACK enabled/disabled */ VNET_DECLARE(int, tcp_sc_rst_sock_fail); /* RST on sock alloc failure */ #define V_tcp_do_sack VNET(tcp_do_sack) #define V_tcp_sc_rst_sock_fail VNET(tcp_sc_rst_sock_fail) VNET_DECLARE(int, tcp_do_ecn); /* TCP ECN enabled/disabled */ VNET_DECLARE(int, tcp_ecn_maxretries); #define V_tcp_do_ecn VNET(tcp_do_ecn) #define V_tcp_ecn_maxretries VNET(tcp_ecn_maxretries) VNET_DECLARE(struct hhook_head *, tcp_hhh[HHOOK_TCP_LAST + 1]); #define V_tcp_hhh VNET(tcp_hhh) VNET_DECLARE(int, tcp_do_rfc6675_pipe); #define V_tcp_do_rfc6675_pipe VNET(tcp_do_rfc6675_pipe) int tcp_addoptions(struct tcpopt *, u_char *); int tcp_ccalgounload(struct cc_algo *unload_algo); struct tcpcb * tcp_close(struct tcpcb *); void tcp_discardcb(struct tcpcb *); void tcp_twstart(struct tcpcb *); void tcp_twclose(struct tcptw *, int); void tcp_ctlinput(int, struct sockaddr *, void *); int tcp_ctloutput(struct socket *, struct sockopt *); struct tcpcb * tcp_drop(struct tcpcb *, int); void tcp_drain(void); void tcp_init(void); #ifdef VIMAGE void tcp_destroy(void); #endif void tcp_fini(void *); char *tcp_log_addrs(struct in_conninfo *, struct tcphdr *, void *, const void *); char *tcp_log_vain(struct in_conninfo *, struct tcphdr *, void *, const void *); int tcp_reass(struct tcpcb *, struct tcphdr *, int *, struct mbuf *); void tcp_reass_global_init(void); void tcp_reass_flush(struct tcpcb *); int tcp_input(struct mbuf **, int *, int); u_long tcp_maxmtu(struct in_conninfo *, struct tcp_ifcap *); u_long tcp_maxmtu6(struct in_conninfo *, struct tcp_ifcap *); void tcp_mss_update(struct tcpcb *, int, int, struct hc_metrics_lite *, struct tcp_ifcap *); void tcp_mss(struct tcpcb *, int); int tcp_mssopt(struct in_conninfo *); struct inpcb * tcp_drop_syn_sent(struct inpcb *, int); struct tcpcb * tcp_newtcpcb(struct inpcb *); int tcp_output(struct tcpcb *); void tcp_state_change(struct tcpcb *, int); void tcp_respond(struct tcpcb *, void *, struct tcphdr *, struct mbuf *, tcp_seq, tcp_seq, int); void tcp_tw_init(void); #ifdef VIMAGE void tcp_tw_destroy(void); #endif void tcp_tw_zone_change(void); int tcp_twcheck(struct inpcb *, struct tcpopt *, struct tcphdr *, struct mbuf *, int); void tcp_setpersist(struct tcpcb *); #ifdef TCP_SIGNATURE struct secasvar; struct secasvar *tcp_get_sav(struct mbuf *, u_int); int tcp_signature_do_compute(struct mbuf *, int, int, u_char *, struct secasvar *); int tcp_signature_compute(struct mbuf *, int, int, int, u_char *, u_int); int tcp_signature_verify(struct mbuf *, int, int, int, struct tcpopt *, struct tcphdr *, u_int); int tcp_signature_check(struct mbuf *m, int off0, int tlen, int optlen, struct tcpopt *to, struct tcphdr *th, u_int tcpbflag); #endif void tcp_slowtimo(void); struct tcptemp * tcpip_maketemplate(struct inpcb *); void tcpip_fillheaders(struct inpcb *, void *, void *); void tcp_timer_activate(struct tcpcb *, uint32_t, u_int); int tcp_timer_active(struct tcpcb *, uint32_t); void tcp_timer_stop(struct tcpcb *, uint32_t); void tcp_trace(short, short, struct tcpcb *, void *, struct tcphdr *, int); /* * All tcp_hc_* functions are IPv4 and IPv6 (via in_conninfo) */ void tcp_hc_init(void); #ifdef VIMAGE void tcp_hc_destroy(void); #endif void tcp_hc_get(struct in_conninfo *, struct hc_metrics_lite *); u_long tcp_hc_getmtu(struct in_conninfo *); void tcp_hc_updatemtu(struct in_conninfo *, u_long); void tcp_hc_update(struct in_conninfo *, struct hc_metrics_lite *); extern struct pr_usrreqs tcp_usrreqs; tcp_seq tcp_new_isn(struct tcpcb *); int tcp_sack_doack(struct tcpcb *, struct tcpopt *, tcp_seq); void tcp_update_sack_list(struct tcpcb *tp, tcp_seq rcv_laststart, tcp_seq rcv_lastend); void tcp_clean_sackreport(struct tcpcb *tp); void tcp_sack_adjust(struct tcpcb *tp); struct sackhole *tcp_sack_output(struct tcpcb *tp, int *sack_bytes_rexmt); void tcp_sack_partialack(struct tcpcb *, struct tcphdr *); void tcp_free_sackholes(struct tcpcb *tp); int tcp_newreno(struct tcpcb *, struct tcphdr *); u_long tcp_seq_subtract(u_long, u_long ); int tcp_compute_pipe(struct tcpcb *); void cc_cong_signal(struct tcpcb *tp, struct tcphdr *th, uint32_t type); static inline void tcp_fields_to_host(struct tcphdr *th) { th->th_seq = ntohl(th->th_seq); th->th_ack = ntohl(th->th_ack); th->th_win = ntohs(th->th_win); th->th_urp = ntohs(th->th_urp); } #ifdef TCP_SIGNATURE static inline void tcp_fields_to_net(struct tcphdr *th) { th->th_seq = htonl(th->th_seq); th->th_ack = htonl(th->th_ack); th->th_win = htons(th->th_win); th->th_urp = htons(th->th_urp); } #endif #endif /* _KERNEL */ #endif /* _NETINET_TCP_VAR_H_ */ Index: user/ngie/more-tests2/sys/netinet6/icmp6.c =================================================================== --- user/ngie/more-tests2/sys/netinet6/icmp6.c (revision 292053) +++ user/ngie/more-tests2/sys/netinet6/icmp6.c (revision 292054) @@ -1,2882 +1,2873 @@ /*- * Copyright (C) 1995, 1996, 1997, and 1998 WIDE Project. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. Neither the name of the project nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE PROJECT AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE PROJECT OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $KAME: icmp6.c,v 1.211 2001/04/04 05:56:20 itojun Exp $ */ /*- * Copyright (c) 1982, 1986, 1988, 1993 * The Regents of the University of California. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * @(#)ip_icmp.c 8.2 (Berkeley) 1/4/94 */ #include __FBSDID("$FreeBSD$"); #define MBUF_PRIVATE /* XXXRW: Optimisation tries to avoid M_EXT mbufs */ #include "opt_inet.h" #include "opt_inet6.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include +#include #include #include #include #include #include #include #include #include extern struct domain inet6domain; VNET_PCPUSTAT_DEFINE(struct icmp6stat, icmp6stat); VNET_PCPUSTAT_SYSINIT(icmp6stat); #ifdef VIMAGE VNET_PCPUSTAT_SYSUNINIT(icmp6stat); #endif /* VIMAGE */ VNET_DECLARE(struct inpcbinfo, ripcbinfo); VNET_DECLARE(struct inpcbhead, ripcb); VNET_DECLARE(int, icmp6errppslim); static VNET_DEFINE(int, icmp6errpps_count) = 0; static VNET_DEFINE(struct timeval, icmp6errppslim_last); VNET_DECLARE(int, icmp6_nodeinfo); #define V_ripcbinfo VNET(ripcbinfo) #define V_ripcb VNET(ripcb) #define V_icmp6errppslim VNET(icmp6errppslim) #define V_icmp6errpps_count VNET(icmp6errpps_count) #define V_icmp6errppslim_last VNET(icmp6errppslim_last) #define V_icmp6_nodeinfo VNET(icmp6_nodeinfo) static void icmp6_errcount(int, int); static int icmp6_rip6_input(struct mbuf **, int); static int icmp6_ratelimit(const struct in6_addr *, const int, const int); static const char *icmp6_redirect_diag(struct in6_addr *, struct in6_addr *, struct in6_addr *); static struct mbuf *ni6_input(struct mbuf *, int); static struct mbuf *ni6_nametodns(const char *, int, int); static int ni6_dnsmatch(const char *, int, const char *, int); static int ni6_addrs(struct icmp6_nodeinfo *, struct mbuf *, struct ifnet **, struct in6_addr *); static int ni6_store_addrs(struct icmp6_nodeinfo *, struct icmp6_nodeinfo *, struct ifnet *, int); static int icmp6_notify_error(struct mbuf **, int, int, int); /* * Kernel module interface for updating icmp6stat. The argument is an index * into icmp6stat treated as an array of u_quad_t. While this encodes the * general layout of icmp6stat into the caller, it doesn't encode its * location, so that future changes to add, for example, per-CPU stats * support won't cause binary compatibility problems for kernel modules. */ void kmod_icmp6stat_inc(int statnum) { counter_u64_add(VNET(icmp6stat)[statnum], 1); } static void icmp6_errcount(int type, int code) { switch (type) { case ICMP6_DST_UNREACH: switch (code) { case ICMP6_DST_UNREACH_NOROUTE: ICMP6STAT_INC(icp6s_odst_unreach_noroute); return; case ICMP6_DST_UNREACH_ADMIN: ICMP6STAT_INC(icp6s_odst_unreach_admin); return; case ICMP6_DST_UNREACH_BEYONDSCOPE: ICMP6STAT_INC(icp6s_odst_unreach_beyondscope); return; case ICMP6_DST_UNREACH_ADDR: ICMP6STAT_INC(icp6s_odst_unreach_addr); return; case ICMP6_DST_UNREACH_NOPORT: ICMP6STAT_INC(icp6s_odst_unreach_noport); return; } break; case ICMP6_PACKET_TOO_BIG: ICMP6STAT_INC(icp6s_opacket_too_big); return; case ICMP6_TIME_EXCEEDED: switch (code) { case ICMP6_TIME_EXCEED_TRANSIT: ICMP6STAT_INC(icp6s_otime_exceed_transit); return; case ICMP6_TIME_EXCEED_REASSEMBLY: ICMP6STAT_INC(icp6s_otime_exceed_reassembly); return; } break; case ICMP6_PARAM_PROB: switch (code) { case ICMP6_PARAMPROB_HEADER: ICMP6STAT_INC(icp6s_oparamprob_header); return; case ICMP6_PARAMPROB_NEXTHEADER: ICMP6STAT_INC(icp6s_oparamprob_nextheader); return; case ICMP6_PARAMPROB_OPTION: ICMP6STAT_INC(icp6s_oparamprob_option); return; } break; case ND_REDIRECT: ICMP6STAT_INC(icp6s_oredirect); return; } ICMP6STAT_INC(icp6s_ounknown); } /* * A wrapper function for icmp6_error() necessary when the erroneous packet * may not contain enough scope zone information. */ void icmp6_error2(struct mbuf *m, int type, int code, int param, struct ifnet *ifp) { struct ip6_hdr *ip6; if (ifp == NULL) return; #ifndef PULLDOWN_TEST IP6_EXTHDR_CHECK(m, 0, sizeof(struct ip6_hdr), ); #else if (m->m_len < sizeof(struct ip6_hdr)) { m = m_pullup(m, sizeof(struct ip6_hdr)); if (m == NULL) return; } #endif ip6 = mtod(m, struct ip6_hdr *); if (in6_setscope(&ip6->ip6_src, ifp, NULL) != 0) return; if (in6_setscope(&ip6->ip6_dst, ifp, NULL) != 0) return; icmp6_error(m, type, code, param); } /* * Generate an error packet of type error in response to bad IP6 packet. */ void icmp6_error(struct mbuf *m, int type, int code, int param) { struct ip6_hdr *oip6, *nip6; struct icmp6_hdr *icmp6; u_int preplen; int off; int nxt; ICMP6STAT_INC(icp6s_error); /* count per-type-code statistics */ icmp6_errcount(type, code); #ifdef M_DECRYPTED /*not openbsd*/ if (m->m_flags & M_DECRYPTED) { ICMP6STAT_INC(icp6s_canterror); goto freeit; } #endif #ifndef PULLDOWN_TEST IP6_EXTHDR_CHECK(m, 0, sizeof(struct ip6_hdr), ); #else if (m->m_len < sizeof(struct ip6_hdr)) { m = m_pullup(m, sizeof(struct ip6_hdr)); if (m == NULL) return; } #endif oip6 = mtod(m, struct ip6_hdr *); /* * If the destination address of the erroneous packet is a multicast * address, or the packet was sent using link-layer multicast, * we should basically suppress sending an error (RFC 2463, Section * 2.4). * We have two exceptions (the item e.2 in that section): * - the Packet Too Big message can be sent for path MTU discovery. * - the Parameter Problem Message that can be allowed an icmp6 error * in the option type field. This check has been done in * ip6_unknown_opt(), so we can just check the type and code. */ if ((m->m_flags & (M_BCAST|M_MCAST) || IN6_IS_ADDR_MULTICAST(&oip6->ip6_dst)) && (type != ICMP6_PACKET_TOO_BIG && (type != ICMP6_PARAM_PROB || code != ICMP6_PARAMPROB_OPTION))) goto freeit; /* * RFC 2463, 2.4 (e.5): source address check. * XXX: the case of anycast source? */ if (IN6_IS_ADDR_UNSPECIFIED(&oip6->ip6_src) || IN6_IS_ADDR_MULTICAST(&oip6->ip6_src)) goto freeit; /* * If we are about to send ICMPv6 against ICMPv6 error/redirect, * don't do it. */ nxt = -1; off = ip6_lasthdr(m, 0, IPPROTO_IPV6, &nxt); if (off >= 0 && nxt == IPPROTO_ICMPV6) { struct icmp6_hdr *icp; #ifndef PULLDOWN_TEST IP6_EXTHDR_CHECK(m, 0, off + sizeof(struct icmp6_hdr), ); icp = (struct icmp6_hdr *)(mtod(m, caddr_t) + off); #else IP6_EXTHDR_GET(icp, struct icmp6_hdr *, m, off, sizeof(*icp)); if (icp == NULL) { ICMP6STAT_INC(icp6s_tooshort); return; } #endif if (icp->icmp6_type < ICMP6_ECHO_REQUEST || icp->icmp6_type == ND_REDIRECT) { /* * ICMPv6 error * Special case: for redirect (which is * informational) we must not send icmp6 error. */ ICMP6STAT_INC(icp6s_canterror); goto freeit; } else { /* ICMPv6 informational - send the error */ } } else { /* non-ICMPv6 - send the error */ } oip6 = mtod(m, struct ip6_hdr *); /* adjust pointer */ /* Finally, do rate limitation check. */ if (icmp6_ratelimit(&oip6->ip6_src, type, code)) { ICMP6STAT_INC(icp6s_toofreq); goto freeit; } /* * OK, ICMP6 can be generated. */ if (m->m_pkthdr.len >= ICMPV6_PLD_MAXLEN) m_adj(m, ICMPV6_PLD_MAXLEN - m->m_pkthdr.len); preplen = sizeof(struct ip6_hdr) + sizeof(struct icmp6_hdr); M_PREPEND(m, preplen, M_NOWAIT); /* FIB is also copied over. */ if (m == NULL) { nd6log((LOG_DEBUG, "ENOBUFS in icmp6_error %d\n", __LINE__)); return; } nip6 = mtod(m, struct ip6_hdr *); nip6->ip6_src = oip6->ip6_src; nip6->ip6_dst = oip6->ip6_dst; in6_clearscope(&oip6->ip6_src); in6_clearscope(&oip6->ip6_dst); icmp6 = (struct icmp6_hdr *)(nip6 + 1); icmp6->icmp6_type = type; icmp6->icmp6_code = code; icmp6->icmp6_pptr = htonl((u_int32_t)param); /* * icmp6_reflect() is designed to be in the input path. * icmp6_error() can be called from both input and output path, * and if we are in output path rcvif could contain bogus value. * clear m->m_pkthdr.rcvif for safety, we should have enough scope * information in ip header (nip6). */ m->m_pkthdr.rcvif = NULL; ICMP6STAT_INC(icp6s_outhist[type]); icmp6_reflect(m, sizeof(struct ip6_hdr)); /* header order: IPv6 - ICMPv6 */ return; freeit: /* * If we can't tell whether or not we can generate ICMP6, free it. */ m_freem(m); } /* * Process a received ICMP6 message. */ int icmp6_input(struct mbuf **mp, int *offp, int proto) { struct mbuf *m = *mp, *n; struct ifnet *ifp; struct ip6_hdr *ip6, *nip6; struct icmp6_hdr *icmp6, *nicmp6; int off = *offp; int icmp6len = m->m_pkthdr.len - *offp; int code, sum, noff; char ip6bufs[INET6_ADDRSTRLEN], ip6bufd[INET6_ADDRSTRLEN]; int ip6len, error; ifp = m->m_pkthdr.rcvif; #ifndef PULLDOWN_TEST IP6_EXTHDR_CHECK(m, off, sizeof(struct icmp6_hdr), IPPROTO_DONE); /* m might change if M_LOOP. So, call mtod after this */ #endif /* * Locate icmp6 structure in mbuf, and check * that not corrupted and of at least minimum length */ ip6 = mtod(m, struct ip6_hdr *); ip6len = sizeof(struct ip6_hdr) + ntohs(ip6->ip6_plen); if (icmp6len < sizeof(struct icmp6_hdr)) { ICMP6STAT_INC(icp6s_tooshort); goto freeit; } /* * Check multicast group membership. * Note: SSM filters are not applied for ICMPv6 traffic. */ if (IN6_IS_ADDR_MULTICAST(&ip6->ip6_dst)) { struct in6_multi *inm; inm = in6m_lookup(ifp, &ip6->ip6_dst); if (inm == NULL) { IP6STAT_INC(ip6s_notmember); in6_ifstat_inc(m->m_pkthdr.rcvif, ifs6_in_discard); goto freeit; } } /* * calculate the checksum */ #ifndef PULLDOWN_TEST icmp6 = (struct icmp6_hdr *)((caddr_t)ip6 + off); #else IP6_EXTHDR_GET(icmp6, struct icmp6_hdr *, m, off, sizeof(*icmp6)); if (icmp6 == NULL) { ICMP6STAT_INC(icp6s_tooshort); return IPPROTO_DONE; } #endif code = icmp6->icmp6_code; if ((sum = in6_cksum(m, IPPROTO_ICMPV6, off, icmp6len)) != 0) { nd6log((LOG_ERR, "ICMP6 checksum error(%d|%x) %s\n", icmp6->icmp6_type, sum, ip6_sprintf(ip6bufs, &ip6->ip6_src))); ICMP6STAT_INC(icp6s_checksum); goto freeit; } ICMP6STAT_INC(icp6s_inhist[icmp6->icmp6_type]); icmp6_ifstat_inc(ifp, ifs6_in_msg); if (icmp6->icmp6_type < ICMP6_INFOMSG_MASK) icmp6_ifstat_inc(ifp, ifs6_in_error); switch (icmp6->icmp6_type) { case ICMP6_DST_UNREACH: icmp6_ifstat_inc(ifp, ifs6_in_dstunreach); switch (code) { case ICMP6_DST_UNREACH_NOROUTE: code = PRC_UNREACH_NET; break; case ICMP6_DST_UNREACH_ADMIN: icmp6_ifstat_inc(ifp, ifs6_in_adminprohib); code = PRC_UNREACH_PROTOCOL; /* is this a good code? */ break; case ICMP6_DST_UNREACH_ADDR: code = PRC_HOSTDEAD; break; case ICMP6_DST_UNREACH_BEYONDSCOPE: /* I mean "source address was incorrect." */ code = PRC_PARAMPROB; break; case ICMP6_DST_UNREACH_NOPORT: code = PRC_UNREACH_PORT; break; default: goto badcode; } goto deliver; break; case ICMP6_PACKET_TOO_BIG: icmp6_ifstat_inc(ifp, ifs6_in_pkttoobig); /* validation is made in icmp6_mtudisc_update */ code = PRC_MSGSIZE; /* * Updating the path MTU will be done after examining * intermediate extension headers. */ goto deliver; break; case ICMP6_TIME_EXCEEDED: icmp6_ifstat_inc(ifp, ifs6_in_timeexceed); switch (code) { case ICMP6_TIME_EXCEED_TRANSIT: code = PRC_TIMXCEED_INTRANS; break; case ICMP6_TIME_EXCEED_REASSEMBLY: code = PRC_TIMXCEED_REASS; break; default: goto badcode; } goto deliver; break; case ICMP6_PARAM_PROB: icmp6_ifstat_inc(ifp, ifs6_in_paramprob); switch (code) { case ICMP6_PARAMPROB_NEXTHEADER: code = PRC_UNREACH_PROTOCOL; break; case ICMP6_PARAMPROB_HEADER: case ICMP6_PARAMPROB_OPTION: code = PRC_PARAMPROB; break; default: goto badcode; } goto deliver; break; case ICMP6_ECHO_REQUEST: icmp6_ifstat_inc(ifp, ifs6_in_echo); if (code != 0) goto badcode; if ((n = m_copy(m, 0, M_COPYALL)) == NULL) { /* Give up remote */ break; } if (!M_WRITABLE(n) || n->m_len < off + sizeof(struct icmp6_hdr)) { struct mbuf *n0 = n; int n0len; CTASSERT(sizeof(*nip6) + sizeof(*nicmp6) <= MHLEN); n = m_gethdr(M_NOWAIT, n0->m_type); if (n == NULL) { /* Give up remote */ m_freem(n0); break; } m_move_pkthdr(n, n0); /* FIB copied. */ n0len = n0->m_pkthdr.len; /* save for use below */ /* * Copy IPv6 and ICMPv6 only. */ nip6 = mtod(n, struct ip6_hdr *); bcopy(ip6, nip6, sizeof(struct ip6_hdr)); nicmp6 = (struct icmp6_hdr *)(nip6 + 1); bcopy(icmp6, nicmp6, sizeof(struct icmp6_hdr)); noff = sizeof(struct ip6_hdr); /* new mbuf contains only ipv6+icmpv6 headers */ n->m_len = noff + sizeof(struct icmp6_hdr); /* * Adjust mbuf. ip6_plen will be adjusted in * ip6_output(). */ m_adj(n0, off + sizeof(struct icmp6_hdr)); /* recalculate complete packet size */ n->m_pkthdr.len = n0len + (noff - off); n->m_next = n0; } else { nip6 = mtod(n, struct ip6_hdr *); IP6_EXTHDR_GET(nicmp6, struct icmp6_hdr *, n, off, sizeof(*nicmp6)); noff = off; } nicmp6->icmp6_type = ICMP6_ECHO_REPLY; nicmp6->icmp6_code = 0; if (n) { ICMP6STAT_INC(icp6s_reflect); ICMP6STAT_INC(icp6s_outhist[ICMP6_ECHO_REPLY]); icmp6_reflect(n, noff); } break; case ICMP6_ECHO_REPLY: icmp6_ifstat_inc(ifp, ifs6_in_echoreply); if (code != 0) goto badcode; break; case MLD_LISTENER_QUERY: case MLD_LISTENER_REPORT: case MLD_LISTENER_DONE: case MLDV2_LISTENER_REPORT: /* * Drop MLD traffic which is not link-local, has a hop limit * of greater than 1 hop, or which does not have the * IPv6 HBH Router Alert option. * As IPv6 HBH options are stripped in ip6_input() we must * check an mbuf header flag. * XXX Should we also sanity check that these messages * were directed to a link-local multicast prefix? */ if ((ip6->ip6_hlim != 1) || (m->m_flags & M_RTALERT_MLD) == 0) goto freeit; if (mld_input(m, off, icmp6len) != 0) return (IPPROTO_DONE); /* m stays. */ break; case ICMP6_WRUREQUEST: /* ICMP6_FQDN_QUERY */ { enum { WRU, FQDN } mode; if (!V_icmp6_nodeinfo) break; if (icmp6len == sizeof(struct icmp6_hdr) + 4) mode = WRU; else if (icmp6len >= sizeof(struct icmp6_nodeinfo)) mode = FQDN; else goto badlen; if (mode == FQDN) { #ifndef PULLDOWN_TEST IP6_EXTHDR_CHECK(m, off, sizeof(struct icmp6_nodeinfo), IPPROTO_DONE); #endif n = m_copy(m, 0, M_COPYALL); if (n) n = ni6_input(n, off); /* XXX meaningless if n == NULL */ noff = sizeof(struct ip6_hdr); } else { struct prison *pr; u_char *p; int maxhlen, hlen; /* * XXX: this combination of flags is pointless, * but should we keep this for compatibility? */ if ((V_icmp6_nodeinfo & 5) != 5) break; if (code != 0) goto badcode; CTASSERT(sizeof(*nip6) + sizeof(*nicmp6) + 4 <= MHLEN); n = m_gethdr(M_NOWAIT, m->m_type); if (n == NULL) { /* Give up remote */ break; } if (!m_dup_pkthdr(n, m, M_NOWAIT)) { /* * Previous code did a blind M_COPY_PKTHDR * and said "just for rcvif". If true, then * we could tolerate the dup failing (due to * the deep copy of the tag chain). For now * be conservative and just fail. */ m_free(n); n = NULL; } maxhlen = M_TRAILINGSPACE(n) - (sizeof(*nip6) + sizeof(*nicmp6) + 4); pr = curthread->td_ucred->cr_prison; mtx_lock(&pr->pr_mtx); hlen = strlen(pr->pr_hostname); if (maxhlen > hlen) maxhlen = hlen; /* * Copy IPv6 and ICMPv6 only. */ nip6 = mtod(n, struct ip6_hdr *); bcopy(ip6, nip6, sizeof(struct ip6_hdr)); nicmp6 = (struct icmp6_hdr *)(nip6 + 1); bcopy(icmp6, nicmp6, sizeof(struct icmp6_hdr)); p = (u_char *)(nicmp6 + 1); bzero(p, 4); /* meaningless TTL */ bcopy(pr->pr_hostname, p + 4, maxhlen); mtx_unlock(&pr->pr_mtx); noff = sizeof(struct ip6_hdr); n->m_pkthdr.len = n->m_len = sizeof(struct ip6_hdr) + sizeof(struct icmp6_hdr) + 4 + maxhlen; nicmp6->icmp6_type = ICMP6_WRUREPLY; nicmp6->icmp6_code = 0; } if (n) { ICMP6STAT_INC(icp6s_reflect); ICMP6STAT_INC(icp6s_outhist[ICMP6_WRUREPLY]); icmp6_reflect(n, noff); } break; } case ICMP6_WRUREPLY: if (code != 0) goto badcode; break; case ND_ROUTER_SOLICIT: icmp6_ifstat_inc(ifp, ifs6_in_routersolicit); if (code != 0) goto badcode; if (icmp6len < sizeof(struct nd_router_solicit)) goto badlen; if ((n = m_copym(m, 0, M_COPYALL, M_NOWAIT)) == NULL) { /* give up local */ /* Send incoming SeND packet to user space. */ if (send_sendso_input_hook != NULL) { IP6_EXTHDR_CHECK(m, off, icmp6len, IPPROTO_DONE); error = send_sendso_input_hook(m, ifp, SND_IN, ip6len); /* -1 == no app on SEND socket */ if (error == 0) return (IPPROTO_DONE); nd6_rs_input(m, off, icmp6len); } else nd6_rs_input(m, off, icmp6len); m = NULL; goto freeit; } if (send_sendso_input_hook != NULL) { IP6_EXTHDR_CHECK(n, off, icmp6len, IPPROTO_DONE); error = send_sendso_input_hook(n, ifp, SND_IN, ip6len); if (error == 0) goto freeit; /* -1 == no app on SEND socket */ nd6_rs_input(n, off, icmp6len); } else nd6_rs_input(n, off, icmp6len); /* m stays. */ break; case ND_ROUTER_ADVERT: icmp6_ifstat_inc(ifp, ifs6_in_routeradvert); if (code != 0) goto badcode; if (icmp6len < sizeof(struct nd_router_advert)) goto badlen; if ((n = m_copym(m, 0, M_COPYALL, M_NOWAIT)) == NULL) { /* Send incoming SeND-protected/ND packet to user space. */ if (send_sendso_input_hook != NULL) { error = send_sendso_input_hook(m, ifp, SND_IN, ip6len); if (error == 0) return (IPPROTO_DONE); nd6_ra_input(m, off, icmp6len); } else nd6_ra_input(m, off, icmp6len); m = NULL; goto freeit; } if (send_sendso_input_hook != NULL) { error = send_sendso_input_hook(n, ifp, SND_IN, ip6len); if (error == 0) goto freeit; nd6_ra_input(n, off, icmp6len); } else nd6_ra_input(n, off, icmp6len); /* m stays. */ break; case ND_NEIGHBOR_SOLICIT: icmp6_ifstat_inc(ifp, ifs6_in_neighborsolicit); if (code != 0) goto badcode; if (icmp6len < sizeof(struct nd_neighbor_solicit)) goto badlen; if ((n = m_copym(m, 0, M_COPYALL, M_NOWAIT)) == NULL) { if (send_sendso_input_hook != NULL) { error = send_sendso_input_hook(m, ifp, SND_IN, ip6len); if (error == 0) return (IPPROTO_DONE); nd6_ns_input(m, off, icmp6len); } else nd6_ns_input(m, off, icmp6len); m = NULL; goto freeit; } if (send_sendso_input_hook != NULL) { error = send_sendso_input_hook(n, ifp, SND_IN, ip6len); if (error == 0) goto freeit; nd6_ns_input(n, off, icmp6len); } else nd6_ns_input(n, off, icmp6len); /* m stays. */ break; case ND_NEIGHBOR_ADVERT: icmp6_ifstat_inc(ifp, ifs6_in_neighboradvert); if (code != 0) goto badcode; if (icmp6len < sizeof(struct nd_neighbor_advert)) goto badlen; if ((n = m_copym(m, 0, M_COPYALL, M_NOWAIT)) == NULL) { /* Send incoming SeND-protected/ND packet to user space. */ if (send_sendso_input_hook != NULL) { error = send_sendso_input_hook(m, ifp, SND_IN, ip6len); if (error == 0) return (IPPROTO_DONE); nd6_na_input(m, off, icmp6len); } else nd6_na_input(m, off, icmp6len); m = NULL; goto freeit; } if (send_sendso_input_hook != NULL) { error = send_sendso_input_hook(n, ifp, SND_IN, ip6len); if (error == 0) goto freeit; nd6_na_input(n, off, icmp6len); } else nd6_na_input(n, off, icmp6len); /* m stays. */ break; case ND_REDIRECT: icmp6_ifstat_inc(ifp, ifs6_in_redirect); if (code != 0) goto badcode; if (icmp6len < sizeof(struct nd_redirect)) goto badlen; if ((n = m_copym(m, 0, M_COPYALL, M_NOWAIT)) == NULL) { if (send_sendso_input_hook != NULL) { error = send_sendso_input_hook(m, ifp, SND_IN, ip6len); if (error == 0) return (IPPROTO_DONE); icmp6_redirect_input(m, off); } else icmp6_redirect_input(m, off); m = NULL; goto freeit; } if (send_sendso_input_hook != NULL) { error = send_sendso_input_hook(n, ifp, SND_IN, ip6len); if (error == 0) goto freeit; icmp6_redirect_input(n, off); } else icmp6_redirect_input(n, off); /* m stays. */ break; case ICMP6_ROUTER_RENUMBERING: if (code != ICMP6_ROUTER_RENUMBERING_COMMAND && code != ICMP6_ROUTER_RENUMBERING_RESULT) goto badcode; if (icmp6len < sizeof(struct icmp6_router_renum)) goto badlen; break; default: nd6log((LOG_DEBUG, "icmp6_input: unknown type %d(src=%s, dst=%s, ifid=%d)\n", icmp6->icmp6_type, ip6_sprintf(ip6bufs, &ip6->ip6_src), ip6_sprintf(ip6bufd, &ip6->ip6_dst), ifp ? ifp->if_index : 0)); if (icmp6->icmp6_type < ICMP6_ECHO_REQUEST) { /* ICMPv6 error: MUST deliver it by spec... */ code = PRC_NCMDS; /* deliver */ } else { /* ICMPv6 informational: MUST not deliver */ break; } deliver: if (icmp6_notify_error(&m, off, icmp6len, code) != 0) { /* In this case, m should've been freed. */ return (IPPROTO_DONE); } break; badcode: ICMP6STAT_INC(icp6s_badcode); break; badlen: ICMP6STAT_INC(icp6s_badlen); break; } /* deliver the packet to appropriate sockets */ icmp6_rip6_input(&m, *offp); return IPPROTO_DONE; freeit: m_freem(m); return IPPROTO_DONE; } static int icmp6_notify_error(struct mbuf **mp, int off, int icmp6len, int code) { struct mbuf *m = *mp; struct icmp6_hdr *icmp6; struct ip6_hdr *eip6; u_int32_t notifymtu; struct sockaddr_in6 icmp6src, icmp6dst; if (icmp6len < sizeof(struct icmp6_hdr) + sizeof(struct ip6_hdr)) { ICMP6STAT_INC(icp6s_tooshort); goto freeit; } #ifndef PULLDOWN_TEST IP6_EXTHDR_CHECK(m, off, sizeof(struct icmp6_hdr) + sizeof(struct ip6_hdr), -1); icmp6 = (struct icmp6_hdr *)(mtod(m, caddr_t) + off); #else IP6_EXTHDR_GET(icmp6, struct icmp6_hdr *, m, off, sizeof(*icmp6) + sizeof(struct ip6_hdr)); if (icmp6 == NULL) { ICMP6STAT_INC(icp6s_tooshort); return (-1); } #endif eip6 = (struct ip6_hdr *)(icmp6 + 1); /* Detect the upper level protocol */ { void (*ctlfunc)(int, struct sockaddr *, void *); u_int8_t nxt = eip6->ip6_nxt; int eoff = off + sizeof(struct icmp6_hdr) + sizeof(struct ip6_hdr); struct ip6ctlparam ip6cp; struct in6_addr *finaldst = NULL; int icmp6type = icmp6->icmp6_type; struct ip6_frag *fh; struct ip6_rthdr *rth; struct ip6_rthdr0 *rth0; int rthlen; while (1) { /* XXX: should avoid infinite loop explicitly? */ struct ip6_ext *eh; switch (nxt) { case IPPROTO_HOPOPTS: case IPPROTO_DSTOPTS: case IPPROTO_AH: #ifndef PULLDOWN_TEST IP6_EXTHDR_CHECK(m, 0, eoff + sizeof(struct ip6_ext), -1); eh = (struct ip6_ext *)(mtod(m, caddr_t) + eoff); #else IP6_EXTHDR_GET(eh, struct ip6_ext *, m, eoff, sizeof(*eh)); if (eh == NULL) { ICMP6STAT_INC(icp6s_tooshort); return (-1); } #endif if (nxt == IPPROTO_AH) eoff += (eh->ip6e_len + 2) << 2; else eoff += (eh->ip6e_len + 1) << 3; nxt = eh->ip6e_nxt; break; case IPPROTO_ROUTING: /* * When the erroneous packet contains a * routing header, we should examine the * header to determine the final destination. * Otherwise, we can't properly update * information that depends on the final * destination (e.g. path MTU). */ #ifndef PULLDOWN_TEST IP6_EXTHDR_CHECK(m, 0, eoff + sizeof(*rth), -1); rth = (struct ip6_rthdr *) (mtod(m, caddr_t) + eoff); #else IP6_EXTHDR_GET(rth, struct ip6_rthdr *, m, eoff, sizeof(*rth)); if (rth == NULL) { ICMP6STAT_INC(icp6s_tooshort); return (-1); } #endif rthlen = (rth->ip6r_len + 1) << 3; /* * XXX: currently there is no * officially defined type other * than type-0. * Note that if the segment left field * is 0, all intermediate hops must * have been passed. */ if (rth->ip6r_segleft && rth->ip6r_type == IPV6_RTHDR_TYPE_0) { int hops; #ifndef PULLDOWN_TEST IP6_EXTHDR_CHECK(m, 0, eoff + rthlen, -1); rth0 = (struct ip6_rthdr0 *) (mtod(m, caddr_t) + eoff); #else IP6_EXTHDR_GET(rth0, struct ip6_rthdr0 *, m, eoff, rthlen); if (rth0 == NULL) { ICMP6STAT_INC(icp6s_tooshort); return (-1); } #endif /* just ignore a bogus header */ if ((rth0->ip6r0_len % 2) == 0 && (hops = rth0->ip6r0_len/2)) finaldst = (struct in6_addr *)(rth0 + 1) + (hops - 1); } eoff += rthlen; nxt = rth->ip6r_nxt; break; case IPPROTO_FRAGMENT: #ifndef PULLDOWN_TEST IP6_EXTHDR_CHECK(m, 0, eoff + sizeof(struct ip6_frag), -1); fh = (struct ip6_frag *)(mtod(m, caddr_t) + eoff); #else IP6_EXTHDR_GET(fh, struct ip6_frag *, m, eoff, sizeof(*fh)); if (fh == NULL) { ICMP6STAT_INC(icp6s_tooshort); return (-1); } #endif /* * Data after a fragment header is meaningless * unless it is the first fragment, but * we'll go to the notify label for path MTU * discovery. */ if (fh->ip6f_offlg & IP6F_OFF_MASK) goto notify; eoff += sizeof(struct ip6_frag); nxt = fh->ip6f_nxt; break; default: /* * This case includes ESP and the No Next * Header. In such cases going to the notify * label does not have any meaning * (i.e. ctlfunc will be NULL), but we go * anyway since we might have to update * path MTU information. */ goto notify; } } notify: #ifndef PULLDOWN_TEST icmp6 = (struct icmp6_hdr *)(mtod(m, caddr_t) + off); #else IP6_EXTHDR_GET(icmp6, struct icmp6_hdr *, m, off, sizeof(*icmp6) + sizeof(struct ip6_hdr)); if (icmp6 == NULL) { ICMP6STAT_INC(icp6s_tooshort); return (-1); } #endif /* * retrieve parameters from the inner IPv6 header, and convert * them into sockaddr structures. * XXX: there is no guarantee that the source or destination * addresses of the inner packet are in the same scope as * the addresses of the icmp packet. But there is no other * way to determine the zone. */ eip6 = (struct ip6_hdr *)(icmp6 + 1); bzero(&icmp6dst, sizeof(icmp6dst)); icmp6dst.sin6_len = sizeof(struct sockaddr_in6); icmp6dst.sin6_family = AF_INET6; if (finaldst == NULL) icmp6dst.sin6_addr = eip6->ip6_dst; else icmp6dst.sin6_addr = *finaldst; if (in6_setscope(&icmp6dst.sin6_addr, m->m_pkthdr.rcvif, NULL)) goto freeit; bzero(&icmp6src, sizeof(icmp6src)); icmp6src.sin6_len = sizeof(struct sockaddr_in6); icmp6src.sin6_family = AF_INET6; icmp6src.sin6_addr = eip6->ip6_src; if (in6_setscope(&icmp6src.sin6_addr, m->m_pkthdr.rcvif, NULL)) goto freeit; icmp6src.sin6_flowinfo = (eip6->ip6_flow & IPV6_FLOWLABEL_MASK); if (finaldst == NULL) finaldst = &eip6->ip6_dst; ip6cp.ip6c_m = m; ip6cp.ip6c_icmp6 = icmp6; ip6cp.ip6c_ip6 = (struct ip6_hdr *)(icmp6 + 1); ip6cp.ip6c_off = eoff; ip6cp.ip6c_finaldst = finaldst; ip6cp.ip6c_src = &icmp6src; ip6cp.ip6c_nxt = nxt; if (icmp6type == ICMP6_PACKET_TOO_BIG) { notifymtu = ntohl(icmp6->icmp6_mtu); ip6cp.ip6c_cmdarg = (void *)¬ifymtu; icmp6_mtudisc_update(&ip6cp, 1); /*XXX*/ } ctlfunc = (void (*)(int, struct sockaddr *, void *)) (inet6sw[ip6_protox[nxt]].pr_ctlinput); if (ctlfunc) { (void) (*ctlfunc)(code, (struct sockaddr *)&icmp6dst, &ip6cp); } } *mp = m; return (0); freeit: m_freem(m); return (-1); } void icmp6_mtudisc_update(struct ip6ctlparam *ip6cp, int validated) { struct in6_addr *dst = ip6cp->ip6c_finaldst; struct icmp6_hdr *icmp6 = ip6cp->ip6c_icmp6; struct mbuf *m = ip6cp->ip6c_m; /* will be necessary for scope issue */ u_int mtu = ntohl(icmp6->icmp6_mtu); struct in_conninfo inc; #if 0 /* * RFC2460 section 5, last paragraph. * even though minimum link MTU for IPv6 is IPV6_MMTU, * we may see ICMPv6 too big with mtu < IPV6_MMTU * due to packet translator in the middle. * see ip6_output() and ip6_getpmtu() "alwaysfrag" case for * special handling. */ if (mtu < IPV6_MMTU) return; #endif /* * we reject ICMPv6 too big with abnormally small value. * XXX what is the good definition of "abnormally small"? */ if (mtu < sizeof(struct ip6_hdr) + sizeof(struct ip6_frag) + 8) return; if (!validated) return; /* * In case the suggested mtu is less than IPV6_MMTU, we * only need to remember that it was for above mentioned * "alwaysfrag" case. * Try to be as close to the spec as possible. */ if (mtu < IPV6_MMTU) mtu = IPV6_MMTU - 8; bzero(&inc, sizeof(inc)); inc.inc_fibnum = M_GETFIB(m); inc.inc_flags |= INC_ISIPV6; inc.inc6_faddr = *dst; if (in6_setscope(&inc.inc6_faddr, m->m_pkthdr.rcvif, NULL)) return; if (mtu < tcp_maxmtu6(&inc, NULL)) { tcp_hc_updatemtu(&inc, mtu); ICMP6STAT_INC(icp6s_pmtuchg); } } /* * Process a Node Information Query packet, based on * draft-ietf-ipngwg-icmp-name-lookups-07. * * Spec incompatibilities: * - IPv6 Subject address handling * - IPv4 Subject address handling support missing * - Proxy reply (answer even if it's not for me) * - joins NI group address at in6_ifattach() time only, does not cope * with hostname changes by sethostname(3) */ static struct mbuf * ni6_input(struct mbuf *m, int off) { struct icmp6_nodeinfo *ni6, *nni6; struct mbuf *n = NULL; struct prison *pr; u_int16_t qtype; int subjlen; int replylen = sizeof(struct ip6_hdr) + sizeof(struct icmp6_nodeinfo); struct ni_reply_fqdn *fqdn; int addrs; /* for NI_QTYPE_NODEADDR */ struct ifnet *ifp = NULL; /* for NI_QTYPE_NODEADDR */ struct in6_addr in6_subj; /* subject address */ struct ip6_hdr *ip6; int oldfqdn = 0; /* if 1, return pascal string (03 draft) */ char *subj = NULL; struct in6_ifaddr *ia6 = NULL; ip6 = mtod(m, struct ip6_hdr *); #ifndef PULLDOWN_TEST ni6 = (struct icmp6_nodeinfo *)(mtod(m, caddr_t) + off); #else IP6_EXTHDR_GET(ni6, struct icmp6_nodeinfo *, m, off, sizeof(*ni6)); if (ni6 == NULL) { /* m is already reclaimed */ return (NULL); } #endif /* * Validate IPv6 source address. * The default configuration MUST be to refuse answering queries from * global-scope addresses according to RFC4602. * Notes: * - it's not very clear what "refuse" means; this implementation * simply drops it. * - it's not very easy to identify global-scope (unicast) addresses * since there are many prefixes for them. It should be safer * and in practice sufficient to check "all" but loopback and * link-local (note that site-local unicast was deprecated and * ULA is defined as global scope-wise) */ if ((V_icmp6_nodeinfo & ICMP6_NODEINFO_GLOBALOK) == 0 && !IN6_IS_ADDR_LOOPBACK(&ip6->ip6_src) && !IN6_IS_ADDR_LINKLOCAL(&ip6->ip6_src)) goto bad; /* * Validate IPv6 destination address. * * The Responder must discard the Query without further processing * unless it is one of the Responder's unicast or anycast addresses, or * a link-local scope multicast address which the Responder has joined. * [RFC4602, Section 5.] */ if (IN6_IS_ADDR_MULTICAST(&ip6->ip6_dst)) { if (!IN6_IS_ADDR_MC_LINKLOCAL(&ip6->ip6_dst)) goto bad; /* else it's a link-local multicast, fine */ } else { /* unicast or anycast */ ia6 = in6ifa_ifwithaddr(&ip6->ip6_dst, 0 /* XXX */); if (ia6 == NULL) goto bad; /* XXX impossible */ if ((ia6->ia6_flags & IN6_IFF_TEMPORARY) && !(V_icmp6_nodeinfo & ICMP6_NODEINFO_TMPADDROK)) { ifa_free(&ia6->ia_ifa); nd6log((LOG_DEBUG, "ni6_input: ignore node info to " "a temporary address in %s:%d", __FILE__, __LINE__)); goto bad; } ifa_free(&ia6->ia_ifa); } /* validate query Subject field. */ qtype = ntohs(ni6->ni_qtype); subjlen = m->m_pkthdr.len - off - sizeof(struct icmp6_nodeinfo); switch (qtype) { case NI_QTYPE_NOOP: case NI_QTYPE_SUPTYPES: /* 07 draft */ if (ni6->ni_code == ICMP6_NI_SUBJ_FQDN && subjlen == 0) break; /* FALLTHROUGH */ case NI_QTYPE_FQDN: case NI_QTYPE_NODEADDR: case NI_QTYPE_IPV4ADDR: switch (ni6->ni_code) { case ICMP6_NI_SUBJ_IPV6: #if ICMP6_NI_SUBJ_IPV6 != 0 case 0: #endif /* * backward compatibility - try to accept 03 draft * format, where no Subject is present. */ if (qtype == NI_QTYPE_FQDN && ni6->ni_code == 0 && subjlen == 0) { oldfqdn++; break; } #if ICMP6_NI_SUBJ_IPV6 != 0 if (ni6->ni_code != ICMP6_NI_SUBJ_IPV6) goto bad; #endif if (subjlen != sizeof(struct in6_addr)) goto bad; /* * Validate Subject address. * * Not sure what exactly "address belongs to the node" * means in the spec, is it just unicast, or what? * * At this moment we consider Subject address as * "belong to the node" if the Subject address equals * to the IPv6 destination address; validation for * IPv6 destination address should have done enough * check for us. * * We do not do proxy at this moment. */ /* m_pulldown instead of copy? */ m_copydata(m, off + sizeof(struct icmp6_nodeinfo), subjlen, (caddr_t)&in6_subj); if (in6_setscope(&in6_subj, m->m_pkthdr.rcvif, NULL)) goto bad; subj = (char *)&in6_subj; if (IN6_ARE_ADDR_EQUAL(&ip6->ip6_dst, &in6_subj)) break; /* * XXX if we are to allow other cases, we should really * be careful about scope here. * basically, we should disallow queries toward IPv6 * destination X with subject Y, * if scope(X) > scope(Y). * if we allow scope(X) > scope(Y), it will result in * information leakage across scope boundary. */ goto bad; case ICMP6_NI_SUBJ_FQDN: /* * Validate Subject name with gethostname(3). * * The behavior may need some debate, since: * - we are not sure if the node has FQDN as * hostname (returned by gethostname(3)). * - the code does wildcard match for truncated names. * however, we are not sure if we want to perform * wildcard match, if gethostname(3) side has * truncated hostname. */ pr = curthread->td_ucred->cr_prison; mtx_lock(&pr->pr_mtx); n = ni6_nametodns(pr->pr_hostname, strlen(pr->pr_hostname), 0); mtx_unlock(&pr->pr_mtx); if (!n || n->m_next || n->m_len == 0) goto bad; IP6_EXTHDR_GET(subj, char *, m, off + sizeof(struct icmp6_nodeinfo), subjlen); if (subj == NULL) goto bad; if (!ni6_dnsmatch(subj, subjlen, mtod(n, const char *), n->m_len)) { goto bad; } m_freem(n); n = NULL; break; case ICMP6_NI_SUBJ_IPV4: /* XXX: to be implemented? */ default: goto bad; } break; } /* refuse based on configuration. XXX ICMP6_NI_REFUSED? */ switch (qtype) { case NI_QTYPE_FQDN: if ((V_icmp6_nodeinfo & ICMP6_NODEINFO_FQDNOK) == 0) goto bad; break; case NI_QTYPE_NODEADDR: case NI_QTYPE_IPV4ADDR: if ((V_icmp6_nodeinfo & ICMP6_NODEINFO_NODEADDROK) == 0) goto bad; break; } /* guess reply length */ switch (qtype) { case NI_QTYPE_NOOP: break; /* no reply data */ case NI_QTYPE_SUPTYPES: replylen += sizeof(u_int32_t); break; case NI_QTYPE_FQDN: /* XXX will append an mbuf */ replylen += offsetof(struct ni_reply_fqdn, ni_fqdn_namelen); break; case NI_QTYPE_NODEADDR: addrs = ni6_addrs(ni6, m, &ifp, (struct in6_addr *)subj); if ((replylen += addrs * (sizeof(struct in6_addr) + sizeof(u_int32_t))) > MCLBYTES) replylen = MCLBYTES; /* XXX: will truncate pkt later */ break; case NI_QTYPE_IPV4ADDR: /* unsupported - should respond with unknown Qtype? */ break; default: /* * XXX: We must return a reply with the ICMP6 code * `unknown Qtype' in this case. However we regard the case * as an FQDN query for backward compatibility. * Older versions set a random value to this field, * so it rarely varies in the defined qtypes. * But the mechanism is not reliable... * maybe we should obsolete older versions. */ qtype = NI_QTYPE_FQDN; /* XXX will append an mbuf */ replylen += offsetof(struct ni_reply_fqdn, ni_fqdn_namelen); oldfqdn++; break; } /* Allocate an mbuf to reply. */ if (replylen > MCLBYTES) { /* * XXX: should we try to allocate more? But MCLBYTES * is probably much larger than IPV6_MMTU... */ goto bad; } if (replylen > MHLEN) n = m_getcl(M_NOWAIT, m->m_type, M_PKTHDR); else n = m_gethdr(M_NOWAIT, m->m_type); if (n == NULL) { m_freem(m); return (NULL); } m_move_pkthdr(n, m); /* just for recvif and FIB */ n->m_pkthdr.len = n->m_len = replylen; /* copy mbuf header and IPv6 + Node Information base headers */ bcopy(mtod(m, caddr_t), mtod(n, caddr_t), sizeof(struct ip6_hdr)); nni6 = (struct icmp6_nodeinfo *)(mtod(n, struct ip6_hdr *) + 1); bcopy((caddr_t)ni6, (caddr_t)nni6, sizeof(struct icmp6_nodeinfo)); /* qtype dependent procedure */ switch (qtype) { case NI_QTYPE_NOOP: nni6->ni_code = ICMP6_NI_SUCCESS; nni6->ni_flags = 0; break; case NI_QTYPE_SUPTYPES: { u_int32_t v; nni6->ni_code = ICMP6_NI_SUCCESS; nni6->ni_flags = htons(0x0000); /* raw bitmap */ /* supports NOOP, SUPTYPES, FQDN, and NODEADDR */ v = (u_int32_t)htonl(0x0000000f); bcopy(&v, nni6 + 1, sizeof(u_int32_t)); break; } case NI_QTYPE_FQDN: nni6->ni_code = ICMP6_NI_SUCCESS; fqdn = (struct ni_reply_fqdn *)(mtod(n, caddr_t) + sizeof(struct ip6_hdr) + sizeof(struct icmp6_nodeinfo)); nni6->ni_flags = 0; /* XXX: meaningless TTL */ fqdn->ni_fqdn_ttl = 0; /* ditto. */ /* * XXX do we really have FQDN in hostname? */ pr = curthread->td_ucred->cr_prison; mtx_lock(&pr->pr_mtx); n->m_next = ni6_nametodns(pr->pr_hostname, strlen(pr->pr_hostname), oldfqdn); mtx_unlock(&pr->pr_mtx); if (n->m_next == NULL) goto bad; /* XXX we assume that n->m_next is not a chain */ if (n->m_next->m_next != NULL) goto bad; n->m_pkthdr.len += n->m_next->m_len; break; case NI_QTYPE_NODEADDR: { int lenlim, copied; nni6->ni_code = ICMP6_NI_SUCCESS; n->m_pkthdr.len = n->m_len = sizeof(struct ip6_hdr) + sizeof(struct icmp6_nodeinfo); lenlim = M_TRAILINGSPACE(n); copied = ni6_store_addrs(ni6, nni6, ifp, lenlim); /* XXX: reset mbuf length */ n->m_pkthdr.len = n->m_len = sizeof(struct ip6_hdr) + sizeof(struct icmp6_nodeinfo) + copied; break; } default: break; /* XXX impossible! */ } nni6->ni_type = ICMP6_NI_REPLY; m_freem(m); return (n); bad: m_freem(m); if (n) m_freem(n); return (NULL); } /* * make a mbuf with DNS-encoded string. no compression support. * * XXX names with less than 2 dots (like "foo" or "foo.section") will be * treated as truncated name (two \0 at the end). this is a wild guess. * * old - return pascal string if non-zero */ static struct mbuf * ni6_nametodns(const char *name, int namelen, int old) { struct mbuf *m; char *cp, *ep; const char *p, *q; int i, len, nterm; if (old) len = namelen + 1; else len = MCLBYTES; /* Because MAXHOSTNAMELEN is usually 256, we use cluster mbuf. */ if (len > MLEN) m = m_getcl(M_NOWAIT, MT_DATA, 0); else m = m_get(M_NOWAIT, MT_DATA); if (m == NULL) goto fail; if (old) { m->m_len = len; *mtod(m, char *) = namelen; bcopy(name, mtod(m, char *) + 1, namelen); return m; } else { m->m_len = 0; cp = mtod(m, char *); ep = mtod(m, char *) + M_TRAILINGSPACE(m); /* if not certain about my name, return empty buffer */ if (namelen == 0) return m; /* * guess if it looks like shortened hostname, or FQDN. * shortened hostname needs two trailing "\0". */ i = 0; for (p = name; p < name + namelen; p++) { if (*p && *p == '.') i++; } if (i < 2) nterm = 2; else nterm = 1; p = name; while (cp < ep && p < name + namelen) { i = 0; for (q = p; q < name + namelen && *q && *q != '.'; q++) i++; /* result does not fit into mbuf */ if (cp + i + 1 >= ep) goto fail; /* * DNS label length restriction, RFC1035 page 8. * "i == 0" case is included here to avoid returning * 0-length label on "foo..bar". */ if (i <= 0 || i >= 64) goto fail; *cp++ = i; bcopy(p, cp, i); cp += i; p = q; if (p < name + namelen && *p == '.') p++; } /* termination */ if (cp + nterm >= ep) goto fail; while (nterm-- > 0) *cp++ = '\0'; m->m_len = cp - mtod(m, char *); return m; } panic("should not reach here"); /* NOTREACHED */ fail: if (m) m_freem(m); return NULL; } /* * check if two DNS-encoded string matches. takes care of truncated * form (with \0\0 at the end). no compression support. * XXX upper/lowercase match (see RFC2065) */ static int ni6_dnsmatch(const char *a, int alen, const char *b, int blen) { const char *a0, *b0; int l; /* simplest case - need validation? */ if (alen == blen && bcmp(a, b, alen) == 0) return 1; a0 = a; b0 = b; /* termination is mandatory */ if (alen < 2 || blen < 2) return 0; if (a0[alen - 1] != '\0' || b0[blen - 1] != '\0') return 0; alen--; blen--; while (a - a0 < alen && b - b0 < blen) { if (a - a0 + 1 > alen || b - b0 + 1 > blen) return 0; if ((signed char)a[0] < 0 || (signed char)b[0] < 0) return 0; /* we don't support compression yet */ if (a[0] >= 64 || b[0] >= 64) return 0; /* truncated case */ if (a[0] == 0 && a - a0 == alen - 1) return 1; if (b[0] == 0 && b - b0 == blen - 1) return 1; if (a[0] == 0 || b[0] == 0) return 0; if (a[0] != b[0]) return 0; l = a[0]; if (a - a0 + 1 + l > alen || b - b0 + 1 + l > blen) return 0; if (bcmp(a + 1, b + 1, l) != 0) return 0; a += 1 + l; b += 1 + l; } if (a - a0 == alen && b - b0 == blen) return 1; else return 0; } /* * calculate the number of addresses to be returned in the node info reply. */ static int ni6_addrs(struct icmp6_nodeinfo *ni6, struct mbuf *m, struct ifnet **ifpp, struct in6_addr *subj) { struct ifnet *ifp; struct in6_ifaddr *ifa6; struct ifaddr *ifa; int addrs = 0, addrsofif, iffound = 0; int niflags = ni6->ni_flags; if ((niflags & NI_NODEADDR_FLAG_ALL) == 0) { switch (ni6->ni_code) { case ICMP6_NI_SUBJ_IPV6: if (subj == NULL) /* must be impossible... */ return (0); break; default: /* * XXX: we only support IPv6 subject address for * this Qtype. */ return (0); } } IFNET_RLOCK_NOSLEEP(); TAILQ_FOREACH(ifp, &V_ifnet, if_link) { addrsofif = 0; IF_ADDR_RLOCK(ifp); TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) { if (ifa->ifa_addr->sa_family != AF_INET6) continue; ifa6 = (struct in6_ifaddr *)ifa; if ((niflags & NI_NODEADDR_FLAG_ALL) == 0 && IN6_ARE_ADDR_EQUAL(subj, &ifa6->ia_addr.sin6_addr)) iffound = 1; /* * IPv4-mapped addresses can only be returned by a * Node Information proxy, since they represent * addresses of IPv4-only nodes, which perforce do * not implement this protocol. * [icmp-name-lookups-07, Section 5.4] * So we don't support NI_NODEADDR_FLAG_COMPAT in * this function at this moment. */ /* What do we have to do about ::1? */ switch (in6_addrscope(&ifa6->ia_addr.sin6_addr)) { case IPV6_ADDR_SCOPE_LINKLOCAL: if ((niflags & NI_NODEADDR_FLAG_LINKLOCAL) == 0) continue; break; case IPV6_ADDR_SCOPE_SITELOCAL: if ((niflags & NI_NODEADDR_FLAG_SITELOCAL) == 0) continue; break; case IPV6_ADDR_SCOPE_GLOBAL: if ((niflags & NI_NODEADDR_FLAG_GLOBAL) == 0) continue; break; default: continue; } /* * check if anycast is okay. * XXX: just experimental. not in the spec. */ if ((ifa6->ia6_flags & IN6_IFF_ANYCAST) != 0 && (niflags & NI_NODEADDR_FLAG_ANYCAST) == 0) continue; /* we need only unicast addresses */ if ((ifa6->ia6_flags & IN6_IFF_TEMPORARY) != 0 && (V_icmp6_nodeinfo & ICMP6_NODEINFO_TMPADDROK) == 0) { continue; } addrsofif++; /* count the address */ } IF_ADDR_RUNLOCK(ifp); if (iffound) { *ifpp = ifp; IFNET_RUNLOCK_NOSLEEP(); return (addrsofif); } addrs += addrsofif; } IFNET_RUNLOCK_NOSLEEP(); return (addrs); } static int ni6_store_addrs(struct icmp6_nodeinfo *ni6, struct icmp6_nodeinfo *nni6, struct ifnet *ifp0, int resid) { struct ifnet *ifp; struct in6_ifaddr *ifa6; struct ifaddr *ifa; struct ifnet *ifp_dep = NULL; int copied = 0, allow_deprecated = 0; u_char *cp = (u_char *)(nni6 + 1); int niflags = ni6->ni_flags; u_int32_t ltime; if (ifp0 == NULL && !(niflags & NI_NODEADDR_FLAG_ALL)) return (0); /* needless to copy */ IFNET_RLOCK_NOSLEEP(); ifp = ifp0 ? ifp0 : TAILQ_FIRST(&V_ifnet); again: for (; ifp; ifp = TAILQ_NEXT(ifp, if_link)) { IF_ADDR_RLOCK(ifp); TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) { if (ifa->ifa_addr->sa_family != AF_INET6) continue; ifa6 = (struct in6_ifaddr *)ifa; if ((ifa6->ia6_flags & IN6_IFF_DEPRECATED) != 0 && allow_deprecated == 0) { /* * prefererred address should be put before * deprecated addresses. */ /* record the interface for later search */ if (ifp_dep == NULL) ifp_dep = ifp; continue; } else if ((ifa6->ia6_flags & IN6_IFF_DEPRECATED) == 0 && allow_deprecated != 0) continue; /* we now collect deprecated addrs */ /* What do we have to do about ::1? */ switch (in6_addrscope(&ifa6->ia_addr.sin6_addr)) { case IPV6_ADDR_SCOPE_LINKLOCAL: if ((niflags & NI_NODEADDR_FLAG_LINKLOCAL) == 0) continue; break; case IPV6_ADDR_SCOPE_SITELOCAL: if ((niflags & NI_NODEADDR_FLAG_SITELOCAL) == 0) continue; break; case IPV6_ADDR_SCOPE_GLOBAL: if ((niflags & NI_NODEADDR_FLAG_GLOBAL) == 0) continue; break; default: continue; } /* * check if anycast is okay. * XXX: just experimental. not in the spec. */ if ((ifa6->ia6_flags & IN6_IFF_ANYCAST) != 0 && (niflags & NI_NODEADDR_FLAG_ANYCAST) == 0) continue; if ((ifa6->ia6_flags & IN6_IFF_TEMPORARY) != 0 && (V_icmp6_nodeinfo & ICMP6_NODEINFO_TMPADDROK) == 0) { continue; } /* now we can copy the address */ if (resid < sizeof(struct in6_addr) + sizeof(u_int32_t)) { IF_ADDR_RUNLOCK(ifp); /* * We give up much more copy. * Set the truncate flag and return. */ nni6->ni_flags |= NI_NODEADDR_FLAG_TRUNCATE; IFNET_RUNLOCK_NOSLEEP(); return (copied); } /* * Set the TTL of the address. * The TTL value should be one of the following * according to the specification: * * 1. The remaining lifetime of a DHCP lease on the * address, or * 2. The remaining Valid Lifetime of a prefix from * which the address was derived through Stateless * Autoconfiguration. * * Note that we currently do not support stateful * address configuration by DHCPv6, so the former * case can't happen. */ if (ifa6->ia6_lifetime.ia6t_expire == 0) ltime = ND6_INFINITE_LIFETIME; else { if (ifa6->ia6_lifetime.ia6t_expire > time_uptime) ltime = htonl(ifa6->ia6_lifetime.ia6t_expire - time_uptime); else ltime = 0; } bcopy(<ime, cp, sizeof(u_int32_t)); cp += sizeof(u_int32_t); /* copy the address itself */ bcopy(&ifa6->ia_addr.sin6_addr, cp, sizeof(struct in6_addr)); in6_clearscope((struct in6_addr *)cp); /* XXX */ cp += sizeof(struct in6_addr); resid -= (sizeof(struct in6_addr) + sizeof(u_int32_t)); copied += (sizeof(struct in6_addr) + sizeof(u_int32_t)); } IF_ADDR_RUNLOCK(ifp); if (ifp0) /* we need search only on the specified IF */ break; } if (allow_deprecated == 0 && ifp_dep != NULL) { ifp = ifp_dep; allow_deprecated = 1; goto again; } IFNET_RUNLOCK_NOSLEEP(); return (copied); } /* * XXX almost dup'ed code with rip6_input. */ static int icmp6_rip6_input(struct mbuf **mp, int off) { struct mbuf *m = *mp; struct ip6_hdr *ip6 = mtod(m, struct ip6_hdr *); struct inpcb *in6p; struct inpcb *last = NULL; struct sockaddr_in6 fromsa; struct icmp6_hdr *icmp6; struct mbuf *opts = NULL; #ifndef PULLDOWN_TEST /* this is assumed to be safe. */ icmp6 = (struct icmp6_hdr *)((caddr_t)ip6 + off); #else IP6_EXTHDR_GET(icmp6, struct icmp6_hdr *, m, off, sizeof(*icmp6)); if (icmp6 == NULL) { /* m is already reclaimed */ return (IPPROTO_DONE); } #endif /* * XXX: the address may have embedded scope zone ID, which should be * hidden from applications. */ bzero(&fromsa, sizeof(fromsa)); fromsa.sin6_family = AF_INET6; fromsa.sin6_len = sizeof(struct sockaddr_in6); fromsa.sin6_addr = ip6->ip6_src; if (sa6_recoverscope(&fromsa)) { m_freem(m); return (IPPROTO_DONE); } INP_INFO_RLOCK(&V_ripcbinfo); LIST_FOREACH(in6p, &V_ripcb, inp_list) { if ((in6p->inp_vflag & INP_IPV6) == 0) continue; if (in6p->inp_ip_p != IPPROTO_ICMPV6) continue; if (!IN6_IS_ADDR_UNSPECIFIED(&in6p->in6p_laddr) && !IN6_ARE_ADDR_EQUAL(&in6p->in6p_laddr, &ip6->ip6_dst)) continue; if (!IN6_IS_ADDR_UNSPECIFIED(&in6p->in6p_faddr) && !IN6_ARE_ADDR_EQUAL(&in6p->in6p_faddr, &ip6->ip6_src)) continue; INP_RLOCK(in6p); if (ICMP6_FILTER_WILLBLOCK(icmp6->icmp6_type, in6p->in6p_icmp6filt)) { INP_RUNLOCK(in6p); continue; } if (last != NULL) { struct mbuf *n = NULL; /* * Recent network drivers tend to allocate a single * mbuf cluster, rather than to make a couple of * mbufs without clusters. Also, since the IPv6 code * path tries to avoid m_pullup(), it is highly * probable that we still have an mbuf cluster here * even though the necessary length can be stored in an * mbuf's internal buffer. * Meanwhile, the default size of the receive socket * buffer for raw sockets is not so large. This means * the possibility of packet loss is relatively higher * than before. To avoid this scenario, we copy the * received data to a separate mbuf that does not use * a cluster, if possible. * XXX: it is better to copy the data after stripping * intermediate headers. */ if ((m->m_flags & M_EXT) && m->m_next == NULL && m->m_len <= MHLEN) { n = m_get(M_NOWAIT, m->m_type); if (n != NULL) { if (m_dup_pkthdr(n, m, M_NOWAIT)) { bcopy(m->m_data, n->m_data, m->m_len); n->m_len = m->m_len; } else { m_free(n); n = NULL; } } } if (n != NULL || (n = m_copy(m, 0, (int)M_COPYALL)) != NULL) { if (last->inp_flags & INP_CONTROLOPTS) ip6_savecontrol(last, n, &opts); /* strip intermediate headers */ m_adj(n, off); SOCKBUF_LOCK(&last->inp_socket->so_rcv); if (sbappendaddr_locked( &last->inp_socket->so_rcv, (struct sockaddr *)&fromsa, n, opts) == 0) { /* should notify about lost packet */ m_freem(n); if (opts) { m_freem(opts); } SOCKBUF_UNLOCK( &last->inp_socket->so_rcv); } else sorwakeup_locked(last->inp_socket); opts = NULL; } INP_RUNLOCK(last); } last = in6p; } INP_INFO_RUNLOCK(&V_ripcbinfo); if (last != NULL) { if (last->inp_flags & INP_CONTROLOPTS) ip6_savecontrol(last, m, &opts); /* strip intermediate headers */ m_adj(m, off); /* avoid using mbuf clusters if possible (see above) */ if ((m->m_flags & M_EXT) && m->m_next == NULL && m->m_len <= MHLEN) { struct mbuf *n; n = m_get(M_NOWAIT, m->m_type); if (n != NULL) { if (m_dup_pkthdr(n, m, M_NOWAIT)) { bcopy(m->m_data, n->m_data, m->m_len); n->m_len = m->m_len; m_freem(m); m = n; } else { m_freem(n); n = NULL; } } } SOCKBUF_LOCK(&last->inp_socket->so_rcv); if (sbappendaddr_locked(&last->inp_socket->so_rcv, (struct sockaddr *)&fromsa, m, opts) == 0) { m_freem(m); if (opts) m_freem(opts); SOCKBUF_UNLOCK(&last->inp_socket->so_rcv); } else sorwakeup_locked(last->inp_socket); INP_RUNLOCK(last); } else { m_freem(m); IP6STAT_DEC(ip6s_delivered); } return IPPROTO_DONE; } /* * Reflect the ip6 packet back to the source. * OFF points to the icmp6 header, counted from the top of the mbuf. */ void icmp6_reflect(struct mbuf *m, size_t off) { struct in6_addr src, *srcp = NULL; struct ip6_hdr *ip6; struct icmp6_hdr *icmp6; struct in6_ifaddr *ia = NULL; struct ifnet *outif = NULL; int plen; int type, code; /* too short to reflect */ if (off < sizeof(struct ip6_hdr)) { nd6log((LOG_DEBUG, "sanity fail: off=%lx, sizeof(ip6)=%lx in %s:%d\n", (u_long)off, (u_long)sizeof(struct ip6_hdr), __FILE__, __LINE__)); goto bad; } /* * If there are extra headers between IPv6 and ICMPv6, strip * off that header first. */ #ifdef DIAGNOSTIC if (sizeof(struct ip6_hdr) + sizeof(struct icmp6_hdr) > MHLEN) panic("assumption failed in icmp6_reflect"); #endif if (off > sizeof(struct ip6_hdr)) { size_t l; struct ip6_hdr nip6; l = off - sizeof(struct ip6_hdr); m_copydata(m, 0, sizeof(nip6), (caddr_t)&nip6); m_adj(m, l); l = sizeof(struct ip6_hdr) + sizeof(struct icmp6_hdr); if (m->m_len < l) { if ((m = m_pullup(m, l)) == NULL) return; } bcopy((caddr_t)&nip6, mtod(m, caddr_t), sizeof(nip6)); } else /* off == sizeof(struct ip6_hdr) */ { size_t l; l = sizeof(struct ip6_hdr) + sizeof(struct icmp6_hdr); if (m->m_len < l) { if ((m = m_pullup(m, l)) == NULL) return; } } plen = m->m_pkthdr.len - sizeof(struct ip6_hdr); ip6 = mtod(m, struct ip6_hdr *); ip6->ip6_nxt = IPPROTO_ICMPV6; icmp6 = (struct icmp6_hdr *)(ip6 + 1); type = icmp6->icmp6_type; /* keep type for statistics */ code = icmp6->icmp6_code; /* ditto. */ /* * If the incoming packet was addressed directly to us (i.e. unicast), * use dst as the src for the reply. * The IN6_IFF_NOTREADY case should be VERY rare, but is possible * (for example) when we encounter an error while forwarding procedure * destined to a duplicated address of ours. */ if (!IN6_IS_ADDR_MULTICAST(&ip6->ip6_dst)) { ia = in6ifa_ifwithaddr(&ip6->ip6_dst, 0 /* XXX */); if (ia != NULL && !(ia->ia6_flags & (IN6_IFF_ANYCAST|IN6_IFF_NOTREADY))) srcp = &ia->ia_addr.sin6_addr; } if (srcp == NULL) { int e; struct sockaddr_in6 sin6; struct route_in6 ro; /* * This case matches to multicasts, our anycast, or unicasts * that we do not own. Select a source address based on the * source address of the erroneous packet. */ bzero(&sin6, sizeof(sin6)); sin6.sin6_family = AF_INET6; sin6.sin6_len = sizeof(sin6); sin6.sin6_addr = ip6->ip6_dst; /* zone ID should be embedded */ bzero(&ro, sizeof(ro)); e = in6_selectsrc(&sin6, NULL, NULL, &ro, NULL, &outif, &src); if (ro.ro_rt) RTFREE(ro.ro_rt); /* XXX: we could use this */ if (e) { char ip6buf[INET6_ADDRSTRLEN]; nd6log((LOG_DEBUG, "icmp6_reflect: source can't be determined: " "dst=%s, error=%d\n", ip6_sprintf(ip6buf, &sin6.sin6_addr), e)); goto bad; } srcp = &src; } /* * ip6_input() drops a packet if its src is multicast. * So, the src is never multicast. */ ip6->ip6_dst = ip6->ip6_src; ip6->ip6_src = *srcp; ip6->ip6_flow = 0; ip6->ip6_vfc &= ~IPV6_VERSION_MASK; ip6->ip6_vfc |= IPV6_VERSION; ip6->ip6_nxt = IPPROTO_ICMPV6; if (outif) ip6->ip6_hlim = ND_IFINFO(outif)->chlim; else if (m->m_pkthdr.rcvif) { /* XXX: This may not be the outgoing interface */ ip6->ip6_hlim = ND_IFINFO(m->m_pkthdr.rcvif)->chlim; } else ip6->ip6_hlim = V_ip6_defhlim; icmp6->icmp6_cksum = 0; icmp6->icmp6_cksum = in6_cksum(m, IPPROTO_ICMPV6, sizeof(struct ip6_hdr), plen); /* * XXX option handling */ m->m_flags &= ~(M_BCAST|M_MCAST); ip6_output(m, NULL, NULL, 0, NULL, &outif, NULL); if (outif) icmp6_ifoutstat_inc(outif, type, code); if (ia != NULL) ifa_free(&ia->ia_ifa); return; bad: if (ia != NULL) ifa_free(&ia->ia_ifa); m_freem(m); return; } void icmp6_fasttimo(void) { mld_fasttimo(); } void icmp6_slowtimo(void) { mld_slowtimo(); } static const char * icmp6_redirect_diag(struct in6_addr *src6, struct in6_addr *dst6, struct in6_addr *tgt6) { static char buf[1024]; char ip6bufs[INET6_ADDRSTRLEN]; char ip6bufd[INET6_ADDRSTRLEN]; char ip6buft[INET6_ADDRSTRLEN]; snprintf(buf, sizeof(buf), "(src=%s dst=%s tgt=%s)", ip6_sprintf(ip6bufs, src6), ip6_sprintf(ip6bufd, dst6), ip6_sprintf(ip6buft, tgt6)); return buf; } void icmp6_redirect_input(struct mbuf *m, int off) { struct ifnet *ifp; struct ip6_hdr *ip6 = mtod(m, struct ip6_hdr *); struct nd_redirect *nd_rd; int icmp6len = ntohs(ip6->ip6_plen); char *lladdr = NULL; int lladdrlen = 0; - struct rtentry *rt = NULL; int is_router; int is_onlink; struct in6_addr src6 = ip6->ip6_src; struct in6_addr redtgt6; struct in6_addr reddst6; union nd_opts ndopts; char ip6buf[INET6_ADDRSTRLEN]; M_ASSERTPKTHDR(m); KASSERT(m->m_pkthdr.rcvif != NULL, ("%s: no rcvif", __func__)); ifp = m->m_pkthdr.rcvif; /* XXX if we are router, we don't update route by icmp6 redirect */ if (V_ip6_forwarding) goto freeit; if (!V_icmp6_rediraccept) goto freeit; #ifndef PULLDOWN_TEST IP6_EXTHDR_CHECK(m, off, icmp6len,); nd_rd = (struct nd_redirect *)((caddr_t)ip6 + off); #else IP6_EXTHDR_GET(nd_rd, struct nd_redirect *, m, off, icmp6len); if (nd_rd == NULL) { ICMP6STAT_INC(icp6s_tooshort); return; } #endif redtgt6 = nd_rd->nd_rd_target; reddst6 = nd_rd->nd_rd_dst; if (in6_setscope(&redtgt6, m->m_pkthdr.rcvif, NULL) || in6_setscope(&reddst6, m->m_pkthdr.rcvif, NULL)) { goto freeit; } /* validation */ if (!IN6_IS_ADDR_LINKLOCAL(&src6)) { nd6log((LOG_ERR, "ICMP6 redirect sent from %s rejected; " "must be from linklocal\n", ip6_sprintf(ip6buf, &src6))); goto bad; } if (ip6->ip6_hlim != 255) { nd6log((LOG_ERR, "ICMP6 redirect sent from %s rejected; " "hlim=%d (must be 255)\n", ip6_sprintf(ip6buf, &src6), ip6->ip6_hlim)); goto bad; } { /* ip6->ip6_src must be equal to gw for icmp6->icmp6_reddst */ - struct sockaddr_in6 sin6; - struct in6_addr *gw6; + struct nhop6_basic nh6; + struct in6_addr kdst; + uint32_t scopeid; - bzero(&sin6, sizeof(sin6)); - sin6.sin6_family = AF_INET6; - sin6.sin6_len = sizeof(struct sockaddr_in6); - bcopy(&reddst6, &sin6.sin6_addr, sizeof(reddst6)); - rt = in6_rtalloc1((struct sockaddr *)&sin6, 0, 0UL, RT_DEFAULT_FIB); - if (rt) { - if (rt->rt_gateway == NULL || - rt->rt_gateway->sa_family != AF_INET6) { - RTFREE_LOCKED(rt); + in6_splitscope(&reddst6, &kdst, &scopeid); + if (fib6_lookup_nh_basic(RT_DEFAULT_FIB, &kdst, scopeid, 0, 0,&nh6)==0){ + if ((nh6.nh_flags & NHF_GATEWAY) == 0) { nd6log((LOG_ERR, "ICMP6 redirect rejected; no route " "with inet6 gateway found for redirect dst: %s\n", icmp6_redirect_diag(&src6, &reddst6, &redtgt6))); goto bad; } - gw6 = &(((struct sockaddr_in6 *)rt->rt_gateway)->sin6_addr); - if (bcmp(&src6, gw6, sizeof(struct in6_addr)) != 0) { - RTFREE_LOCKED(rt); + if (IN6_ARE_ADDR_EQUAL(&src6, &nh6.nh_addr) == 0) { nd6log((LOG_ERR, "ICMP6 redirect rejected; " "not equal to gw-for-src=%s (must be same): " "%s\n", - ip6_sprintf(ip6buf, gw6), + ip6_sprintf(ip6buf, &nh6.nh_addr), icmp6_redirect_diag(&src6, &reddst6, &redtgt6))); goto bad; } } else { nd6log((LOG_ERR, "ICMP6 redirect rejected; " "no route found for redirect dst: %s\n", icmp6_redirect_diag(&src6, &reddst6, &redtgt6))); goto bad; } - RTFREE_LOCKED(rt); - rt = NULL; } if (IN6_IS_ADDR_MULTICAST(&reddst6)) { nd6log((LOG_ERR, "ICMP6 redirect rejected; " "redirect dst must be unicast: %s\n", icmp6_redirect_diag(&src6, &reddst6, &redtgt6))); goto bad; } is_router = is_onlink = 0; if (IN6_IS_ADDR_LINKLOCAL(&redtgt6)) is_router = 1; /* router case */ if (bcmp(&redtgt6, &reddst6, sizeof(redtgt6)) == 0) is_onlink = 1; /* on-link destination case */ if (!is_router && !is_onlink) { nd6log((LOG_ERR, "ICMP6 redirect rejected; " "neither router case nor onlink case: %s\n", icmp6_redirect_diag(&src6, &reddst6, &redtgt6))); goto bad; } icmp6len -= sizeof(*nd_rd); nd6_option_init(nd_rd + 1, icmp6len, &ndopts); if (nd6_options(&ndopts) < 0) { nd6log((LOG_INFO, "%s: invalid ND option, rejected: %s\n", __func__, icmp6_redirect_diag(&src6, &reddst6, &redtgt6))); /* nd6_options have incremented stats */ goto freeit; } if (ndopts.nd_opts_tgt_lladdr) { lladdr = (char *)(ndopts.nd_opts_tgt_lladdr + 1); lladdrlen = ndopts.nd_opts_tgt_lladdr->nd_opt_len << 3; } if (lladdr && ((ifp->if_addrlen + 2 + 7) & ~7) != lladdrlen) { nd6log((LOG_INFO, "%s: lladdrlen mismatch for %s " "(if %d, icmp6 packet %d): %s\n", __func__, ip6_sprintf(ip6buf, &redtgt6), ifp->if_addrlen, lladdrlen - 2, icmp6_redirect_diag(&src6, &reddst6, &redtgt6))); goto bad; } /* Validation passed. */ /* RFC 2461 8.3 */ nd6_cache_lladdr(ifp, &redtgt6, lladdr, lladdrlen, ND_REDIRECT, is_onlink ? ND_REDIRECT_ONLINK : ND_REDIRECT_ROUTER); /* * Install a gateway route in the better-router case or an interface * route in the on-link-destination case. */ { struct sockaddr_in6 sdst; struct sockaddr_in6 sgw; struct sockaddr_in6 ssrc; struct sockaddr *gw; int rt_flags; u_int fibnum; bzero(&sdst, sizeof(sdst)); bzero(&ssrc, sizeof(ssrc)); sdst.sin6_family = ssrc.sin6_family = AF_INET6; sdst.sin6_len = ssrc.sin6_len = sizeof(struct sockaddr_in6); bcopy(&reddst6, &sdst.sin6_addr, sizeof(struct in6_addr)); bcopy(&src6, &ssrc.sin6_addr, sizeof(struct in6_addr)); rt_flags = RTF_HOST; if (is_router) { bzero(&sgw, sizeof(sgw)); sgw.sin6_family = AF_INET6; sgw.sin6_len = sizeof(struct sockaddr_in6); bcopy(&redtgt6, &sgw.sin6_addr, sizeof(struct in6_addr)); gw = (struct sockaddr *)&sgw; rt_flags |= RTF_GATEWAY; } else gw = ifp->if_addr->ifa_addr; for (fibnum = 0; fibnum < rt_numfibs; fibnum++) in6_rtredirect((struct sockaddr *)&sdst, gw, (struct sockaddr *)NULL, rt_flags, (struct sockaddr *)&ssrc, fibnum); } /* finally update cached route in each socket via pfctlinput */ { struct sockaddr_in6 sdst; bzero(&sdst, sizeof(sdst)); sdst.sin6_family = AF_INET6; sdst.sin6_len = sizeof(struct sockaddr_in6); bcopy(&reddst6, &sdst.sin6_addr, sizeof(struct in6_addr)); pfctlinput(PRC_REDIRECT_HOST, (struct sockaddr *)&sdst); } freeit: m_freem(m); return; bad: ICMP6STAT_INC(icp6s_badredirect); m_freem(m); } void icmp6_redirect_output(struct mbuf *m0, struct rtentry *rt) { struct ifnet *ifp; /* my outgoing interface */ struct in6_addr *ifp_ll6; struct in6_addr *router_ll6; struct ip6_hdr *sip6; /* m0 as struct ip6_hdr */ struct mbuf *m = NULL; /* newly allocated one */ struct m_tag *mtag; struct ip6_hdr *ip6; /* m as struct ip6_hdr */ struct nd_redirect *nd_rd; struct llentry *ln = NULL; size_t maxlen; u_char *p; struct ifnet *outif = NULL; struct sockaddr_in6 src_sa; icmp6_errcount(ND_REDIRECT, 0); /* if we are not router, we don't send icmp6 redirect */ if (!V_ip6_forwarding) goto fail; /* sanity check */ if (!m0 || !rt || !(rt->rt_flags & RTF_UP) || !(ifp = rt->rt_ifp)) goto fail; /* * Address check: * the source address must identify a neighbor, and * the destination address must not be a multicast address * [RFC 2461, sec 8.2] */ sip6 = mtod(m0, struct ip6_hdr *); bzero(&src_sa, sizeof(src_sa)); src_sa.sin6_family = AF_INET6; src_sa.sin6_len = sizeof(src_sa); src_sa.sin6_addr = sip6->ip6_src; if (nd6_is_addr_neighbor(&src_sa, ifp) == 0) goto fail; if (IN6_IS_ADDR_MULTICAST(&sip6->ip6_dst)) goto fail; /* what should we do here? */ /* rate limit */ if (icmp6_ratelimit(&sip6->ip6_src, ND_REDIRECT, 0)) goto fail; /* * Since we are going to append up to 1280 bytes (= IPV6_MMTU), * we almost always ask for an mbuf cluster for simplicity. * (MHLEN < IPV6_MMTU is almost always true) */ #if IPV6_MMTU >= MCLBYTES # error assumption failed about IPV6_MMTU and MCLBYTES #endif m = m_getcl(M_NOWAIT, MT_DATA, M_PKTHDR); if (m == NULL) goto fail; M_SETFIB(m, rt->rt_fibnum); maxlen = M_TRAILINGSPACE(m); maxlen = min(IPV6_MMTU, maxlen); /* just for safety */ if (maxlen < sizeof(struct ip6_hdr) + sizeof(struct icmp6_hdr) + ((sizeof(struct nd_opt_hdr) + ifp->if_addrlen + 7) & ~7)) { goto fail; } { /* get ip6 linklocal address for ifp(my outgoing interface). */ struct in6_ifaddr *ia; if ((ia = in6ifa_ifpforlinklocal(ifp, IN6_IFF_NOTREADY| IN6_IFF_ANYCAST)) == NULL) goto fail; ifp_ll6 = &ia->ia_addr.sin6_addr; /* XXXRW: reference released prematurely. */ ifa_free(&ia->ia_ifa); } /* get ip6 linklocal address for the router. */ if (rt->rt_gateway && (rt->rt_flags & RTF_GATEWAY)) { struct sockaddr_in6 *sin6; sin6 = (struct sockaddr_in6 *)rt->rt_gateway; router_ll6 = &sin6->sin6_addr; if (!IN6_IS_ADDR_LINKLOCAL(router_ll6)) router_ll6 = (struct in6_addr *)NULL; } else router_ll6 = (struct in6_addr *)NULL; /* ip6 */ ip6 = mtod(m, struct ip6_hdr *); ip6->ip6_flow = 0; ip6->ip6_vfc &= ~IPV6_VERSION_MASK; ip6->ip6_vfc |= IPV6_VERSION; /* ip6->ip6_plen will be set later */ ip6->ip6_nxt = IPPROTO_ICMPV6; ip6->ip6_hlim = 255; /* ip6->ip6_src must be linklocal addr for my outgoing if. */ bcopy(ifp_ll6, &ip6->ip6_src, sizeof(struct in6_addr)); bcopy(&sip6->ip6_src, &ip6->ip6_dst, sizeof(struct in6_addr)); /* ND Redirect */ nd_rd = (struct nd_redirect *)(ip6 + 1); nd_rd->nd_rd_type = ND_REDIRECT; nd_rd->nd_rd_code = 0; nd_rd->nd_rd_reserved = 0; if (rt->rt_flags & RTF_GATEWAY) { /* * nd_rd->nd_rd_target must be a link-local address in * better router cases. */ if (!router_ll6) goto fail; bcopy(router_ll6, &nd_rd->nd_rd_target, sizeof(nd_rd->nd_rd_target)); bcopy(&sip6->ip6_dst, &nd_rd->nd_rd_dst, sizeof(nd_rd->nd_rd_dst)); } else { /* make sure redtgt == reddst */ bcopy(&sip6->ip6_dst, &nd_rd->nd_rd_target, sizeof(nd_rd->nd_rd_target)); bcopy(&sip6->ip6_dst, &nd_rd->nd_rd_dst, sizeof(nd_rd->nd_rd_dst)); } p = (u_char *)(nd_rd + 1); if (!router_ll6) goto nolladdropt; { /* target lladdr option */ int len; struct nd_opt_hdr *nd_opt; char *lladdr; IF_AFDATA_RLOCK(ifp); ln = nd6_lookup(router_ll6, 0, ifp); IF_AFDATA_RUNLOCK(ifp); if (ln == NULL) goto nolladdropt; len = sizeof(*nd_opt) + ifp->if_addrlen; len = (len + 7) & ~7; /* round by 8 */ /* safety check */ if (len + (p - (u_char *)ip6) > maxlen) goto nolladdropt; if (ln->la_flags & LLE_VALID) { nd_opt = (struct nd_opt_hdr *)p; nd_opt->nd_opt_type = ND_OPT_TARGET_LINKADDR; nd_opt->nd_opt_len = len >> 3; lladdr = (char *)(nd_opt + 1); bcopy(&ln->ll_addr, lladdr, ifp->if_addrlen); p += len; } } nolladdropt: if (ln != NULL) LLE_RUNLOCK(ln); m->m_pkthdr.len = m->m_len = p - (u_char *)ip6; /* just to be safe */ #ifdef M_DECRYPTED /*not openbsd*/ if (m0->m_flags & M_DECRYPTED) goto noredhdropt; #endif if (p - (u_char *)ip6 > maxlen) goto noredhdropt; { /* redirected header option */ int len; struct nd_opt_rd_hdr *nd_opt_rh; /* * compute the maximum size for icmp6 redirect header option. * XXX room for auth header? */ len = maxlen - (p - (u_char *)ip6); len &= ~7; /* This is just for simplicity. */ if (m0->m_pkthdr.len != m0->m_len) { if (m0->m_next) { m_freem(m0->m_next); m0->m_next = NULL; } m0->m_pkthdr.len = m0->m_len; } /* * Redirected header option spec (RFC2461 4.6.3) talks nothing * about padding/truncate rule for the original IP packet. * From the discussion on IPv6imp in Feb 1999, * the consensus was: * - "attach as much as possible" is the goal * - pad if not aligned (original size can be guessed by * original ip6 header) * Following code adds the padding if it is simple enough, * and truncates if not. */ if (m0->m_next || m0->m_pkthdr.len != m0->m_len) panic("assumption failed in %s:%d", __FILE__, __LINE__); if (len - sizeof(*nd_opt_rh) < m0->m_pkthdr.len) { /* not enough room, truncate */ m0->m_pkthdr.len = m0->m_len = len - sizeof(*nd_opt_rh); } else { /* enough room, pad or truncate */ size_t extra; extra = m0->m_pkthdr.len % 8; if (extra) { /* pad if easy enough, truncate if not */ if (8 - extra <= M_TRAILINGSPACE(m0)) { /* pad */ m0->m_len += (8 - extra); m0->m_pkthdr.len += (8 - extra); } else { /* truncate */ m0->m_pkthdr.len -= extra; m0->m_len -= extra; } } len = m0->m_pkthdr.len + sizeof(*nd_opt_rh); m0->m_pkthdr.len = m0->m_len = len - sizeof(*nd_opt_rh); } nd_opt_rh = (struct nd_opt_rd_hdr *)p; bzero(nd_opt_rh, sizeof(*nd_opt_rh)); nd_opt_rh->nd_opt_rh_type = ND_OPT_REDIRECTED_HEADER; nd_opt_rh->nd_opt_rh_len = len >> 3; p += sizeof(*nd_opt_rh); m->m_pkthdr.len = m->m_len = p - (u_char *)ip6; /* connect m0 to m */ m_tag_delete_chain(m0, NULL); m0->m_flags &= ~M_PKTHDR; m->m_next = m0; m->m_pkthdr.len = m->m_len + m0->m_len; m0 = NULL; } noredhdropt:; if (m0) { m_freem(m0); m0 = NULL; } /* XXX: clear embedded link IDs in the inner header */ in6_clearscope(&sip6->ip6_src); in6_clearscope(&sip6->ip6_dst); in6_clearscope(&nd_rd->nd_rd_target); in6_clearscope(&nd_rd->nd_rd_dst); ip6->ip6_plen = htons(m->m_pkthdr.len - sizeof(struct ip6_hdr)); nd_rd->nd_rd_cksum = 0; nd_rd->nd_rd_cksum = in6_cksum(m, IPPROTO_ICMPV6, sizeof(*ip6), ntohs(ip6->ip6_plen)); if (send_sendso_input_hook != NULL) { mtag = m_tag_get(PACKET_TAG_ND_OUTGOING, sizeof(unsigned short), M_NOWAIT); if (mtag == NULL) goto fail; *(unsigned short *)(mtag + 1) = nd_rd->nd_rd_type; m_tag_prepend(m, mtag); } /* send the packet to outside... */ ip6_output(m, NULL, NULL, 0, NULL, &outif, NULL); if (outif) { icmp6_ifstat_inc(outif, ifs6_out_msg); icmp6_ifstat_inc(outif, ifs6_out_redirect); } ICMP6STAT_INC(icp6s_outhist[ND_REDIRECT]); return; fail: if (m) m_freem(m); if (m0) m_freem(m0); } /* * ICMPv6 socket option processing. */ int icmp6_ctloutput(struct socket *so, struct sockopt *sopt) { int error = 0; int optlen; struct inpcb *inp = sotoinpcb(so); int level, op, optname; if (sopt) { level = sopt->sopt_level; op = sopt->sopt_dir; optname = sopt->sopt_name; optlen = sopt->sopt_valsize; } else level = op = optname = optlen = 0; if (level != IPPROTO_ICMPV6) { return EINVAL; } switch (op) { case PRCO_SETOPT: switch (optname) { case ICMP6_FILTER: { struct icmp6_filter ic6f; if (optlen != sizeof(ic6f)) { error = EMSGSIZE; break; } error = sooptcopyin(sopt, &ic6f, optlen, optlen); if (error == 0) { INP_WLOCK(inp); *inp->in6p_icmp6filt = ic6f; INP_WUNLOCK(inp); } break; } default: error = ENOPROTOOPT; break; } break; case PRCO_GETOPT: switch (optname) { case ICMP6_FILTER: { struct icmp6_filter ic6f; INP_RLOCK(inp); ic6f = *inp->in6p_icmp6filt; INP_RUNLOCK(inp); error = sooptcopyout(sopt, &ic6f, sizeof(ic6f)); break; } default: error = ENOPROTOOPT; break; } break; } return (error); } /* * Perform rate limit check. * Returns 0 if it is okay to send the icmp6 packet. * Returns 1 if the router SHOULD NOT send this icmp6 packet due to rate * limitation. * * XXX per-destination/type check necessary? * * dst - not used at this moment * type - not used at this moment * code - not used at this moment */ static int icmp6_ratelimit(const struct in6_addr *dst, const int type, const int code) { int ret; ret = 0; /* okay to send */ /* PPS limit */ if (!ppsratecheck(&V_icmp6errppslim_last, &V_icmp6errpps_count, V_icmp6errppslim)) { /* The packet is subject to rate limit */ ret++; } return ret; } Index: user/ngie/more-tests2/sys/netinet6/in6.c =================================================================== --- user/ngie/more-tests2/sys/netinet6/in6.c (revision 292053) +++ user/ngie/more-tests2/sys/netinet6/in6.c (revision 292054) @@ -1,2504 +1,2505 @@ /*- * Copyright (C) 1995, 1996, 1997, and 1998 WIDE Project. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. Neither the name of the project nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE PROJECT AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE PROJECT OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $KAME: in6.c,v 1.259 2002/01/21 11:37:50 keiichi Exp $ */ /*- * Copyright (c) 1982, 1986, 1991, 1993 * The Regents of the University of California. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * @(#)in.c 8.2 (Berkeley) 11/15/93 */ #include __FBSDID("$FreeBSD$"); #include "opt_compat.h" #include "opt_inet.h" #include "opt_inet6.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include +#include #include VNET_DECLARE(int, icmp6_nodeinfo_oldmcprefix); #define V_icmp6_nodeinfo_oldmcprefix VNET(icmp6_nodeinfo_oldmcprefix) /* * Definitions of some costant IP6 addresses. */ const struct in6_addr in6addr_any = IN6ADDR_ANY_INIT; const struct in6_addr in6addr_loopback = IN6ADDR_LOOPBACK_INIT; const struct in6_addr in6addr_nodelocal_allnodes = IN6ADDR_NODELOCAL_ALLNODES_INIT; const struct in6_addr in6addr_linklocal_allnodes = IN6ADDR_LINKLOCAL_ALLNODES_INIT; const struct in6_addr in6addr_linklocal_allrouters = IN6ADDR_LINKLOCAL_ALLROUTERS_INIT; const struct in6_addr in6addr_linklocal_allv2routers = IN6ADDR_LINKLOCAL_ALLV2ROUTERS_INIT; const struct in6_addr in6mask0 = IN6MASK0; const struct in6_addr in6mask32 = IN6MASK32; const struct in6_addr in6mask64 = IN6MASK64; const struct in6_addr in6mask96 = IN6MASK96; const struct in6_addr in6mask128 = IN6MASK128; const struct sockaddr_in6 sa6_any = { sizeof(sa6_any), AF_INET6, 0, 0, IN6ADDR_ANY_INIT, 0 }; static int in6_notify_ifa(struct ifnet *, struct in6_ifaddr *, struct in6_aliasreq *, int); static void in6_unlink_ifa(struct in6_ifaddr *, struct ifnet *); static int in6_validate_ifra(struct ifnet *, struct in6_aliasreq *, struct in6_ifaddr *, int); static struct in6_ifaddr *in6_alloc_ifa(struct ifnet *, struct in6_aliasreq *, int flags); static int in6_update_ifa_internal(struct ifnet *, struct in6_aliasreq *, struct in6_ifaddr *, int, int); static int in6_broadcast_ifa(struct ifnet *, struct in6_aliasreq *, struct in6_ifaddr *, int); #define ifa2ia6(ifa) ((struct in6_ifaddr *)(ifa)) #define ia62ifa(ia6) (&((ia6)->ia_ifa)) void in6_newaddrmsg(struct in6_ifaddr *ia, int cmd) { struct sockaddr_dl gateway; struct sockaddr_in6 mask, addr; struct rtentry rt; /* * initialize for rtmsg generation */ bzero(&gateway, sizeof(gateway)); gateway.sdl_len = sizeof(gateway); gateway.sdl_family = AF_LINK; bzero(&rt, sizeof(rt)); rt.rt_gateway = (struct sockaddr *)&gateway; memcpy(&mask, &ia->ia_prefixmask, sizeof(ia->ia_prefixmask)); memcpy(&addr, &ia->ia_addr, sizeof(ia->ia_addr)); rt_mask(&rt) = (struct sockaddr *)&mask; rt_key(&rt) = (struct sockaddr *)&addr; rt.rt_flags = RTF_HOST | RTF_STATIC; if (cmd == RTM_ADD) rt.rt_flags |= RTF_UP; /* Announce arrival of local address to all FIBs. */ rt_newaddrmsg(cmd, &ia->ia_ifa, 0, &rt); } int in6_mask2len(struct in6_addr *mask, u_char *lim0) { int x = 0, y; u_char *lim = lim0, *p; /* ignore the scope_id part */ if (lim0 == NULL || lim0 - (u_char *)mask > sizeof(*mask)) lim = (u_char *)mask + sizeof(*mask); for (p = (u_char *)mask; p < lim; x++, p++) { if (*p != 0xff) break; } y = 0; if (p < lim) { for (y = 0; y < 8; y++) { if ((*p & (0x80 >> y)) == 0) break; } } /* * when the limit pointer is given, do a stricter check on the * remaining bits. */ if (p < lim) { if (y != 0 && (*p & (0x00ff >> y)) != 0) return (-1); for (p = p + 1; p < lim; p++) if (*p != 0) return (-1); } return x * 8 + y; } #ifdef COMPAT_FREEBSD32 struct in6_ndifreq32 { char ifname[IFNAMSIZ]; uint32_t ifindex; }; #define SIOCGDEFIFACE32_IN6 _IOWR('i', 86, struct in6_ndifreq32) #endif int in6_control(struct socket *so, u_long cmd, caddr_t data, struct ifnet *ifp, struct thread *td) { struct in6_ifreq *ifr = (struct in6_ifreq *)data; struct in6_ifaddr *ia = NULL; struct in6_aliasreq *ifra = (struct in6_aliasreq *)data; struct sockaddr_in6 *sa6; int carp_attached = 0; int error; u_long ocmd = cmd; /* * Compat to make pre-10.x ifconfig(8) operable. */ if (cmd == OSIOCAIFADDR_IN6) cmd = SIOCAIFADDR_IN6; switch (cmd) { case SIOCGETSGCNT_IN6: case SIOCGETMIFCNT_IN6: /* * XXX mrt_ioctl has a 3rd, unused, FIB argument in route.c. * We cannot see how that would be needed, so do not adjust the * KPI blindly; more likely should clean up the IPv4 variant. */ return (mrt6_ioctl ? mrt6_ioctl(cmd, data) : EOPNOTSUPP); } switch (cmd) { case SIOCAADDRCTL_POLICY: case SIOCDADDRCTL_POLICY: if (td != NULL) { error = priv_check(td, PRIV_NETINET_ADDRCTRL6); if (error) return (error); } return (in6_src_ioctl(cmd, data)); } if (ifp == NULL) return (EOPNOTSUPP); switch (cmd) { case SIOCSNDFLUSH_IN6: case SIOCSPFXFLUSH_IN6: case SIOCSRTRFLUSH_IN6: case SIOCSDEFIFACE_IN6: case SIOCSIFINFO_FLAGS: case SIOCSIFINFO_IN6: if (td != NULL) { error = priv_check(td, PRIV_NETINET_ND6); if (error) return (error); } /* FALLTHROUGH */ case OSIOCGIFINFO_IN6: case SIOCGIFINFO_IN6: case SIOCGNBRINFO_IN6: case SIOCGDEFIFACE_IN6: return (nd6_ioctl(cmd, data, ifp)); #ifdef COMPAT_FREEBSD32 case SIOCGDEFIFACE32_IN6: { struct in6_ndifreq ndif; struct in6_ndifreq32 *ndif32; error = nd6_ioctl(SIOCGDEFIFACE_IN6, (caddr_t)&ndif, ifp); if (error) return (error); ndif32 = (struct in6_ndifreq32 *)data; ndif32->ifindex = ndif.ifindex; return (0); } #endif } switch (cmd) { case SIOCSIFPREFIX_IN6: case SIOCDIFPREFIX_IN6: case SIOCAIFPREFIX_IN6: case SIOCCIFPREFIX_IN6: case SIOCSGIFPREFIX_IN6: case SIOCGIFPREFIX_IN6: log(LOG_NOTICE, "prefix ioctls are now invalidated. " "please use ifconfig.\n"); return (EOPNOTSUPP); } switch (cmd) { case SIOCSSCOPE6: if (td != NULL) { error = priv_check(td, PRIV_NETINET_SCOPE6); if (error) return (error); } /* FALLTHROUGH */ case SIOCGSCOPE6: case SIOCGSCOPE6DEF: return (scope6_ioctl(cmd, data, ifp)); } /* * Find address for this interface, if it exists. * * In netinet code, we have checked ifra_addr in SIOCSIF*ADDR operation * only, and used the first interface address as the target of other * operations (without checking ifra_addr). This was because netinet * code/API assumed at most 1 interface address per interface. * Since IPv6 allows a node to assign multiple addresses * on a single interface, we almost always look and check the * presence of ifra_addr, and reject invalid ones here. * It also decreases duplicated code among SIOC*_IN6 operations. */ switch (cmd) { case SIOCAIFADDR_IN6: case SIOCSIFPHYADDR_IN6: sa6 = &ifra->ifra_addr; break; case SIOCSIFADDR_IN6: case SIOCGIFADDR_IN6: case SIOCSIFDSTADDR_IN6: case SIOCSIFNETMASK_IN6: case SIOCGIFDSTADDR_IN6: case SIOCGIFNETMASK_IN6: case SIOCDIFADDR_IN6: case SIOCGIFPSRCADDR_IN6: case SIOCGIFPDSTADDR_IN6: case SIOCGIFAFLAG_IN6: case SIOCSNDFLUSH_IN6: case SIOCSPFXFLUSH_IN6: case SIOCSRTRFLUSH_IN6: case SIOCGIFALIFETIME_IN6: case SIOCSIFALIFETIME_IN6: case SIOCGIFSTAT_IN6: case SIOCGIFSTAT_ICMP6: sa6 = &ifr->ifr_addr; break; case SIOCSIFADDR: case SIOCSIFBRDADDR: case SIOCSIFDSTADDR: case SIOCSIFNETMASK: /* * Although we should pass any non-INET6 ioctl requests * down to driver, we filter some legacy INET requests. * Drivers trust SIOCSIFADDR et al to come from an already * privileged layer, and do not perform any credentials * checks or input validation. */ return (EINVAL); default: sa6 = NULL; break; } if (sa6 && sa6->sin6_family == AF_INET6) { if (sa6->sin6_scope_id != 0) error = sa6_embedscope(sa6, 0); else error = in6_setscope(&sa6->sin6_addr, ifp, NULL); if (error != 0) return (error); if (td != NULL && (error = prison_check_ip6(td->td_ucred, &sa6->sin6_addr)) != 0) return (error); ia = in6ifa_ifpwithaddr(ifp, &sa6->sin6_addr); } else ia = NULL; switch (cmd) { case SIOCSIFADDR_IN6: case SIOCSIFDSTADDR_IN6: case SIOCSIFNETMASK_IN6: /* * Since IPv6 allows a node to assign multiple addresses * on a single interface, SIOCSIFxxx ioctls are deprecated. */ /* we decided to obsolete this command (20000704) */ error = EINVAL; goto out; case SIOCDIFADDR_IN6: /* * for IPv4, we look for existing in_ifaddr here to allow * "ifconfig if0 delete" to remove the first IPv4 address on * the interface. For IPv6, as the spec allows multiple * interface address from the day one, we consider "remove the * first one" semantics to be not preferable. */ if (ia == NULL) { error = EADDRNOTAVAIL; goto out; } /* FALLTHROUGH */ case SIOCAIFADDR_IN6: /* * We always require users to specify a valid IPv6 address for * the corresponding operation. */ if (ifra->ifra_addr.sin6_family != AF_INET6 || ifra->ifra_addr.sin6_len != sizeof(struct sockaddr_in6)) { error = EAFNOSUPPORT; goto out; } if (td != NULL) { error = priv_check(td, (cmd == SIOCDIFADDR_IN6) ? PRIV_NET_DELIFADDR : PRIV_NET_ADDIFADDR); if (error) goto out; } /* FALLTHROUGH */ case SIOCGIFSTAT_IN6: case SIOCGIFSTAT_ICMP6: if (ifp->if_afdata[AF_INET6] == NULL) { error = EPFNOSUPPORT; goto out; } break; case SIOCGIFADDR_IN6: /* This interface is basically deprecated. use SIOCGIFCONF. */ /* FALLTHROUGH */ case SIOCGIFAFLAG_IN6: case SIOCGIFNETMASK_IN6: case SIOCGIFDSTADDR_IN6: case SIOCGIFALIFETIME_IN6: /* must think again about its semantics */ if (ia == NULL) { error = EADDRNOTAVAIL; goto out; } break; case SIOCSIFALIFETIME_IN6: { struct in6_addrlifetime *lt; if (td != NULL) { error = priv_check(td, PRIV_NETINET_ALIFETIME6); if (error) goto out; } if (ia == NULL) { error = EADDRNOTAVAIL; goto out; } /* sanity for overflow - beware unsigned */ lt = &ifr->ifr_ifru.ifru_lifetime; if (lt->ia6t_vltime != ND6_INFINITE_LIFETIME && lt->ia6t_vltime + time_uptime < time_uptime) { error = EINVAL; goto out; } if (lt->ia6t_pltime != ND6_INFINITE_LIFETIME && lt->ia6t_pltime + time_uptime < time_uptime) { error = EINVAL; goto out; } break; } } switch (cmd) { case SIOCGIFADDR_IN6: ifr->ifr_addr = ia->ia_addr; if ((error = sa6_recoverscope(&ifr->ifr_addr)) != 0) goto out; break; case SIOCGIFDSTADDR_IN6: if ((ifp->if_flags & IFF_POINTOPOINT) == 0) { error = EINVAL; goto out; } /* * XXX: should we check if ifa_dstaddr is NULL and return * an error? */ ifr->ifr_dstaddr = ia->ia_dstaddr; if ((error = sa6_recoverscope(&ifr->ifr_dstaddr)) != 0) goto out; break; case SIOCGIFNETMASK_IN6: ifr->ifr_addr = ia->ia_prefixmask; break; case SIOCGIFAFLAG_IN6: ifr->ifr_ifru.ifru_flags6 = ia->ia6_flags; break; case SIOCGIFSTAT_IN6: COUNTER_ARRAY_COPY(((struct in6_ifextra *) ifp->if_afdata[AF_INET6])->in6_ifstat, &ifr->ifr_ifru.ifru_stat, sizeof(struct in6_ifstat) / sizeof(uint64_t)); break; case SIOCGIFSTAT_ICMP6: COUNTER_ARRAY_COPY(((struct in6_ifextra *) ifp->if_afdata[AF_INET6])->icmp6_ifstat, &ifr->ifr_ifru.ifru_icmp6stat, sizeof(struct icmp6_ifstat) / sizeof(uint64_t)); break; case SIOCGIFALIFETIME_IN6: ifr->ifr_ifru.ifru_lifetime = ia->ia6_lifetime; if (ia->ia6_lifetime.ia6t_vltime != ND6_INFINITE_LIFETIME) { time_t maxexpire; struct in6_addrlifetime *retlt = &ifr->ifr_ifru.ifru_lifetime; /* * XXX: adjust expiration time assuming time_t is * signed. */ maxexpire = (-1) & ~((time_t)1 << ((sizeof(maxexpire) * 8) - 1)); if (ia->ia6_lifetime.ia6t_vltime < maxexpire - ia->ia6_updatetime) { retlt->ia6t_expire = ia->ia6_updatetime + ia->ia6_lifetime.ia6t_vltime; } else retlt->ia6t_expire = maxexpire; } if (ia->ia6_lifetime.ia6t_pltime != ND6_INFINITE_LIFETIME) { time_t maxexpire; struct in6_addrlifetime *retlt = &ifr->ifr_ifru.ifru_lifetime; /* * XXX: adjust expiration time assuming time_t is * signed. */ maxexpire = (-1) & ~((time_t)1 << ((sizeof(maxexpire) * 8) - 1)); if (ia->ia6_lifetime.ia6t_pltime < maxexpire - ia->ia6_updatetime) { retlt->ia6t_preferred = ia->ia6_updatetime + ia->ia6_lifetime.ia6t_pltime; } else retlt->ia6t_preferred = maxexpire; } break; case SIOCSIFALIFETIME_IN6: ia->ia6_lifetime = ifr->ifr_ifru.ifru_lifetime; /* for sanity */ if (ia->ia6_lifetime.ia6t_vltime != ND6_INFINITE_LIFETIME) { ia->ia6_lifetime.ia6t_expire = time_uptime + ia->ia6_lifetime.ia6t_vltime; } else ia->ia6_lifetime.ia6t_expire = 0; if (ia->ia6_lifetime.ia6t_pltime != ND6_INFINITE_LIFETIME) { ia->ia6_lifetime.ia6t_preferred = time_uptime + ia->ia6_lifetime.ia6t_pltime; } else ia->ia6_lifetime.ia6t_preferred = 0; break; case SIOCAIFADDR_IN6: { struct nd_prefixctl pr0; struct nd_prefix *pr; /* * first, make or update the interface address structure, * and link it to the list. */ if ((error = in6_update_ifa(ifp, ifra, ia, 0)) != 0) goto out; if (ia != NULL) ifa_free(&ia->ia_ifa); if ((ia = in6ifa_ifpwithaddr(ifp, &ifra->ifra_addr.sin6_addr)) == NULL) { /* * this can happen when the user specify the 0 valid * lifetime. */ break; } if (cmd == ocmd && ifra->ifra_vhid > 0) { if (carp_attach_p != NULL) error = (*carp_attach_p)(&ia->ia_ifa, ifra->ifra_vhid); else error = EPROTONOSUPPORT; if (error) goto out; else carp_attached = 1; } /* * then, make the prefix on-link on the interface. * XXX: we'd rather create the prefix before the address, but * we need at least one address to install the corresponding * interface route, so we configure the address first. */ /* * convert mask to prefix length (prefixmask has already * been validated in in6_update_ifa(). */ bzero(&pr0, sizeof(pr0)); pr0.ndpr_ifp = ifp; pr0.ndpr_plen = in6_mask2len(&ifra->ifra_prefixmask.sin6_addr, NULL); if (pr0.ndpr_plen == 128) { /* we don't need to install a host route. */ goto aifaddr_out; } pr0.ndpr_prefix = ifra->ifra_addr; /* apply the mask for safety. */ IN6_MASK_ADDR(&pr0.ndpr_prefix.sin6_addr, &ifra->ifra_prefixmask.sin6_addr); /* * XXX: since we don't have an API to set prefix (not address) * lifetimes, we just use the same lifetimes as addresses. * The (temporarily) installed lifetimes can be overridden by * later advertised RAs (when accept_rtadv is non 0), which is * an intended behavior. */ pr0.ndpr_raf_onlink = 1; /* should be configurable? */ pr0.ndpr_raf_auto = ((ifra->ifra_flags & IN6_IFF_AUTOCONF) != 0); pr0.ndpr_vltime = ifra->ifra_lifetime.ia6t_vltime; pr0.ndpr_pltime = ifra->ifra_lifetime.ia6t_pltime; /* add the prefix if not yet. */ if ((pr = nd6_prefix_lookup(&pr0)) == NULL) { /* * nd6_prelist_add will install the corresponding * interface route. */ if ((error = nd6_prelist_add(&pr0, NULL, &pr)) != 0) { if (carp_attached) (*carp_detach_p)(&ia->ia_ifa); goto out; } if (pr == NULL) { if (carp_attached) (*carp_detach_p)(&ia->ia_ifa); log(LOG_ERR, "nd6_prelist_add succeeded but " "no prefix\n"); error = EINVAL; goto out; } } /* relate the address to the prefix */ if (ia->ia6_ndpr == NULL) { ia->ia6_ndpr = pr; pr->ndpr_refcnt++; /* * If this is the first autoconf address from the * prefix, create a temporary address as well * (when required). */ if ((ia->ia6_flags & IN6_IFF_AUTOCONF) && V_ip6_use_tempaddr && pr->ndpr_refcnt == 1) { int e; if ((e = in6_tmpifadd(ia, 1, 0)) != 0) { log(LOG_NOTICE, "in6_control: failed " "to create a temporary address, " "errno=%d\n", e); } } } /* * this might affect the status of autoconfigured addresses, * that is, this address might make other addresses detached. */ pfxlist_onlink_check(); aifaddr_out: if (error != 0 || ia == NULL) break; /* * Try to clear the flag when a new IPv6 address is added * onto an IFDISABLED interface and it succeeds. */ if (ND_IFINFO(ifp)->flags & ND6_IFF_IFDISABLED) { struct in6_ndireq nd; memset(&nd, 0, sizeof(nd)); nd.ndi.flags = ND_IFINFO(ifp)->flags; nd.ndi.flags &= ~ND6_IFF_IFDISABLED; if (nd6_ioctl(SIOCSIFINFO_FLAGS, (caddr_t)&nd, ifp) < 0) log(LOG_NOTICE, "SIOCAIFADDR_IN6: " "SIOCSIFINFO_FLAGS for -ifdisabled " "failed."); /* * Ignore failure of clearing the flag intentionally. * The failure means address duplication was detected. */ } EVENTHANDLER_INVOKE(ifaddr_event, ifp); break; } case SIOCDIFADDR_IN6: { struct nd_prefix *pr; /* * If the address being deleted is the only one that owns * the corresponding prefix, expire the prefix as well. * XXX: theoretically, we don't have to worry about such * relationship, since we separate the address management * and the prefix management. We do this, however, to provide * as much backward compatibility as possible in terms of * the ioctl operation. * Note that in6_purgeaddr() will decrement ndpr_refcnt. */ pr = ia->ia6_ndpr; in6_purgeaddr(&ia->ia_ifa); if (pr && pr->ndpr_refcnt == 0) prelist_remove(pr); EVENTHANDLER_INVOKE(ifaddr_event, ifp); break; } default: if (ifp->if_ioctl == NULL) { error = EOPNOTSUPP; goto out; } error = (*ifp->if_ioctl)(ifp, cmd, data); goto out; } error = 0; out: if (ia != NULL) ifa_free(&ia->ia_ifa); return (error); } /* * Join necessary multicast groups. Factored out from in6_update_ifa(). * This entire work should only be done once, for the default FIB. */ static int in6_update_ifa_join_mc(struct ifnet *ifp, struct in6_aliasreq *ifra, struct in6_ifaddr *ia, int flags, struct in6_multi **in6m_sol) { char ip6buf[INET6_ADDRSTRLEN]; struct in6_addr mltaddr; struct in6_multi_mship *imm; int delay, error; KASSERT(in6m_sol != NULL, ("%s: in6m_sol is NULL", __func__)); /* Join solicited multicast addr for new host id. */ bzero(&mltaddr, sizeof(struct in6_addr)); mltaddr.s6_addr32[0] = IPV6_ADDR_INT32_MLL; mltaddr.s6_addr32[2] = htonl(1); mltaddr.s6_addr32[3] = ifra->ifra_addr.sin6_addr.s6_addr32[3]; mltaddr.s6_addr8[12] = 0xff; if ((error = in6_setscope(&mltaddr, ifp, NULL)) != 0) { /* XXX: should not happen */ log(LOG_ERR, "%s: in6_setscope failed\n", __func__); goto cleanup; } delay = error = 0; if ((flags & IN6_IFAUPDATE_DADDELAY)) { /* * We need a random delay for DAD on the address being * configured. It also means delaying transmission of the * corresponding MLD report to avoid report collision. * [RFC 4861, Section 6.3.7] */ delay = arc4random() % (MAX_RTR_SOLICITATION_DELAY * hz); } imm = in6_joingroup(ifp, &mltaddr, &error, delay); if (imm == NULL) { nd6log((LOG_WARNING, "%s: in6_joingroup failed for %s on %s " "(errno=%d)\n", __func__, ip6_sprintf(ip6buf, &mltaddr), if_name(ifp), error)); goto cleanup; } LIST_INSERT_HEAD(&ia->ia6_memberships, imm, i6mm_chain); *in6m_sol = imm->i6mm_maddr; /* * Join link-local all-nodes address. */ mltaddr = in6addr_linklocal_allnodes; if ((error = in6_setscope(&mltaddr, ifp, NULL)) != 0) goto cleanup; /* XXX: should not fail */ imm = in6_joingroup(ifp, &mltaddr, &error, 0); if (imm == NULL) { nd6log((LOG_WARNING, "%s: in6_joingroup failed for %s on %s " "(errno=%d)\n", __func__, ip6_sprintf(ip6buf, &mltaddr), if_name(ifp), error)); goto cleanup; } LIST_INSERT_HEAD(&ia->ia6_memberships, imm, i6mm_chain); /* * Join node information group address. */ delay = 0; if ((flags & IN6_IFAUPDATE_DADDELAY)) { /* * The spec does not say anything about delay for this group, * but the same logic should apply. */ delay = arc4random() % (MAX_RTR_SOLICITATION_DELAY * hz); } if (in6_nigroup(ifp, NULL, -1, &mltaddr) == 0) { /* XXX jinmei */ imm = in6_joingroup(ifp, &mltaddr, &error, delay); if (imm == NULL) nd6log((LOG_WARNING, "%s: in6_joingroup failed for %s on %s " "(errno=%d)\n", __func__, ip6_sprintf(ip6buf, &mltaddr), if_name(ifp), error)); /* XXX not very fatal, go on... */ else LIST_INSERT_HEAD(&ia->ia6_memberships, imm, i6mm_chain); } if (V_icmp6_nodeinfo_oldmcprefix && in6_nigroup_oldmcprefix(ifp, NULL, -1, &mltaddr) == 0) { imm = in6_joingroup(ifp, &mltaddr, &error, delay); if (imm == NULL) nd6log((LOG_WARNING, "%s: in6_joingroup failed for %s on %s " "(errno=%d)\n", __func__, ip6_sprintf(ip6buf, &mltaddr), if_name(ifp), error)); /* XXX not very fatal, go on... */ else LIST_INSERT_HEAD(&ia->ia6_memberships, imm, i6mm_chain); } /* * Join interface-local all-nodes address. * (ff01::1%ifN, and ff01::%ifN/32) */ mltaddr = in6addr_nodelocal_allnodes; if ((error = in6_setscope(&mltaddr, ifp, NULL)) != 0) goto cleanup; /* XXX: should not fail */ imm = in6_joingroup(ifp, &mltaddr, &error, 0); if (imm == NULL) { nd6log((LOG_WARNING, "%s: in6_joingroup failed for %s on %s " "(errno=%d)\n", __func__, ip6_sprintf(ip6buf, &mltaddr), if_name(ifp), error)); goto cleanup; } LIST_INSERT_HEAD(&ia->ia6_memberships, imm, i6mm_chain); cleanup: return (error); } /* * Update parameters of an IPv6 interface address. * If necessary, a new entry is created and linked into address chains. * This function is separated from in6_control(). */ int in6_update_ifa(struct ifnet *ifp, struct in6_aliasreq *ifra, struct in6_ifaddr *ia, int flags) { int error, hostIsNew = 0; if ((error = in6_validate_ifra(ifp, ifra, ia, flags)) != 0) return (error); if (ia == NULL) { hostIsNew = 1; if ((ia = in6_alloc_ifa(ifp, ifra, flags)) == NULL) return (ENOBUFS); } error = in6_update_ifa_internal(ifp, ifra, ia, hostIsNew, flags); if (error != 0) { if (hostIsNew != 0) { in6_unlink_ifa(ia, ifp); ifa_free(&ia->ia_ifa); } return (error); } if (hostIsNew) error = in6_broadcast_ifa(ifp, ifra, ia, flags); return (error); } /* * Fill in basic IPv6 address request info. */ void in6_prepare_ifra(struct in6_aliasreq *ifra, const struct in6_addr *addr, const struct in6_addr *mask) { memset(ifra, 0, sizeof(struct in6_aliasreq)); ifra->ifra_addr.sin6_family = AF_INET6; ifra->ifra_addr.sin6_len = sizeof(struct sockaddr_in6); if (addr != NULL) ifra->ifra_addr.sin6_addr = *addr; ifra->ifra_prefixmask.sin6_family = AF_INET6; ifra->ifra_prefixmask.sin6_len = sizeof(struct sockaddr_in6); if (mask != NULL) ifra->ifra_prefixmask.sin6_addr = *mask; } static int in6_validate_ifra(struct ifnet *ifp, struct in6_aliasreq *ifra, struct in6_ifaddr *ia, int flags) { int plen = -1; struct sockaddr_in6 dst6; struct in6_addrlifetime *lt; char ip6buf[INET6_ADDRSTRLEN]; /* Validate parameters */ if (ifp == NULL || ifra == NULL) /* this maybe redundant */ return (EINVAL); /* * The destination address for a p2p link must have a family * of AF_UNSPEC or AF_INET6. */ if ((ifp->if_flags & IFF_POINTOPOINT) != 0 && ifra->ifra_dstaddr.sin6_family != AF_INET6 && ifra->ifra_dstaddr.sin6_family != AF_UNSPEC) return (EAFNOSUPPORT); /* * Validate address */ if (ifra->ifra_addr.sin6_len != sizeof(struct sockaddr_in6) || ifra->ifra_addr.sin6_family != AF_INET6) return (EINVAL); /* * validate ifra_prefixmask. don't check sin6_family, netmask * does not carry fields other than sin6_len. */ if (ifra->ifra_prefixmask.sin6_len > sizeof(struct sockaddr_in6)) return (EINVAL); /* * Because the IPv6 address architecture is classless, we require * users to specify a (non 0) prefix length (mask) for a new address. * We also require the prefix (when specified) mask is valid, and thus * reject a non-consecutive mask. */ if (ia == NULL && ifra->ifra_prefixmask.sin6_len == 0) return (EINVAL); if (ifra->ifra_prefixmask.sin6_len != 0) { plen = in6_mask2len(&ifra->ifra_prefixmask.sin6_addr, (u_char *)&ifra->ifra_prefixmask + ifra->ifra_prefixmask.sin6_len); if (plen <= 0) return (EINVAL); } else { /* * In this case, ia must not be NULL. We just use its prefix * length. */ plen = in6_mask2len(&ia->ia_prefixmask.sin6_addr, NULL); } /* * If the destination address on a p2p interface is specified, * and the address is a scoped one, validate/set the scope * zone identifier. */ dst6 = ifra->ifra_dstaddr; if ((ifp->if_flags & (IFF_POINTOPOINT|IFF_LOOPBACK)) != 0 && (dst6.sin6_family == AF_INET6)) { struct in6_addr in6_tmp; u_int32_t zoneid; in6_tmp = dst6.sin6_addr; if (in6_setscope(&in6_tmp, ifp, &zoneid)) return (EINVAL); /* XXX: should be impossible */ if (dst6.sin6_scope_id != 0) { if (dst6.sin6_scope_id != zoneid) return (EINVAL); } else /* user omit to specify the ID. */ dst6.sin6_scope_id = zoneid; /* convert into the internal form */ if (sa6_embedscope(&dst6, 0)) return (EINVAL); /* XXX: should be impossible */ } /* Modify original ifra_dstaddr to reflect changes */ ifra->ifra_dstaddr = dst6; /* * The destination address can be specified only for a p2p or a * loopback interface. If specified, the corresponding prefix length * must be 128. */ if (ifra->ifra_dstaddr.sin6_family == AF_INET6) { if ((ifp->if_flags & (IFF_POINTOPOINT|IFF_LOOPBACK)) == 0) { /* XXX: noisy message */ nd6log((LOG_INFO, "in6_update_ifa: a destination can " "be specified for a p2p or a loopback IF only\n")); return (EINVAL); } if (plen != 128) { nd6log((LOG_INFO, "in6_update_ifa: prefixlen should " "be 128 when dstaddr is specified\n")); return (EINVAL); } } /* lifetime consistency check */ lt = &ifra->ifra_lifetime; if (lt->ia6t_pltime > lt->ia6t_vltime) return (EINVAL); if (lt->ia6t_vltime == 0) { /* * the following log might be noisy, but this is a typical * configuration mistake or a tool's bug. */ nd6log((LOG_INFO, "in6_update_ifa: valid lifetime is 0 for %s\n", ip6_sprintf(ip6buf, &ifra->ifra_addr.sin6_addr))); if (ia == NULL) return (0); /* there's nothing to do */ } /* Check prefix mask */ if (ia != NULL && ifra->ifra_prefixmask.sin6_len != 0) { /* * We prohibit changing the prefix length of an existing * address, because * + such an operation should be rare in IPv6, and * + the operation would confuse prefix management. */ if (ia->ia_prefixmask.sin6_len != 0 && in6_mask2len(&ia->ia_prefixmask.sin6_addr, NULL) != plen) { nd6log((LOG_INFO, "in6_validate_ifa: the prefix length " "of an existing %s address should not be changed\n", ip6_sprintf(ip6buf, &ia->ia_addr.sin6_addr))); return (EINVAL); } } return (0); } /* * Allocate a new ifaddr and link it into chains. */ static struct in6_ifaddr * in6_alloc_ifa(struct ifnet *ifp, struct in6_aliasreq *ifra, int flags) { struct in6_ifaddr *ia; /* * When in6_alloc_ifa() is called in a process of a received * RA, it is called under an interrupt context. So, we should * call malloc with M_NOWAIT. */ ia = (struct in6_ifaddr *)ifa_alloc(sizeof(*ia), M_NOWAIT); if (ia == NULL) return (NULL); LIST_INIT(&ia->ia6_memberships); /* Initialize the address and masks, and put time stamp */ ia->ia_ifa.ifa_addr = (struct sockaddr *)&ia->ia_addr; ia->ia_addr.sin6_family = AF_INET6; ia->ia_addr.sin6_len = sizeof(ia->ia_addr); /* XXX: Can we assign ,sin6_addr and skip the rest? */ ia->ia_addr = ifra->ifra_addr; ia->ia6_createtime = time_uptime; if ((ifp->if_flags & (IFF_POINTOPOINT | IFF_LOOPBACK)) != 0) { /* * Some functions expect that ifa_dstaddr is not * NULL for p2p interfaces. */ ia->ia_ifa.ifa_dstaddr = (struct sockaddr *)&ia->ia_dstaddr; } else { ia->ia_ifa.ifa_dstaddr = NULL; } /* set prefix mask if any */ ia->ia_ifa.ifa_netmask = (struct sockaddr *)&ia->ia_prefixmask; if (ifra->ifra_prefixmask.sin6_len != 0) { ia->ia_prefixmask.sin6_family = AF_INET6; ia->ia_prefixmask.sin6_len = ifra->ifra_prefixmask.sin6_len; ia->ia_prefixmask.sin6_addr = ifra->ifra_prefixmask.sin6_addr; } ia->ia_ifp = ifp; ifa_ref(&ia->ia_ifa); /* if_addrhead */ IF_ADDR_WLOCK(ifp); TAILQ_INSERT_TAIL(&ifp->if_addrhead, &ia->ia_ifa, ifa_link); IF_ADDR_WUNLOCK(ifp); ifa_ref(&ia->ia_ifa); /* in6_ifaddrhead */ IN6_IFADDR_WLOCK(); TAILQ_INSERT_TAIL(&V_in6_ifaddrhead, ia, ia_link); LIST_INSERT_HEAD(IN6ADDR_HASH(&ia->ia_addr.sin6_addr), ia, ia6_hash); IN6_IFADDR_WUNLOCK(); return (ia); } /* * Update/configure interface address parameters: * * 1) Update lifetime * 2) Update interface metric ad flags * 3) Notify other subsystems */ static int in6_update_ifa_internal(struct ifnet *ifp, struct in6_aliasreq *ifra, struct in6_ifaddr *ia, int hostIsNew, int flags) { int error; /* update timestamp */ ia->ia6_updatetime = time_uptime; /* * Set lifetimes. We do not refer to ia6t_expire and ia6t_preferred * to see if the address is deprecated or invalidated, but initialize * these members for applications. */ ia->ia6_lifetime = ifra->ifra_lifetime; if (ia->ia6_lifetime.ia6t_vltime != ND6_INFINITE_LIFETIME) { ia->ia6_lifetime.ia6t_expire = time_uptime + ia->ia6_lifetime.ia6t_vltime; } else ia->ia6_lifetime.ia6t_expire = 0; if (ia->ia6_lifetime.ia6t_pltime != ND6_INFINITE_LIFETIME) { ia->ia6_lifetime.ia6t_preferred = time_uptime + ia->ia6_lifetime.ia6t_pltime; } else ia->ia6_lifetime.ia6t_preferred = 0; /* * backward compatibility - if IN6_IFF_DEPRECATED is set from the * userland, make it deprecated. */ if ((ifra->ifra_flags & IN6_IFF_DEPRECATED) != 0) { ia->ia6_lifetime.ia6t_pltime = 0; ia->ia6_lifetime.ia6t_preferred = time_uptime; } /* * configure address flags. */ ia->ia6_flags = ifra->ifra_flags; /* * Make the address tentative before joining multicast addresses, * so that corresponding MLD responses would not have a tentative * source address. */ ia->ia6_flags &= ~IN6_IFF_DUPLICATED; /* safety */ /* * DAD should be performed for an new address or addresses on * an interface with ND6_IFF_IFDISABLED. */ if (in6if_do_dad(ifp) && (hostIsNew || (ND_IFINFO(ifp)->flags & ND6_IFF_IFDISABLED))) ia->ia6_flags |= IN6_IFF_TENTATIVE; /* notify other subsystems */ error = in6_notify_ifa(ifp, ia, ifra, hostIsNew); return (error); } /* * Do link-level ifa job: * 1) Add lle entry for added address * 2) Notifies routing socket users about new address * 3) join appropriate multicast group * 4) start DAD if enabled */ static int in6_broadcast_ifa(struct ifnet *ifp, struct in6_aliasreq *ifra, struct in6_ifaddr *ia, int flags) { struct in6_multi *in6m_sol; int error = 0; /* Add local address to lltable, if necessary (ex. on p2p link). */ if ((error = nd6_add_ifa_lle(ia)) != 0) { in6_purgeaddr(&ia->ia_ifa); ifa_free(&ia->ia_ifa); return (error); } /* Join necessary multicast groups. */ in6m_sol = NULL; if ((ifp->if_flags & IFF_MULTICAST) != 0) { error = in6_update_ifa_join_mc(ifp, ifra, ia, flags, &in6m_sol); if (error != 0) { in6_purgeaddr(&ia->ia_ifa); ifa_free(&ia->ia_ifa); return (error); } } /* Perform DAD, if the address is TENTATIVE. */ if ((ia->ia6_flags & IN6_IFF_TENTATIVE)) { int delay, mindelay, maxdelay; delay = 0; if ((flags & IN6_IFAUPDATE_DADDELAY)) { /* * We need to impose a delay before sending an NS * for DAD. Check if we also needed a delay for the * corresponding MLD message. If we did, the delay * should be larger than the MLD delay (this could be * relaxed a bit, but this simple logic is at least * safe). * XXX: Break data hiding guidelines and look at * state for the solicited multicast group. */ mindelay = 0; if (in6m_sol != NULL && in6m_sol->in6m_state == MLD_REPORTING_MEMBER) { mindelay = in6m_sol->in6m_timer; } maxdelay = MAX_RTR_SOLICITATION_DELAY * hz; if (maxdelay - mindelay == 0) delay = 0; else { delay = (arc4random() % (maxdelay - mindelay)) + mindelay; } } nd6_dad_start((struct ifaddr *)ia, delay); } in6_newaddrmsg(ia, RTM_ADD); ifa_free(&ia->ia_ifa); return (error); } void in6_purgeaddr(struct ifaddr *ifa) { struct ifnet *ifp = ifa->ifa_ifp; struct in6_ifaddr *ia = (struct in6_ifaddr *) ifa; struct in6_multi_mship *imm; int plen, error; if (ifa->ifa_carp) (*carp_detach_p)(ifa); /* * Remove the loopback route to the interface address. * The check for the current setting of "nd6_useloopback" * is not needed. */ if (ia->ia_flags & IFA_RTSELF) { error = ifa_del_loopback_route((struct ifaddr *)ia, (struct sockaddr *)&ia->ia_addr); if (error == 0) ia->ia_flags &= ~IFA_RTSELF; } /* stop DAD processing */ nd6_dad_stop(ifa); /* Leave multicast groups. */ while ((imm = LIST_FIRST(&ia->ia6_memberships)) != NULL) { LIST_REMOVE(imm, i6mm_chain); in6_leavegroup(imm); } plen = in6_mask2len(&ia->ia_prefixmask.sin6_addr, NULL); /* XXX */ if ((ia->ia_flags & IFA_ROUTE) && plen == 128) { error = rtinit(&(ia->ia_ifa), RTM_DELETE, ia->ia_flags | (ia->ia_dstaddr.sin6_family == AF_INET6) ? RTF_HOST : 0); if (error != 0) log(LOG_INFO, "%s: err=%d, destination address delete " "failed\n", __func__, error); ia->ia_flags &= ~IFA_ROUTE; } in6_newaddrmsg(ia, RTM_DELETE); in6_unlink_ifa(ia, ifp); } static void in6_unlink_ifa(struct in6_ifaddr *ia, struct ifnet *ifp) { char ip6buf[INET6_ADDRSTRLEN]; int remove_lle; IF_ADDR_WLOCK(ifp); TAILQ_REMOVE(&ifp->if_addrhead, &ia->ia_ifa, ifa_link); IF_ADDR_WUNLOCK(ifp); ifa_free(&ia->ia_ifa); /* if_addrhead */ /* * Defer the release of what might be the last reference to the * in6_ifaddr so that it can't be freed before the remainder of the * cleanup. */ IN6_IFADDR_WLOCK(); TAILQ_REMOVE(&V_in6_ifaddrhead, ia, ia_link); LIST_REMOVE(ia, ia6_hash); IN6_IFADDR_WUNLOCK(); /* * Release the reference to the base prefix. There should be a * positive reference. */ remove_lle = 0; if (ia->ia6_ndpr == NULL) { nd6log((LOG_NOTICE, "in6_unlink_ifa: autoconf'ed address " "%s has no prefix\n", ip6_sprintf(ip6buf, IA6_IN6(ia)))); } else { ia->ia6_ndpr->ndpr_refcnt--; /* Do not delete lles within prefix if refcont != 0 */ if (ia->ia6_ndpr->ndpr_refcnt == 0) remove_lle = 1; ia->ia6_ndpr = NULL; } nd6_rem_ifa_lle(ia, remove_lle); /* * Also, if the address being removed is autoconf'ed, call * pfxlist_onlink_check() since the release might affect the status of * other (detached) addresses. */ if ((ia->ia6_flags & IN6_IFF_AUTOCONF)) { pfxlist_onlink_check(); } ifa_free(&ia->ia_ifa); /* in6_ifaddrhead */ } /* * Notifies other subsystems about address change/arrival: * 1) Notifies device handler on the first IPv6 address assignment * 2) Handle routing table changes for P2P links and route * 3) Handle routing table changes for address host route */ static int in6_notify_ifa(struct ifnet *ifp, struct in6_ifaddr *ia, struct in6_aliasreq *ifra, int hostIsNew) { int error = 0, plen, ifacount = 0; struct ifaddr *ifa; struct sockaddr_in6 *pdst; char ip6buf[INET6_ADDRSTRLEN]; /* * Give the interface a chance to initialize * if this is its first address, */ if (hostIsNew != 0) { IF_ADDR_RLOCK(ifp); TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) { if (ifa->ifa_addr->sa_family != AF_INET6) continue; ifacount++; } IF_ADDR_RUNLOCK(ifp); } if (ifacount <= 1 && ifp->if_ioctl) { error = (*ifp->if_ioctl)(ifp, SIOCSIFADDR, (caddr_t)ia); if (error) return (error); } /* * If a new destination address is specified, scrub the old one and * install the new destination. Note that the interface must be * p2p or loopback. */ pdst = &ifra->ifra_dstaddr; if (pdst->sin6_family == AF_INET6 && !IN6_ARE_ADDR_EQUAL(&pdst->sin6_addr, &ia->ia_dstaddr.sin6_addr)) { if ((ia->ia_flags & IFA_ROUTE) != 0 && (rtinit(&(ia->ia_ifa), (int)RTM_DELETE, RTF_HOST) != 0)) { nd6log((LOG_ERR, "in6_update_ifa_internal: failed to " "remove a route to the old destination: %s\n", ip6_sprintf(ip6buf, &ia->ia_addr.sin6_addr))); /* proceed anyway... */ } else ia->ia_flags &= ~IFA_ROUTE; ia->ia_dstaddr = *pdst; } /* * If a new destination address is specified for a point-to-point * interface, install a route to the destination as an interface * direct route. * XXX: the logic below rejects assigning multiple addresses on a p2p * interface that share the same destination. */ plen = in6_mask2len(&ia->ia_prefixmask.sin6_addr, NULL); /* XXX */ if (!(ia->ia_flags & IFA_ROUTE) && plen == 128 && ia->ia_dstaddr.sin6_family == AF_INET6) { int rtflags = RTF_UP | RTF_HOST; /* * Handle the case for ::1 . */ if (ifp->if_flags & IFF_LOOPBACK) ia->ia_flags |= IFA_RTSELF; error = rtinit(&ia->ia_ifa, RTM_ADD, ia->ia_flags | rtflags); if (error) return (error); ia->ia_flags |= IFA_ROUTE; } /* * add a loopback route to self if not exists */ if (!(ia->ia_flags & IFA_RTSELF) && V_nd6_useloopback) { error = ifa_add_loopback_route((struct ifaddr *)ia, (struct sockaddr *)&ia->ia_addr); if (error == 0) ia->ia_flags |= IFA_RTSELF; } return (error); } /* * Find an IPv6 interface link-local address specific to an interface. * ifaddr is returned referenced. */ struct in6_ifaddr * in6ifa_ifpforlinklocal(struct ifnet *ifp, int ignoreflags) { struct ifaddr *ifa; IF_ADDR_RLOCK(ifp); TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) { if (ifa->ifa_addr->sa_family != AF_INET6) continue; if (IN6_IS_ADDR_LINKLOCAL(IFA_IN6(ifa))) { if ((((struct in6_ifaddr *)ifa)->ia6_flags & ignoreflags) != 0) continue; ifa_ref(ifa); break; } } IF_ADDR_RUNLOCK(ifp); return ((struct in6_ifaddr *)ifa); } /* * find the internet address corresponding to a given address. * ifaddr is returned referenced. */ struct in6_ifaddr * in6ifa_ifwithaddr(const struct in6_addr *addr, uint32_t zoneid) { struct rm_priotracker in6_ifa_tracker; struct in6_ifaddr *ia; IN6_IFADDR_RLOCK(&in6_ifa_tracker); LIST_FOREACH(ia, IN6ADDR_HASH(addr), ia6_hash) { if (IN6_ARE_ADDR_EQUAL(IA6_IN6(ia), addr)) { if (zoneid != 0 && zoneid != ia->ia_addr.sin6_scope_id) continue; ifa_ref(&ia->ia_ifa); break; } } IN6_IFADDR_RUNLOCK(&in6_ifa_tracker); return (ia); } /* * find the internet address corresponding to a given interface and address. * ifaddr is returned referenced. */ struct in6_ifaddr * in6ifa_ifpwithaddr(struct ifnet *ifp, const struct in6_addr *addr) { struct ifaddr *ifa; IF_ADDR_RLOCK(ifp); TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) { if (ifa->ifa_addr->sa_family != AF_INET6) continue; if (IN6_ARE_ADDR_EQUAL(addr, IFA_IN6(ifa))) { ifa_ref(ifa); break; } } IF_ADDR_RUNLOCK(ifp); return ((struct in6_ifaddr *)ifa); } /* * Find a link-local scoped address on ifp and return it if any. */ struct in6_ifaddr * in6ifa_llaonifp(struct ifnet *ifp) { struct sockaddr_in6 *sin6; struct ifaddr *ifa; if (ND_IFINFO(ifp)->flags & ND6_IFF_IFDISABLED) return (NULL); if_addr_rlock(ifp); TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) { if (ifa->ifa_addr->sa_family != AF_INET6) continue; sin6 = (struct sockaddr_in6 *)ifa->ifa_addr; if (IN6_IS_SCOPE_LINKLOCAL(&sin6->sin6_addr) || IN6_IS_ADDR_MC_INTFACELOCAL(&sin6->sin6_addr) || IN6_IS_ADDR_MC_NODELOCAL(&sin6->sin6_addr)) break; } if_addr_runlock(ifp); return ((struct in6_ifaddr *)ifa); } /* * Convert IP6 address to printable (loggable) representation. Caller * has to make sure that ip6buf is at least INET6_ADDRSTRLEN long. */ static char digits[] = "0123456789abcdef"; char * ip6_sprintf(char *ip6buf, const struct in6_addr *addr) { int i, cnt = 0, maxcnt = 0, idx = 0, index = 0; char *cp; const u_int16_t *a = (const u_int16_t *)addr; const u_int8_t *d; int dcolon = 0, zero = 0; cp = ip6buf; for (i = 0; i < 8; i++) { if (*(a + i) == 0) { cnt++; if (cnt == 1) idx = i; } else if (maxcnt < cnt) { maxcnt = cnt; index = idx; cnt = 0; } } if (maxcnt < cnt) { maxcnt = cnt; index = idx; } for (i = 0; i < 8; i++) { if (dcolon == 1) { if (*a == 0) { if (i == 7) *cp++ = ':'; a++; continue; } else dcolon = 2; } if (*a == 0) { if (dcolon == 0 && *(a + 1) == 0 && i == index) { if (i == 0) *cp++ = ':'; *cp++ = ':'; dcolon = 1; } else { *cp++ = '0'; *cp++ = ':'; } a++; continue; } d = (const u_char *)a; /* Try to eliminate leading zeros in printout like in :0001. */ zero = 1; *cp = digits[*d >> 4]; if (*cp != '0') { zero = 0; cp++; } *cp = digits[*d++ & 0xf]; if (zero == 0 || (*cp != '0')) { zero = 0; cp++; } *cp = digits[*d >> 4]; if (zero == 0 || (*cp != '0')) { zero = 0; cp++; } *cp++ = digits[*d & 0xf]; *cp++ = ':'; a++; } *--cp = '\0'; return (ip6buf); } int in6_localaddr(struct in6_addr *in6) { struct rm_priotracker in6_ifa_tracker; struct in6_ifaddr *ia; if (IN6_IS_ADDR_LOOPBACK(in6) || IN6_IS_ADDR_LINKLOCAL(in6)) return 1; IN6_IFADDR_RLOCK(&in6_ifa_tracker); TAILQ_FOREACH(ia, &V_in6_ifaddrhead, ia_link) { if (IN6_ARE_MASKED_ADDR_EQUAL(in6, &ia->ia_addr.sin6_addr, &ia->ia_prefixmask.sin6_addr)) { IN6_IFADDR_RUNLOCK(&in6_ifa_tracker); return 1; } } IN6_IFADDR_RUNLOCK(&in6_ifa_tracker); return (0); } /* * Return 1 if an internet address is for the local host and configured * on one of its interfaces. */ int in6_localip(struct in6_addr *in6) { struct rm_priotracker in6_ifa_tracker; struct in6_ifaddr *ia; IN6_IFADDR_RLOCK(&in6_ifa_tracker); LIST_FOREACH(ia, IN6ADDR_HASH(in6), ia6_hash) { if (IN6_ARE_ADDR_EQUAL(in6, &ia->ia_addr.sin6_addr)) { IN6_IFADDR_RUNLOCK(&in6_ifa_tracker); return (1); } } IN6_IFADDR_RUNLOCK(&in6_ifa_tracker); return (0); } /* * Return 1 if an internet address is configured on an interface. */ int in6_ifhasaddr(struct ifnet *ifp, struct in6_addr *addr) { struct in6_addr in6; struct ifaddr *ifa; struct in6_ifaddr *ia6; in6 = *addr; if (in6_clearscope(&in6)) return (0); in6_setscope(&in6, ifp, NULL); IF_ADDR_RLOCK(ifp); TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) { if (ifa->ifa_addr->sa_family != AF_INET6) continue; ia6 = (struct in6_ifaddr *)ifa; if (IN6_ARE_ADDR_EQUAL(&ia6->ia_addr.sin6_addr, &in6)) { IF_ADDR_RUNLOCK(ifp); return (1); } } IF_ADDR_RUNLOCK(ifp); return (0); } int in6_is_addr_deprecated(struct sockaddr_in6 *sa6) { struct rm_priotracker in6_ifa_tracker; struct in6_ifaddr *ia; IN6_IFADDR_RLOCK(&in6_ifa_tracker); LIST_FOREACH(ia, IN6ADDR_HASH(&sa6->sin6_addr), ia6_hash) { if (IN6_ARE_ADDR_EQUAL(IA6_IN6(ia), &sa6->sin6_addr)) { if (ia->ia6_flags & IN6_IFF_DEPRECATED) { IN6_IFADDR_RUNLOCK(&in6_ifa_tracker); return (1); /* true */ } break; } } IN6_IFADDR_RUNLOCK(&in6_ifa_tracker); return (0); /* false */ } /* * return length of part which dst and src are equal * hard coding... */ int in6_matchlen(struct in6_addr *src, struct in6_addr *dst) { int match = 0; u_char *s = (u_char *)src, *d = (u_char *)dst; u_char *lim = s + 16, r; while (s < lim) if ((r = (*d++ ^ *s++)) != 0) { while (r < 128) { match++; r <<= 1; } break; } else match += 8; return match; } /* XXX: to be scope conscious */ int in6_are_prefix_equal(struct in6_addr *p1, struct in6_addr *p2, int len) { int bytelen, bitlen; /* sanity check */ if (0 > len || len > 128) { log(LOG_ERR, "in6_are_prefix_equal: invalid prefix length(%d)\n", len); return (0); } bytelen = len / 8; bitlen = len % 8; if (bcmp(&p1->s6_addr, &p2->s6_addr, bytelen)) return (0); if (bitlen != 0 && p1->s6_addr[bytelen] >> (8 - bitlen) != p2->s6_addr[bytelen] >> (8 - bitlen)) return (0); return (1); } void in6_prefixlen2mask(struct in6_addr *maskp, int len) { u_char maskarray[8] = {0x80, 0xc0, 0xe0, 0xf0, 0xf8, 0xfc, 0xfe, 0xff}; int bytelen, bitlen, i; /* sanity check */ if (0 > len || len > 128) { log(LOG_ERR, "in6_prefixlen2mask: invalid prefix length(%d)\n", len); return; } bzero(maskp, sizeof(*maskp)); bytelen = len / 8; bitlen = len % 8; for (i = 0; i < bytelen; i++) maskp->s6_addr[i] = 0xff; if (bitlen) maskp->s6_addr[bytelen] = maskarray[bitlen - 1]; } /* * return the best address out of the same scope. if no address was * found, return the first valid address from designated IF. */ struct in6_ifaddr * in6_ifawithifp(struct ifnet *ifp, struct in6_addr *dst) { int dst_scope = in6_addrscope(dst), blen = -1, tlen; struct ifaddr *ifa; struct in6_ifaddr *besta = 0; struct in6_ifaddr *dep[2]; /* last-resort: deprecated */ dep[0] = dep[1] = NULL; /* * We first look for addresses in the same scope. * If there is one, return it. * If two or more, return one which matches the dst longest. * If none, return one of global addresses assigned other ifs. */ IF_ADDR_RLOCK(ifp); TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) { if (ifa->ifa_addr->sa_family != AF_INET6) continue; if (((struct in6_ifaddr *)ifa)->ia6_flags & IN6_IFF_ANYCAST) continue; /* XXX: is there any case to allow anycast? */ if (((struct in6_ifaddr *)ifa)->ia6_flags & IN6_IFF_NOTREADY) continue; /* don't use this interface */ if (((struct in6_ifaddr *)ifa)->ia6_flags & IN6_IFF_DETACHED) continue; if (((struct in6_ifaddr *)ifa)->ia6_flags & IN6_IFF_DEPRECATED) { if (V_ip6_use_deprecated) dep[0] = (struct in6_ifaddr *)ifa; continue; } if (dst_scope == in6_addrscope(IFA_IN6(ifa))) { /* * call in6_matchlen() as few as possible */ if (besta) { if (blen == -1) blen = in6_matchlen(&besta->ia_addr.sin6_addr, dst); tlen = in6_matchlen(IFA_IN6(ifa), dst); if (tlen > blen) { blen = tlen; besta = (struct in6_ifaddr *)ifa; } } else besta = (struct in6_ifaddr *)ifa; } } if (besta) { ifa_ref(&besta->ia_ifa); IF_ADDR_RUNLOCK(ifp); return (besta); } TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) { if (ifa->ifa_addr->sa_family != AF_INET6) continue; if (((struct in6_ifaddr *)ifa)->ia6_flags & IN6_IFF_ANYCAST) continue; /* XXX: is there any case to allow anycast? */ if (((struct in6_ifaddr *)ifa)->ia6_flags & IN6_IFF_NOTREADY) continue; /* don't use this interface */ if (((struct in6_ifaddr *)ifa)->ia6_flags & IN6_IFF_DETACHED) continue; if (((struct in6_ifaddr *)ifa)->ia6_flags & IN6_IFF_DEPRECATED) { if (V_ip6_use_deprecated) dep[1] = (struct in6_ifaddr *)ifa; continue; } if (ifa != NULL) ifa_ref(ifa); IF_ADDR_RUNLOCK(ifp); return (struct in6_ifaddr *)ifa; } /* use the last-resort values, that are, deprecated addresses */ if (dep[0]) { ifa_ref((struct ifaddr *)dep[0]); IF_ADDR_RUNLOCK(ifp); return dep[0]; } if (dep[1]) { ifa_ref((struct ifaddr *)dep[1]); IF_ADDR_RUNLOCK(ifp); return dep[1]; } IF_ADDR_RUNLOCK(ifp); return NULL; } /* * perform DAD when interface becomes IFF_UP. */ void in6_if_up(struct ifnet *ifp) { struct ifaddr *ifa; struct in6_ifaddr *ia; IF_ADDR_RLOCK(ifp); TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) { if (ifa->ifa_addr->sa_family != AF_INET6) continue; ia = (struct in6_ifaddr *)ifa; if (ia->ia6_flags & IN6_IFF_TENTATIVE) { /* * The TENTATIVE flag was likely set by hand * beforehand, implicitly indicating the need for DAD. * We may be able to skip the random delay in this * case, but we impose delays just in case. */ nd6_dad_start(ifa, arc4random() % (MAX_RTR_SOLICITATION_DELAY * hz)); } } IF_ADDR_RUNLOCK(ifp); /* * special cases, like 6to4, are handled in in6_ifattach */ in6_ifattach(ifp, NULL); } int in6if_do_dad(struct ifnet *ifp) { if ((ifp->if_flags & IFF_LOOPBACK) != 0) return (0); if ((ND_IFINFO(ifp)->flags & ND6_IFF_IFDISABLED) || (ND_IFINFO(ifp)->flags & ND6_IFF_NO_DAD)) return (0); /* * Our DAD routine requires the interface up and running. * However, some interfaces can be up before the RUNNING * status. Additionaly, users may try to assign addresses * before the interface becomes up (or running). * This function returns EAGAIN in that case. * The caller should mark "tentative" on the address instead of * performing DAD immediately. */ if (!((ifp->if_flags & IFF_UP) && (ifp->if_drv_flags & IFF_DRV_RUNNING))) return (EAGAIN); return (1); } /* * Calculate max IPv6 MTU through all the interfaces and store it * to in6_maxmtu. */ void in6_setmaxmtu(void) { unsigned long maxmtu = 0; struct ifnet *ifp; IFNET_RLOCK_NOSLEEP(); TAILQ_FOREACH(ifp, &V_ifnet, if_link) { /* this function can be called during ifnet initialization */ if (!ifp->if_afdata[AF_INET6]) continue; if ((ifp->if_flags & IFF_LOOPBACK) == 0 && IN6_LINKMTU(ifp) > maxmtu) maxmtu = IN6_LINKMTU(ifp); } IFNET_RUNLOCK_NOSLEEP(); if (maxmtu) /* update only when maxmtu is positive */ V_in6_maxmtu = maxmtu; } /* * Provide the length of interface identifiers to be used for the link attached * to the given interface. The length should be defined in "IPv6 over * xxx-link" document. Note that address architecture might also define * the length for a particular set of address prefixes, regardless of the * link type. As clarified in rfc2462bis, those two definitions should be * consistent, and those really are as of August 2004. */ int in6_if2idlen(struct ifnet *ifp) { switch (ifp->if_type) { case IFT_ETHER: /* RFC2464 */ case IFT_PROPVIRTUAL: /* XXX: no RFC. treat it as ether */ case IFT_L2VLAN: /* ditto */ case IFT_IEEE80211: /* ditto */ case IFT_INFINIBAND: return (64); case IFT_FDDI: /* RFC2467 */ return (64); case IFT_ISO88025: /* RFC2470 (IPv6 over Token Ring) */ return (64); case IFT_PPP: /* RFC2472 */ return (64); case IFT_ARCNET: /* RFC2497 */ return (64); case IFT_FRELAY: /* RFC2590 */ return (64); case IFT_IEEE1394: /* RFC3146 */ return (64); case IFT_GIF: return (64); /* draft-ietf-v6ops-mech-v2-07 */ case IFT_LOOP: return (64); /* XXX: is this really correct? */ default: /* * Unknown link type: * It might be controversial to use the today's common constant * of 64 for these cases unconditionally. For full compliance, * we should return an error in this case. On the other hand, * if we simply miss the standard for the link type or a new * standard is defined for a new link type, the IFID length * is very likely to be the common constant. As a compromise, * we always use the constant, but make an explicit notice * indicating the "unknown" case. */ printf("in6_if2idlen: unknown link type (%d)\n", ifp->if_type); return (64); } } #include struct in6_llentry { struct llentry base; }; #define IN6_LLTBL_DEFAULT_HSIZE 32 #define IN6_LLTBL_HASH(k, h) \ (((((((k >> 8) ^ k) >> 8) ^ k) >> 8) ^ k) & ((h) - 1)) /* * Do actual deallocation of @lle. * Called by LLE_FREE_LOCKED when number of references * drops to zero. */ static void in6_lltable_destroy_lle(struct llentry *lle) { LLE_WUNLOCK(lle); LLE_LOCK_DESTROY(lle); free(lle, M_LLTABLE); } static struct llentry * in6_lltable_new(const struct in6_addr *addr6, u_int flags) { struct in6_llentry *lle; lle = malloc(sizeof(struct in6_llentry), M_LLTABLE, M_NOWAIT | M_ZERO); if (lle == NULL) /* NB: caller generates msg */ return NULL; lle->base.r_l3addr.addr6 = *addr6; lle->base.lle_refcnt = 1; lle->base.lle_free = in6_lltable_destroy_lle; LLE_LOCK_INIT(&lle->base); callout_init(&lle->base.lle_timer, 1); return (&lle->base); } static int in6_lltable_match_prefix(const struct sockaddr *saddr, const struct sockaddr *smask, u_int flags, struct llentry *lle) { const struct in6_addr *addr, *mask, *lle_addr; addr = &((const struct sockaddr_in6 *)saddr)->sin6_addr; mask = &((const struct sockaddr_in6 *)smask)->sin6_addr; lle_addr = &lle->r_l3addr.addr6; if (IN6_ARE_MASKED_ADDR_EQUAL(lle_addr, addr, mask) == 0) return (0); if (lle->la_flags & LLE_IFADDR) { /* * Delete LLE_IFADDR records IFF address & flag matches. * Note that addr is the interface address within prefix * being matched. */ if (IN6_ARE_ADDR_EQUAL(addr, lle_addr) && (flags & LLE_STATIC) != 0) return (1); return (0); } /* flags & LLE_STATIC means deleting both dynamic and static entries */ if ((flags & LLE_STATIC) || !(lle->la_flags & LLE_STATIC)) return (1); return (0); } static void in6_lltable_free_entry(struct lltable *llt, struct llentry *lle) { struct ifnet *ifp; LLE_WLOCK_ASSERT(lle); KASSERT(llt != NULL, ("lltable is NULL")); /* Unlink entry from table */ if ((lle->la_flags & LLE_LINKED) != 0) { ifp = llt->llt_ifp; IF_AFDATA_WLOCK_ASSERT(ifp); lltable_unlink_entry(llt, lle); } if (callout_stop(&lle->lle_timer) > 0) LLE_REMREF(lle); llentry_free(lle); } static int in6_lltable_rtcheck(struct ifnet *ifp, u_int flags, const struct sockaddr *l3addr) { - struct rtentry *rt; + const struct sockaddr_in6 *sin6; + struct nhop6_basic nh6; + struct in6_addr dst; + uint32_t scopeid; + int error; char ip6buf[INET6_ADDRSTRLEN]; KASSERT(l3addr->sa_family == AF_INET6, ("sin_family %d", l3addr->sa_family)); /* Our local addresses are always only installed on the default FIB. */ - /* XXX rtalloc1 should take a const param */ - rt = in6_rtalloc1(__DECONST(struct sockaddr *, l3addr), 0, 0, - RT_DEFAULT_FIB); - if (rt == NULL || (rt->rt_flags & RTF_GATEWAY) || rt->rt_ifp != ifp) { + + sin6 = (const struct sockaddr_in6 *)l3addr; + in6_splitscope(&sin6->sin6_addr, &dst, &scopeid); + error = fib6_lookup_nh_basic(RT_DEFAULT_FIB, &dst, scopeid, 0, 0, &nh6); + if (error != 0 || (nh6.nh_flags & NHF_GATEWAY) || nh6.nh_ifp != ifp) { struct ifaddr *ifa; /* * Create an ND6 cache for an IPv6 neighbor * that is not covered by our own prefix. */ ifa = ifaof_ifpforaddr(l3addr, ifp); if (ifa != NULL) { ifa_free(ifa); - if (rt != NULL) - RTFREE_LOCKED(rt); return 0; } log(LOG_INFO, "IPv6 address: \"%s\" is not on the network\n", - ip6_sprintf(ip6buf, &((const struct sockaddr_in6 *)l3addr)->sin6_addr)); - if (rt != NULL) - RTFREE_LOCKED(rt); + ip6_sprintf(ip6buf, &sin6->sin6_addr)); return EINVAL; } - RTFREE_LOCKED(rt); return 0; } static inline uint32_t in6_lltable_hash_dst(const struct in6_addr *dst, uint32_t hsize) { return (IN6_LLTBL_HASH(dst->s6_addr32[3], hsize)); } static uint32_t in6_lltable_hash(const struct llentry *lle, uint32_t hsize) { return (in6_lltable_hash_dst(&lle->r_l3addr.addr6, hsize)); } static void in6_lltable_fill_sa_entry(const struct llentry *lle, struct sockaddr *sa) { struct sockaddr_in6 *sin6; sin6 = (struct sockaddr_in6 *)sa; bzero(sin6, sizeof(*sin6)); sin6->sin6_family = AF_INET6; sin6->sin6_len = sizeof(*sin6); sin6->sin6_addr = lle->r_l3addr.addr6; } static inline struct llentry * in6_lltable_find_dst(struct lltable *llt, const struct in6_addr *dst) { struct llentry *lle; struct llentries *lleh; u_int hashidx; hashidx = in6_lltable_hash_dst(dst, llt->llt_hsize); lleh = &llt->lle_head[hashidx]; LIST_FOREACH(lle, lleh, lle_next) { if (lle->la_flags & LLE_DELETED) continue; if (IN6_ARE_ADDR_EQUAL(&lle->r_l3addr.addr6, dst)) break; } return (lle); } static void in6_lltable_delete_entry(struct lltable *llt, struct llentry *lle) { lle->la_flags |= LLE_DELETED; EVENTHANDLER_INVOKE(lle_event, lle, LLENTRY_DELETED); #ifdef DIAGNOSTIC log(LOG_INFO, "ifaddr cache = %p is deleted\n", lle); #endif llentry_free(lle); } static struct llentry * in6_lltable_alloc(struct lltable *llt, u_int flags, const struct sockaddr *l3addr) { const struct sockaddr_in6 *sin6 = (const struct sockaddr_in6 *)l3addr; struct ifnet *ifp = llt->llt_ifp; struct llentry *lle; KASSERT(l3addr->sa_family == AF_INET6, ("sin_family %d", l3addr->sa_family)); /* * A route that covers the given address must have * been installed 1st because we are doing a resolution, * verify this. */ if (!(flags & LLE_IFADDR) && in6_lltable_rtcheck(ifp, flags, l3addr) != 0) return (NULL); lle = in6_lltable_new(&sin6->sin6_addr, flags); if (lle == NULL) { log(LOG_INFO, "lla_lookup: new lle malloc failed\n"); return (NULL); } lle->la_flags = flags; if ((flags & LLE_IFADDR) == LLE_IFADDR) { lltable_set_entry_addr(ifp, lle, IF_LLADDR(ifp)); lle->la_flags |= LLE_STATIC; } if ((lle->la_flags & LLE_STATIC) != 0) lle->ln_state = ND6_LLINFO_REACHABLE; return (lle); } static struct llentry * in6_lltable_lookup(struct lltable *llt, u_int flags, const struct sockaddr *l3addr) { const struct sockaddr_in6 *sin6 = (const struct sockaddr_in6 *)l3addr; struct llentry *lle; IF_AFDATA_LOCK_ASSERT(llt->llt_ifp); KASSERT(l3addr->sa_family == AF_INET6, ("sin_family %d", l3addr->sa_family)); lle = in6_lltable_find_dst(llt, &sin6->sin6_addr); if (lle == NULL) return (NULL); if (flags & LLE_EXCLUSIVE) LLE_WLOCK(lle); else LLE_RLOCK(lle); return (lle); } static int in6_lltable_dump_entry(struct lltable *llt, struct llentry *lle, struct sysctl_req *wr) { struct ifnet *ifp = llt->llt_ifp; /* XXX stack use */ struct { struct rt_msghdr rtm; struct sockaddr_in6 sin6; /* * ndp.c assumes that sdl is word aligned */ #ifdef __LP64__ uint32_t pad; #endif struct sockaddr_dl sdl; } ndpc; struct sockaddr_dl *sdl; int error; bzero(&ndpc, sizeof(ndpc)); /* skip deleted entries */ if ((lle->la_flags & LLE_DELETED) == LLE_DELETED) return (0); /* Skip if jailed and not a valid IP of the prison. */ lltable_fill_sa_entry(lle, (struct sockaddr *)&ndpc.sin6); if (prison_if(wr->td->td_ucred, (struct sockaddr *)&ndpc.sin6) != 0) return (0); /* * produce a msg made of: * struct rt_msghdr; * struct sockaddr_in6 (IPv6) * struct sockaddr_dl; */ ndpc.rtm.rtm_msglen = sizeof(ndpc); ndpc.rtm.rtm_version = RTM_VERSION; ndpc.rtm.rtm_type = RTM_GET; ndpc.rtm.rtm_flags = RTF_UP; ndpc.rtm.rtm_addrs = RTA_DST | RTA_GATEWAY; if (V_deembed_scopeid) sa6_recoverscope(&ndpc.sin6); /* publish */ if (lle->la_flags & LLE_PUB) ndpc.rtm.rtm_flags |= RTF_ANNOUNCE; sdl = &ndpc.sdl; sdl->sdl_family = AF_LINK; sdl->sdl_len = sizeof(*sdl); sdl->sdl_alen = ifp->if_addrlen; sdl->sdl_index = ifp->if_index; sdl->sdl_type = ifp->if_type; bcopy(&lle->ll_addr, LLADDR(sdl), ifp->if_addrlen); ndpc.rtm.rtm_rmx.rmx_expire = lle->la_flags & LLE_STATIC ? 0 : lle->la_expire; ndpc.rtm.rtm_flags |= (RTF_HOST | RTF_LLDATA); if (lle->la_flags & LLE_STATIC) ndpc.rtm.rtm_flags |= RTF_STATIC; if (lle->la_flags & LLE_IFADDR) ndpc.rtm.rtm_flags |= RTF_PINNED; ndpc.rtm.rtm_index = ifp->if_index; error = SYSCTL_OUT(wr, &ndpc, sizeof(ndpc)); return (error); } static struct lltable * in6_lltattach(struct ifnet *ifp) { struct lltable *llt; llt = lltable_allocate_htbl(IN6_LLTBL_DEFAULT_HSIZE); llt->llt_af = AF_INET6; llt->llt_ifp = ifp; llt->llt_lookup = in6_lltable_lookup; llt->llt_alloc_entry = in6_lltable_alloc; llt->llt_delete_entry = in6_lltable_delete_entry; llt->llt_dump_entry = in6_lltable_dump_entry; llt->llt_hash = in6_lltable_hash; llt->llt_fill_sa_entry = in6_lltable_fill_sa_entry; llt->llt_free_entry = in6_lltable_free_entry; llt->llt_match_prefix = in6_lltable_match_prefix; lltable_link(llt); return (llt); } void * in6_domifattach(struct ifnet *ifp) { struct in6_ifextra *ext; /* There are not IPv6-capable interfaces. */ switch (ifp->if_type) { case IFT_PFLOG: case IFT_PFSYNC: case IFT_USB: return (NULL); } ext = (struct in6_ifextra *)malloc(sizeof(*ext), M_IFADDR, M_WAITOK); bzero(ext, sizeof(*ext)); ext->in6_ifstat = malloc(sizeof(counter_u64_t) * sizeof(struct in6_ifstat) / sizeof(uint64_t), M_IFADDR, M_WAITOK); COUNTER_ARRAY_ALLOC(ext->in6_ifstat, sizeof(struct in6_ifstat) / sizeof(uint64_t), M_WAITOK); ext->icmp6_ifstat = malloc(sizeof(counter_u64_t) * sizeof(struct icmp6_ifstat) / sizeof(uint64_t), M_IFADDR, M_WAITOK); COUNTER_ARRAY_ALLOC(ext->icmp6_ifstat, sizeof(struct icmp6_ifstat) / sizeof(uint64_t), M_WAITOK); ext->nd_ifinfo = nd6_ifattach(ifp); ext->scope6_id = scope6_ifattach(ifp); ext->lltable = in6_lltattach(ifp); ext->mld_ifinfo = mld_domifattach(ifp); return ext; } int in6_domifmtu(struct ifnet *ifp) { return (IN6_LINKMTU(ifp)); } void in6_domifdetach(struct ifnet *ifp, void *aux) { struct in6_ifextra *ext = (struct in6_ifextra *)aux; mld_domifdetach(ifp); scope6_ifdetach(ext->scope6_id); nd6_ifdetach(ext->nd_ifinfo); lltable_free(ext->lltable); COUNTER_ARRAY_FREE(ext->in6_ifstat, sizeof(struct in6_ifstat) / sizeof(uint64_t)); free(ext->in6_ifstat, M_IFADDR); COUNTER_ARRAY_FREE(ext->icmp6_ifstat, sizeof(struct icmp6_ifstat) / sizeof(uint64_t)); free(ext->icmp6_ifstat, M_IFADDR); free(ext, M_IFADDR); } /* * Convert sockaddr_in6 to sockaddr_in. Original sockaddr_in6 must be * v4 mapped addr or v4 compat addr */ void in6_sin6_2_sin(struct sockaddr_in *sin, struct sockaddr_in6 *sin6) { bzero(sin, sizeof(*sin)); sin->sin_len = sizeof(struct sockaddr_in); sin->sin_family = AF_INET; sin->sin_port = sin6->sin6_port; sin->sin_addr.s_addr = sin6->sin6_addr.s6_addr32[3]; } /* Convert sockaddr_in to sockaddr_in6 in v4 mapped addr format. */ void in6_sin_2_v4mapsin6(struct sockaddr_in *sin, struct sockaddr_in6 *sin6) { bzero(sin6, sizeof(*sin6)); sin6->sin6_len = sizeof(struct sockaddr_in6); sin6->sin6_family = AF_INET6; sin6->sin6_port = sin->sin_port; sin6->sin6_addr.s6_addr32[0] = 0; sin6->sin6_addr.s6_addr32[1] = 0; sin6->sin6_addr.s6_addr32[2] = IPV6_ADDR_INT32_SMP; sin6->sin6_addr.s6_addr32[3] = sin->sin_addr.s_addr; } /* Convert sockaddr_in6 into sockaddr_in. */ void in6_sin6_2_sin_in_sock(struct sockaddr *nam) { struct sockaddr_in *sin_p; struct sockaddr_in6 sin6; /* * Save original sockaddr_in6 addr and convert it * to sockaddr_in. */ sin6 = *(struct sockaddr_in6 *)nam; sin_p = (struct sockaddr_in *)nam; in6_sin6_2_sin(sin_p, &sin6); } /* Convert sockaddr_in into sockaddr_in6 in v4 mapped addr format. */ void in6_sin_2_v4mapsin6_in_sock(struct sockaddr **nam) { struct sockaddr_in *sin_p; struct sockaddr_in6 *sin6_p; sin6_p = malloc(sizeof *sin6_p, M_SONAME, M_WAITOK); sin_p = (struct sockaddr_in *)*nam; in6_sin_2_v4mapsin6(sin_p, sin6_p); free(*nam, M_SONAME); *nam = (struct sockaddr *)sin6_p; } Index: user/ngie/more-tests2/sys/netinet6/in6_fib.c =================================================================== --- user/ngie/more-tests2/sys/netinet6/in6_fib.c (revision 292053) +++ user/ngie/more-tests2/sys/netinet6/in6_fib.c (revision 292054) @@ -1,264 +1,268 @@ /*- * Copyright (c) 2015 * Alexander V. Chernikov * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include "opt_inet.h" #include "opt_inet6.h" #include "opt_route.h" #include "opt_mpath.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef RADIX_MPATH #include #endif #include #include #include #include #include #include #include #include #include #ifdef INET6 static void fib6_rte_to_nh_extended(struct rtentry *rte, const struct in6_addr *dst, uint32_t flags, struct nhop6_extended *pnh6); static void fib6_rte_to_nh_basic(struct rtentry *rte, const struct in6_addr *dst, uint32_t flags, struct nhop6_basic *pnh6); static struct ifnet *fib6_get_ifaifp(struct rtentry *rte); #define RNTORT(p) ((struct rtentry *)(p)) /* * Gets real interface for the @rte. * Returns rt_ifp for !IFF_LOOPBACK routers. * Extracts "real" address interface from interface address * loopback routes. */ static struct ifnet * fib6_get_ifaifp(struct rtentry *rte) { struct ifnet *ifp; struct sockaddr_dl *sdl; ifp = rte->rt_ifp; if ((ifp->if_flags & IFF_LOOPBACK) && rte->rt_gateway->sa_family == AF_LINK) { sdl = (struct sockaddr_dl *)rte->rt_gateway; return (ifnet_byindex(sdl->sdl_index)); } return (ifp); } static void fib6_rte_to_nh_basic(struct rtentry *rte, const struct in6_addr *dst, uint32_t flags, struct nhop6_basic *pnh6) { struct sockaddr_in6 *gw; /* Do explicit nexthop zero unless we're copying it */ memset(pnh6, 0, sizeof(*pnh6)); if ((flags & NHR_IFAIF) != 0) pnh6->nh_ifp = fib6_get_ifaifp(rte); else pnh6->nh_ifp = rte->rt_ifp; pnh6->nh_mtu = min(rte->rt_mtu, IN6_LINKMTU(rte->rt_ifp)); if (rte->rt_flags & RTF_GATEWAY) { gw = (struct sockaddr_in6 *)rte->rt_gateway; pnh6->nh_addr = gw->sin6_addr; in6_clearscope(&pnh6->nh_addr); } else pnh6->nh_addr = *dst; /* Set flags */ pnh6->nh_flags = fib_rte_to_nh_flags(rte->rt_flags); gw = (struct sockaddr_in6 *)rt_key(rte); if (IN6_IS_ADDR_UNSPECIFIED(&gw->sin6_addr)) pnh6->nh_flags |= NHF_DEFAULT; } static void fib6_rte_to_nh_extended(struct rtentry *rte, const struct in6_addr *dst, uint32_t flags, struct nhop6_extended *pnh6) { struct sockaddr_in6 *gw; /* Do explicit nexthop zero unless we're copying it */ memset(pnh6, 0, sizeof(*pnh6)); if ((flags & NHR_IFAIF) != 0) pnh6->nh_ifp = fib6_get_ifaifp(rte); else pnh6->nh_ifp = rte->rt_ifp; pnh6->nh_mtu = min(rte->rt_mtu, IN6_LINKMTU(rte->rt_ifp)); if (rte->rt_flags & RTF_GATEWAY) { gw = (struct sockaddr_in6 *)rte->rt_gateway; pnh6->nh_addr = gw->sin6_addr; in6_clearscope(&pnh6->nh_addr); } else pnh6->nh_addr = *dst; /* Set flags */ pnh6->nh_flags = fib_rte_to_nh_flags(rte->rt_flags); gw = (struct sockaddr_in6 *)rt_key(rte); if (IN6_IS_ADDR_UNSPECIFIED(&gw->sin6_addr)) pnh6->nh_flags |= NHF_DEFAULT; } /* * Performs IPv6 route table lookup on @dst. Returns 0 on success. * Stores basic nexthop info into provided @pnh6 structure. * Note that * - nh_ifp represents logical transmit interface (rt_ifp) by default * - nh_ifp represents "address" interface if NHR_IFAIF flag is passed * - mtu from logical transmit interface will be returned. * - nh_ifp cannot be safely dereferenced * - nh_ifp represents rt_ifp (e.g. if looking up address on * interface "ix0" pointer to "ix0" interface will be returned instead * of "lo0") * - howewer mtu from "transmit" interface will be returned. + * - scope will be embedded in nh_addr */ int fib6_lookup_nh_basic(uint32_t fibnum, const struct in6_addr *dst, uint32_t scopeid, uint32_t flags, uint32_t flowid, struct nhop6_basic *pnh6) { struct radix_node_head *rh; struct radix_node *rn; struct sockaddr_in6 sin6; struct rtentry *rte; KASSERT((fibnum < rt_numfibs), ("fib6_lookup_nh_basic: bad fibnum")); rh = rt_tables_get_rnh(fibnum, AF_INET6); if (rh == NULL) return (ENOENT); /* Prepare lookup key */ memset(&sin6, 0, sizeof(sin6)); sin6.sin6_addr = *dst; + sin6.sin6_len = sizeof(struct sockaddr_in6); /* Assume scopeid is valid and embed it directly */ if (IN6_IS_SCOPE_LINKLOCAL(dst)) sin6.sin6_addr.s6_addr16[1] = htons(scopeid & 0xffff); RADIX_NODE_HEAD_RLOCK(rh); rn = rh->rnh_matchaddr((void *)&sin6, rh); if (rn != NULL && ((rn->rn_flags & RNF_ROOT) == 0)) { rte = RNTORT(rn); /* Ensure route & ifp is UP */ if (RT_LINK_IS_UP(rte->rt_ifp)) { - fib6_rte_to_nh_basic(rte, dst, flags, pnh6); + fib6_rte_to_nh_basic(rte, &sin6.sin6_addr, flags, pnh6); RADIX_NODE_HEAD_RUNLOCK(rh); return (0); } } RADIX_NODE_HEAD_RUNLOCK(rh); return (ENOENT); } /* * Performs IPv6 route table lookup on @dst. Returns 0 on success. * Stores extended nexthop info into provided @pnh6 structure. * Note that * - nh_ifp cannot be safely dereferenced unless NHR_REF is specified. * - in that case you need to call fib6_free_nh_ext() * - nh_ifp represents logical transmit interface (rt_ifp) by default * - nh_ifp represents "address" interface if NHR_IFAIF flag is passed * - mtu from logical transmit interface will be returned. + * - scope will be embedded in nh_addr */ int fib6_lookup_nh_ext(uint32_t fibnum, const struct in6_addr *dst,uint32_t scopeid, uint32_t flags, uint32_t flowid, struct nhop6_extended *pnh6) { struct radix_node_head *rh; struct radix_node *rn; struct sockaddr_in6 sin6; struct rtentry *rte; KASSERT((fibnum < rt_numfibs), ("fib6_lookup_nh_ext: bad fibnum")); rh = rt_tables_get_rnh(fibnum, AF_INET6); if (rh == NULL) return (ENOENT); /* Prepare lookup key */ memset(&sin6, 0, sizeof(sin6)); sin6.sin6_len = sizeof(struct sockaddr_in6); sin6.sin6_addr = *dst; /* Assume scopeid is valid and embed it directly */ if (IN6_IS_SCOPE_LINKLOCAL(dst)) sin6.sin6_addr.s6_addr16[1] = htons(scopeid & 0xffff); RADIX_NODE_HEAD_RLOCK(rh); rn = rh->rnh_matchaddr((void *)&sin6, rh); if (rn != NULL && ((rn->rn_flags & RNF_ROOT) == 0)) { rte = RNTORT(rn); /* Ensure route & ifp is UP */ if (RT_LINK_IS_UP(rte->rt_ifp)) { - fib6_rte_to_nh_extended(rte, dst, flags, pnh6); + fib6_rte_to_nh_extended(rte, &sin6.sin6_addr, flags, + pnh6); if ((flags & NHR_REF) != 0) { /* TODO: Do lwref on egress ifp's */ } RADIX_NODE_HEAD_RUNLOCK(rh); return (0); } } RADIX_NODE_HEAD_RUNLOCK(rh); return (ENOENT); } void fib6_free_nh_ext(uint32_t fibnum, struct nhop6_extended *pnh6) { } #endif Index: user/ngie/more-tests2/sys/netinet6/in6_gif.c =================================================================== --- user/ngie/more-tests2/sys/netinet6/in6_gif.c (revision 292053) +++ user/ngie/more-tests2/sys/netinet6/in6_gif.c (revision 292054) @@ -1,230 +1,230 @@ /*- * Copyright (C) 1995, 1996, 1997, and 1998 WIDE Project. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. Neither the name of the project nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE PROJECT AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE PROJECT OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $KAME: in6_gif.c,v 1.49 2001/05/14 14:02:17 itojun Exp $ */ #include __FBSDID("$FreeBSD$"); #include "opt_inet.h" #include "opt_inet6.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef INET #include #endif #include #ifdef INET6 #include #include #include #endif #include #ifdef INET6 #include #include #endif #include #define GIF_HLIM 30 static VNET_DEFINE(int, ip6_gif_hlim) = GIF_HLIM; #define V_ip6_gif_hlim VNET(ip6_gif_hlim) SYSCTL_DECL(_net_inet6_ip6); SYSCTL_INT(_net_inet6_ip6, IPV6CTL_GIF_HLIM, gifhlim, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(ip6_gif_hlim), 0, ""); static int in6_gif_input(struct mbuf **, int *, int); extern struct domain inet6domain; static struct protosw in6_gif_protosw = { .pr_type = SOCK_RAW, .pr_domain = &inet6domain, .pr_protocol = 0, /* IPPROTO_IPV[46] */ .pr_flags = PR_ATOMIC|PR_ADDR, .pr_input = in6_gif_input, .pr_output = rip6_output, .pr_ctloutput = rip6_ctloutput, .pr_usrreqs = &rip6_usrreqs }; int in6_gif_output(struct ifnet *ifp, struct mbuf *m, int proto, uint8_t ecn) { GIF_RLOCK_TRACKER; struct gif_softc *sc = ifp->if_softc; struct ip6_hdr *ip6; int len; /* prepend new IP header */ len = sizeof(struct ip6_hdr); #ifndef __NO_STRICT_ALIGNMENT if (proto == IPPROTO_ETHERIP) len += ETHERIP_ALIGN; #endif M_PREPEND(m, len, M_NOWAIT); if (m == NULL) return (ENOBUFS); #ifndef __NO_STRICT_ALIGNMENT if (proto == IPPROTO_ETHERIP) { len = mtod(m, vm_offset_t) & 3; KASSERT(len == 0 || len == ETHERIP_ALIGN, ("in6_gif_output: unexpected misalignment")); m->m_data += len; m->m_len -= ETHERIP_ALIGN; } #endif ip6 = mtod(m, struct ip6_hdr *); GIF_RLOCK(sc); if (sc->gif_family != AF_INET6) { m_freem(m); GIF_RUNLOCK(sc); return (ENETDOWN); } bcopy(sc->gif_ip6hdr, ip6, sizeof(struct ip6_hdr)); GIF_RUNLOCK(sc); ip6->ip6_flow |= htonl((uint32_t)ecn << 20); ip6->ip6_nxt = proto; ip6->ip6_hlim = V_ip6_gif_hlim; /* * force fragmentation to minimum MTU, to avoid path MTU discovery. * it is too painful to ask for resend of inner packet, to achieve * path MTU discovery for encapsulated packets. */ return (ip6_output(m, 0, NULL, IPV6_MINMTU, 0, NULL, NULL)); } static int in6_gif_input(struct mbuf **mp, int *offp, int proto) { struct mbuf *m = *mp; struct ifnet *gifp; struct gif_softc *sc; struct ip6_hdr *ip6; uint8_t ecn; sc = encap_getarg(m); if (sc == NULL) { m_freem(m); IP6STAT_INC(ip6s_nogif); return (IPPROTO_DONE); } gifp = GIF2IFP(sc); if ((gifp->if_flags & IFF_UP) != 0) { ip6 = mtod(m, struct ip6_hdr *); ecn = (ntohl(ip6->ip6_flow) >> 20) & 0xff; m_adj(m, *offp); gif_input(m, gifp, proto, ecn); } else { m_freem(m); IP6STAT_INC(ip6s_nogif); } return (IPPROTO_DONE); } /* * we know that we are in IFF_UP, outer address available, and outer family * matched the physical addr family. see gif_encapcheck(). */ int in6_gif_encapcheck(const struct mbuf *m, int off, int proto, void *arg) { const struct ip6_hdr *ip6; struct gif_softc *sc; int ret; /* sanity check done in caller */ sc = (struct gif_softc *)arg; GIF_RLOCK_ASSERT(sc); /* * Check for address match. Note that the check is for an incoming * packet. We should compare the *source* address in our configuration * and the *destination* address of the packet, and vice versa. */ ip6 = mtod(m, const struct ip6_hdr *); if (!IN6_ARE_ADDR_EQUAL(&sc->gif_ip6hdr->ip6_src, &ip6->ip6_dst)) return (0); ret = 128; if (!IN6_ARE_ADDR_EQUAL(&sc->gif_ip6hdr->ip6_dst, &ip6->ip6_src)) { if ((sc->gif_options & GIF_IGNORE_SOURCE) == 0) return (0); } else ret += 128; /* ingress filters on outer source */ if ((GIF2IFP(sc)->if_flags & IFF_LINK2) == 0) { struct nhop6_basic nh6; - struct in6_addr *dst; /* XXX empty scope id */ - if (fib6_lookup_nh_basic(sc->gif_fibnum, dst, 0, 0, 0, &nh6)!=0) + if (fib6_lookup_nh_basic(sc->gif_fibnum, &ip6->ip6_src, 0, 0, 0, + &nh6) != 0) return (0); if (nh6.nh_ifp != m->m_pkthdr.rcvif) return (0); } return (ret); } int in6_gif_attach(struct gif_softc *sc) { KASSERT(sc->gif_ecookie == NULL, ("gif_ecookie isn't NULL")); sc->gif_ecookie = encap_attach_func(AF_INET6, -1, gif_encapcheck, (void *)&in6_gif_protosw, sc); if (sc->gif_ecookie == NULL) return (EEXIST); return (0); } Index: user/ngie/more-tests2/sys/netinet6/in6_mcast.c =================================================================== --- user/ngie/more-tests2/sys/netinet6/in6_mcast.c (revision 292053) +++ user/ngie/more-tests2/sys/netinet6/in6_mcast.c (revision 292054) @@ -1,2836 +1,2833 @@ /* * Copyright (c) 2009 Bruce Simpson. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. The name of the author may not be used to endorse or promote * products derived from this software without specific prior written * permission. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * IPv6 multicast socket, group, and socket option processing module. * Normative references: RFC 2292, RFC 3492, RFC 3542, RFC 3678, RFC 3810. */ #include __FBSDID("$FreeBSD$"); #include "opt_inet6.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include +#include #include #include #include #include #include #include #include #include #include #ifndef KTR_MLD #define KTR_MLD KTR_INET6 #endif #ifndef __SOCKUNION_DECLARED union sockunion { struct sockaddr_storage ss; struct sockaddr sa; struct sockaddr_dl sdl; struct sockaddr_in6 sin6; }; typedef union sockunion sockunion_t; #define __SOCKUNION_DECLARED #endif /* __SOCKUNION_DECLARED */ static MALLOC_DEFINE(M_IN6MFILTER, "in6_mfilter", "IPv6 multicast PCB-layer source filter"); static MALLOC_DEFINE(M_IP6MADDR, "in6_multi", "IPv6 multicast group"); static MALLOC_DEFINE(M_IP6MOPTS, "ip6_moptions", "IPv6 multicast options"); static MALLOC_DEFINE(M_IP6MSOURCE, "ip6_msource", "IPv6 multicast MLD-layer source filter"); RB_GENERATE(ip6_msource_tree, ip6_msource, im6s_link, ip6_msource_cmp); /* * Locking: * - Lock order is: Giant, INP_WLOCK, IN6_MULTI_LOCK, MLD_LOCK, IF_ADDR_LOCK. * - The IF_ADDR_LOCK is implicitly taken by in6m_lookup() earlier, however * it can be taken by code in net/if.c also. * - ip6_moptions and in6_mfilter are covered by the INP_WLOCK. * * struct in6_multi is covered by IN6_MULTI_LOCK. There isn't strictly * any need for in6_multi itself to be virtualized -- it is bound to an ifp * anyway no matter what happens. */ struct mtx in6_multi_mtx; MTX_SYSINIT(in6_multi_mtx, &in6_multi_mtx, "in6_multi_mtx", MTX_DEF); static void im6f_commit(struct in6_mfilter *); static int im6f_get_source(struct in6_mfilter *imf, const struct sockaddr_in6 *psin, struct in6_msource **); static struct in6_msource * im6f_graft(struct in6_mfilter *, const uint8_t, const struct sockaddr_in6 *); static void im6f_leave(struct in6_mfilter *); static int im6f_prune(struct in6_mfilter *, const struct sockaddr_in6 *); static void im6f_purge(struct in6_mfilter *); static void im6f_rollback(struct in6_mfilter *); static void im6f_reap(struct in6_mfilter *); static int im6o_grow(struct ip6_moptions *); static size_t im6o_match_group(const struct ip6_moptions *, const struct ifnet *, const struct sockaddr *); static struct in6_msource * im6o_match_source(const struct ip6_moptions *, const size_t, const struct sockaddr *); static void im6s_merge(struct ip6_msource *ims, const struct in6_msource *lims, const int rollback); static int in6_mc_get(struct ifnet *, const struct in6_addr *, struct in6_multi **); static int in6m_get_source(struct in6_multi *inm, const struct in6_addr *addr, const int noalloc, struct ip6_msource **pims); #ifdef KTR static int in6m_is_ifp_detached(const struct in6_multi *); #endif static int in6m_merge(struct in6_multi *, /*const*/ struct in6_mfilter *); static void in6m_purge(struct in6_multi *); static void in6m_reap(struct in6_multi *); static struct ip6_moptions * in6p_findmoptions(struct inpcb *); static int in6p_get_source_filters(struct inpcb *, struct sockopt *); static int in6p_join_group(struct inpcb *, struct sockopt *); static int in6p_leave_group(struct inpcb *, struct sockopt *); static struct ifnet * in6p_lookup_mcast_ifp(const struct inpcb *, const struct sockaddr_in6 *); static int in6p_block_unblock_source(struct inpcb *, struct sockopt *); static int in6p_set_multicast_if(struct inpcb *, struct sockopt *); static int in6p_set_source_filters(struct inpcb *, struct sockopt *); static int sysctl_ip6_mcast_filters(SYSCTL_HANDLER_ARGS); SYSCTL_DECL(_net_inet6_ip6); /* XXX Not in any common header. */ static SYSCTL_NODE(_net_inet6_ip6, OID_AUTO, mcast, CTLFLAG_RW, 0, "IPv6 multicast"); static u_long in6_mcast_maxgrpsrc = IPV6_MAX_GROUP_SRC_FILTER; SYSCTL_ULONG(_net_inet6_ip6_mcast, OID_AUTO, maxgrpsrc, CTLFLAG_RWTUN, &in6_mcast_maxgrpsrc, 0, "Max source filters per group"); static u_long in6_mcast_maxsocksrc = IPV6_MAX_SOCK_SRC_FILTER; SYSCTL_ULONG(_net_inet6_ip6_mcast, OID_AUTO, maxsocksrc, CTLFLAG_RWTUN, &in6_mcast_maxsocksrc, 0, "Max source filters per socket"); /* TODO Virtualize this switch. */ int in6_mcast_loop = IPV6_DEFAULT_MULTICAST_LOOP; SYSCTL_INT(_net_inet6_ip6_mcast, OID_AUTO, loop, CTLFLAG_RWTUN, &in6_mcast_loop, 0, "Loopback multicast datagrams by default"); static SYSCTL_NODE(_net_inet6_ip6_mcast, OID_AUTO, filters, CTLFLAG_RD | CTLFLAG_MPSAFE, sysctl_ip6_mcast_filters, "Per-interface stack-wide source filters"); #ifdef KTR /* * Inline function which wraps assertions for a valid ifp. * The ifnet layer will set the ifma's ifp pointer to NULL if the ifp * is detached. */ static int __inline in6m_is_ifp_detached(const struct in6_multi *inm) { struct ifnet *ifp; KASSERT(inm->in6m_ifma != NULL, ("%s: no ifma", __func__)); ifp = inm->in6m_ifma->ifma_ifp; if (ifp != NULL) { /* * Sanity check that network-layer notion of ifp is the * same as that of link-layer. */ KASSERT(inm->in6m_ifp == ifp, ("%s: bad ifp", __func__)); } return (ifp == NULL); } #endif /* * Initialize an in6_mfilter structure to a known state at t0, t1 * with an empty source filter list. */ static __inline void im6f_init(struct in6_mfilter *imf, const int st0, const int st1) { memset(imf, 0, sizeof(struct in6_mfilter)); RB_INIT(&imf->im6f_sources); imf->im6f_st[0] = st0; imf->im6f_st[1] = st1; } /* * Resize the ip6_moptions vector to the next power-of-two minus 1. * May be called with locks held; do not sleep. */ static int im6o_grow(struct ip6_moptions *imo) { struct in6_multi **nmships; struct in6_multi **omships; struct in6_mfilter *nmfilters; struct in6_mfilter *omfilters; size_t idx; size_t newmax; size_t oldmax; nmships = NULL; nmfilters = NULL; omships = imo->im6o_membership; omfilters = imo->im6o_mfilters; oldmax = imo->im6o_max_memberships; newmax = ((oldmax + 1) * 2) - 1; if (newmax <= IPV6_MAX_MEMBERSHIPS) { nmships = (struct in6_multi **)realloc(omships, sizeof(struct in6_multi *) * newmax, M_IP6MOPTS, M_NOWAIT); nmfilters = (struct in6_mfilter *)realloc(omfilters, sizeof(struct in6_mfilter) * newmax, M_IN6MFILTER, M_NOWAIT); if (nmships != NULL && nmfilters != NULL) { /* Initialize newly allocated source filter heads. */ for (idx = oldmax; idx < newmax; idx++) { im6f_init(&nmfilters[idx], MCAST_UNDEFINED, MCAST_EXCLUDE); } imo->im6o_max_memberships = newmax; imo->im6o_membership = nmships; imo->im6o_mfilters = nmfilters; } } if (nmships == NULL || nmfilters == NULL) { if (nmships != NULL) free(nmships, M_IP6MOPTS); if (nmfilters != NULL) free(nmfilters, M_IN6MFILTER); return (ETOOMANYREFS); } return (0); } /* * Find an IPv6 multicast group entry for this ip6_moptions instance * which matches the specified group, and optionally an interface. * Return its index into the array, or -1 if not found. */ static size_t im6o_match_group(const struct ip6_moptions *imo, const struct ifnet *ifp, const struct sockaddr *group) { const struct sockaddr_in6 *gsin6; struct in6_multi **pinm; int idx; int nmships; gsin6 = (const struct sockaddr_in6 *)group; /* The im6o_membership array may be lazy allocated. */ if (imo->im6o_membership == NULL || imo->im6o_num_memberships == 0) return (-1); nmships = imo->im6o_num_memberships; pinm = &imo->im6o_membership[0]; for (idx = 0; idx < nmships; idx++, pinm++) { if (*pinm == NULL) continue; if ((ifp == NULL || ((*pinm)->in6m_ifp == ifp)) && IN6_ARE_ADDR_EQUAL(&(*pinm)->in6m_addr, &gsin6->sin6_addr)) { break; } } if (idx >= nmships) idx = -1; return (idx); } /* * Find an IPv6 multicast source entry for this imo which matches * the given group index for this socket, and source address. * * XXX TODO: The scope ID, if present in src, is stripped before * any comparison. We SHOULD enforce scope/zone checks where the source * filter entry has a link scope. * * NOTE: This does not check if the entry is in-mode, merely if * it exists, which may not be the desired behaviour. */ static struct in6_msource * im6o_match_source(const struct ip6_moptions *imo, const size_t gidx, const struct sockaddr *src) { struct ip6_msource find; struct in6_mfilter *imf; struct ip6_msource *ims; const sockunion_t *psa; KASSERT(src->sa_family == AF_INET6, ("%s: !AF_INET6", __func__)); KASSERT(gidx != -1 && gidx < imo->im6o_num_memberships, ("%s: invalid index %d\n", __func__, (int)gidx)); /* The im6o_mfilters array may be lazy allocated. */ if (imo->im6o_mfilters == NULL) return (NULL); imf = &imo->im6o_mfilters[gidx]; psa = (const sockunion_t *)src; find.im6s_addr = psa->sin6.sin6_addr; in6_clearscope(&find.im6s_addr); /* XXX */ ims = RB_FIND(ip6_msource_tree, &imf->im6f_sources, &find); return ((struct in6_msource *)ims); } /* * Perform filtering for multicast datagrams on a socket by group and source. * * Returns 0 if a datagram should be allowed through, or various error codes * if the socket was not a member of the group, or the source was muted, etc. */ int im6o_mc_filter(const struct ip6_moptions *imo, const struct ifnet *ifp, const struct sockaddr *group, const struct sockaddr *src) { size_t gidx; struct in6_msource *ims; int mode; KASSERT(ifp != NULL, ("%s: null ifp", __func__)); gidx = im6o_match_group(imo, ifp, group); if (gidx == -1) return (MCAST_NOTGMEMBER); /* * Check if the source was included in an (S,G) join. * Allow reception on exclusive memberships by default, * reject reception on inclusive memberships by default. * Exclude source only if an in-mode exclude filter exists. * Include source only if an in-mode include filter exists. * NOTE: We are comparing group state here at MLD t1 (now) * with socket-layer t0 (since last downcall). */ mode = imo->im6o_mfilters[gidx].im6f_st[1]; ims = im6o_match_source(imo, gidx, src); if ((ims == NULL && mode == MCAST_INCLUDE) || (ims != NULL && ims->im6sl_st[0] != mode)) return (MCAST_NOTSMEMBER); return (MCAST_PASS); } /* * Find and return a reference to an in6_multi record for (ifp, group), * and bump its reference count. * If one does not exist, try to allocate it, and update link-layer multicast * filters on ifp to listen for group. * Assumes the IN6_MULTI lock is held across the call. * Return 0 if successful, otherwise return an appropriate error code. */ static int in6_mc_get(struct ifnet *ifp, const struct in6_addr *group, struct in6_multi **pinm) { struct sockaddr_in6 gsin6; struct ifmultiaddr *ifma; struct in6_multi *inm; int error; error = 0; /* * XXX: Accesses to ifma_protospec must be covered by IF_ADDR_LOCK; * if_addmulti() takes this mutex itself, so we must drop and * re-acquire around the call. */ IN6_MULTI_LOCK_ASSERT(); IF_ADDR_WLOCK(ifp); inm = in6m_lookup_locked(ifp, group); if (inm != NULL) { /* * If we already joined this group, just bump the * refcount and return it. */ KASSERT(inm->in6m_refcount >= 1, ("%s: bad refcount %d", __func__, inm->in6m_refcount)); ++inm->in6m_refcount; *pinm = inm; goto out_locked; } memset(&gsin6, 0, sizeof(gsin6)); gsin6.sin6_family = AF_INET6; gsin6.sin6_len = sizeof(struct sockaddr_in6); gsin6.sin6_addr = *group; /* * Check if a link-layer group is already associated * with this network-layer group on the given ifnet. */ IF_ADDR_WUNLOCK(ifp); error = if_addmulti(ifp, (struct sockaddr *)&gsin6, &ifma); if (error != 0) return (error); IF_ADDR_WLOCK(ifp); /* * If something other than netinet6 is occupying the link-layer * group, print a meaningful error message and back out of * the allocation. * Otherwise, bump the refcount on the existing network-layer * group association and return it. */ if (ifma->ifma_protospec != NULL) { inm = (struct in6_multi *)ifma->ifma_protospec; #ifdef INVARIANTS KASSERT(ifma->ifma_addr != NULL, ("%s: no ifma_addr", __func__)); KASSERT(ifma->ifma_addr->sa_family == AF_INET6, ("%s: ifma not AF_INET6", __func__)); KASSERT(inm != NULL, ("%s: no ifma_protospec", __func__)); if (inm->in6m_ifma != ifma || inm->in6m_ifp != ifp || !IN6_ARE_ADDR_EQUAL(&inm->in6m_addr, group)) panic("%s: ifma %p is inconsistent with %p (%p)", __func__, ifma, inm, group); #endif ++inm->in6m_refcount; *pinm = inm; goto out_locked; } IF_ADDR_WLOCK_ASSERT(ifp); /* * A new in6_multi record is needed; allocate and initialize it. * We DO NOT perform an MLD join as the in6_ layer may need to * push an initial source list down to MLD to support SSM. * * The initial source filter state is INCLUDE, {} as per the RFC. * Pending state-changes per group are subject to a bounds check. */ inm = malloc(sizeof(*inm), M_IP6MADDR, M_NOWAIT | M_ZERO); if (inm == NULL) { IF_ADDR_WUNLOCK(ifp); if_delmulti_ifma(ifma); return (ENOMEM); } inm->in6m_addr = *group; inm->in6m_ifp = ifp; inm->in6m_mli = MLD_IFINFO(ifp); inm->in6m_ifma = ifma; inm->in6m_refcount = 1; inm->in6m_state = MLD_NOT_MEMBER; mbufq_init(&inm->in6m_scq, MLD_MAX_STATE_CHANGES); inm->in6m_st[0].iss_fmode = MCAST_UNDEFINED; inm->in6m_st[1].iss_fmode = MCAST_UNDEFINED; RB_INIT(&inm->in6m_srcs); ifma->ifma_protospec = inm; *pinm = inm; out_locked: IF_ADDR_WUNLOCK(ifp); return (error); } /* * Drop a reference to an in6_multi record. * * If the refcount drops to 0, free the in6_multi record and * delete the underlying link-layer membership. */ void in6m_release_locked(struct in6_multi *inm) { struct ifmultiaddr *ifma; IN6_MULTI_LOCK_ASSERT(); CTR2(KTR_MLD, "%s: refcount is %d", __func__, inm->in6m_refcount); if (--inm->in6m_refcount > 0) { CTR2(KTR_MLD, "%s: refcount is now %d", __func__, inm->in6m_refcount); return; } CTR2(KTR_MLD, "%s: freeing inm %p", __func__, inm); ifma = inm->in6m_ifma; /* XXX this access is not covered by IF_ADDR_LOCK */ CTR2(KTR_MLD, "%s: purging ifma %p", __func__, ifma); KASSERT(ifma->ifma_protospec == inm, ("%s: ifma_protospec != inm", __func__)); ifma->ifma_protospec = NULL; in6m_purge(inm); free(inm, M_IP6MADDR); if_delmulti_ifma(ifma); } /* * Clear recorded source entries for a group. * Used by the MLD code. Caller must hold the IN6_MULTI lock. * FIXME: Should reap. */ void in6m_clear_recorded(struct in6_multi *inm) { struct ip6_msource *ims; IN6_MULTI_LOCK_ASSERT(); RB_FOREACH(ims, ip6_msource_tree, &inm->in6m_srcs) { if (ims->im6s_stp) { ims->im6s_stp = 0; --inm->in6m_st[1].iss_rec; } } KASSERT(inm->in6m_st[1].iss_rec == 0, ("%s: iss_rec %d not 0", __func__, inm->in6m_st[1].iss_rec)); } /* * Record a source as pending for a Source-Group MLDv2 query. * This lives here as it modifies the shared tree. * * inm is the group descriptor. * naddr is the address of the source to record in network-byte order. * * If the net.inet6.mld.sgalloc sysctl is non-zero, we will * lazy-allocate a source node in response to an SG query. * Otherwise, no allocation is performed. This saves some memory * with the trade-off that the source will not be reported to the * router if joined in the window between the query response and * the group actually being joined on the local host. * * VIMAGE: XXX: Currently the mld_sgalloc feature has been removed. * This turns off the allocation of a recorded source entry if * the group has not been joined. * * Return 0 if the source didn't exist or was already marked as recorded. * Return 1 if the source was marked as recorded by this function. * Return <0 if any error occured (negated errno code). */ int in6m_record_source(struct in6_multi *inm, const struct in6_addr *addr) { struct ip6_msource find; struct ip6_msource *ims, *nims; IN6_MULTI_LOCK_ASSERT(); find.im6s_addr = *addr; ims = RB_FIND(ip6_msource_tree, &inm->in6m_srcs, &find); if (ims && ims->im6s_stp) return (0); if (ims == NULL) { if (inm->in6m_nsrc == in6_mcast_maxgrpsrc) return (-ENOSPC); nims = malloc(sizeof(struct ip6_msource), M_IP6MSOURCE, M_NOWAIT | M_ZERO); if (nims == NULL) return (-ENOMEM); nims->im6s_addr = find.im6s_addr; RB_INSERT(ip6_msource_tree, &inm->in6m_srcs, nims); ++inm->in6m_nsrc; ims = nims; } /* * Mark the source as recorded and update the recorded * source count. */ ++ims->im6s_stp; ++inm->in6m_st[1].iss_rec; return (1); } /* * Return a pointer to an in6_msource owned by an in6_mfilter, * given its source address. * Lazy-allocate if needed. If this is a new entry its filter state is * undefined at t0. * * imf is the filter set being modified. * addr is the source address. * * SMPng: May be called with locks held; malloc must not block. */ static int im6f_get_source(struct in6_mfilter *imf, const struct sockaddr_in6 *psin, struct in6_msource **plims) { struct ip6_msource find; struct ip6_msource *ims, *nims; struct in6_msource *lims; int error; error = 0; ims = NULL; lims = NULL; find.im6s_addr = psin->sin6_addr; ims = RB_FIND(ip6_msource_tree, &imf->im6f_sources, &find); lims = (struct in6_msource *)ims; if (lims == NULL) { if (imf->im6f_nsrc == in6_mcast_maxsocksrc) return (ENOSPC); nims = malloc(sizeof(struct in6_msource), M_IN6MFILTER, M_NOWAIT | M_ZERO); if (nims == NULL) return (ENOMEM); lims = (struct in6_msource *)nims; lims->im6s_addr = find.im6s_addr; lims->im6sl_st[0] = MCAST_UNDEFINED; RB_INSERT(ip6_msource_tree, &imf->im6f_sources, nims); ++imf->im6f_nsrc; } *plims = lims; return (error); } /* * Graft a source entry into an existing socket-layer filter set, * maintaining any required invariants and checking allocations. * * The source is marked as being in the new filter mode at t1. * * Return the pointer to the new node, otherwise return NULL. */ static struct in6_msource * im6f_graft(struct in6_mfilter *imf, const uint8_t st1, const struct sockaddr_in6 *psin) { struct ip6_msource *nims; struct in6_msource *lims; nims = malloc(sizeof(struct in6_msource), M_IN6MFILTER, M_NOWAIT | M_ZERO); if (nims == NULL) return (NULL); lims = (struct in6_msource *)nims; lims->im6s_addr = psin->sin6_addr; lims->im6sl_st[0] = MCAST_UNDEFINED; lims->im6sl_st[1] = st1; RB_INSERT(ip6_msource_tree, &imf->im6f_sources, nims); ++imf->im6f_nsrc; return (lims); } /* * Prune a source entry from an existing socket-layer filter set, * maintaining any required invariants and checking allocations. * * The source is marked as being left at t1, it is not freed. * * Return 0 if no error occurred, otherwise return an errno value. */ static int im6f_prune(struct in6_mfilter *imf, const struct sockaddr_in6 *psin) { struct ip6_msource find; struct ip6_msource *ims; struct in6_msource *lims; find.im6s_addr = psin->sin6_addr; ims = RB_FIND(ip6_msource_tree, &imf->im6f_sources, &find); if (ims == NULL) return (ENOENT); lims = (struct in6_msource *)ims; lims->im6sl_st[1] = MCAST_UNDEFINED; return (0); } /* * Revert socket-layer filter set deltas at t1 to t0 state. */ static void im6f_rollback(struct in6_mfilter *imf) { struct ip6_msource *ims, *tims; struct in6_msource *lims; RB_FOREACH_SAFE(ims, ip6_msource_tree, &imf->im6f_sources, tims) { lims = (struct in6_msource *)ims; if (lims->im6sl_st[0] == lims->im6sl_st[1]) { /* no change at t1 */ continue; } else if (lims->im6sl_st[0] != MCAST_UNDEFINED) { /* revert change to existing source at t1 */ lims->im6sl_st[1] = lims->im6sl_st[0]; } else { /* revert source added t1 */ CTR2(KTR_MLD, "%s: free ims %p", __func__, ims); RB_REMOVE(ip6_msource_tree, &imf->im6f_sources, ims); free(ims, M_IN6MFILTER); imf->im6f_nsrc--; } } imf->im6f_st[1] = imf->im6f_st[0]; } /* * Mark socket-layer filter set as INCLUDE {} at t1. */ static void im6f_leave(struct in6_mfilter *imf) { struct ip6_msource *ims; struct in6_msource *lims; RB_FOREACH(ims, ip6_msource_tree, &imf->im6f_sources) { lims = (struct in6_msource *)ims; lims->im6sl_st[1] = MCAST_UNDEFINED; } imf->im6f_st[1] = MCAST_INCLUDE; } /* * Mark socket-layer filter set deltas as committed. */ static void im6f_commit(struct in6_mfilter *imf) { struct ip6_msource *ims; struct in6_msource *lims; RB_FOREACH(ims, ip6_msource_tree, &imf->im6f_sources) { lims = (struct in6_msource *)ims; lims->im6sl_st[0] = lims->im6sl_st[1]; } imf->im6f_st[0] = imf->im6f_st[1]; } /* * Reap unreferenced sources from socket-layer filter set. */ static void im6f_reap(struct in6_mfilter *imf) { struct ip6_msource *ims, *tims; struct in6_msource *lims; RB_FOREACH_SAFE(ims, ip6_msource_tree, &imf->im6f_sources, tims) { lims = (struct in6_msource *)ims; if ((lims->im6sl_st[0] == MCAST_UNDEFINED) && (lims->im6sl_st[1] == MCAST_UNDEFINED)) { CTR2(KTR_MLD, "%s: free lims %p", __func__, ims); RB_REMOVE(ip6_msource_tree, &imf->im6f_sources, ims); free(ims, M_IN6MFILTER); imf->im6f_nsrc--; } } } /* * Purge socket-layer filter set. */ static void im6f_purge(struct in6_mfilter *imf) { struct ip6_msource *ims, *tims; RB_FOREACH_SAFE(ims, ip6_msource_tree, &imf->im6f_sources, tims) { CTR2(KTR_MLD, "%s: free ims %p", __func__, ims); RB_REMOVE(ip6_msource_tree, &imf->im6f_sources, ims); free(ims, M_IN6MFILTER); imf->im6f_nsrc--; } imf->im6f_st[0] = imf->im6f_st[1] = MCAST_UNDEFINED; KASSERT(RB_EMPTY(&imf->im6f_sources), ("%s: im6f_sources not empty", __func__)); } /* * Look up a source filter entry for a multicast group. * * inm is the group descriptor to work with. * addr is the IPv6 address to look up. * noalloc may be non-zero to suppress allocation of sources. * *pims will be set to the address of the retrieved or allocated source. * * SMPng: NOTE: may be called with locks held. * Return 0 if successful, otherwise return a non-zero error code. */ static int in6m_get_source(struct in6_multi *inm, const struct in6_addr *addr, const int noalloc, struct ip6_msource **pims) { struct ip6_msource find; struct ip6_msource *ims, *nims; #ifdef KTR char ip6tbuf[INET6_ADDRSTRLEN]; #endif find.im6s_addr = *addr; ims = RB_FIND(ip6_msource_tree, &inm->in6m_srcs, &find); if (ims == NULL && !noalloc) { if (inm->in6m_nsrc == in6_mcast_maxgrpsrc) return (ENOSPC); nims = malloc(sizeof(struct ip6_msource), M_IP6MSOURCE, M_NOWAIT | M_ZERO); if (nims == NULL) return (ENOMEM); nims->im6s_addr = *addr; RB_INSERT(ip6_msource_tree, &inm->in6m_srcs, nims); ++inm->in6m_nsrc; ims = nims; CTR3(KTR_MLD, "%s: allocated %s as %p", __func__, ip6_sprintf(ip6tbuf, addr), ims); } *pims = ims; return (0); } /* * Merge socket-layer source into MLD-layer source. * If rollback is non-zero, perform the inverse of the merge. */ static void im6s_merge(struct ip6_msource *ims, const struct in6_msource *lims, const int rollback) { int n = rollback ? -1 : 1; #ifdef KTR char ip6tbuf[INET6_ADDRSTRLEN]; ip6_sprintf(ip6tbuf, &lims->im6s_addr); #endif if (lims->im6sl_st[0] == MCAST_EXCLUDE) { CTR3(KTR_MLD, "%s: t1 ex -= %d on %s", __func__, n, ip6tbuf); ims->im6s_st[1].ex -= n; } else if (lims->im6sl_st[0] == MCAST_INCLUDE) { CTR3(KTR_MLD, "%s: t1 in -= %d on %s", __func__, n, ip6tbuf); ims->im6s_st[1].in -= n; } if (lims->im6sl_st[1] == MCAST_EXCLUDE) { CTR3(KTR_MLD, "%s: t1 ex += %d on %s", __func__, n, ip6tbuf); ims->im6s_st[1].ex += n; } else if (lims->im6sl_st[1] == MCAST_INCLUDE) { CTR3(KTR_MLD, "%s: t1 in += %d on %s", __func__, n, ip6tbuf); ims->im6s_st[1].in += n; } } /* * Atomically update the global in6_multi state, when a membership's * filter list is being updated in any way. * * imf is the per-inpcb-membership group filter pointer. * A fake imf may be passed for in-kernel consumers. * * XXX This is a candidate for a set-symmetric-difference style loop * which would eliminate the repeated lookup from root of ims nodes, * as they share the same key space. * * If any error occurred this function will back out of refcounts * and return a non-zero value. */ static int in6m_merge(struct in6_multi *inm, /*const*/ struct in6_mfilter *imf) { struct ip6_msource *ims, *nims; struct in6_msource *lims; int schanged, error; int nsrc0, nsrc1; schanged = 0; error = 0; nsrc1 = nsrc0 = 0; /* * Update the source filters first, as this may fail. * Maintain count of in-mode filters at t0, t1. These are * used to work out if we transition into ASM mode or not. * Maintain a count of source filters whose state was * actually modified by this operation. */ RB_FOREACH(ims, ip6_msource_tree, &imf->im6f_sources) { lims = (struct in6_msource *)ims; if (lims->im6sl_st[0] == imf->im6f_st[0]) nsrc0++; if (lims->im6sl_st[1] == imf->im6f_st[1]) nsrc1++; if (lims->im6sl_st[0] == lims->im6sl_st[1]) continue; error = in6m_get_source(inm, &lims->im6s_addr, 0, &nims); ++schanged; if (error) break; im6s_merge(nims, lims, 0); } if (error) { struct ip6_msource *bims; RB_FOREACH_REVERSE_FROM(ims, ip6_msource_tree, nims) { lims = (struct in6_msource *)ims; if (lims->im6sl_st[0] == lims->im6sl_st[1]) continue; (void)in6m_get_source(inm, &lims->im6s_addr, 1, &bims); if (bims == NULL) continue; im6s_merge(bims, lims, 1); } goto out_reap; } CTR3(KTR_MLD, "%s: imf filters in-mode: %d at t0, %d at t1", __func__, nsrc0, nsrc1); /* Handle transition between INCLUDE {n} and INCLUDE {} on socket. */ if (imf->im6f_st[0] == imf->im6f_st[1] && imf->im6f_st[1] == MCAST_INCLUDE) { if (nsrc1 == 0) { CTR1(KTR_MLD, "%s: --in on inm at t1", __func__); --inm->in6m_st[1].iss_in; } } /* Handle filter mode transition on socket. */ if (imf->im6f_st[0] != imf->im6f_st[1]) { CTR3(KTR_MLD, "%s: imf transition %d to %d", __func__, imf->im6f_st[0], imf->im6f_st[1]); if (imf->im6f_st[0] == MCAST_EXCLUDE) { CTR1(KTR_MLD, "%s: --ex on inm at t1", __func__); --inm->in6m_st[1].iss_ex; } else if (imf->im6f_st[0] == MCAST_INCLUDE) { CTR1(KTR_MLD, "%s: --in on inm at t1", __func__); --inm->in6m_st[1].iss_in; } if (imf->im6f_st[1] == MCAST_EXCLUDE) { CTR1(KTR_MLD, "%s: ex++ on inm at t1", __func__); inm->in6m_st[1].iss_ex++; } else if (imf->im6f_st[1] == MCAST_INCLUDE && nsrc1 > 0) { CTR1(KTR_MLD, "%s: in++ on inm at t1", __func__); inm->in6m_st[1].iss_in++; } } /* * Track inm filter state in terms of listener counts. * If there are any exclusive listeners, stack-wide * membership is exclusive. * Otherwise, if only inclusive listeners, stack-wide is inclusive. * If no listeners remain, state is undefined at t1, * and the MLD lifecycle for this group should finish. */ if (inm->in6m_st[1].iss_ex > 0) { CTR1(KTR_MLD, "%s: transition to EX", __func__); inm->in6m_st[1].iss_fmode = MCAST_EXCLUDE; } else if (inm->in6m_st[1].iss_in > 0) { CTR1(KTR_MLD, "%s: transition to IN", __func__); inm->in6m_st[1].iss_fmode = MCAST_INCLUDE; } else { CTR1(KTR_MLD, "%s: transition to UNDEF", __func__); inm->in6m_st[1].iss_fmode = MCAST_UNDEFINED; } /* Decrement ASM listener count on transition out of ASM mode. */ if (imf->im6f_st[0] == MCAST_EXCLUDE && nsrc0 == 0) { if ((imf->im6f_st[1] != MCAST_EXCLUDE) || (imf->im6f_st[1] == MCAST_EXCLUDE && nsrc1 > 0)) CTR1(KTR_MLD, "%s: --asm on inm at t1", __func__); --inm->in6m_st[1].iss_asm; } /* Increment ASM listener count on transition to ASM mode. */ if (imf->im6f_st[1] == MCAST_EXCLUDE && nsrc1 == 0) { CTR1(KTR_MLD, "%s: asm++ on inm at t1", __func__); inm->in6m_st[1].iss_asm++; } CTR3(KTR_MLD, "%s: merged imf %p to inm %p", __func__, imf, inm); in6m_print(inm); out_reap: if (schanged > 0) { CTR1(KTR_MLD, "%s: sources changed; reaping", __func__); in6m_reap(inm); } return (error); } /* * Mark an in6_multi's filter set deltas as committed. * Called by MLD after a state change has been enqueued. */ void in6m_commit(struct in6_multi *inm) { struct ip6_msource *ims; CTR2(KTR_MLD, "%s: commit inm %p", __func__, inm); CTR1(KTR_MLD, "%s: pre commit:", __func__); in6m_print(inm); RB_FOREACH(ims, ip6_msource_tree, &inm->in6m_srcs) { ims->im6s_st[0] = ims->im6s_st[1]; } inm->in6m_st[0] = inm->in6m_st[1]; } /* * Reap unreferenced nodes from an in6_multi's filter set. */ static void in6m_reap(struct in6_multi *inm) { struct ip6_msource *ims, *tims; RB_FOREACH_SAFE(ims, ip6_msource_tree, &inm->in6m_srcs, tims) { if (ims->im6s_st[0].ex > 0 || ims->im6s_st[0].in > 0 || ims->im6s_st[1].ex > 0 || ims->im6s_st[1].in > 0 || ims->im6s_stp != 0) continue; CTR2(KTR_MLD, "%s: free ims %p", __func__, ims); RB_REMOVE(ip6_msource_tree, &inm->in6m_srcs, ims); free(ims, M_IP6MSOURCE); inm->in6m_nsrc--; } } /* * Purge all source nodes from an in6_multi's filter set. */ static void in6m_purge(struct in6_multi *inm) { struct ip6_msource *ims, *tims; RB_FOREACH_SAFE(ims, ip6_msource_tree, &inm->in6m_srcs, tims) { CTR2(KTR_MLD, "%s: free ims %p", __func__, ims); RB_REMOVE(ip6_msource_tree, &inm->in6m_srcs, ims); free(ims, M_IP6MSOURCE); inm->in6m_nsrc--; } /* Free state-change requests that might be queued. */ mbufq_drain(&inm->in6m_scq); } /* * Join a multicast address w/o sources. * KAME compatibility entry point. * * SMPng: Assume no mc locks held by caller. */ struct in6_multi_mship * in6_joingroup(struct ifnet *ifp, struct in6_addr *mcaddr, int *errorp, int delay) { struct in6_multi_mship *imm; int error; imm = malloc(sizeof(*imm), M_IP6MADDR, M_NOWAIT); if (imm == NULL) { *errorp = ENOBUFS; return (NULL); } delay = (delay * PR_FASTHZ) / hz; error = in6_mc_join(ifp, mcaddr, NULL, &imm->i6mm_maddr, delay); if (error) { *errorp = error; free(imm, M_IP6MADDR); return (NULL); } return (imm); } /* * Leave a multicast address w/o sources. * KAME compatibility entry point. * * SMPng: Assume no mc locks held by caller. */ int in6_leavegroup(struct in6_multi_mship *imm) { if (imm->i6mm_maddr != NULL) in6_mc_leave(imm->i6mm_maddr, NULL); free(imm, M_IP6MADDR); return 0; } /* * Join a multicast group; unlocked entry point. * * SMPng: XXX: in6_mc_join() is called from in6_control() when upper * locks are not held. Fortunately, ifp is unlikely to have been detached * at this point, so we assume it's OK to recurse. */ int in6_mc_join(struct ifnet *ifp, const struct in6_addr *mcaddr, /*const*/ struct in6_mfilter *imf, struct in6_multi **pinm, const int delay) { int error; IN6_MULTI_LOCK(); error = in6_mc_join_locked(ifp, mcaddr, imf, pinm, delay); IN6_MULTI_UNLOCK(); return (error); } /* * Join a multicast group; real entry point. * * Only preserves atomicity at inm level. * NOTE: imf argument cannot be const due to sys/tree.h limitations. * * If the MLD downcall fails, the group is not joined, and an error * code is returned. */ int in6_mc_join_locked(struct ifnet *ifp, const struct in6_addr *mcaddr, /*const*/ struct in6_mfilter *imf, struct in6_multi **pinm, const int delay) { struct in6_mfilter timf; struct in6_multi *inm; int error; #ifdef KTR char ip6tbuf[INET6_ADDRSTRLEN]; #endif #ifdef INVARIANTS /* * Sanity: Check scope zone ID was set for ifp, if and * only if group is scoped to an interface. */ KASSERT(IN6_IS_ADDR_MULTICAST(mcaddr), ("%s: not a multicast address", __func__)); if (IN6_IS_ADDR_MC_LINKLOCAL(mcaddr) || IN6_IS_ADDR_MC_INTFACELOCAL(mcaddr)) { KASSERT(mcaddr->s6_addr16[1] != 0, ("%s: scope zone ID not set", __func__)); } #endif IN6_MULTI_LOCK_ASSERT(); CTR4(KTR_MLD, "%s: join %s on %p(%s))", __func__, ip6_sprintf(ip6tbuf, mcaddr), ifp, if_name(ifp)); error = 0; inm = NULL; /* * If no imf was specified (i.e. kernel consumer), * fake one up and assume it is an ASM join. */ if (imf == NULL) { im6f_init(&timf, MCAST_UNDEFINED, MCAST_EXCLUDE); imf = &timf; } error = in6_mc_get(ifp, mcaddr, &inm); if (error) { CTR1(KTR_MLD, "%s: in6_mc_get() failure", __func__); return (error); } CTR1(KTR_MLD, "%s: merge inm state", __func__); error = in6m_merge(inm, imf); if (error) { CTR1(KTR_MLD, "%s: failed to merge inm state", __func__); goto out_in6m_release; } CTR1(KTR_MLD, "%s: doing mld downcall", __func__); error = mld_change_state(inm, delay); if (error) { CTR1(KTR_MLD, "%s: failed to update source", __func__); goto out_in6m_release; } out_in6m_release: if (error) { CTR2(KTR_MLD, "%s: dropping ref on %p", __func__, inm); in6m_release_locked(inm); } else { *pinm = inm; } return (error); } /* * Leave a multicast group; unlocked entry point. */ int in6_mc_leave(struct in6_multi *inm, /*const*/ struct in6_mfilter *imf) { struct ifnet *ifp; int error; ifp = inm->in6m_ifp; IN6_MULTI_LOCK(); error = in6_mc_leave_locked(inm, imf); IN6_MULTI_UNLOCK(); return (error); } /* * Leave a multicast group; real entry point. * All source filters will be expunged. * * Only preserves atomicity at inm level. * * Holding the write lock for the INP which contains imf * is highly advisable. We can't assert for it as imf does not * contain a back-pointer to the owning inp. * * Note: This is not the same as in6m_release(*) as this function also * makes a state change downcall into MLD. */ int in6_mc_leave_locked(struct in6_multi *inm, /*const*/ struct in6_mfilter *imf) { struct in6_mfilter timf; int error; #ifdef KTR char ip6tbuf[INET6_ADDRSTRLEN]; #endif error = 0; IN6_MULTI_LOCK_ASSERT(); CTR5(KTR_MLD, "%s: leave inm %p, %s/%s, imf %p", __func__, inm, ip6_sprintf(ip6tbuf, &inm->in6m_addr), (in6m_is_ifp_detached(inm) ? "null" : if_name(inm->in6m_ifp)), imf); /* * If no imf was specified (i.e. kernel consumer), * fake one up and assume it is an ASM join. */ if (imf == NULL) { im6f_init(&timf, MCAST_EXCLUDE, MCAST_UNDEFINED); imf = &timf; } /* * Begin state merge transaction at MLD layer. * * As this particular invocation should not cause any memory * to be allocated, and there is no opportunity to roll back * the transaction, it MUST NOT fail. */ CTR1(KTR_MLD, "%s: merge inm state", __func__); error = in6m_merge(inm, imf); KASSERT(error == 0, ("%s: failed to merge inm state", __func__)); CTR1(KTR_MLD, "%s: doing mld downcall", __func__); error = mld_change_state(inm, 0); if (error) CTR1(KTR_MLD, "%s: failed mld downcall", __func__); CTR2(KTR_MLD, "%s: dropping ref on %p", __func__, inm); in6m_release_locked(inm); return (error); } /* * Block or unblock an ASM multicast source on an inpcb. * This implements the delta-based API described in RFC 3678. * * The delta-based API applies only to exclusive-mode memberships. * An MLD downcall will be performed. * * SMPng: NOTE: Must take Giant as a join may create a new ifma. * * Return 0 if successful, otherwise return an appropriate error code. */ static int in6p_block_unblock_source(struct inpcb *inp, struct sockopt *sopt) { struct group_source_req gsr; sockunion_t *gsa, *ssa; struct ifnet *ifp; struct in6_mfilter *imf; struct ip6_moptions *imo; struct in6_msource *ims; struct in6_multi *inm; size_t idx; uint16_t fmode; int error, doblock; #ifdef KTR char ip6tbuf[INET6_ADDRSTRLEN]; #endif ifp = NULL; error = 0; doblock = 0; memset(&gsr, 0, sizeof(struct group_source_req)); gsa = (sockunion_t *)&gsr.gsr_group; ssa = (sockunion_t *)&gsr.gsr_source; switch (sopt->sopt_name) { case MCAST_BLOCK_SOURCE: case MCAST_UNBLOCK_SOURCE: error = sooptcopyin(sopt, &gsr, sizeof(struct group_source_req), sizeof(struct group_source_req)); if (error) return (error); if (gsa->sin6.sin6_family != AF_INET6 || gsa->sin6.sin6_len != sizeof(struct sockaddr_in6)) return (EINVAL); if (ssa->sin6.sin6_family != AF_INET6 || ssa->sin6.sin6_len != sizeof(struct sockaddr_in6)) return (EINVAL); if (gsr.gsr_interface == 0 || V_if_index < gsr.gsr_interface) return (EADDRNOTAVAIL); ifp = ifnet_byindex(gsr.gsr_interface); if (sopt->sopt_name == MCAST_BLOCK_SOURCE) doblock = 1; break; default: CTR2(KTR_MLD, "%s: unknown sopt_name %d", __func__, sopt->sopt_name); return (EOPNOTSUPP); break; } if (!IN6_IS_ADDR_MULTICAST(&gsa->sin6.sin6_addr)) return (EINVAL); (void)in6_setscope(&gsa->sin6.sin6_addr, ifp, NULL); /* * Check if we are actually a member of this group. */ imo = in6p_findmoptions(inp); idx = im6o_match_group(imo, ifp, &gsa->sa); if (idx == -1 || imo->im6o_mfilters == NULL) { error = EADDRNOTAVAIL; goto out_in6p_locked; } KASSERT(imo->im6o_mfilters != NULL, ("%s: im6o_mfilters not allocated", __func__)); imf = &imo->im6o_mfilters[idx]; inm = imo->im6o_membership[idx]; /* * Attempting to use the delta-based API on an * non exclusive-mode membership is an error. */ fmode = imf->im6f_st[0]; if (fmode != MCAST_EXCLUDE) { error = EINVAL; goto out_in6p_locked; } /* * Deal with error cases up-front: * Asked to block, but already blocked; or * Asked to unblock, but nothing to unblock. * If adding a new block entry, allocate it. */ ims = im6o_match_source(imo, idx, &ssa->sa); if ((ims != NULL && doblock) || (ims == NULL && !doblock)) { CTR3(KTR_MLD, "%s: source %s %spresent", __func__, ip6_sprintf(ip6tbuf, &ssa->sin6.sin6_addr), doblock ? "" : "not "); error = EADDRNOTAVAIL; goto out_in6p_locked; } INP_WLOCK_ASSERT(inp); /* * Begin state merge transaction at socket layer. */ if (doblock) { CTR2(KTR_MLD, "%s: %s source", __func__, "block"); ims = im6f_graft(imf, fmode, &ssa->sin6); if (ims == NULL) error = ENOMEM; } else { CTR2(KTR_MLD, "%s: %s source", __func__, "allow"); error = im6f_prune(imf, &ssa->sin6); } if (error) { CTR1(KTR_MLD, "%s: merge imf state failed", __func__); goto out_im6f_rollback; } /* * Begin state merge transaction at MLD layer. */ IN6_MULTI_LOCK(); CTR1(KTR_MLD, "%s: merge inm state", __func__); error = in6m_merge(inm, imf); if (error) CTR1(KTR_MLD, "%s: failed to merge inm state", __func__); else { CTR1(KTR_MLD, "%s: doing mld downcall", __func__); error = mld_change_state(inm, 0); if (error) CTR1(KTR_MLD, "%s: failed mld downcall", __func__); } IN6_MULTI_UNLOCK(); out_im6f_rollback: if (error) im6f_rollback(imf); else im6f_commit(imf); im6f_reap(imf); out_in6p_locked: INP_WUNLOCK(inp); return (error); } /* * Given an inpcb, return its multicast options structure pointer. Accepts * an unlocked inpcb pointer, but will return it locked. May sleep. * * SMPng: NOTE: Potentially calls malloc(M_WAITOK) with Giant held. * SMPng: NOTE: Returns with the INP write lock held. */ static struct ip6_moptions * in6p_findmoptions(struct inpcb *inp) { struct ip6_moptions *imo; struct in6_multi **immp; struct in6_mfilter *imfp; size_t idx; INP_WLOCK(inp); if (inp->in6p_moptions != NULL) return (inp->in6p_moptions); INP_WUNLOCK(inp); imo = malloc(sizeof(*imo), M_IP6MOPTS, M_WAITOK); immp = malloc(sizeof(*immp) * IPV6_MIN_MEMBERSHIPS, M_IP6MOPTS, M_WAITOK | M_ZERO); imfp = malloc(sizeof(struct in6_mfilter) * IPV6_MIN_MEMBERSHIPS, M_IN6MFILTER, M_WAITOK); imo->im6o_multicast_ifp = NULL; imo->im6o_multicast_hlim = V_ip6_defmcasthlim; imo->im6o_multicast_loop = in6_mcast_loop; imo->im6o_num_memberships = 0; imo->im6o_max_memberships = IPV6_MIN_MEMBERSHIPS; imo->im6o_membership = immp; /* Initialize per-group source filters. */ for (idx = 0; idx < IPV6_MIN_MEMBERSHIPS; idx++) im6f_init(&imfp[idx], MCAST_UNDEFINED, MCAST_EXCLUDE); imo->im6o_mfilters = imfp; INP_WLOCK(inp); if (inp->in6p_moptions != NULL) { free(imfp, M_IN6MFILTER); free(immp, M_IP6MOPTS); free(imo, M_IP6MOPTS); return (inp->in6p_moptions); } inp->in6p_moptions = imo; return (imo); } /* * Discard the IPv6 multicast options (and source filters). * * SMPng: NOTE: assumes INP write lock is held. */ void ip6_freemoptions(struct ip6_moptions *imo) { struct in6_mfilter *imf; size_t idx, nmships; KASSERT(imo != NULL, ("%s: ip6_moptions is NULL", __func__)); nmships = imo->im6o_num_memberships; for (idx = 0; idx < nmships; ++idx) { imf = imo->im6o_mfilters ? &imo->im6o_mfilters[idx] : NULL; if (imf) im6f_leave(imf); /* XXX this will thrash the lock(s) */ (void)in6_mc_leave(imo->im6o_membership[idx], imf); if (imf) im6f_purge(imf); } if (imo->im6o_mfilters) free(imo->im6o_mfilters, M_IN6MFILTER); free(imo->im6o_membership, M_IP6MOPTS); free(imo, M_IP6MOPTS); } /* * Atomically get source filters on a socket for an IPv6 multicast group. * Called with INP lock held; returns with lock released. */ static int in6p_get_source_filters(struct inpcb *inp, struct sockopt *sopt) { struct __msfilterreq msfr; sockunion_t *gsa; struct ifnet *ifp; struct ip6_moptions *imo; struct in6_mfilter *imf; struct ip6_msource *ims; struct in6_msource *lims; struct sockaddr_in6 *psin; struct sockaddr_storage *ptss; struct sockaddr_storage *tss; int error; size_t idx, nsrcs, ncsrcs; INP_WLOCK_ASSERT(inp); imo = inp->in6p_moptions; KASSERT(imo != NULL, ("%s: null ip6_moptions", __func__)); INP_WUNLOCK(inp); error = sooptcopyin(sopt, &msfr, sizeof(struct __msfilterreq), sizeof(struct __msfilterreq)); if (error) return (error); if (msfr.msfr_group.ss_family != AF_INET6 || msfr.msfr_group.ss_len != sizeof(struct sockaddr_in6)) return (EINVAL); gsa = (sockunion_t *)&msfr.msfr_group; if (!IN6_IS_ADDR_MULTICAST(&gsa->sin6.sin6_addr)) return (EINVAL); if (msfr.msfr_ifindex == 0 || V_if_index < msfr.msfr_ifindex) return (EADDRNOTAVAIL); ifp = ifnet_byindex(msfr.msfr_ifindex); if (ifp == NULL) return (EADDRNOTAVAIL); (void)in6_setscope(&gsa->sin6.sin6_addr, ifp, NULL); INP_WLOCK(inp); /* * Lookup group on the socket. */ idx = im6o_match_group(imo, ifp, &gsa->sa); if (idx == -1 || imo->im6o_mfilters == NULL) { INP_WUNLOCK(inp); return (EADDRNOTAVAIL); } imf = &imo->im6o_mfilters[idx]; /* * Ignore memberships which are in limbo. */ if (imf->im6f_st[1] == MCAST_UNDEFINED) { INP_WUNLOCK(inp); return (EAGAIN); } msfr.msfr_fmode = imf->im6f_st[1]; /* * If the user specified a buffer, copy out the source filter * entries to userland gracefully. * We only copy out the number of entries which userland * has asked for, but we always tell userland how big the * buffer really needs to be. */ if (msfr.msfr_nsrcs > in6_mcast_maxsocksrc) msfr.msfr_nsrcs = in6_mcast_maxsocksrc; tss = NULL; if (msfr.msfr_srcs != NULL && msfr.msfr_nsrcs > 0) { tss = malloc(sizeof(struct sockaddr_storage) * msfr.msfr_nsrcs, M_TEMP, M_NOWAIT | M_ZERO); if (tss == NULL) { INP_WUNLOCK(inp); return (ENOBUFS); } } /* * Count number of sources in-mode at t0. * If buffer space exists and remains, copy out source entries. */ nsrcs = msfr.msfr_nsrcs; ncsrcs = 0; ptss = tss; RB_FOREACH(ims, ip6_msource_tree, &imf->im6f_sources) { lims = (struct in6_msource *)ims; if (lims->im6sl_st[0] == MCAST_UNDEFINED || lims->im6sl_st[0] != imf->im6f_st[0]) continue; ++ncsrcs; if (tss != NULL && nsrcs > 0) { psin = (struct sockaddr_in6 *)ptss; psin->sin6_family = AF_INET6; psin->sin6_len = sizeof(struct sockaddr_in6); psin->sin6_addr = lims->im6s_addr; psin->sin6_port = 0; --nsrcs; ++ptss; } } INP_WUNLOCK(inp); if (tss != NULL) { error = copyout(tss, msfr.msfr_srcs, sizeof(struct sockaddr_storage) * msfr.msfr_nsrcs); free(tss, M_TEMP); if (error) return (error); } msfr.msfr_nsrcs = ncsrcs; error = sooptcopyout(sopt, &msfr, sizeof(struct __msfilterreq)); return (error); } /* * Return the IP multicast options in response to user getsockopt(). */ int ip6_getmoptions(struct inpcb *inp, struct sockopt *sopt) { struct ip6_moptions *im6o; int error; u_int optval; INP_WLOCK(inp); im6o = inp->in6p_moptions; /* * If socket is neither of type SOCK_RAW or SOCK_DGRAM, * or is a divert socket, reject it. */ if (inp->inp_socket->so_proto->pr_protocol == IPPROTO_DIVERT || (inp->inp_socket->so_proto->pr_type != SOCK_RAW && inp->inp_socket->so_proto->pr_type != SOCK_DGRAM)) { INP_WUNLOCK(inp); return (EOPNOTSUPP); } error = 0; switch (sopt->sopt_name) { case IPV6_MULTICAST_IF: if (im6o == NULL || im6o->im6o_multicast_ifp == NULL) { optval = 0; } else { optval = im6o->im6o_multicast_ifp->if_index; } INP_WUNLOCK(inp); error = sooptcopyout(sopt, &optval, sizeof(u_int)); break; case IPV6_MULTICAST_HOPS: if (im6o == NULL) optval = V_ip6_defmcasthlim; else optval = im6o->im6o_multicast_hlim; INP_WUNLOCK(inp); error = sooptcopyout(sopt, &optval, sizeof(u_int)); break; case IPV6_MULTICAST_LOOP: if (im6o == NULL) optval = in6_mcast_loop; /* XXX VIMAGE */ else optval = im6o->im6o_multicast_loop; INP_WUNLOCK(inp); error = sooptcopyout(sopt, &optval, sizeof(u_int)); break; case IPV6_MSFILTER: if (im6o == NULL) { error = EADDRNOTAVAIL; INP_WUNLOCK(inp); } else { error = in6p_get_source_filters(inp, sopt); } break; default: INP_WUNLOCK(inp); error = ENOPROTOOPT; break; } INP_UNLOCK_ASSERT(inp); return (error); } /* * Look up the ifnet to use for a multicast group membership, * given the address of an IPv6 group. * * This routine exists to support legacy IPv6 multicast applications. * * If inp is non-NULL, use this socket's current FIB number for any * required FIB lookup. Look up the group address in the unicast FIB, * and use its ifp; usually, this points to the default next-hop. * If the FIB lookup fails, return NULL. * * FUTURE: Support multiple forwarding tables for IPv6. * * Returns NULL if no ifp could be found. */ static struct ifnet * in6p_lookup_mcast_ifp(const struct inpcb *in6p, const struct sockaddr_in6 *gsin6) { - struct route_in6 ro6; - struct ifnet *ifp; + struct nhop6_basic nh6; + struct in6_addr dst; + uint32_t scopeid; + uint32_t fibnum; KASSERT(in6p->inp_vflag & INP_IPV6, ("%s: not INP_IPV6 inpcb", __func__)); KASSERT(gsin6->sin6_family == AF_INET6, ("%s: not AF_INET6 group", __func__)); - ifp = NULL; - memset(&ro6, 0, sizeof(struct route_in6)); - memcpy(&ro6.ro_dst, gsin6, sizeof(struct sockaddr_in6)); - rtalloc_ign_fib((struct route *)&ro6, 0, - in6p ? in6p->inp_inc.inc_fibnum : RT_DEFAULT_FIB); - if (ro6.ro_rt != NULL) { - ifp = ro6.ro_rt->rt_ifp; - KASSERT(ifp != NULL, ("%s: null ifp", __func__)); - RTFREE(ro6.ro_rt); - } + in6_splitscope(&gsin6->sin6_addr, &dst, &scopeid); + fibnum = in6p ? in6p->inp_inc.inc_fibnum : RT_DEFAULT_FIB; + if (fib6_lookup_nh_basic(fibnum, &dst, scopeid, 0, 0, &nh6) != 0) + return (NULL); - return (ifp); + return (nh6.nh_ifp); } /* * Join an IPv6 multicast group, possibly with a source. * * FIXME: The KAME use of the unspecified address (::) * to join *all* multicast groups is currently unsupported. */ static int in6p_join_group(struct inpcb *inp, struct sockopt *sopt) { struct group_source_req gsr; sockunion_t *gsa, *ssa; struct ifnet *ifp; struct in6_mfilter *imf; struct ip6_moptions *imo; struct in6_multi *inm; struct in6_msource *lims; size_t idx; int error, is_new; ifp = NULL; imf = NULL; lims = NULL; error = 0; is_new = 0; memset(&gsr, 0, sizeof(struct group_source_req)); gsa = (sockunion_t *)&gsr.gsr_group; gsa->ss.ss_family = AF_UNSPEC; ssa = (sockunion_t *)&gsr.gsr_source; ssa->ss.ss_family = AF_UNSPEC; /* * Chew everything into struct group_source_req. * Overwrite the port field if present, as the sockaddr * being copied in may be matched with a binary comparison. * Ignore passed-in scope ID. */ switch (sopt->sopt_name) { case IPV6_JOIN_GROUP: { struct ipv6_mreq mreq; error = sooptcopyin(sopt, &mreq, sizeof(struct ipv6_mreq), sizeof(struct ipv6_mreq)); if (error) return (error); gsa->sin6.sin6_family = AF_INET6; gsa->sin6.sin6_len = sizeof(struct sockaddr_in6); gsa->sin6.sin6_addr = mreq.ipv6mr_multiaddr; if (mreq.ipv6mr_interface == 0) { ifp = in6p_lookup_mcast_ifp(inp, &gsa->sin6); } else { if (V_if_index < mreq.ipv6mr_interface) return (EADDRNOTAVAIL); ifp = ifnet_byindex(mreq.ipv6mr_interface); } CTR3(KTR_MLD, "%s: ipv6mr_interface = %d, ifp = %p", __func__, mreq.ipv6mr_interface, ifp); } break; case MCAST_JOIN_GROUP: case MCAST_JOIN_SOURCE_GROUP: if (sopt->sopt_name == MCAST_JOIN_GROUP) { error = sooptcopyin(sopt, &gsr, sizeof(struct group_req), sizeof(struct group_req)); } else if (sopt->sopt_name == MCAST_JOIN_SOURCE_GROUP) { error = sooptcopyin(sopt, &gsr, sizeof(struct group_source_req), sizeof(struct group_source_req)); } if (error) return (error); if (gsa->sin6.sin6_family != AF_INET6 || gsa->sin6.sin6_len != sizeof(struct sockaddr_in6)) return (EINVAL); if (sopt->sopt_name == MCAST_JOIN_SOURCE_GROUP) { if (ssa->sin6.sin6_family != AF_INET6 || ssa->sin6.sin6_len != sizeof(struct sockaddr_in6)) return (EINVAL); if (IN6_IS_ADDR_MULTICAST(&ssa->sin6.sin6_addr)) return (EINVAL); /* * TODO: Validate embedded scope ID in source * list entry against passed-in ifp, if and only * if source list filter entry is iface or node local. */ in6_clearscope(&ssa->sin6.sin6_addr); ssa->sin6.sin6_port = 0; ssa->sin6.sin6_scope_id = 0; } if (gsr.gsr_interface == 0 || V_if_index < gsr.gsr_interface) return (EADDRNOTAVAIL); ifp = ifnet_byindex(gsr.gsr_interface); break; default: CTR2(KTR_MLD, "%s: unknown sopt_name %d", __func__, sopt->sopt_name); return (EOPNOTSUPP); break; } if (!IN6_IS_ADDR_MULTICAST(&gsa->sin6.sin6_addr)) return (EINVAL); if (ifp == NULL || (ifp->if_flags & IFF_MULTICAST) == 0) return (EADDRNOTAVAIL); gsa->sin6.sin6_port = 0; gsa->sin6.sin6_scope_id = 0; /* * Always set the scope zone ID on memberships created from userland. * Use the passed-in ifp to do this. * XXX The in6_setscope() return value is meaningless. * XXX SCOPE6_LOCK() is taken by in6_setscope(). */ (void)in6_setscope(&gsa->sin6.sin6_addr, ifp, NULL); imo = in6p_findmoptions(inp); idx = im6o_match_group(imo, ifp, &gsa->sa); if (idx == -1) { is_new = 1; } else { inm = imo->im6o_membership[idx]; imf = &imo->im6o_mfilters[idx]; if (ssa->ss.ss_family != AF_UNSPEC) { /* * MCAST_JOIN_SOURCE_GROUP on an exclusive membership * is an error. On an existing inclusive membership, * it just adds the source to the filter list. */ if (imf->im6f_st[1] != MCAST_INCLUDE) { error = EINVAL; goto out_in6p_locked; } /* * Throw out duplicates. * * XXX FIXME: This makes a naive assumption that * even if entries exist for *ssa in this imf, * they will be rejected as dupes, even if they * are not valid in the current mode (in-mode). * * in6_msource is transactioned just as for anything * else in SSM -- but note naive use of in6m_graft() * below for allocating new filter entries. * * This is only an issue if someone mixes the * full-state SSM API with the delta-based API, * which is discouraged in the relevant RFCs. */ lims = im6o_match_source(imo, idx, &ssa->sa); if (lims != NULL /*&& lims->im6sl_st[1] == MCAST_INCLUDE*/) { error = EADDRNOTAVAIL; goto out_in6p_locked; } } else { /* * MCAST_JOIN_GROUP alone, on any existing membership, * is rejected, to stop the same inpcb tying up * multiple refs to the in_multi. * On an existing inclusive membership, this is also * an error; if you want to change filter mode, * you must use the userland API setsourcefilter(). * XXX We don't reject this for imf in UNDEFINED * state at t1, because allocation of a filter * is atomic with allocation of a membership. */ error = EINVAL; goto out_in6p_locked; } } /* * Begin state merge transaction at socket layer. */ INP_WLOCK_ASSERT(inp); if (is_new) { if (imo->im6o_num_memberships == imo->im6o_max_memberships) { error = im6o_grow(imo); if (error) goto out_in6p_locked; } /* * Allocate the new slot upfront so we can deal with * grafting the new source filter in same code path * as for join-source on existing membership. */ idx = imo->im6o_num_memberships; imo->im6o_membership[idx] = NULL; imo->im6o_num_memberships++; KASSERT(imo->im6o_mfilters != NULL, ("%s: im6f_mfilters vector was not allocated", __func__)); imf = &imo->im6o_mfilters[idx]; KASSERT(RB_EMPTY(&imf->im6f_sources), ("%s: im6f_sources not empty", __func__)); } /* * Graft new source into filter list for this inpcb's * membership of the group. The in6_multi may not have * been allocated yet if this is a new membership, however, * the in_mfilter slot will be allocated and must be initialized. * * Note: Grafting of exclusive mode filters doesn't happen * in this path. * XXX: Should check for non-NULL lims (node exists but may * not be in-mode) for interop with full-state API. */ if (ssa->ss.ss_family != AF_UNSPEC) { /* Membership starts in IN mode */ if (is_new) { CTR1(KTR_MLD, "%s: new join w/source", __func__); im6f_init(imf, MCAST_UNDEFINED, MCAST_INCLUDE); } else { CTR2(KTR_MLD, "%s: %s source", __func__, "allow"); } lims = im6f_graft(imf, MCAST_INCLUDE, &ssa->sin6); if (lims == NULL) { CTR1(KTR_MLD, "%s: merge imf state failed", __func__); error = ENOMEM; goto out_im6o_free; } } else { /* No address specified; Membership starts in EX mode */ if (is_new) { CTR1(KTR_MLD, "%s: new join w/o source", __func__); im6f_init(imf, MCAST_UNDEFINED, MCAST_EXCLUDE); } } /* * Begin state merge transaction at MLD layer. */ IN6_MULTI_LOCK(); if (is_new) { error = in6_mc_join_locked(ifp, &gsa->sin6.sin6_addr, imf, &inm, 0); if (error) { IN6_MULTI_UNLOCK(); goto out_im6o_free; } imo->im6o_membership[idx] = inm; } else { CTR1(KTR_MLD, "%s: merge inm state", __func__); error = in6m_merge(inm, imf); if (error) CTR1(KTR_MLD, "%s: failed to merge inm state", __func__); else { CTR1(KTR_MLD, "%s: doing mld downcall", __func__); error = mld_change_state(inm, 0); if (error) CTR1(KTR_MLD, "%s: failed mld downcall", __func__); } } IN6_MULTI_UNLOCK(); INP_WLOCK_ASSERT(inp); if (error) { im6f_rollback(imf); if (is_new) im6f_purge(imf); else im6f_reap(imf); } else { im6f_commit(imf); } out_im6o_free: if (error && is_new) { imo->im6o_membership[idx] = NULL; --imo->im6o_num_memberships; } out_in6p_locked: INP_WUNLOCK(inp); return (error); } /* * Leave an IPv6 multicast group on an inpcb, possibly with a source. */ static int in6p_leave_group(struct inpcb *inp, struct sockopt *sopt) { struct ipv6_mreq mreq; struct group_source_req gsr; sockunion_t *gsa, *ssa; struct ifnet *ifp; struct in6_mfilter *imf; struct ip6_moptions *imo; struct in6_msource *ims; struct in6_multi *inm; uint32_t ifindex; size_t idx; int error, is_final; #ifdef KTR char ip6tbuf[INET6_ADDRSTRLEN]; #endif ifp = NULL; ifindex = 0; error = 0; is_final = 1; memset(&gsr, 0, sizeof(struct group_source_req)); gsa = (sockunion_t *)&gsr.gsr_group; gsa->ss.ss_family = AF_UNSPEC; ssa = (sockunion_t *)&gsr.gsr_source; ssa->ss.ss_family = AF_UNSPEC; /* * Chew everything passed in up into a struct group_source_req * as that is easier to process. * Note: Any embedded scope ID in the multicast group passed * in by userland is ignored, the interface index is the recommended * mechanism to specify an interface; see below. */ switch (sopt->sopt_name) { case IPV6_LEAVE_GROUP: error = sooptcopyin(sopt, &mreq, sizeof(struct ipv6_mreq), sizeof(struct ipv6_mreq)); if (error) return (error); gsa->sin6.sin6_family = AF_INET6; gsa->sin6.sin6_len = sizeof(struct sockaddr_in6); gsa->sin6.sin6_addr = mreq.ipv6mr_multiaddr; gsa->sin6.sin6_port = 0; gsa->sin6.sin6_scope_id = 0; ifindex = mreq.ipv6mr_interface; break; case MCAST_LEAVE_GROUP: case MCAST_LEAVE_SOURCE_GROUP: if (sopt->sopt_name == MCAST_LEAVE_GROUP) { error = sooptcopyin(sopt, &gsr, sizeof(struct group_req), sizeof(struct group_req)); } else if (sopt->sopt_name == MCAST_LEAVE_SOURCE_GROUP) { error = sooptcopyin(sopt, &gsr, sizeof(struct group_source_req), sizeof(struct group_source_req)); } if (error) return (error); if (gsa->sin6.sin6_family != AF_INET6 || gsa->sin6.sin6_len != sizeof(struct sockaddr_in6)) return (EINVAL); if (sopt->sopt_name == MCAST_LEAVE_SOURCE_GROUP) { if (ssa->sin6.sin6_family != AF_INET6 || ssa->sin6.sin6_len != sizeof(struct sockaddr_in6)) return (EINVAL); if (IN6_IS_ADDR_MULTICAST(&ssa->sin6.sin6_addr)) return (EINVAL); /* * TODO: Validate embedded scope ID in source * list entry against passed-in ifp, if and only * if source list filter entry is iface or node local. */ in6_clearscope(&ssa->sin6.sin6_addr); } gsa->sin6.sin6_port = 0; gsa->sin6.sin6_scope_id = 0; ifindex = gsr.gsr_interface; break; default: CTR2(KTR_MLD, "%s: unknown sopt_name %d", __func__, sopt->sopt_name); return (EOPNOTSUPP); break; } if (!IN6_IS_ADDR_MULTICAST(&gsa->sin6.sin6_addr)) return (EINVAL); /* * Validate interface index if provided. If no interface index * was provided separately, attempt to look the membership up * from the default scope as a last resort to disambiguate * the membership we are being asked to leave. * XXX SCOPE6 lock potentially taken here. */ if (ifindex != 0) { if (V_if_index < ifindex) return (EADDRNOTAVAIL); ifp = ifnet_byindex(ifindex); if (ifp == NULL) return (EADDRNOTAVAIL); (void)in6_setscope(&gsa->sin6.sin6_addr, ifp, NULL); } else { error = sa6_embedscope(&gsa->sin6, V_ip6_use_defzone); if (error) return (EADDRNOTAVAIL); /* * Some badly behaved applications don't pass an ifindex * or a scope ID, which is an API violation. In this case, * perform a lookup as per a v6 join. * * XXX For now, stomp on zone ID for the corner case. * This is not the 'KAME way', but we need to see the ifp * directly until such time as this implementation is * refactored, assuming the scope IDs are the way to go. */ ifindex = ntohs(gsa->sin6.sin6_addr.s6_addr16[1]); if (ifindex == 0) { CTR2(KTR_MLD, "%s: warning: no ifindex, looking up " "ifp for group %s.", __func__, ip6_sprintf(ip6tbuf, &gsa->sin6.sin6_addr)); ifp = in6p_lookup_mcast_ifp(inp, &gsa->sin6); } else { ifp = ifnet_byindex(ifindex); } if (ifp == NULL) return (EADDRNOTAVAIL); } CTR2(KTR_MLD, "%s: ifp = %p", __func__, ifp); KASSERT(ifp != NULL, ("%s: ifp did not resolve", __func__)); /* * Find the membership in the membership array. */ imo = in6p_findmoptions(inp); idx = im6o_match_group(imo, ifp, &gsa->sa); if (idx == -1) { error = EADDRNOTAVAIL; goto out_in6p_locked; } inm = imo->im6o_membership[idx]; imf = &imo->im6o_mfilters[idx]; if (ssa->ss.ss_family != AF_UNSPEC) is_final = 0; /* * Begin state merge transaction at socket layer. */ INP_WLOCK_ASSERT(inp); /* * If we were instructed only to leave a given source, do so. * MCAST_LEAVE_SOURCE_GROUP is only valid for inclusive memberships. */ if (is_final) { im6f_leave(imf); } else { if (imf->im6f_st[0] == MCAST_EXCLUDE) { error = EADDRNOTAVAIL; goto out_in6p_locked; } ims = im6o_match_source(imo, idx, &ssa->sa); if (ims == NULL) { CTR3(KTR_MLD, "%s: source %p %spresent", __func__, ip6_sprintf(ip6tbuf, &ssa->sin6.sin6_addr), "not "); error = EADDRNOTAVAIL; goto out_in6p_locked; } CTR2(KTR_MLD, "%s: %s source", __func__, "block"); error = im6f_prune(imf, &ssa->sin6); if (error) { CTR1(KTR_MLD, "%s: merge imf state failed", __func__); goto out_in6p_locked; } } /* * Begin state merge transaction at MLD layer. */ IN6_MULTI_LOCK(); if (is_final) { /* * Give up the multicast address record to which * the membership points. */ (void)in6_mc_leave_locked(inm, imf); } else { CTR1(KTR_MLD, "%s: merge inm state", __func__); error = in6m_merge(inm, imf); if (error) CTR1(KTR_MLD, "%s: failed to merge inm state", __func__); else { CTR1(KTR_MLD, "%s: doing mld downcall", __func__); error = mld_change_state(inm, 0); if (error) CTR1(KTR_MLD, "%s: failed mld downcall", __func__); } } IN6_MULTI_UNLOCK(); if (error) im6f_rollback(imf); else im6f_commit(imf); im6f_reap(imf); if (is_final) { /* Remove the gap in the membership array. */ for (++idx; idx < imo->im6o_num_memberships; ++idx) { imo->im6o_membership[idx-1] = imo->im6o_membership[idx]; imo->im6o_mfilters[idx-1] = imo->im6o_mfilters[idx]; } imo->im6o_num_memberships--; } out_in6p_locked: INP_WUNLOCK(inp); return (error); } /* * Select the interface for transmitting IPv6 multicast datagrams. * * Either an instance of struct in6_addr or an instance of struct ipv6_mreqn * may be passed to this socket option. An address of in6addr_any or an * interface index of 0 is used to remove a previous selection. * When no interface is selected, one is chosen for every send. */ static int in6p_set_multicast_if(struct inpcb *inp, struct sockopt *sopt) { struct ifnet *ifp; struct ip6_moptions *imo; u_int ifindex; int error; if (sopt->sopt_valsize != sizeof(u_int)) return (EINVAL); error = sooptcopyin(sopt, &ifindex, sizeof(u_int), sizeof(u_int)); if (error) return (error); if (V_if_index < ifindex) return (EINVAL); if (ifindex == 0) ifp = NULL; else { ifp = ifnet_byindex(ifindex); if (ifp == NULL) return (EINVAL); if ((ifp->if_flags & IFF_MULTICAST) == 0) return (EADDRNOTAVAIL); } imo = in6p_findmoptions(inp); imo->im6o_multicast_ifp = ifp; INP_WUNLOCK(inp); return (0); } /* * Atomically set source filters on a socket for an IPv6 multicast group. * * SMPng: NOTE: Potentially calls malloc(M_WAITOK) with Giant held. */ static int in6p_set_source_filters(struct inpcb *inp, struct sockopt *sopt) { struct __msfilterreq msfr; sockunion_t *gsa; struct ifnet *ifp; struct in6_mfilter *imf; struct ip6_moptions *imo; struct in6_multi *inm; size_t idx; int error; error = sooptcopyin(sopt, &msfr, sizeof(struct __msfilterreq), sizeof(struct __msfilterreq)); if (error) return (error); if (msfr.msfr_nsrcs > in6_mcast_maxsocksrc) return (ENOBUFS); if (msfr.msfr_fmode != MCAST_EXCLUDE && msfr.msfr_fmode != MCAST_INCLUDE) return (EINVAL); if (msfr.msfr_group.ss_family != AF_INET6 || msfr.msfr_group.ss_len != sizeof(struct sockaddr_in6)) return (EINVAL); gsa = (sockunion_t *)&msfr.msfr_group; if (!IN6_IS_ADDR_MULTICAST(&gsa->sin6.sin6_addr)) return (EINVAL); gsa->sin6.sin6_port = 0; /* ignore port */ if (msfr.msfr_ifindex == 0 || V_if_index < msfr.msfr_ifindex) return (EADDRNOTAVAIL); ifp = ifnet_byindex(msfr.msfr_ifindex); if (ifp == NULL) return (EADDRNOTAVAIL); (void)in6_setscope(&gsa->sin6.sin6_addr, ifp, NULL); /* * Take the INP write lock. * Check if this socket is a member of this group. */ imo = in6p_findmoptions(inp); idx = im6o_match_group(imo, ifp, &gsa->sa); if (idx == -1 || imo->im6o_mfilters == NULL) { error = EADDRNOTAVAIL; goto out_in6p_locked; } inm = imo->im6o_membership[idx]; imf = &imo->im6o_mfilters[idx]; /* * Begin state merge transaction at socket layer. */ INP_WLOCK_ASSERT(inp); imf->im6f_st[1] = msfr.msfr_fmode; /* * Apply any new source filters, if present. * Make a copy of the user-space source vector so * that we may copy them with a single copyin. This * allows us to deal with page faults up-front. */ if (msfr.msfr_nsrcs > 0) { struct in6_msource *lims; struct sockaddr_in6 *psin; struct sockaddr_storage *kss, *pkss; int i; INP_WUNLOCK(inp); CTR2(KTR_MLD, "%s: loading %lu source list entries", __func__, (unsigned long)msfr.msfr_nsrcs); kss = malloc(sizeof(struct sockaddr_storage) * msfr.msfr_nsrcs, M_TEMP, M_WAITOK); error = copyin(msfr.msfr_srcs, kss, sizeof(struct sockaddr_storage) * msfr.msfr_nsrcs); if (error) { free(kss, M_TEMP); return (error); } INP_WLOCK(inp); /* * Mark all source filters as UNDEFINED at t1. * Restore new group filter mode, as im6f_leave() * will set it to INCLUDE. */ im6f_leave(imf); imf->im6f_st[1] = msfr.msfr_fmode; /* * Update socket layer filters at t1, lazy-allocating * new entries. This saves a bunch of memory at the * cost of one RB_FIND() per source entry; duplicate * entries in the msfr_nsrcs vector are ignored. * If we encounter an error, rollback transaction. * * XXX This too could be replaced with a set-symmetric * difference like loop to avoid walking from root * every time, as the key space is common. */ for (i = 0, pkss = kss; i < msfr.msfr_nsrcs; i++, pkss++) { psin = (struct sockaddr_in6 *)pkss; if (psin->sin6_family != AF_INET6) { error = EAFNOSUPPORT; break; } if (psin->sin6_len != sizeof(struct sockaddr_in6)) { error = EINVAL; break; } if (IN6_IS_ADDR_MULTICAST(&psin->sin6_addr)) { error = EINVAL; break; } /* * TODO: Validate embedded scope ID in source * list entry against passed-in ifp, if and only * if source list filter entry is iface or node local. */ in6_clearscope(&psin->sin6_addr); error = im6f_get_source(imf, psin, &lims); if (error) break; lims->im6sl_st[1] = imf->im6f_st[1]; } free(kss, M_TEMP); } if (error) goto out_im6f_rollback; INP_WLOCK_ASSERT(inp); IN6_MULTI_LOCK(); /* * Begin state merge transaction at MLD layer. */ CTR1(KTR_MLD, "%s: merge inm state", __func__); error = in6m_merge(inm, imf); if (error) CTR1(KTR_MLD, "%s: failed to merge inm state", __func__); else { CTR1(KTR_MLD, "%s: doing mld downcall", __func__); error = mld_change_state(inm, 0); if (error) CTR1(KTR_MLD, "%s: failed mld downcall", __func__); } IN6_MULTI_UNLOCK(); out_im6f_rollback: if (error) im6f_rollback(imf); else im6f_commit(imf); im6f_reap(imf); out_in6p_locked: INP_WUNLOCK(inp); return (error); } /* * Set the IP multicast options in response to user setsockopt(). * * Many of the socket options handled in this function duplicate the * functionality of socket options in the regular unicast API. However, * it is not possible to merge the duplicate code, because the idempotence * of the IPv6 multicast part of the BSD Sockets API must be preserved; * the effects of these options must be treated as separate and distinct. * * SMPng: XXX: Unlocked read of inp_socket believed OK. */ int ip6_setmoptions(struct inpcb *inp, struct sockopt *sopt) { struct ip6_moptions *im6o; int error; error = 0; /* * If socket is neither of type SOCK_RAW or SOCK_DGRAM, * or is a divert socket, reject it. */ if (inp->inp_socket->so_proto->pr_protocol == IPPROTO_DIVERT || (inp->inp_socket->so_proto->pr_type != SOCK_RAW && inp->inp_socket->so_proto->pr_type != SOCK_DGRAM)) return (EOPNOTSUPP); switch (sopt->sopt_name) { case IPV6_MULTICAST_IF: error = in6p_set_multicast_if(inp, sopt); break; case IPV6_MULTICAST_HOPS: { int hlim; if (sopt->sopt_valsize != sizeof(int)) { error = EINVAL; break; } error = sooptcopyin(sopt, &hlim, sizeof(hlim), sizeof(int)); if (error) break; if (hlim < -1 || hlim > 255) { error = EINVAL; break; } else if (hlim == -1) { hlim = V_ip6_defmcasthlim; } im6o = in6p_findmoptions(inp); im6o->im6o_multicast_hlim = hlim; INP_WUNLOCK(inp); break; } case IPV6_MULTICAST_LOOP: { u_int loop; /* * Set the loopback flag for outgoing multicast packets. * Must be zero or one. */ if (sopt->sopt_valsize != sizeof(u_int)) { error = EINVAL; break; } error = sooptcopyin(sopt, &loop, sizeof(u_int), sizeof(u_int)); if (error) break; if (loop > 1) { error = EINVAL; break; } im6o = in6p_findmoptions(inp); im6o->im6o_multicast_loop = loop; INP_WUNLOCK(inp); break; } case IPV6_JOIN_GROUP: case MCAST_JOIN_GROUP: case MCAST_JOIN_SOURCE_GROUP: error = in6p_join_group(inp, sopt); break; case IPV6_LEAVE_GROUP: case MCAST_LEAVE_GROUP: case MCAST_LEAVE_SOURCE_GROUP: error = in6p_leave_group(inp, sopt); break; case MCAST_BLOCK_SOURCE: case MCAST_UNBLOCK_SOURCE: error = in6p_block_unblock_source(inp, sopt); break; case IPV6_MSFILTER: error = in6p_set_source_filters(inp, sopt); break; default: error = EOPNOTSUPP; break; } INP_UNLOCK_ASSERT(inp); return (error); } /* * Expose MLD's multicast filter mode and source list(s) to userland, * keyed by (ifindex, group). * The filter mode is written out as a uint32_t, followed by * 0..n of struct in6_addr. * For use by ifmcstat(8). * SMPng: NOTE: unlocked read of ifindex space. */ static int sysctl_ip6_mcast_filters(SYSCTL_HANDLER_ARGS) { struct in6_addr mcaddr; struct in6_addr src; struct ifnet *ifp; struct ifmultiaddr *ifma; struct in6_multi *inm; struct ip6_msource *ims; int *name; int retval; u_int namelen; uint32_t fmode, ifindex; #ifdef KTR char ip6tbuf[INET6_ADDRSTRLEN]; #endif name = (int *)arg1; namelen = arg2; if (req->newptr != NULL) return (EPERM); /* int: ifindex + 4 * 32 bits of IPv6 address */ if (namelen != 5) return (EINVAL); ifindex = name[0]; if (ifindex <= 0 || ifindex > V_if_index) { CTR2(KTR_MLD, "%s: ifindex %u out of range", __func__, ifindex); return (ENOENT); } memcpy(&mcaddr, &name[1], sizeof(struct in6_addr)); if (!IN6_IS_ADDR_MULTICAST(&mcaddr)) { CTR2(KTR_MLD, "%s: group %s is not multicast", __func__, ip6_sprintf(ip6tbuf, &mcaddr)); return (EINVAL); } ifp = ifnet_byindex(ifindex); if (ifp == NULL) { CTR2(KTR_MLD, "%s: no ifp for ifindex %u", __func__, ifindex); return (ENOENT); } /* * Internal MLD lookups require that scope/zone ID is set. */ (void)in6_setscope(&mcaddr, ifp, NULL); retval = sysctl_wire_old_buffer(req, sizeof(uint32_t) + (in6_mcast_maxgrpsrc * sizeof(struct in6_addr))); if (retval) return (retval); IN6_MULTI_LOCK(); IF_ADDR_RLOCK(ifp); TAILQ_FOREACH(ifma, &ifp->if_multiaddrs, ifma_link) { if (ifma->ifma_addr->sa_family != AF_INET6 || ifma->ifma_protospec == NULL) continue; inm = (struct in6_multi *)ifma->ifma_protospec; if (!IN6_ARE_ADDR_EQUAL(&inm->in6m_addr, &mcaddr)) continue; fmode = inm->in6m_st[1].iss_fmode; retval = SYSCTL_OUT(req, &fmode, sizeof(uint32_t)); if (retval != 0) break; RB_FOREACH(ims, ip6_msource_tree, &inm->in6m_srcs) { CTR2(KTR_MLD, "%s: visit node %p", __func__, ims); /* * Only copy-out sources which are in-mode. */ if (fmode != im6s_get_mode(inm, ims, 1)) { CTR1(KTR_MLD, "%s: skip non-in-mode", __func__); continue; } src = ims->im6s_addr; retval = SYSCTL_OUT(req, &src, sizeof(struct in6_addr)); if (retval != 0) break; } } IF_ADDR_RUNLOCK(ifp); IN6_MULTI_UNLOCK(); return (retval); } #ifdef KTR static const char *in6m_modestrs[] = { "un", "in", "ex" }; static const char * in6m_mode_str(const int mode) { if (mode >= MCAST_UNDEFINED && mode <= MCAST_EXCLUDE) return (in6m_modestrs[mode]); return ("??"); } static const char *in6m_statestrs[] = { "not-member", "silent", "idle", "lazy", "sleeping", "awakening", "query-pending", "sg-query-pending", "leaving" }; static const char * in6m_state_str(const int state) { if (state >= MLD_NOT_MEMBER && state <= MLD_LEAVING_MEMBER) return (in6m_statestrs[state]); return ("??"); } /* * Dump an in6_multi structure to the console. */ void in6m_print(const struct in6_multi *inm) { int t; char ip6tbuf[INET6_ADDRSTRLEN]; if ((ktr_mask & KTR_MLD) == 0) return; printf("%s: --- begin in6m %p ---\n", __func__, inm); printf("addr %s ifp %p(%s) ifma %p\n", ip6_sprintf(ip6tbuf, &inm->in6m_addr), inm->in6m_ifp, if_name(inm->in6m_ifp), inm->in6m_ifma); printf("timer %u state %s refcount %u scq.len %u\n", inm->in6m_timer, in6m_state_str(inm->in6m_state), inm->in6m_refcount, mbufq_len(&inm->in6m_scq)); printf("mli %p nsrc %lu sctimer %u scrv %u\n", inm->in6m_mli, inm->in6m_nsrc, inm->in6m_sctimer, inm->in6m_scrv); for (t = 0; t < 2; t++) { printf("t%d: fmode %s asm %u ex %u in %u rec %u\n", t, in6m_mode_str(inm->in6m_st[t].iss_fmode), inm->in6m_st[t].iss_asm, inm->in6m_st[t].iss_ex, inm->in6m_st[t].iss_in, inm->in6m_st[t].iss_rec); } printf("%s: --- end in6m %p ---\n", __func__, inm); } #else /* !KTR */ void in6m_print(const struct in6_multi *inm) { } #endif /* KTR */ Index: user/ngie/more-tests2/sys/netinet6/in6_src.c =================================================================== --- user/ngie/more-tests2/sys/netinet6/in6_src.c (revision 292053) +++ user/ngie/more-tests2/sys/netinet6/in6_src.c (revision 292054) @@ -1,1171 +1,1169 @@ /*- * Copyright (C) 1995, 1996, 1997, and 1998 WIDE Project. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. Neither the name of the project nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE PROJECT AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE PROJECT OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $KAME: in6_src.c,v 1.132 2003/08/26 04:42:27 keiichi Exp $ */ /*- * Copyright (c) 1982, 1986, 1991, 1993 * The Regents of the University of California. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * @(#)in_pcb.c 8.2 (Berkeley) 1/4/94 */ #include __FBSDID("$FreeBSD$"); #include "opt_inet.h" #include "opt_inet6.h" #include "opt_mpath.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef RADIX_MPATH #include #endif #include #include #include #include #include #include #include #include #include #include +#include #include #include #include #include static struct mtx addrsel_lock; #define ADDRSEL_LOCK_INIT() mtx_init(&addrsel_lock, "addrsel_lock", NULL, MTX_DEF) #define ADDRSEL_LOCK() mtx_lock(&addrsel_lock) #define ADDRSEL_UNLOCK() mtx_unlock(&addrsel_lock) #define ADDRSEL_LOCK_ASSERT() mtx_assert(&addrsel_lock, MA_OWNED) static struct sx addrsel_sxlock; #define ADDRSEL_SXLOCK_INIT() sx_init(&addrsel_sxlock, "addrsel_sxlock") #define ADDRSEL_SLOCK() sx_slock(&addrsel_sxlock) #define ADDRSEL_SUNLOCK() sx_sunlock(&addrsel_sxlock) #define ADDRSEL_XLOCK() sx_xlock(&addrsel_sxlock) #define ADDRSEL_XUNLOCK() sx_xunlock(&addrsel_sxlock) #define ADDR_LABEL_NOTAPP (-1) static VNET_DEFINE(struct in6_addrpolicy, defaultaddrpolicy); #define V_defaultaddrpolicy VNET(defaultaddrpolicy) VNET_DEFINE(int, ip6_prefer_tempaddr) = 0; static int selectroute(struct sockaddr_in6 *, struct ip6_pktopts *, struct ip6_moptions *, struct route_in6 *, struct ifnet **, struct rtentry **, int, u_int); static int in6_selectif(struct sockaddr_in6 *, struct ip6_pktopts *, struct ip6_moptions *, struct route_in6 *ro, struct ifnet **, struct ifnet *, u_int); static struct in6_addrpolicy *lookup_addrsel_policy(struct sockaddr_in6 *); static void init_policy_queue(void); static int add_addrsel_policyent(struct in6_addrpolicy *); static int delete_addrsel_policyent(struct in6_addrpolicy *); static int walk_addrsel_policy(int (*)(struct in6_addrpolicy *, void *), void *); static int dump_addrsel_policyent(struct in6_addrpolicy *, void *); static struct in6_addrpolicy *match_addrsel_policy(struct sockaddr_in6 *); /* * Return an IPv6 address, which is the most appropriate for a given * destination and user specified options. * If necessary, this function lookups the routing table and returns * an entry to the caller for later use. */ #define REPLACE(r) do {\ IP6STAT_INC(ip6s_sources_rule[(r)]); \ rule = (r); \ /* { \ char ip6buf[INET6_ADDRSTRLEN], ip6b[INET6_ADDRSTRLEN]; \ printf("in6_selectsrc: replace %s with %s by %d\n", ia_best ? ip6_sprintf(ip6buf, &ia_best->ia_addr.sin6_addr) : "none", ip6_sprintf(ip6b, &ia->ia_addr.sin6_addr), (r)); \ } */ \ goto replace; \ } while(0) #define NEXT(r) do {\ /* { \ char ip6buf[INET6_ADDRSTRLEN], ip6b[INET6_ADDRSTRLEN]; \ printf("in6_selectsrc: keep %s against %s by %d\n", ia_best ? ip6_sprintf(ip6buf, &ia_best->ia_addr.sin6_addr) : "none", ip6_sprintf(ip6b, &ia->ia_addr.sin6_addr), (r)); \ } */ \ goto next; /* XXX: we can't use 'continue' here */ \ } while(0) #define BREAK(r) do { \ IP6STAT_INC(ip6s_sources_rule[(r)]); \ rule = (r); \ goto out; /* XXX: we can't use 'break' here */ \ } while(0) int in6_selectsrc(struct sockaddr_in6 *dstsock, struct ip6_pktopts *opts, struct inpcb *inp, struct route_in6 *ro, struct ucred *cred, struct ifnet **ifpp, struct in6_addr *srcp) { struct rm_priotracker in6_ifa_tracker; struct in6_addr dst, tmp; struct ifnet *ifp = NULL, *oifp = NULL; struct in6_ifaddr *ia = NULL, *ia_best = NULL; struct in6_pktinfo *pi = NULL; int dst_scope = -1, best_scope = -1, best_matchlen = -1; struct in6_addrpolicy *dst_policy = NULL, *best_policy = NULL; u_int32_t odstzone; int prefer_tempaddr; int error, rule; struct ip6_moptions *mopts; KASSERT(srcp != NULL, ("%s: srcp is NULL", __func__)); dst = dstsock->sin6_addr; /* make a copy for local operation */ if (ifpp) { /* * Save a possibly passed in ifp for in6_selectsrc. Only * neighbor discovery code should use this feature, where * we may know the interface but not the FIB number holding * the connected subnet in case someone deleted it from the * default FIB and we need to check the interface. */ if (*ifpp != NULL) oifp = *ifpp; *ifpp = NULL; } if (inp != NULL) { INP_LOCK_ASSERT(inp); mopts = inp->in6p_moptions; } else { mopts = NULL; } /* * If the source address is explicitly specified by the caller, * check if the requested source address is indeed a unicast address * assigned to the node, and can be used as the packet's source * address. If everything is okay, use the address as source. */ if (opts && (pi = opts->ip6po_pktinfo) && !IN6_IS_ADDR_UNSPECIFIED(&pi->ipi6_addr)) { struct sockaddr_in6 srcsock; struct in6_ifaddr *ia6; /* get the outgoing interface */ if ((error = in6_selectif(dstsock, opts, mopts, ro, &ifp, oifp, (inp != NULL) ? inp->inp_inc.inc_fibnum : RT_DEFAULT_FIB)) != 0) return (error); /* * determine the appropriate zone id of the source based on * the zone of the destination and the outgoing interface. * If the specified address is ambiguous wrt the scope zone, * the interface must be specified; otherwise, ifa_ifwithaddr() * will fail matching the address. */ bzero(&srcsock, sizeof(srcsock)); srcsock.sin6_family = AF_INET6; srcsock.sin6_len = sizeof(srcsock); srcsock.sin6_addr = pi->ipi6_addr; if (ifp) { error = in6_setscope(&srcsock.sin6_addr, ifp, NULL); if (error) return (error); } if (cred != NULL && (error = prison_local_ip6(cred, &srcsock.sin6_addr, (inp != NULL && (inp->inp_flags & IN6P_IPV6_V6ONLY) != 0))) != 0) return (error); ia6 = (struct in6_ifaddr *)ifa_ifwithaddr( (struct sockaddr *)&srcsock); if (ia6 == NULL || (ia6->ia6_flags & (IN6_IFF_ANYCAST | IN6_IFF_NOTREADY))) { if (ia6 != NULL) ifa_free(&ia6->ia_ifa); return (EADDRNOTAVAIL); } pi->ipi6_addr = srcsock.sin6_addr; /* XXX: this overrides pi */ if (ifpp) *ifpp = ifp; bcopy(&ia6->ia_addr.sin6_addr, srcp, sizeof(*srcp)); ifa_free(&ia6->ia_ifa); return (0); } /* * Otherwise, if the socket has already bound the source, just use it. */ if (inp != NULL && !IN6_IS_ADDR_UNSPECIFIED(&inp->in6p_laddr)) { if (cred != NULL && (error = prison_local_ip6(cred, &inp->in6p_laddr, ((inp->inp_flags & IN6P_IPV6_V6ONLY) != 0))) != 0) return (error); bcopy(&inp->in6p_laddr, srcp, sizeof(*srcp)); return (0); } /* * Bypass source address selection and use the primary jail IP * if requested. */ if (cred != NULL && !prison_saddrsel_ip6(cred, srcp)) return (0); /* * If the address is not specified, choose the best one based on * the outgoing interface and the destination address. */ /* get the outgoing interface */ if ((error = in6_selectif(dstsock, opts, mopts, ro, &ifp, oifp, (inp != NULL) ? inp->inp_inc.inc_fibnum : RT_DEFAULT_FIB)) != 0) return (error); #ifdef DIAGNOSTIC if (ifp == NULL) /* this should not happen */ panic("in6_selectsrc: NULL ifp"); #endif error = in6_setscope(&dst, ifp, &odstzone); if (error) return (error); rule = 0; IN6_IFADDR_RLOCK(&in6_ifa_tracker); TAILQ_FOREACH(ia, &V_in6_ifaddrhead, ia_link) { int new_scope = -1, new_matchlen = -1; struct in6_addrpolicy *new_policy = NULL; u_int32_t srczone, osrczone, dstzone; struct in6_addr src; struct ifnet *ifp1 = ia->ia_ifp; /* * We'll never take an address that breaks the scope zone * of the destination. We also skip an address if its zone * does not contain the outgoing interface. * XXX: we should probably use sin6_scope_id here. */ if (in6_setscope(&dst, ifp1, &dstzone) || odstzone != dstzone) { continue; } src = ia->ia_addr.sin6_addr; if (in6_setscope(&src, ifp, &osrczone) || in6_setscope(&src, ifp1, &srczone) || osrczone != srczone) { continue; } /* avoid unusable addresses */ if ((ia->ia6_flags & (IN6_IFF_NOTREADY | IN6_IFF_ANYCAST | IN6_IFF_DETACHED))) { continue; } if (!V_ip6_use_deprecated && IFA6_IS_DEPRECATED(ia)) continue; /* If jailed only take addresses of the jail into account. */ if (cred != NULL && prison_check_ip6(cred, &ia->ia_addr.sin6_addr) != 0) continue; /* Rule 1: Prefer same address */ if (IN6_ARE_ADDR_EQUAL(&dst, &ia->ia_addr.sin6_addr)) { ia_best = ia; BREAK(1); /* there should be no better candidate */ } if (ia_best == NULL) REPLACE(0); /* Rule 2: Prefer appropriate scope */ if (dst_scope < 0) dst_scope = in6_addrscope(&dst); new_scope = in6_addrscope(&ia->ia_addr.sin6_addr); if (IN6_ARE_SCOPE_CMP(best_scope, new_scope) < 0) { if (IN6_ARE_SCOPE_CMP(best_scope, dst_scope) < 0) REPLACE(2); NEXT(2); } else if (IN6_ARE_SCOPE_CMP(new_scope, best_scope) < 0) { if (IN6_ARE_SCOPE_CMP(new_scope, dst_scope) < 0) NEXT(2); REPLACE(2); } /* * Rule 3: Avoid deprecated addresses. Note that the case of * !ip6_use_deprecated is already rejected above. */ if (!IFA6_IS_DEPRECATED(ia_best) && IFA6_IS_DEPRECATED(ia)) NEXT(3); if (IFA6_IS_DEPRECATED(ia_best) && !IFA6_IS_DEPRECATED(ia)) REPLACE(3); /* Rule 4: Prefer home addresses */ /* * XXX: This is a TODO. We should probably merge the MIP6 * case above. */ /* Rule 5: Prefer outgoing interface */ if (!(ND_IFINFO(ifp)->flags & ND6_IFF_NO_PREFER_IFACE)) { if (ia_best->ia_ifp == ifp && ia->ia_ifp != ifp) NEXT(5); if (ia_best->ia_ifp != ifp && ia->ia_ifp == ifp) REPLACE(5); } /* * Rule 6: Prefer matching label * Note that best_policy should be non-NULL here. */ if (dst_policy == NULL) dst_policy = lookup_addrsel_policy(dstsock); if (dst_policy->label != ADDR_LABEL_NOTAPP) { new_policy = lookup_addrsel_policy(&ia->ia_addr); if (dst_policy->label == best_policy->label && dst_policy->label != new_policy->label) NEXT(6); if (dst_policy->label != best_policy->label && dst_policy->label == new_policy->label) REPLACE(6); } /* * Rule 7: Prefer public addresses. * We allow users to reverse the logic by configuring * a sysctl variable, so that privacy conscious users can * always prefer temporary addresses. */ if (opts == NULL || opts->ip6po_prefer_tempaddr == IP6PO_TEMPADDR_SYSTEM) { prefer_tempaddr = V_ip6_prefer_tempaddr; } else if (opts->ip6po_prefer_tempaddr == IP6PO_TEMPADDR_NOTPREFER) { prefer_tempaddr = 0; } else prefer_tempaddr = 1; if (!(ia_best->ia6_flags & IN6_IFF_TEMPORARY) && (ia->ia6_flags & IN6_IFF_TEMPORARY)) { if (prefer_tempaddr) REPLACE(7); else NEXT(7); } if ((ia_best->ia6_flags & IN6_IFF_TEMPORARY) && !(ia->ia6_flags & IN6_IFF_TEMPORARY)) { if (prefer_tempaddr) NEXT(7); else REPLACE(7); } /* * Rule 8: prefer addresses on alive interfaces. * This is a KAME specific rule. */ if ((ia_best->ia_ifp->if_flags & IFF_UP) && !(ia->ia_ifp->if_flags & IFF_UP)) NEXT(8); if (!(ia_best->ia_ifp->if_flags & IFF_UP) && (ia->ia_ifp->if_flags & IFF_UP)) REPLACE(8); /* * Rule 9: prefer address with better virtual status. */ if (ifa_preferred(&ia_best->ia_ifa, &ia->ia_ifa)) REPLACE(9); if (ifa_preferred(&ia->ia_ifa, &ia_best->ia_ifa)) NEXT(9); /* * Rule 10: prefer address with `prefer_source' flag. */ if ((ia_best->ia6_flags & IN6_IFF_PREFER_SOURCE) == 0 && (ia->ia6_flags & IN6_IFF_PREFER_SOURCE) != 0) REPLACE(10); if ((ia_best->ia6_flags & IN6_IFF_PREFER_SOURCE) != 0 && (ia->ia6_flags & IN6_IFF_PREFER_SOURCE) == 0) NEXT(10); /* * Rule 14: Use longest matching prefix. * Note: in the address selection draft, this rule is * documented as "Rule 8". However, since it is also * documented that this rule can be overridden, we assign * a large number so that it is easy to assign smaller numbers * to more preferred rules. */ new_matchlen = in6_matchlen(&ia->ia_addr.sin6_addr, &dst); if (best_matchlen < new_matchlen) REPLACE(14); if (new_matchlen < best_matchlen) NEXT(14); /* Rule 15 is reserved. */ /* * Last resort: just keep the current candidate. * Or, do we need more rules? */ continue; replace: ia_best = ia; best_scope = (new_scope >= 0 ? new_scope : in6_addrscope(&ia_best->ia_addr.sin6_addr)); best_policy = (new_policy ? new_policy : lookup_addrsel_policy(&ia_best->ia_addr)); best_matchlen = (new_matchlen >= 0 ? new_matchlen : in6_matchlen(&ia_best->ia_addr.sin6_addr, &dst)); next: continue; out: break; } if ((ia = ia_best) == NULL) { IN6_IFADDR_RUNLOCK(&in6_ifa_tracker); IP6STAT_INC(ip6s_sources_none); return (EADDRNOTAVAIL); } /* * At this point at least one of the addresses belonged to the jail * but it could still be, that we want to further restrict it, e.g. * theoratically IN6_IS_ADDR_LOOPBACK. * It must not be IN6_IS_ADDR_UNSPECIFIED anymore. * prison_local_ip6() will fix an IN6_IS_ADDR_LOOPBACK but should * let all others previously selected pass. * Use tmp to not change ::1 on lo0 to the primary jail address. */ tmp = ia->ia_addr.sin6_addr; if (cred != NULL && prison_local_ip6(cred, &tmp, (inp != NULL && (inp->inp_flags & IN6P_IPV6_V6ONLY) != 0)) != 0) { IN6_IFADDR_RUNLOCK(&in6_ifa_tracker); IP6STAT_INC(ip6s_sources_none); return (EADDRNOTAVAIL); } if (ifpp) *ifpp = ifp; bcopy(&tmp, srcp, sizeof(*srcp)); if (ia->ia_ifp == ifp) IP6STAT_INC(ip6s_sources_sameif[best_scope]); else IP6STAT_INC(ip6s_sources_otherif[best_scope]); if (dst_scope == best_scope) IP6STAT_INC(ip6s_sources_samescope[best_scope]); else IP6STAT_INC(ip6s_sources_otherscope[best_scope]); if (IFA6_IS_DEPRECATED(ia)) IP6STAT_INC(ip6s_sources_deprecated[best_scope]); IN6_IFADDR_RUNLOCK(&in6_ifa_tracker); return (0); } /* * clone - meaningful only for bsdi and freebsd */ static int selectroute(struct sockaddr_in6 *dstsock, struct ip6_pktopts *opts, struct ip6_moptions *mopts, struct route_in6 *ro, struct ifnet **retifp, struct rtentry **retrt, int norouteok, u_int fibnum) { int error = 0; struct ifnet *ifp = NULL; struct rtentry *rt = NULL; struct sockaddr_in6 *sin6_next; struct in6_pktinfo *pi = NULL; struct in6_addr *dst = &dstsock->sin6_addr; uint32_t zoneid; #if 0 char ip6buf[INET6_ADDRSTRLEN]; if (dstsock->sin6_addr.s6_addr32[0] == 0 && dstsock->sin6_addr.s6_addr32[1] == 0 && !IN6_IS_ADDR_LOOPBACK(&dstsock->sin6_addr)) { printf("in6_selectroute: strange destination %s\n", ip6_sprintf(ip6buf, &dstsock->sin6_addr)); } else { printf("in6_selectroute: destination = %s%%%d\n", ip6_sprintf(ip6buf, &dstsock->sin6_addr), dstsock->sin6_scope_id); /* for debug */ } #endif /* If the caller specify the outgoing interface explicitly, use it. */ if (opts && (pi = opts->ip6po_pktinfo) != NULL && pi->ipi6_ifindex) { /* XXX boundary check is assumed to be already done. */ ifp = ifnet_byindex(pi->ipi6_ifindex); if (ifp != NULL && (norouteok || retrt == NULL || IN6_IS_ADDR_MULTICAST(dst))) { /* * we do not have to check or get the route for * multicast. */ goto done; } else goto getroute; } /* * If the destination address is a multicast address and the outgoing * interface for the address is specified by the caller, use it. */ if (IN6_IS_ADDR_MULTICAST(dst) && mopts != NULL && (ifp = mopts->im6o_multicast_ifp) != NULL) { goto done; /* we do not need a route for multicast. */ } /* * If destination address is LLA or link- or node-local multicast, * use it's embedded scope zone id to determine outgoing interface. */ if (IN6_IS_ADDR_MC_LINKLOCAL(dst) || IN6_IS_ADDR_MC_NODELOCAL(dst)) { zoneid = ntohs(in6_getscope(dst)); if (zoneid > 0) { ifp = in6_getlinkifnet(zoneid); goto done; } } getroute: /* * If the next hop address for the packet is specified by the caller, * use it as the gateway. */ if (opts && opts->ip6po_nexthop) { struct route_in6 *ron; sin6_next = satosin6(opts->ip6po_nexthop); if (IN6_IS_ADDR_LINKLOCAL(&sin6_next->sin6_addr)) { /* * Next hop is LLA, thus it should be neighbor. * Determine outgoing interface by zone index. */ zoneid = ntohs(in6_getscope(&sin6_next->sin6_addr)); if (zoneid > 0) { ifp = in6_getlinkifnet(zoneid); goto done; } } ron = &opts->ip6po_nextroute; /* Use a cached route if it exists and is valid. */ if (ron->ro_rt != NULL && ( (ron->ro_rt->rt_flags & RTF_UP) == 0 || ron->ro_dst.sin6_family != AF_INET6 || !IN6_ARE_ADDR_EQUAL(&ron->ro_dst.sin6_addr, &sin6_next->sin6_addr))) RO_RTFREE(ron); if (ron->ro_rt == NULL) { ron->ro_dst = *sin6_next; in6_rtalloc(ron, fibnum); /* multi path case? */ } /* * The node identified by that address must be a * neighbor of the sending host. */ if (ron->ro_rt == NULL || (ron->ro_rt->rt_flags & RTF_GATEWAY) != 0) error = EHOSTUNREACH; goto done; } /* * Use a cached route if it exists and is valid, else try to allocate * a new one. Note that we should check the address family of the * cached destination, in case of sharing the cache with IPv4. */ if (ro) { if (ro->ro_rt && (!(ro->ro_rt->rt_flags & RTF_UP) || ((struct sockaddr *)(&ro->ro_dst))->sa_family != AF_INET6 || !IN6_ARE_ADDR_EQUAL(&satosin6(&ro->ro_dst)->sin6_addr, dst))) { RTFREE(ro->ro_rt); ro->ro_rt = (struct rtentry *)NULL; } if (ro->ro_rt == (struct rtentry *)NULL) { struct sockaddr_in6 *sa6; /* No route yet, so try to acquire one */ bzero(&ro->ro_dst, sizeof(struct sockaddr_in6)); sa6 = (struct sockaddr_in6 *)&ro->ro_dst; *sa6 = *dstsock; sa6->sin6_scope_id = 0; #ifdef RADIX_MPATH rtalloc_mpath_fib((struct route *)ro, ntohl(sa6->sin6_addr.s6_addr32[3]), fibnum); #else ro->ro_rt = in6_rtalloc1((struct sockaddr *) &ro->ro_dst, 0, 0UL, fibnum); if (ro->ro_rt) RT_UNLOCK(ro->ro_rt); #endif } /* * do not care about the result if we have the nexthop * explicitly specified. */ if (opts && opts->ip6po_nexthop) goto done; if (ro->ro_rt) { ifp = ro->ro_rt->rt_ifp; if (ifp == NULL) { /* can this really happen? */ RTFREE(ro->ro_rt); ro->ro_rt = NULL; } } if (ro->ro_rt == NULL) error = EHOSTUNREACH; rt = ro->ro_rt; /* * Check if the outgoing interface conflicts with * the interface specified by ipi6_ifindex (if specified). * Note that loopback interface is always okay. * (this may happen when we are sending a packet to one of * our own addresses.) */ if (ifp && opts && opts->ip6po_pktinfo && opts->ip6po_pktinfo->ipi6_ifindex) { if (!(ifp->if_flags & IFF_LOOPBACK) && ifp->if_index != opts->ip6po_pktinfo->ipi6_ifindex) { error = EHOSTUNREACH; goto done; } } } done: if (ifp == NULL && rt == NULL) { /* * This can happen if the caller did not pass a cached route * nor any other hints. We treat this case an error. */ error = EHOSTUNREACH; } if (error == EHOSTUNREACH) IP6STAT_INC(ip6s_noroute); if (retifp != NULL) { *retifp = ifp; /* * Adjust the "outgoing" interface. If we're going to loop * the packet back to ourselves, the ifp would be the loopback * interface. However, we'd rather know the interface associated * to the destination address (which should probably be one of * our own addresses.) */ if (rt) { if ((rt->rt_ifp->if_flags & IFF_LOOPBACK) && (rt->rt_gateway->sa_family == AF_LINK)) *retifp = ifnet_byindex(((struct sockaddr_dl *) rt->rt_gateway)->sdl_index); } } if (retrt != NULL) *retrt = rt; /* rt may be NULL */ return (error); } static int in6_selectif(struct sockaddr_in6 *dstsock, struct ip6_pktopts *opts, struct ip6_moptions *mopts, struct route_in6 *ro, struct ifnet **retifp, struct ifnet *oifp, u_int fibnum) { int error; struct route_in6 sro; struct rtentry *rt = NULL; KASSERT(retifp != NULL, ("%s: retifp is NULL", __func__)); if (ro == NULL) { bzero(&sro, sizeof(sro)); ro = &sro; } if ((error = selectroute(dstsock, opts, mopts, ro, retifp, &rt, 1, fibnum)) != 0) { if (ro == &sro && rt && rt == sro.ro_rt) RTFREE(rt); /* Help ND. See oifp comment in in6_selectsrc(). */ if (oifp != NULL && fibnum == RT_DEFAULT_FIB) { *retifp = oifp; error = 0; } return (error); } /* * do not use a rejected or black hole route. * XXX: this check should be done in the L2 output routine. * However, if we skipped this check here, we'd see the following * scenario: * - install a rejected route for a scoped address prefix * (like fe80::/10) * - send a packet to a destination that matches the scoped prefix, * with ambiguity about the scope zone. * - pick the outgoing interface from the route, and disambiguate the * scope zone with the interface. * - ip6_output() would try to get another route with the "new" * destination, which may be valid. * - we'd see no error on output. * Although this may not be very harmful, it should still be confusing. * We thus reject the case here. */ if (rt && (rt->rt_flags & (RTF_REJECT | RTF_BLACKHOLE))) { int flags = (rt->rt_flags & RTF_HOST ? EHOSTUNREACH : ENETUNREACH); if (ro == &sro && rt && rt == sro.ro_rt) RTFREE(rt); return (flags); } if (ro == &sro && rt && rt == sro.ro_rt) RTFREE(rt); return (0); } /* * Public wrapper function to selectroute(). * * XXX-BZ in6_selectroute() should and will grow the FIB argument. The * in6_selectroute_fib() function is only there for backward compat on stable. */ int in6_selectroute(struct sockaddr_in6 *dstsock, struct ip6_pktopts *opts, struct ip6_moptions *mopts, struct route_in6 *ro, struct ifnet **retifp, struct rtentry **retrt) { return (selectroute(dstsock, opts, mopts, ro, retifp, retrt, 0, RT_DEFAULT_FIB)); } #ifndef BURN_BRIDGES int in6_selectroute_fib(struct sockaddr_in6 *dstsock, struct ip6_pktopts *opts, struct ip6_moptions *mopts, struct route_in6 *ro, struct ifnet **retifp, struct rtentry **retrt, u_int fibnum) { return (selectroute(dstsock, opts, mopts, ro, retifp, retrt, 0, fibnum)); } #endif /* * Default hop limit selection. The precedence is as follows: * 1. Hoplimit value specified via ioctl. * 2. (If the outgoing interface is detected) the current * hop limit of the interface specified by router advertisement. * 3. The system default hoplimit. */ int in6_selecthlim(struct inpcb *in6p, struct ifnet *ifp) { if (in6p && in6p->in6p_hops >= 0) return (in6p->in6p_hops); else if (ifp) return (ND_IFINFO(ifp)->chlim); else if (in6p && !IN6_IS_ADDR_UNSPECIFIED(&in6p->in6p_faddr)) { - struct route_in6 ro6; - struct ifnet *lifp; + struct nhop6_basic nh6; + struct in6_addr dst; + uint32_t fibnum, scopeid; + int hlim; - bzero(&ro6, sizeof(ro6)); - ro6.ro_dst.sin6_family = AF_INET6; - ro6.ro_dst.sin6_len = sizeof(struct sockaddr_in6); - ro6.ro_dst.sin6_addr = in6p->in6p_faddr; - in6_rtalloc(&ro6, in6p->inp_inc.inc_fibnum); - if (ro6.ro_rt) { - lifp = ro6.ro_rt->rt_ifp; - RTFREE(ro6.ro_rt); - if (lifp) - return (ND_IFINFO(lifp)->chlim); + fibnum = in6p->inp_inc.inc_fibnum; + in6_splitscope(&in6p->in6p_faddr, &dst, &scopeid); + if (fib6_lookup_nh_basic(fibnum, &dst, scopeid, 0, 0, &nh6)==0){ + hlim = ND_IFINFO(nh6.nh_ifp)->chlim; + return (hlim); } } return (V_ip6_defhlim); } /* * XXX: this is borrowed from in6_pcbbind(). If possible, we should * share this function by all *bsd*... */ int in6_pcbsetport(struct in6_addr *laddr, struct inpcb *inp, struct ucred *cred) { struct socket *so = inp->inp_socket; u_int16_t lport = 0; int error, lookupflags = 0; #ifdef INVARIANTS struct inpcbinfo *pcbinfo = inp->inp_pcbinfo; #endif INP_WLOCK_ASSERT(inp); INP_HASH_WLOCK_ASSERT(pcbinfo); error = prison_local_ip6(cred, laddr, ((inp->inp_flags & IN6P_IPV6_V6ONLY) != 0)); if (error) return(error); /* XXX: this is redundant when called from in6_pcbbind */ if ((so->so_options & (SO_REUSEADDR|SO_REUSEPORT)) == 0) lookupflags = INPLOOKUP_WILDCARD; inp->inp_flags |= INP_ANONPORT; error = in_pcb_lport(inp, NULL, &lport, cred, lookupflags); if (error != 0) return (error); inp->inp_lport = lport; if (in_pcbinshash(inp) != 0) { inp->in6p_laddr = in6addr_any; inp->inp_lport = 0; return (EAGAIN); } return (0); } void addrsel_policy_init(void) { init_policy_queue(); /* initialize the "last resort" policy */ bzero(&V_defaultaddrpolicy, sizeof(V_defaultaddrpolicy)); V_defaultaddrpolicy.label = ADDR_LABEL_NOTAPP; if (!IS_DEFAULT_VNET(curvnet)) return; ADDRSEL_LOCK_INIT(); ADDRSEL_SXLOCK_INIT(); } static struct in6_addrpolicy * lookup_addrsel_policy(struct sockaddr_in6 *key) { struct in6_addrpolicy *match = NULL; ADDRSEL_LOCK(); match = match_addrsel_policy(key); if (match == NULL) match = &V_defaultaddrpolicy; else match->use++; ADDRSEL_UNLOCK(); return (match); } /* * Subroutines to manage the address selection policy table via sysctl. */ struct walkarg { struct sysctl_req *w_req; }; static int in6_src_sysctl(SYSCTL_HANDLER_ARGS); SYSCTL_DECL(_net_inet6_ip6); static SYSCTL_NODE(_net_inet6_ip6, IPV6CTL_ADDRCTLPOLICY, addrctlpolicy, CTLFLAG_RD, in6_src_sysctl, ""); static int in6_src_sysctl(SYSCTL_HANDLER_ARGS) { struct walkarg w; if (req->newptr) return EPERM; bzero(&w, sizeof(w)); w.w_req = req; return (walk_addrsel_policy(dump_addrsel_policyent, &w)); } int in6_src_ioctl(u_long cmd, caddr_t data) { struct in6_addrpolicy ent0; if (cmd != SIOCAADDRCTL_POLICY && cmd != SIOCDADDRCTL_POLICY) return (EOPNOTSUPP); /* check for safety */ ent0 = *(struct in6_addrpolicy *)data; if (ent0.label == ADDR_LABEL_NOTAPP) return (EINVAL); /* check if the prefix mask is consecutive. */ if (in6_mask2len(&ent0.addrmask.sin6_addr, NULL) < 0) return (EINVAL); /* clear trailing garbages (if any) of the prefix address. */ IN6_MASK_ADDR(&ent0.addr.sin6_addr, &ent0.addrmask.sin6_addr); ent0.use = 0; switch (cmd) { case SIOCAADDRCTL_POLICY: return (add_addrsel_policyent(&ent0)); case SIOCDADDRCTL_POLICY: return (delete_addrsel_policyent(&ent0)); } return (0); /* XXX: compromise compilers */ } /* * The followings are implementation of the policy table using a * simple tail queue. * XXX such details should be hidden. * XXX implementation using binary tree should be more efficient. */ struct addrsel_policyent { TAILQ_ENTRY(addrsel_policyent) ape_entry; struct in6_addrpolicy ape_policy; }; TAILQ_HEAD(addrsel_policyhead, addrsel_policyent); static VNET_DEFINE(struct addrsel_policyhead, addrsel_policytab); #define V_addrsel_policytab VNET(addrsel_policytab) static void init_policy_queue(void) { TAILQ_INIT(&V_addrsel_policytab); } static int add_addrsel_policyent(struct in6_addrpolicy *newpolicy) { struct addrsel_policyent *new, *pol; new = malloc(sizeof(*new), M_IFADDR, M_WAITOK); ADDRSEL_XLOCK(); ADDRSEL_LOCK(); /* duplication check */ TAILQ_FOREACH(pol, &V_addrsel_policytab, ape_entry) { if (IN6_ARE_ADDR_EQUAL(&newpolicy->addr.sin6_addr, &pol->ape_policy.addr.sin6_addr) && IN6_ARE_ADDR_EQUAL(&newpolicy->addrmask.sin6_addr, &pol->ape_policy.addrmask.sin6_addr)) { ADDRSEL_UNLOCK(); ADDRSEL_XUNLOCK(); free(new, M_IFADDR); return (EEXIST); /* or override it? */ } } bzero(new, sizeof(*new)); /* XXX: should validate entry */ new->ape_policy = *newpolicy; TAILQ_INSERT_TAIL(&V_addrsel_policytab, new, ape_entry); ADDRSEL_UNLOCK(); ADDRSEL_XUNLOCK(); return (0); } static int delete_addrsel_policyent(struct in6_addrpolicy *key) { struct addrsel_policyent *pol; ADDRSEL_XLOCK(); ADDRSEL_LOCK(); /* search for the entry in the table */ TAILQ_FOREACH(pol, &V_addrsel_policytab, ape_entry) { if (IN6_ARE_ADDR_EQUAL(&key->addr.sin6_addr, &pol->ape_policy.addr.sin6_addr) && IN6_ARE_ADDR_EQUAL(&key->addrmask.sin6_addr, &pol->ape_policy.addrmask.sin6_addr)) { break; } } if (pol == NULL) { ADDRSEL_UNLOCK(); ADDRSEL_XUNLOCK(); return (ESRCH); } TAILQ_REMOVE(&V_addrsel_policytab, pol, ape_entry); ADDRSEL_UNLOCK(); ADDRSEL_XUNLOCK(); free(pol, M_IFADDR); return (0); } static int walk_addrsel_policy(int (*callback)(struct in6_addrpolicy *, void *), void *w) { struct addrsel_policyent *pol; int error = 0; ADDRSEL_SLOCK(); TAILQ_FOREACH(pol, &V_addrsel_policytab, ape_entry) { if ((error = (*callback)(&pol->ape_policy, w)) != 0) { ADDRSEL_SUNLOCK(); return (error); } } ADDRSEL_SUNLOCK(); return (error); } static int dump_addrsel_policyent(struct in6_addrpolicy *pol, void *arg) { int error = 0; struct walkarg *w = arg; error = SYSCTL_OUT(w->w_req, pol, sizeof(*pol)); return (error); } static struct in6_addrpolicy * match_addrsel_policy(struct sockaddr_in6 *key) { struct addrsel_policyent *pent; struct in6_addrpolicy *bestpol = NULL, *pol; int matchlen, bestmatchlen = -1; u_char *mp, *ep, *k, *p, m; TAILQ_FOREACH(pent, &V_addrsel_policytab, ape_entry) { matchlen = 0; pol = &pent->ape_policy; mp = (u_char *)&pol->addrmask.sin6_addr; ep = mp + 16; /* XXX: scope field? */ k = (u_char *)&key->sin6_addr; p = (u_char *)&pol->addr.sin6_addr; for (; mp < ep && *mp; mp++, k++, p++) { m = *mp; if ((*k & m) != *p) goto next; /* not match */ if (m == 0xff) /* short cut for a typical case */ matchlen += 8; else { while (m >= 0x80) { matchlen++; m <<= 1; } } } /* matched. check if this is better than the current best. */ if (bestpol == NULL || matchlen > bestmatchlen) { bestpol = pol; bestmatchlen = matchlen; } next: continue; } return (bestpol); } Index: user/ngie/more-tests2/sys =================================================================== --- user/ngie/more-tests2/sys (revision 292053) +++ user/ngie/more-tests2/sys (revision 292054) Property changes on: user/ngie/more-tests2/sys ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,1 ## Merged /head/sys:r292006-292053 Index: user/ngie/more-tests2/tools/tools/ioat/ioatcontrol.8 =================================================================== --- user/ngie/more-tests2/tools/tools/ioat/ioatcontrol.8 (revision 292053) +++ user/ngie/more-tests2/tools/tools/ioat/ioatcontrol.8 (revision 292054) @@ -1,164 +1,170 @@ .\" Copyright (c) 2015 EMC / Isilon Storage Division .\" All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" $FreeBSD$ .\" -.Dd October 28, 2015 +.Dd December 9, 2015 .Dt IOATCONTROL 8 .Os .Sh NAME .Nm ioatcontrol .Nd Userspace tool to test .Xr ioat 4 .Sh SYNOPSIS .Nm +.Op Fl E .Op Fl f +.Op Fl m .Op Fl V .Ar channel_number .Ar num_txns .Ar [ bufsize .Ar [ chain-len .Ar [ duration ] ] ] .Nm .Fl r .Op Fl v .Op Fl V .Op Fl w .Ar channel_number .Ar address .Ar [ bufsize ] .Sh DESCRIPTION .Nm allows one to issue some number of test operations to the .Xr ioat 4 driver on a specific hardware channel. The arguments are as follows: .Bl -tag -width Ds +.It Fl E +Test non-contiguous 8k copy. .It Fl f Test block fill (by default, .Nm tests copy) +.It Fl m +Test memcpy instead of DMA. .It Fl V Verify copies/fills for accuracy .El .Pp Alternatively one can use .Nm .Fl r to issue DMA to or from a specific .Ar address . The arguments in "raw" mode are: .Bl -tag -width Ds .It Fl v .Ar address is a kernel virtual address (by default, .Ar address is assumed to be a physical address) .It Fl V Dump the resulting hex to syslog .It Fl w Write to the specified .Ar address (by default, .Nm .Fl r reads) .El .Pp .Nm operates in one of two modes; if the .Ar duration argument is passed, .Nm tries to estimate the copy rate in bytes per second by running .Ar num_txns repeatedly in loop. If .Ar duration is not passed, .Nm only runs through .Ar num_txns once and prints the total bytes copied, as well as error information. .Pp The .Ar bufsize argument determines the size of buffers to use for each .Fn ioat_copy invocation. The default is 256 KB. In raw mode, the default is 4 KB. .Pp The .Ar chain-len argument determines the number of copies to chain together in a single DMA transaction. The default is 1, and the maximum is currently 4. .Pp The .Ar duration argument specifies an approximate time limit for the test, in milliseconds. .Pp The test will allocate two chunks of memory for each component of each transaction's chain. It will initialize them with specific data patterns. During the test, it submits DMA requests to copy between pairs of buffers. If the .Fl V flag was specified, it will compare the contents in the callback for a copy error. .Sh FILES .Pa /dev/ioat_test .Pp The interface between .Nm and .Xr ioat 4 . .Xr ioat 4 exposes it with .Cd hw.ioat.enable_ioat_test=1 . .Sh DIAGNOSTICS The wait channel .Va test_submit indicates that the test code has enqueued all requested transactions and is waiting on the IOAT hardware to complete one before issuing another operation. .Sh SEE ALSO .Xr ioat 4 .Sh HISTORY The .Xr ioat 4 driver first appeared in .Fx 11.0 . .Sh AUTHORS The .Xr ioat 4 driver and .Nm tool were developed by .An \&Jim Harris Aq Mt jimharris@FreeBSD.org , .An \&Carl Delsey Aq Mt carl.r.delsey@intel.com , and .An \&Conrad Meyer Aq Mt cem@FreeBSD.org . This manual page was written by .An \&Conrad Meyer Aq Mt cem@FreeBSD.org . Index: user/ngie/more-tests2/tools/tools/ioat/ioatcontrol.c =================================================================== --- user/ngie/more-tests2/tools/tools/ioat/ioatcontrol.c (revision 292053) +++ user/ngie/more-tests2/tools/tools/ioat/ioatcontrol.c (revision 292054) @@ -1,229 +1,248 @@ /*- * Copyright (C) 2012 Intel Corporation * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include "ioat_test.h" static int prettyprint(struct ioat_test *); static void usage(void) { - printf("Usage: %s [-fV] [ " + printf("Usage: %s [-E|-f|-m] [-V] [ " "[ [duration]]]\n", getprogname()); printf(" %s -r [-vV] []\n", getprogname()); exit(EX_USAGE); } static void main_raw(struct ioat_test *t, int argc, char **argv) { int fd; /* Raw DMA defaults */ t->testkind = IOAT_TEST_RAW_DMA; t->transactions = 1; t->chain_depth = 1; t->buffer_size = 4 * 1024; t->raw_target = strtoull(argv[1], NULL, 0); if (t->raw_target == 0) { printf("Target shoudln't be NULL\n"); exit(EX_USAGE); } if (argc >= 3) { t->buffer_size = atoi(argv[2]); if (t->buffer_size == 0) { printf("Buffer size must be greater than zero\n"); exit(EX_USAGE); } } fd = open("/dev/ioat_test", O_RDWR); if (fd < 0) { printf("Cannot open /dev/ioat_test\n"); exit(EX_UNAVAILABLE); } (void)ioctl(fd, IOAT_DMATEST, t); close(fd); exit(prettyprint(t)); } int main(int argc, char **argv) { struct ioat_test t; int fd, ch; - bool fflag, rflag; + bool fflag, rflag, Eflag, mflag; + unsigned modeflags; - while ((ch = getopt(argc, argv, "rfvVw")) != -1) { + fflag = rflag = Eflag = mflag = false; + modeflags = 0; + + while ((ch = getopt(argc, argv, "EfmrvVw")) != -1) { switch (ch) { + case 'E': + Eflag = true; + modeflags++; + break; case 'f': fflag = true; + modeflags++; break; + case 'm': + mflag = true; + modeflags++; + break; case 'r': rflag = true; + modeflags++; break; case 'v': t.raw_is_virtual = true; break; case 'V': t.verify = true; break; case 'w': t.raw_write = true; break; default: usage(); } } argc -= optind; argv += optind; if (argc < 2) usage(); - if (rflag && fflag) { - printf("Invalid: -r and -f\n"); + if (modeflags > 1) { + printf("Invalid: Cannot use >1 mode flag (-E, -f, -m, or -r)\n"); usage(); } /* Defaults for optional args */ t.buffer_size = 256 * 1024; t.chain_depth = 2; t.duration = 0; t.testkind = IOAT_TEST_DMA; if (fflag) t.testkind = IOAT_TEST_FILL; + else if (Eflag) { + t.testkind = IOAT_TEST_DMA_8K; + t.buffer_size = 8 * 1024; + } else if (mflag) + t.testkind = IOAT_TEST_MEMCPY; t.channel_index = atoi(argv[0]); if (t.channel_index > 8) { printf("Channel number must be between 0 and 7.\n"); return (EX_USAGE); } if (rflag) { main_raw(&t, argc, argv); return (EX_OK); } t.transactions = atoi(argv[1]); if (argc >= 3) { t.buffer_size = atoi(argv[2]); if (t.buffer_size == 0) { printf("Buffer size must be greater than zero\n"); return (EX_USAGE); } } if (argc >= 4) { t.chain_depth = atoi(argv[3]); if (t.chain_depth < 1) { printf("Chain length must be greater than zero\n"); return (EX_USAGE); } } if (argc >= 5) { t.duration = atoi(argv[4]); if (t.duration < 1) { printf("Duration must be greater than zero\n"); return (EX_USAGE); } } fd = open("/dev/ioat_test", O_RDWR); if (fd < 0) { printf("Cannot open /dev/ioat_test\n"); return (EX_UNAVAILABLE); } (void)ioctl(fd, IOAT_DMATEST, &t); close(fd); return (prettyprint(&t)); } static int prettyprint(struct ioat_test *t) { char bps[10], bytesh[10]; uintmax_t bytes; if (t->status[IOAT_TEST_NO_DMA_ENGINE] != 0 || t->status[IOAT_TEST_NO_MEMORY] != 0 || t->status[IOAT_TEST_MISCOMPARE] != 0) { printf("Errors:\n"); if (t->status[IOAT_TEST_NO_DMA_ENGINE] != 0) printf("\tNo DMA engine present: %u\n", (unsigned)t->status[IOAT_TEST_NO_DMA_ENGINE]); if (t->status[IOAT_TEST_NO_MEMORY] != 0) printf("\tOut of memory: %u\n", (unsigned)t->status[IOAT_TEST_NO_MEMORY]); if (t->status[IOAT_TEST_MISCOMPARE] != 0) printf("\tMiscompares: %u\n", (unsigned)t->status[IOAT_TEST_MISCOMPARE]); } printf("Processed %u txns\n", (unsigned)t->status[IOAT_TEST_OK] / t->chain_depth); bytes = (uintmax_t)t->buffer_size * t->status[IOAT_TEST_OK]; humanize_number(bytesh, sizeof(bytesh), (int64_t)bytes, "B", HN_AUTOSCALE, HN_DECIMAL); if (t->duration) { humanize_number(bps, sizeof(bps), (int64_t)1000 * bytes / t->duration, "B/s", HN_AUTOSCALE, HN_DECIMAL); printf("%ju (%s) copied in %u ms (%s)\n", bytes, bytesh, (unsigned)t->duration, bps); } else printf("%ju (%s) copied\n", bytes, bytesh); return (EX_OK); } Index: user/ngie/more-tests2/usr.bin/clang/lldb/Makefile =================================================================== --- user/ngie/more-tests2/usr.bin/clang/lldb/Makefile (revision 292053) +++ user/ngie/more-tests2/usr.bin/clang/lldb/Makefile (revision 292054) @@ -1,166 +1,166 @@ # $FreeBSD$ .include PROG_CXX=lldb LLDB_SRCS=${.CURDIR}/../../../contrib/llvm/tools/lldb CFLAGS+= -I${LLDB_SRCS}/include CXXFLAGS+= -std=c++11 SRCDIR= tools/lldb/tools/driver SRCS= Driver.cpp \ Platform.cpp # Man page directory .PATH: ${LLDB_SRCS}/docs LIBADD= edit panel ncursesw execinfo z LLDB_LIBS=\ lldb \ \ lldbAPI \ lldbBreakpoint \ lldbCommands \ lldbCore \ lldbDataFormatters \ lldbExpression \ lldbHostFreeBSD \ lldbHostCommon \ lldbHostPOSIX \ lldbInitialization \ lldbInterpreter \ lldbSymbol \ lldbTarget \ lldbUtility \ \ lldbPluginABISysV_arm \ lldbPluginABISysV_arm64 \ lldbPluginABISysV_i386 \ lldbPluginABISysV_mips \ lldbPluginABISysV_mips64 \ lldbPluginABISysV_ppc \ lldbPluginABISysV_ppc64 \ lldbPluginABISysV_x86_64 \ lldbPluginCXXItaniumABI \ lldbPluginDisassemblerLLVM \ lldbPluginInstructionARM \ lldbPluginInstructionARM64 \ lldbPluginInstructionMIPS \ lldbPluginInstructionMIPS64 \ lldbPluginInstrumentationRuntimeAddressSanitizer \ lldbPluginJITLoaderGDB \ lldbPluginSymbolFileDWARF \ lldbPluginSymbolFileSymtab \ lldbPluginDynamicLoaderStatic \ lldbPluginDynamicLoaderPosixDYLD \ lldbPluginMemoryHistoryASan \ lldbPluginObjectContainerBSDArchive \ lldbPluginObjectFileELF \ lldbPluginObjectFileJIT \ lldbPluginSymbolVendorELF \ lldbPluginPlatformFreeBSD \ lldbPluginPlatformGDB \ lldbPluginProcessElfCore \ lldbPluginProcessFreeBSD \ lldbPluginProcessGDBRemote \ lldbPluginProcessUtility \ lldbPluginProcessPOSIX \ lldbPluginProcessFreeBSD \ lldbPluginUnwindAssemblyInstEmulation \ lldbPluginUnwindAssemblyX86 LDADD+= -Wl,--start-group .for lib in ${LLDB_LIBS} DPADD+= ${.OBJDIR}/../../../lib/clang/lib${lib}/lib${lib}.a LDADD+= ${.OBJDIR}/../../../lib/clang/lib${lib}/lib${lib}.a .endfor LDADD+= -Wl,--end-group # Clang and LLVM libraries LIBDEPS=\ clangfrontend \ clangdriver \ clangserialization \ clangcodegen \ clangparse \ clangsema \ clanganalysis \ clangedit \ clangast \ clanglex \ clangbasic \ \ llvmoption \ llvmobjcarcopts \ llvmlinker \ llvmmcjit \ llvmruntimedyld \ llvmexecutionengine \ llvmirreader \ llvmipo \ llvmvectorize \ llvmbitwriter \ llvmasmparser \ llvmaarch64disassembler \ llvmaarch64codegen \ llvmaarch64asmparser \ llvmaarch64desc \ llvmaarch64info \ llvmaarch64instprinter \ llvmaarch64utils \ llvmarmdisassembler \ llvmarmcodegen \ llvmarmasmparser \ llvmarmdesc \ llvmarminfo \ llvmarminstprinter \ llvmmipsdisassembler \ llvmmipscodegen \ llvmmipsasmparser \ llvmmipsdesc \ llvmmipsinfo \ llvmmipsinstprinter \ llvmpowerpcdisassembler \ llvmpowerpccodegen \ llvmpowerpcasmparser \ llvmpowerpcdesc \ llvmpowerpcinfo \ llvmpowerpcinstprinter \ llvmsparcdisassembler \ llvmsparccodegen \ llvmsparcasmparser \ llvmsparcdesc \ llvmsparcinfo \ llvmsparcinstprinter \ llvmx86disassembler \ llvmx86asmparser \ llvmx86codegen \ llvmselectiondag \ llvmasmprinter \ llvmcodegen \ llvmtarget \ llvmscalaropts \ llvmprofiledata \ llvminstcombine \ llvmtransformutils \ llvmipa \ llvmanalysis \ llvminstrumentation \ llvmx86desc \ llvmobject \ llvmmcparser \ llvmbitreader \ llvmmcdisassembler \ llvmx86info \ llvmx86instprinter \ llvmmc \ llvmx86utils \ llvmcore \ llvmsupport -.include "../clang.prog.mk" +LIBADD+= pthread -LDADD+= -lpthread +.include "../clang.prog.mk" Index: user/ngie/more-tests2/usr.sbin/iostat/iostat.c =================================================================== --- user/ngie/more-tests2/usr.sbin/iostat/iostat.c (revision 292053) +++ user/ngie/more-tests2/usr.sbin/iostat/iostat.c (revision 292054) @@ -1,966 +1,1021 @@ /* * Copyright (c) 1997, 1998, 2000, 2001 Kenneth D. Merry * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. The name of the author may not be used to endorse or promote products * derived from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ /* * Parts of this program are derived from the original FreeBSD iostat * program: */ /*- * Copyright (c) 1986, 1991, 1993 * The Regents of the University of California. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * Ideas for the new iostat statistics output modes taken from the NetBSD * version of iostat: */ /* * Copyright (c) 1996 John M. Vinopal * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. All advertising materials mentioning features or use of this software * must display the following acknowledgement: * This product includes software developed for the NetBSD Project * by John M. Vinopal. * 4. The name of the author may not be used to endorse or promote products * derived from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED * AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include #include #include #include #include #include #include #include #include #include #include #include #include +#include #include #include #include #include #include struct nlist namelist[] = { #define X_TTY_NIN 0 { "_tty_nin" }, #define X_TTY_NOUT 1 { "_tty_nout" }, #define X_BOOTTIME 2 { "_boottime" }, #define X_END 2 { NULL }, }; #define IOSTAT_DEFAULT_ROWS 20 /* Traditional default `wrows' */ struct statinfo cur, last; int num_devices; struct device_selection *dev_select; int maxshowdevs; volatile sig_atomic_t headercount; volatile sig_atomic_t wresized; /* Tty resized, when non-zero. */ +volatile sig_atomic_t alarm_rang; +volatile sig_atomic_t return_requested; unsigned short wrows; /* Current number of tty rows. */ int dflag = 0, Iflag = 0, Cflag = 0, Tflag = 0, oflag = 0, Kflag = 0; int xflag = 0, zflag = 0; /* local function declarations */ static void usage(void); static void needhdr(int signo); static void needresize(int signo); +static void needreturn(int signo); +static void alarm_clock(int signo); static void doresize(void); static void phdr(void); static void devstats(int perf_select, long double etime, int havelast); static void cpustats(void); static int readvar(kvm_t *kd, const char *name, int nlid, void *ptr, size_t len); static void usage(void) { /* * We also support the following 'traditional' syntax: * iostat [drives] [wait [count]] * This isn't mentioned in the man page, or the usage statement, * but it is supported. */ fprintf(stderr, "usage: iostat [-CdhIKoTxz?] [-c count] [-M core]" " [-n devs] [-N system]\n" "\t [-t type,if,pass] [-w wait] [drives]\n"); } int main(int argc, char **argv) { int c, i; int tflag = 0, hflag = 0, cflag = 0, wflag = 0, nflag = 0; int count = 0, waittime = 0; char *memf = NULL, *nlistf = NULL; struct devstat_match *matches; + struct itimerval alarmspec; int num_matches = 0; char errbuf[_POSIX2_LINE_MAX]; kvm_t *kd = NULL; long generation; int num_devices_specified; int num_selected, num_selections; long select_generation; char **specified_devices; devstat_select_mode select_mode; float f; int havelast = 0; matches = NULL; maxshowdevs = 3; while ((c = getopt(argc, argv, "c:CdhIKM:n:N:ot:Tw:xz?")) != -1) { switch(c) { case 'c': cflag++; count = atoi(optarg); if (count < 1) errx(1, "count %d is < 1", count); break; case 'C': Cflag++; break; case 'd': dflag++; break; case 'h': hflag++; break; case 'I': Iflag++; break; case 'K': Kflag++; break; case 'M': memf = optarg; break; case 'n': nflag++; maxshowdevs = atoi(optarg); if (maxshowdevs < 0) errx(1, "number of devices %d is < 0", maxshowdevs); break; case 'N': nlistf = optarg; break; case 'o': oflag++; break; case 't': tflag++; if (devstat_buildmatch(optarg, &matches, &num_matches) != 0) errx(1, "%s", devstat_errbuf); break; case 'T': Tflag++; break; case 'w': wflag++; f = atof(optarg); waittime = f * 1000; if (waittime < 1) errx(1, "wait time is < 1ms"); break; case 'x': xflag++; break; case 'z': zflag++; break; default: usage(); exit(1); break; } } argc -= optind; argv += optind; if (nlistf != NULL || memf != NULL) { kd = kvm_openfiles(nlistf, memf, NULL, O_RDONLY, errbuf); if (kd == NULL) errx(1, "kvm_openfiles: %s", errbuf); if (kvm_nlist(kd, namelist) == -1) errx(1, "kvm_nlist: %s", kvm_geterr(kd)); } /* * Make sure that the userland devstat version matches the kernel * devstat version. If not, exit and print a message informing * the user of his mistake. */ if (devstat_checkversion(kd) < 0) errx(1, "%s", devstat_errbuf); /* * Make sure Tflag and/or Cflag are set if dflag == 0. If dflag is * greater than 0, they may be 0 or non-zero. */ if (dflag == 0 && xflag == 0) { Cflag = 1; Tflag = 1; } /* find out how many devices we have */ if ((num_devices = devstat_getnumdevs(kd)) < 0) err(1, "can't get number of devices"); /* * Figure out how many devices we should display. */ if (nflag == 0) { if (xflag > 0) maxshowdevs = num_devices; else if (oflag > 0) { if ((dflag > 0) && (Cflag == 0) && (Tflag == 0)) maxshowdevs = 5; else if ((dflag > 0) && (Tflag > 0) && (Cflag == 0)) maxshowdevs = 5; else maxshowdevs = 4; } else { if ((dflag > 0) && (Cflag == 0)) maxshowdevs = 4; else maxshowdevs = 3; } } cur.dinfo = (struct devinfo *)calloc(1, sizeof(struct devinfo)); if (cur.dinfo == NULL) err(1, "calloc failed"); last.dinfo = (struct devinfo *)calloc(1, sizeof(struct devinfo)); if (last.dinfo == NULL) err(1, "calloc failed"); /* * Grab all the devices. We don't look to see if the list has * changed here, since it almost certainly has. We only look for * errors. */ if (devstat_getdevs(kd, &cur) == -1) errx(1, "%s", devstat_errbuf); num_devices = cur.dinfo->numdevs; generation = cur.dinfo->generation; /* * If the user specified any devices on the command line, see if * they are in the list of devices we have now. */ specified_devices = (char **)malloc(sizeof(char *)); if (specified_devices == NULL) err(1, "malloc failed"); for (num_devices_specified = 0; *argv; ++argv) { if (isdigit(**argv)) break; num_devices_specified++; specified_devices = (char **)realloc(specified_devices, sizeof(char *) * num_devices_specified); if (specified_devices == NULL) err(1, "realloc failed"); specified_devices[num_devices_specified - 1] = *argv; } if (nflag == 0 && maxshowdevs < num_devices_specified) maxshowdevs = num_devices_specified; dev_select = NULL; if ((num_devices_specified == 0) && (num_matches == 0)) select_mode = DS_SELECT_ADD; else select_mode = DS_SELECT_ONLY; /* * At this point, selectdevs will almost surely indicate that the * device list has changed, so we don't look for return values of 0 * or 1. If we get back -1, though, there is an error. */ if (devstat_selectdevs(&dev_select, &num_selected, &num_selections, &select_generation, generation, cur.dinfo->devices, num_devices, matches, num_matches, specified_devices, num_devices_specified, select_mode, maxshowdevs, hflag) == -1) errx(1, "%s", devstat_errbuf); /* * Look for the traditional wait time and count arguments. */ if (*argv) { f = atof(*argv); waittime = f * 1000; /* Let the user know he goofed, but keep going anyway */ if (wflag != 0) warnx("discarding previous wait interval, using" " %g instead", waittime / 1000.0); wflag++; if (*++argv) { count = atoi(*argv); if (cflag != 0) warnx("discarding previous count, using %d" " instead", count); cflag++; } else count = -1; } /* * If the user specified a count, but not an interval, we default * to an interval of 1 second. */ if ((wflag == 0) && (cflag > 0)) waittime = 1 * 1000; /* * If the user specified a wait time, but not a count, we want to * go on ad infinitum. This can be redundant if the user uses the * traditional method of specifying the wait, since in that case we * already set count = -1 above. Oh well. */ if ((wflag > 0) && (cflag == 0)) count = -1; bzero(cur.cp_time, sizeof(cur.cp_time)); cur.tk_nout = 0; cur.tk_nin = 0; /* * Set the snap time to the system boot time (ie: zero), so the * stats are calculated since system boot. */ cur.snap_time = 0; /* * If the user stops the program (control-Z) and then resumes it, * print out the header again. */ (void)signal(SIGCONT, needhdr); /* * If our standard output is a tty, then install a SIGWINCH handler * and set wresized so that our first iteration through the main * iostat loop will peek at the terminal's current rows to find out * how many lines can fit in a screenful of output. */ if (isatty(fileno(stdout)) != 0) { wresized = 1; (void)signal(SIGWINCH, needresize); } else { wresized = 0; wrows = IOSTAT_DEFAULT_ROWS; } + /* + * Register a SIGINT handler so that we can print out final statistics + * when we get that signal + */ + (void)signal(SIGINT, needreturn); + + /* + * Register a SIGALRM handler to implement sleeps if the user uses the + * -c or -w options + */ + (void)signal(SIGALRM, alarm_clock); + alarmspec.it_interval.tv_sec = waittime / 1000; + alarmspec.it_interval.tv_usec = 1000 * (waittime % 1000); + alarmspec.it_value.tv_sec = waittime / 1000; + alarmspec.it_value.tv_usec = 1000 * (waittime % 1000); + setitimer(ITIMER_REAL, &alarmspec, NULL); + for (headercount = 1;;) { struct devinfo *tmp_dinfo; long tmp; long double etime; + sigset_t sigmask, oldsigmask; if (Tflag > 0) { if ((readvar(kd, "kern.tty_nin", X_TTY_NIN, &cur.tk_nin, sizeof(cur.tk_nin)) != 0) || (readvar(kd, "kern.tty_nout", X_TTY_NOUT, &cur.tk_nout, sizeof(cur.tk_nout))!= 0)) { Tflag = 0; warnx("disabling TTY statistics"); } } if (Cflag > 0) { if (kd == NULL) { if (readvar(kd, "kern.cp_time", 0, &cur.cp_time, sizeof(cur.cp_time)) != 0) Cflag = 0; } else { if (kvm_getcptime(kd, cur.cp_time) < 0) { warnx("kvm_getcptime: %s", kvm_geterr(kd)); Cflag = 0; } } if (Cflag == 0) warnx("disabling CPU time statistics"); } if (!--headercount) { phdr(); if (wresized != 0) doresize(); headercount = wrows; } tmp_dinfo = last.dinfo; last.dinfo = cur.dinfo; cur.dinfo = tmp_dinfo; last.snap_time = cur.snap_time; /* * Here what we want to do is refresh our device stats. * devstat_getdevs() returns 1 when the device list has changed. * If the device list has changed, we want to go through * the selection process again, in case a device that we * were previously displaying has gone away. */ switch (devstat_getdevs(kd, &cur)) { case -1: errx(1, "%s", devstat_errbuf); break; case 1: { int retval; num_devices = cur.dinfo->numdevs; generation = cur.dinfo->generation; retval = devstat_selectdevs(&dev_select, &num_selected, &num_selections, &select_generation, generation, cur.dinfo->devices, num_devices, matches, num_matches, specified_devices, num_devices_specified, select_mode, maxshowdevs, hflag); switch(retval) { case -1: errx(1, "%s", devstat_errbuf); break; case 1: phdr(); if (wresized != 0) doresize(); headercount = wrows; break; default: break; } break; } default: break; } /* * We only want to re-select devices if we're in 'top' * mode. This is the only mode where the devices selected * could actually change. */ if (hflag > 0) { int retval; retval = devstat_selectdevs(&dev_select, &num_selected, &num_selections, &select_generation, generation, cur.dinfo->devices, num_devices, matches, num_matches, specified_devices, num_devices_specified, select_mode, maxshowdevs, hflag); switch(retval) { case -1: errx(1,"%s", devstat_errbuf); break; case 1: phdr(); if (wresized != 0) doresize(); headercount = wrows; break; default: break; } } if (Tflag > 0) { tmp = cur.tk_nin; cur.tk_nin -= last.tk_nin; last.tk_nin = tmp; tmp = cur.tk_nout; cur.tk_nout -= last.tk_nout; last.tk_nout = tmp; } etime = cur.snap_time - last.snap_time; if (etime == 0.0) etime = 1.0; for (i = 0; i < CPUSTATES; i++) { tmp = cur.cp_time[i]; cur.cp_time[i] -= last.cp_time[i]; last.cp_time[i] = tmp; } if (xflag == 0 && Tflag > 0) printf("%4.0Lf %5.0Lf", cur.tk_nin / etime, cur.tk_nout / etime); devstats(hflag, etime, havelast); if (xflag == 0) { if (Cflag > 0) cpustats(); printf("\n"); } fflush(stdout); - if (count >= 0 && --count <= 0) + if ((count >= 0 && --count <= 0) || return_requested) break; - usleep(waittime * 1000); + /* + * Use sigsuspend to safely sleep until either signal is + * received + */ + alarm_rang = 0; + sigemptyset(&sigmask); + sigaddset(&sigmask, SIGINT); + sigaddset(&sigmask, SIGALRM); + sigprocmask(SIG_BLOCK, &sigmask, &oldsigmask); + while (! (alarm_rang || return_requested) ) { + sigsuspend(&oldsigmask); + } + sigprocmask(SIG_UNBLOCK, &sigmask, NULL); + havelast = 1; } exit(0); } /* * Force a header to be prepended to the next output. */ void needhdr(int signo) { headercount = 1; } /* * When the terminal is resized, force an update of the maximum number of rows * printed between each header repetition. Then force a new header to be * prepended to the next output. */ void needresize(int signo) { wresized = 1; headercount = 1; +} + +/* + * Record the alarm so the main loop can break its sleep + */ +void +alarm_clock(int signo) +{ + alarm_rang = 1; +} + +/* + * Request that the main loop exit soon + */ +void +needreturn(int signo) +{ + return_requested = 1; } /* * Update the global `wrows' count of terminal rows. */ void doresize(void) { int status; struct winsize w; for (;;) { status = ioctl(fileno(stdout), TIOCGWINSZ, &w); if (status == -1 && errno == EINTR) continue; else if (status == -1) err(1, "ioctl"); if (w.ws_row > 3) wrows = w.ws_row - 3; else wrows = IOSTAT_DEFAULT_ROWS; break; } /* * Inhibit doresize() calls until we are rescheduled by SIGWINCH. */ wresized = 0; } static void phdr(void) { int i, printed; char devbuf[256]; /* * If xflag is set, we need a per-loop header, not a page header, so * just return. We'll print the header in devstats(). */ if (xflag > 0) return; if (Tflag > 0) (void)printf(" tty"); for (i = 0, printed=0;(i < num_devices) && (printed < maxshowdevs);i++){ int di; if ((dev_select[i].selected != 0) && (dev_select[i].selected <= maxshowdevs)) { di = dev_select[i].position; snprintf(devbuf, sizeof(devbuf), "%s%d", cur.dinfo->devices[di].device_name, cur.dinfo->devices[di].unit_number); if (oflag > 0) (void)printf("%13.6s ", devbuf); else printf("%16.6s ", devbuf); printed++; } } if (Cflag > 0) (void)printf(" cpu\n"); else (void)printf("\n"); if (Tflag > 0) (void)printf(" tin tout"); for (i=0, printed = 0;(i < num_devices) && (printed < maxshowdevs);i++){ if ((dev_select[i].selected != 0) && (dev_select[i].selected <= maxshowdevs)) { if (oflag > 0) { if (Iflag == 0) (void)printf(" sps tps msps "); else (void)printf(" blk xfr msps "); } else { if (Iflag == 0) printf(" KB/t tps MB/s "); else printf(" KB/t xfrs MB "); } printed++; } } if (Cflag > 0) (void)printf(" us ni sy in id\n"); else printf("\n"); } static void devstats(int perf_select, long double etime, int havelast) { int dn; long double transfers_per_second, transfers_per_second_read; long double transfers_per_second_write; long double kb_per_transfer, mb_per_second, mb_per_second_read; long double mb_per_second_write; u_int64_t total_bytes, total_transfers, total_blocks; u_int64_t total_bytes_read, total_transfers_read; u_int64_t total_bytes_write, total_transfers_write; long double busy_pct, busy_time; u_int64_t queue_len; long double total_mb, blocks_per_second, total_duration; long double ms_per_other, ms_per_read, ms_per_write, ms_per_transaction; int firstline = 1; char *devname; if (xflag > 0) { printf(" extended device statistics "); if (Tflag > 0) printf(" tty "); if (Cflag > 0) printf(" cpu "); printf("\n"); if (Iflag == 0) { printf("device r/s w/s kr/s kw/s " " ms/r ms/w ms/o ms/t qlen %%b "); } else { printf("device r/i w/i kr/i" " kw/i qlen tsvc_t/i sb/i "); } if (Tflag > 0) printf("tin tout "); if (Cflag > 0) printf("us ni sy in id "); printf("\n"); } for (dn = 0; dn < num_devices; dn++) { int di; if (((perf_select == 0) && (dev_select[dn].selected == 0)) || (dev_select[dn].selected > maxshowdevs)) continue; di = dev_select[dn].position; if (devstat_compute_statistics(&cur.dinfo->devices[di], havelast ? &last.dinfo->devices[di] : NULL, etime, DSM_TOTAL_BYTES, &total_bytes, DSM_TOTAL_BYTES_READ, &total_bytes_read, DSM_TOTAL_BYTES_WRITE, &total_bytes_write, DSM_TOTAL_TRANSFERS, &total_transfers, DSM_TOTAL_TRANSFERS_READ, &total_transfers_read, DSM_TOTAL_TRANSFERS_WRITE, &total_transfers_write, DSM_TOTAL_BLOCKS, &total_blocks, DSM_KB_PER_TRANSFER, &kb_per_transfer, DSM_TRANSFERS_PER_SECOND, &transfers_per_second, DSM_TRANSFERS_PER_SECOND_READ, &transfers_per_second_read, DSM_TRANSFERS_PER_SECOND_WRITE, &transfers_per_second_write, DSM_MB_PER_SECOND, &mb_per_second, DSM_MB_PER_SECOND_READ, &mb_per_second_read, DSM_MB_PER_SECOND_WRITE, &mb_per_second_write, DSM_BLOCKS_PER_SECOND, &blocks_per_second, DSM_MS_PER_TRANSACTION, &ms_per_transaction, DSM_MS_PER_TRANSACTION_READ, &ms_per_read, DSM_MS_PER_TRANSACTION_WRITE, &ms_per_write, DSM_MS_PER_TRANSACTION_OTHER, &ms_per_other, DSM_BUSY_PCT, &busy_pct, DSM_QUEUE_LENGTH, &queue_len, DSM_TOTAL_DURATION, &total_duration, DSM_TOTAL_BUSY_TIME, &busy_time, DSM_NONE) != 0) errx(1, "%s", devstat_errbuf); if (perf_select != 0) { dev_select[dn].bytes = total_bytes; if ((dev_select[dn].selected == 0) || (dev_select[dn].selected > maxshowdevs)) continue; } if (Kflag > 0 || xflag > 0) { int block_size = cur.dinfo->devices[di].block_size; total_blocks = total_blocks * (block_size ? block_size : 512) / 1024; } if (xflag > 0) { if (asprintf(&devname, "%s%d", cur.dinfo->devices[di].device_name, cur.dinfo->devices[di].unit_number) == -1) err(1, "asprintf"); /* * If zflag is set, skip any devices with zero I/O. */ if (zflag == 0 || transfers_per_second_read > 0.05 || transfers_per_second_write > 0.05 || mb_per_second_read > ((long double).0005)/1024 || mb_per_second_write > ((long double).0005)/1024 || busy_pct > 0.5) { if (Iflag == 0) printf("%-8.8s %5d %5d %8.1Lf " "%8.1Lf %5d %5d %5d %5d " "%4" PRIu64 " %3.0Lf ", devname, (int)transfers_per_second_read, (int)transfers_per_second_write, mb_per_second_read * 1024, mb_per_second_write * 1024, (int)ms_per_read, (int)ms_per_write, (int)ms_per_other, (int)ms_per_transaction, queue_len, busy_pct); else printf("%-8.8s %11.1Lf %11.1Lf " "%12.1Lf %12.1Lf %4" PRIu64 " %10.1Lf %9.1Lf ", devname, (long double)total_transfers_read, (long double)total_transfers_write, (long double) total_bytes_read / 1024, (long double) total_bytes_write / 1024, queue_len, total_duration, busy_time); if (firstline) { /* * If this is the first device * we're printing, also print * CPU or TTY stats if requested. */ firstline = 0; if (Tflag > 0) printf("%4.0Lf%5.0Lf", cur.tk_nin / etime, cur.tk_nout / etime); if (Cflag > 0) cpustats(); } printf("\n"); } free(devname); } else if (oflag > 0) { int msdig = (ms_per_transaction < 100.0) ? 1 : 0; if (Iflag == 0) printf("%4.0Lf%4.0Lf%5.*Lf ", blocks_per_second, transfers_per_second, msdig, ms_per_transaction); else printf("%4.1" PRIu64 "%4.1" PRIu64 "%5.*Lf ", total_blocks, total_transfers, msdig, ms_per_transaction); } else { if (Iflag == 0) printf(" %5.2Lf %3.0Lf %5.2Lf ", kb_per_transfer, transfers_per_second, mb_per_second); else { total_mb = total_bytes; total_mb /= 1024 * 1024; printf(" %5.2Lf %3.1" PRIu64 " %5.2Lf ", kb_per_transfer, total_transfers, total_mb); } } } if (xflag > 0 && zflag > 0 && firstline == 1 && (Tflag > 0 || Cflag > 0)) { /* * If zflag is set and we did not print any device * lines I/O because they were all zero, * print TTY/CPU stats. */ printf("%52s",""); if (Tflag > 0) printf("%4.0Lf %5.0Lf", cur.tk_nin / etime, cur.tk_nout / etime); if (Cflag > 0) cpustats(); printf("\n"); } } static void cpustats(void) { int state; double time; time = 0.0; for (state = 0; state < CPUSTATES; ++state) time += cur.cp_time[state]; for (state = 0; state < CPUSTATES; ++state) printf(" %2.0f", rint(100. * cur.cp_time[state] / (time ? time : 1))); } static int readvar(kvm_t *kd, const char *name, int nlid, void *ptr, size_t len) { if (kd != NULL) { ssize_t nbytes; nbytes = kvm_read(kd, namelist[nlid].n_value, ptr, len); if (nbytes < 0) { warnx("kvm_read(%s): %s", namelist[nlid].n_name, kvm_geterr(kd)); return (1); } if (nbytes != len) { warnx("kvm_read(%s): expected %zu bytes, got %zd bytes", namelist[nlid].n_name, len, nbytes); return (1); } } else { size_t nlen = len; if (sysctlbyname(name, ptr, &nlen, NULL, 0) == -1) { warn("sysctl(%s...) failed", name); return (1); } if (nlen != len) { warnx("sysctl(%s...): expected %lu, got %lu", name, (unsigned long)len, (unsigned long)nlen); return (1); } } return (0); } Index: user/ngie/more-tests2/usr.sbin/pmcstudy/pmcstudy.8 =================================================================== --- user/ngie/more-tests2/usr.sbin/pmcstudy/pmcstudy.8 (revision 292053) +++ user/ngie/more-tests2/usr.sbin/pmcstudy/pmcstudy.8 (revision 292054) @@ -1,143 +1,145 @@ .\" Copyright (c) 2015 .\" Netflix Inc. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" $FreeBSD$ .\" .Dd Mar 26, 2015 .Dt PMCSTUDY 8 .Os .Sh NAME .Nm pmcstudy .Nd Perform various studies on a system's overall PMCs. .Sh SYNOPSIS .Nm -.Oo Fl i Ar inputfile | Fl T | Fl v | Fl m Ar max | Fl e exp | Fl Ar E | Fl h | fl H Oc +.Oo Fl i Ar inputfile | Fl A | Fl T | Fl v | Fl m Ar max | Fl e exp | Fl Ar E | Fl h | fl H Oc .Nm .Fl i Ar inputfile .Nm .Fl v .Nm .Fl m Ar max .Nm .Fl e Ar exp-name .Nm .Fl E Ar your-expr .Nm .Fl h .Nm .Fl H .Nm .Fl T .Sh DESCRIPTION The .Nm program is designed to run various tests against your systems performance. There are roughly 20-22 canned tests that setup specific PMCs and then run various formulas on the output information. These formulas can be found in Intel documentation "Using Intel Vtune amplifier xe on NNN Generation Intel Core Processors". The NNN is either -2nd, 3rd or 4th generation i.e., Sandy Bridge, Ivy Bridge and Haswell. -Currently the program only works on these three Intel processor types. +2nd, 3rd, 4th or 5th generation i.e., Sandy Bridge, Ivy Bridge, Haswell and Broadwell. +Currently the program only works on these four Intel processor types. .Sh OPTIONS The following options are available: .Bl -tag -width indent .It Fl i Ar filename If this option is supplied, instead of running a .Xr pmcstat 8 command to collect the current running information the filename will be read in as input instead. .It Fl H This option will display the complete list of canned formulas that can be run including their names which can be input to the .Fl e option. .It Fl e Ar name Execute the canned test .Ar name on the running kernel. .It Fl h If you add this option to the .Fl e option the test will not execute but instead give you a small description of the test that would run. .It Fl T This option will execute a test of every PMC to validate that they are working on your system. If a PMC does not show up in this test chances are the kernel .Xr hwpmc 4 driver needs updating with new PMC information. .It Fl m Ar num This option can restrict the number of one second samples that will be collected by your system when running a test (it bounds the time the test will run). Without this option the test will run for 1024 seconds or until the user types ctrl-c. .It Fl v The verbose option adds debugging output to the command. .It Fl E Ar expression This option can be used by those that have their own ideas on what formulas they want to run. The expression given to the .Fl E option is a "formula". The formula can declare directly the PMCs by name or you can use an abbreviation %NNN. To find out the abbreviations on your system you may run the .Fl L option. An example of a formula of your own might be .Fl E "FP_ASSIST.ANY / INST_RETIRED.ANY_P" or using the abbreviations on a Haswell machine you would type .Fl E " %176 / %150". You must have spaces between each entry and you may use parentheses to prioritize the operators. Add (+), Subtract (-), Divide (/) and Multiplication (*) are supported. You may also introduce constant numbers. For example you can do a standard efficency test like .Fl E "UOPS_RETIRED.RETIRE_SLOTS / (4 * CPU_CLK_UNHALTED.THREAD_P)". .It Fl L This option will list all known PMCs and their abbreviation (%NNN). +.It Fl A +Run all canned tests. .El .Sh SEE ALSO .Xr pmc 3 , .Xr pmclog 3 , .Xr hwpmc 4 , .Xr pmcstat 8 .Sh HISTORY The .Nm utility first appeared in .Fx 11.0. .Sh AUTHORS .An Randall Stewart Aq Mt rrs@FreeBSD.org Index: user/ngie/more-tests2/usr.sbin/pmcstudy/pmcstudy.c =================================================================== --- user/ngie/more-tests2/usr.sbin/pmcstudy/pmcstudy.c (revision 292053) +++ user/ngie/more-tests2/usr.sbin/pmcstudy/pmcstudy.c (revision 292054) @@ -1,2438 +1,2954 @@ /*- * Copyright (c) 2014, 2015 Netflix Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer, * in this position and unchanged. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. The name of the author may not be used to endorse or promote products * derived from this software without specific prior written permission * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include #include #include #include #include #include #include #include #include #include #include "eval_expr.h" __FBSDID("$FreeBSD$"); +static int max_pmc_counters = 1; +static int run_all = 0; + #define MAX_COUNTER_SLOTS 1024 #define MAX_NLEN 64 #define MAX_CPU 64 static int verbose = 0; extern char **environ; extern struct expression *master_exp; struct expression *master_exp=NULL; #define PMC_INITIAL_ALLOC 512 extern char **valid_pmcs; char **valid_pmcs = NULL; extern int valid_pmc_cnt; int valid_pmc_cnt=0; extern int pmc_allocated_cnt; int pmc_allocated_cnt=0; /* * The following two varients on popen and pclose with * the cavet that they get you the PID so that you * can supply it to pclose so it can send a SIGTERM * to the process. */ static FILE * my_popen(const char *command, const char *dir, pid_t *p_pid) { FILE *io_out, *io_in; int pdesin[2], pdesout[2]; char *argv[4]; pid_t pid; char cmd[4]; char cmd2[1024]; char arg1[4]; if ((strcmp(dir, "r") != 0) && (strcmp(dir, "w") != 0)) { errno = EINVAL; return(NULL); } if (pipe(pdesin) < 0) return (NULL); if (pipe(pdesout) < 0) { (void)close(pdesin[0]); (void)close(pdesin[1]); return (NULL); } strcpy(cmd, "sh"); strcpy(arg1, "-c"); strcpy(cmd2, command); argv[0] = cmd; argv[1] = arg1; argv[2] = cmd2; argv[3] = NULL; switch (pid = fork()) { case -1: /* Error. */ (void)close(pdesin[0]); (void)close(pdesin[1]); (void)close(pdesout[0]); (void)close(pdesout[1]); return (NULL); /* NOTREACHED */ case 0: /* Child. */ /* Close out un-used sides */ (void)close(pdesin[1]); (void)close(pdesout[0]); /* Now prepare the stdin of the process */ close(0); (void)dup(pdesin[0]); (void)close(pdesin[0]); /* Now prepare the stdout of the process */ close(1); (void)dup(pdesout[1]); /* And lets do stderr just in case */ close(2); (void)dup(pdesout[1]); (void)close(pdesout[1]); /* Now run it */ execve("/bin/sh", argv, environ); exit(127); /* NOTREACHED */ } /* Parent; assume fdopen can't fail. */ /* Store the pid */ *p_pid = pid; if (strcmp(dir, "r") != 0) { io_out = fdopen(pdesin[1], "w"); (void)close(pdesin[0]); (void)close(pdesout[0]); (void)close(pdesout[1]); return(io_out); } else { /* Prepare the input stream */ io_in = fdopen(pdesout[0], "r"); (void)close(pdesout[1]); (void)close(pdesin[0]); (void)close(pdesin[1]); return (io_in); } } /* * pclose -- * Pclose returns -1 if stream is not associated with a `popened' command, * if already `pclosed', or waitpid returns an error. */ static void my_pclose(FILE *io, pid_t the_pid) { int pstat; pid_t pid; /* * Find the appropriate file pointer and remove it from the list. */ (void)fclose(io); /* Die if you are not dead! */ kill(the_pid, SIGTERM); do { pid = wait4(the_pid, &pstat, 0, (struct rusage *)0); } while (pid == -1 && errno == EINTR); } struct counters { struct counters *next_cpu; char counter_name[MAX_NLEN]; /* Name of counter */ int cpu; /* CPU we are on */ int pos; /* Index we are filling to. */ uint64_t vals[MAX_COUNTER_SLOTS]; /* Last 64 entries */ uint64_t sum; /* Summary of entries */ }; extern struct counters *glob_cpu[MAX_CPU]; struct counters *glob_cpu[MAX_CPU]; extern struct counters *cnts; struct counters *cnts=NULL; extern int ncnts; int ncnts=0; extern int (*expression)(struct counters *, int); int (*expression)(struct counters *, int); static const char *threshold=NULL; static const char *command; struct cpu_entry { const char *name; const char *thresh; const char *command; int (*func)(struct counters *, int); + int counters_required; }; - struct cpu_type { char cputype[32]; int number; struct cpu_entry *ents; void (*explain)(const char *name); }; extern struct cpu_type the_cpu; struct cpu_type the_cpu; static void explain_name_sb(const char *name) { const char *mythresh; if (strcmp(name, "allocstall1") == 0) { printf("Examine PARTIAL_RAT_STALLS.SLOW_LEA_WINDOW / CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh > .05"; } else if (strcmp(name, "allocstall2") == 0) { printf("Examine PARTIAL_RAT_STALLS.FLAGS_MERGE_UOP_CYCLES/CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh > .05"; } else if (strcmp(name, "br_miss") == 0) { printf("Examine (20 * BR_MISP_RETIRED.ALL_BRANCHES)/CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh >= .2"; } else if (strcmp(name, "splitload") == 0) { - printf("Examine MEM_UOP_RETIRED.SPLIT_LOADS * 5) / CPU_CLK_UNHALTED.THREAD_P\n"); + printf("Examine MEM_UOPS_RETIRED.SPLIT_LOADS * 5) / CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh >= .1"; } else if (strcmp(name, "splitstore") == 0) { - printf("Examine MEM_UOP_RETIRED.SPLIT_STORES / MEM_UOP_RETIRED.ALL_STORES\n"); + printf("Examine MEM_UOPS_RETIRED.SPLIT_STORES / MEM_UOPS_RETIRED.ALL_STORES\n"); mythresh = "thresh >= .01"; } else if (strcmp(name, "contested") == 0) { printf("Examine (MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HITM * 60) / CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh >= .05"; } else if (strcmp(name, "blockstorefwd") == 0) { printf("Examine (LD_BLOCKS_STORE_FORWARD * 13) / CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh >= .05"; } else if (strcmp(name, "cache2") == 0) { printf("Examine ((MEM_LOAD_RETIRED.L3_HIT * 26) + \n"); printf(" (MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HIT * 43) + \n"); printf(" (MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HITM * 60)) / CPU_CLK_UNHALTED.THREAD_P\n"); printf("**Note we have it labeled MEM_LOAD_UOPS_RETIRED.LLC_HIT not MEM_LOAD_RETIRED.L3_HIT\n"); mythresh = "thresh >= .2"; } else if (strcmp(name, "cache1") == 0) { printf("Examine (MEM_LOAD_UOPS_MISC_RETIRED.LLC_MISS * 180) / CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh >= .2"; } else if (strcmp(name, "dtlbmissload") == 0) { printf("Examine (((DTLB_LOAD_MISSES.STLB_HIT * 7) + DTLB_LOAD_MISSES.WALK_DURATION)\n"); printf(" / CPU_CLK_UNHALTED.THREAD_P)\n"); mythresh = "thresh >= .1"; } else if (strcmp(name, "frontendstall") == 0) { printf("Examine IDQ_UOPS_NOT_DELIVERED.CORE / (CPU_CLK_UNHALTED.THREAD_P * 4)\n"); mythresh = "thresh >= .15"; } else if (strcmp(name, "clears") == 0) { printf("Examine ((MACHINE_CLEARS.MEMORY_ORDERING + \n"); printf(" MACHINE_CLEARS.SMC + \n"); printf(" MACHINE_CLEARS.MASKMOV ) * 100 ) / CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh >= .02"; } else if (strcmp(name, "microassist") == 0) { printf("Examine IDQ.MS_CYCLES / (CPU_CLK_UNHALTED.THREAD_P * 4)\n"); printf("***We use IDQ.MS_UOPS,cmask=1 to get cycles\n"); mythresh = "thresh >= .05"; } else if (strcmp(name, "aliasing_4k") == 0) { printf("Examine (LD_BLOCKS_PARTIAL.ADDRESS_ALIAS * 5) / CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh >= .1"; } else if (strcmp(name, "fpassist") == 0) { printf("Examine FP_ASSIST.ANY/INST_RETIRED.ANY_P\n"); mythresh = "look for a excessive value"; } else if (strcmp(name, "otherassistavx") == 0) { printf("Examine (OTHER_ASSISTS.AVX_TO_SSE * 75)/CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "look for a excessive value"; } else if (strcmp(name, "otherassistsse") == 0) { printf("Examine (OTHER_ASSISTS.SSE_TO_AVX * 75)/CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "look for a excessive value"; } else if (strcmp(name, "eff1") == 0) { printf("Examine (UOPS_RETIRED.RETIRE_SLOTS)/(4 *CPU_CLK_UNHALTED.THREAD_P)\n"); mythresh = "thresh < .9"; } else if (strcmp(name, "eff2") == 0) { printf("Examine CPU_CLK_UNHALTED.THREAD_P/INST_RETIRED.ANY_P\n"); mythresh = "thresh > 1.0"; } else if (strcmp(name, "dtlbmissstore") == 0) { printf("Examine (((DTLB_STORE_MISSES.STLB_HIT * 7) + DTLB_STORE_MISSES.WALK_DURATION)\n"); printf(" / CPU_CLK_UNHALTED.THREAD_P)\n"); mythresh = "thresh >= .05"; } else { printf("Unknown name:%s\n", name); mythresh = "unknown entry"; } printf("If the value printed is %s we may have the ability to improve performance\n", mythresh); } static void explain_name_ib(const char *name) { const char *mythresh; if (strcmp(name, "br_miss") == 0) { printf("Examine ((BR_MISP_RETIRED.ALL_BRANCHES /(BR_MISP_RETIRED.ALL_BRANCHES +\n"); printf(" MACHINE_CLEAR.COUNT) * ((UOPS_ISSUED.ANY - UOPS_RETIRED.RETIRE_SLOTS + 4 * INT_MISC.RECOVERY_CYCLES)\n"); printf("/ (4 * CPU_CLK_UNHALTED.THREAD))))\n"); mythresh = "thresh >= .2"; } else if (strcmp(name, "eff1") == 0) { printf("Examine (UOPS_RETIRED.RETIRE_SLOTS)/(4 *CPU_CLK_UNHALTED.THREAD_P)\n"); mythresh = "thresh < .9"; } else if (strcmp(name, "eff2") == 0) { printf("Examine CPU_CLK_UNHALTED.THREAD_P/INST_RETIRED.ANY_P\n"); mythresh = "thresh > 1.0"; } else if (strcmp(name, "cache1") == 0) { printf("Examine (MEM_LOAD_UOPS_LLC_MISS_RETIRED.LOCAL_DRAM * 180) / CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh >= .2"; } else if (strcmp(name, "cache2") == 0) { printf("Examine (MEM_LOAD_UOPS_RETIRED.LLC_HIT / CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh >= .2"; } else if (strcmp(name, "itlbmiss") == 0) { printf("Examine ITLB_MISSES.WALK_DURATION / CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh > .05"; } else if (strcmp(name, "icachemiss") == 0) { printf("Examine (ICACHE.IFETCH_STALL - ITLB_MISSES.WALK_DURATION)/ CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh > .05"; } else if (strcmp(name, "lcpstall") == 0) { printf("Examine ILD_STALL.LCP/CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh > .05"; } else if (strcmp(name, "datashare") == 0) { printf("Examine (MEM_LOAD_UOPS_L3_HIT_RETIRED.XSNP_HIT * 43)/CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh > .05"; } else if (strcmp(name, "blockstorefwd") == 0) { printf("Examine (LD_BLOCKS_STORE_FORWARD * 13) / CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh >= .05"; } else if (strcmp(name, "splitload") == 0) { printf("Examine ((L1D_PEND_MISS.PENDING / MEM_LOAD_UOPS_RETIRED.L1_MISS) *\n"); printf(" LD_BLOCKS.NO_SR)/CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh >= .1"; } else if (strcmp(name, "splitstore") == 0) { - printf("Examine MEM_UOP_RETIRED.SPLIT_STORES / MEM_UOP_RETIRED.ALL_STORES\n"); + printf("Examine MEM_UOPS_RETIRED.SPLIT_STORES / MEM_UOPS_RETIRED.ALL_STORES\n"); mythresh = "thresh >= .01"; } else if (strcmp(name, "aliasing_4k") == 0) { printf("Examine (LD_BLOCKS_PARTIAL.ADDRESS_ALIAS * 5) / CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh >= .1"; } else if (strcmp(name, "dtlbmissload") == 0) { printf("Examine (((DTLB_LOAD_MISSES.STLB_HIT * 7) + DTLB_LOAD_MISSES.WALK_DURATION)\n"); printf(" / CPU_CLK_UNHALTED.THREAD_P)\n"); mythresh = "thresh >= .1"; } else if (strcmp(name, "dtlbmissstore") == 0) { printf("Examine (((DTLB_STORE_MISSES.STLB_HIT * 7) + DTLB_STORE_MISSES.WALK_DURATION)\n"); printf(" / CPU_CLK_UNHALTED.THREAD_P)\n"); mythresh = "thresh >= .05"; } else if (strcmp(name, "contested") == 0) { printf("Examine (MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HITM * 60) / CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh >= .05"; } else if (strcmp(name, "clears") == 0) { printf("Examine ((MACHINE_CLEARS.MEMORY_ORDERING + \n"); printf(" MACHINE_CLEARS.SMC + \n"); printf(" MACHINE_CLEARS.MASKMOV ) * 100 ) / CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh >= .02"; } else if (strcmp(name, "microassist") == 0) { printf("Examine IDQ.MS_CYCLES / (4 * CPU_CLK_UNHALTED.THREAD_P)\n"); printf("***We use IDQ.MS_UOPS,cmask=1 to get cycles\n"); mythresh = "thresh >= .05"; } else if (strcmp(name, "fpassist") == 0) { printf("Examine FP_ASSIST.ANY/INST_RETIRED.ANY_P\n"); mythresh = "look for a excessive value"; } else if (strcmp(name, "otherassistavx") == 0) { printf("Examine (OTHER_ASSISTS.AVX_TO_SSE * 75)/CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "look for a excessive value"; } else if (strcmp(name, "otherassistsse") == 0) { printf("Examine (OTHER_ASSISTS.SSE_TO_AVX * 75)/CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "look for a excessive value"; } else { printf("Unknown name:%s\n", name); mythresh = "unknown entry"; } printf("If the value printed is %s we may have the ability to improve performance\n", mythresh); } static void explain_name_has(const char *name) { const char *mythresh; if (strcmp(name, "eff1") == 0) { printf("Examine (UOPS_RETIRED.RETIRE_SLOTS)/(4 *CPU_CLK_UNHALTED.THREAD_P)\n"); mythresh = "thresh < .75"; } else if (strcmp(name, "eff2") == 0) { printf("Examine CPU_CLK_UNHALTED.THREAD_P/INST_RETIRED.ANY_P\n"); mythresh = "thresh > 1.0"; } else if (strcmp(name, "itlbmiss") == 0) { printf("Examine ITLB_MISSES.WALK_DURATION / CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh > .05"; } else if (strcmp(name, "icachemiss") == 0) { printf("Examine (36 * ICACHE.MISSES)/ CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh > .05"; } else if (strcmp(name, "lcpstall") == 0) { printf("Examine ILD_STALL.LCP/CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh > .05"; } else if (strcmp(name, "cache1") == 0) { printf("Examine (MEM_LOAD_UOPS_LLC_MISS_RETIRED.LOCAL_DRAM * 180) / CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh >= .2"; } else if (strcmp(name, "cache2") == 0) { printf("Examine ((MEM_LOAD_UOPS_RETIRED.LLC_HIT * 36) + \n"); printf(" (MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HIT * 72) + \n"); printf(" (MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HITM * 84))\n"); printf(" / CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh >= .2"; } else if (strcmp(name, "contested") == 0) { printf("Examine (MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HITM * 84) / CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh >= .05"; } else if (strcmp(name, "datashare") == 0) { printf("Examine (MEM_LOAD_UOPS_L3_HIT_RETIRED.XSNP_HIT * 72)/CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh > .05"; } else if (strcmp(name, "blockstorefwd") == 0) { printf("Examine (LD_BLOCKS_STORE_FORWARD * 13) / CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh >= .05"; } else if (strcmp(name, "splitload") == 0) { - printf("Examine (MEM_UOP_RETIRED.SPLIT_LOADS * 5) / CPU_CLK_UNHALTED.THREAD_P\n"); + printf("Examine (MEM_UOPS_RETIRED.SPLIT_LOADS * 5) / CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh >= .1"; } else if (strcmp(name, "splitstore") == 0) { - printf("Examine MEM_UOP_RETIRED.SPLIT_STORES / MEM_UOP_RETIRED.ALL_STORES\n"); + printf("Examine MEM_UOPS_RETIRED.SPLIT_STORES / MEM_UOPS_RETIRED.ALL_STORES\n"); mythresh = "thresh >= .01"; } else if (strcmp(name, "aliasing_4k") == 0) { printf("Examine (LD_BLOCKS_PARTIAL.ADDRESS_ALIAS * 5) / CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh >= .1"; } else if (strcmp(name, "dtlbmissload") == 0) { printf("Examine (((DTLB_LOAD_MISSES.STLB_HIT * 7) + DTLB_LOAD_MISSES.WALK_DURATION)\n"); printf(" / CPU_CLK_UNHALTED.THREAD_P)\n"); mythresh = "thresh >= .1"; } else if (strcmp(name, "br_miss") == 0) { printf("Examine (20 * BR_MISP_RETIRED.ALL_BRANCHES)/CPU_CLK_UNHALTED.THREAD\n"); mythresh = "thresh >= .2"; } else if (strcmp(name, "clears") == 0) { printf("Examine ((MACHINE_CLEARS.MEMORY_ORDERING + \n"); printf(" MACHINE_CLEARS.SMC + \n"); printf(" MACHINE_CLEARS.MASKMOV ) * 100 ) / CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "thresh >= .02"; } else if (strcmp(name, "microassist") == 0) { printf("Examine IDQ.MS_CYCLES / (4 * CPU_CLK_UNHALTED.THREAD_P)\n"); printf("***We use IDQ.MS_UOPS,cmask=1 to get cycles\n"); mythresh = "thresh >= .05"; } else if (strcmp(name, "fpassist") == 0) { printf("Examine FP_ASSIST.ANY/INST_RETIRED.ANY_P\n"); mythresh = "look for a excessive value"; } else if (strcmp(name, "otherassistavx") == 0) { printf("Examine (OTHER_ASSISTS.AVX_TO_SSE * 75)/CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "look for a excessive value"; } else if (strcmp(name, "otherassistsse") == 0) { printf("Examine (OTHER_ASSISTS.SSE_TO_AVX * 75)/CPU_CLK_UNHALTED.THREAD_P\n"); mythresh = "look for a excessive value"; } else { printf("Unknown name:%s\n", name); mythresh = "unknown entry"; } printf("If the value printed is %s we may have the ability to improve performance\n", mythresh); } + static struct counters * find_counter(struct counters *base, const char *name) { struct counters *at; int len; at = base; len = strlen(name); while(at) { if (strncmp(at->counter_name, name, len) == 0) { return(at); } at = at->next_cpu; } printf("Can't find counter %s\n", name); printf("We have:\n"); at = base; while(at) { printf("- %s\n", at->counter_name); at = at->next_cpu; } exit(-1); } static int allocstall1(struct counters *cpu, int pos) { /* 1 - PARTIAL_RAT_STALLS.SLOW_LEA_WINDOW/CPU_CLK_UNHALTED.THREAD_P (thresh > .05)*/ int ret; struct counters *partial; struct counters *unhalt; double un, par, res; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); partial = find_counter(cpu, "PARTIAL_RAT_STALLS.SLOW_LEA_WINDOW"); if (pos != -1) { par = partial->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { par = partial->sum * 1.0; un = unhalt->sum * 1.0; } res = par/un; ret = printf("%1.3f", res); return(ret); } static int allocstall2(struct counters *cpu, int pos) { /* 2 - PARTIAL_RAT_STALLS.FLAGS_MERGE_UOP_CYCLES/CPU_CLK_UNHALTED.THREAD_P (thresh >.05) */ int ret; struct counters *partial; struct counters *unhalt; double un, par, res; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); partial = find_counter(cpu, "PARTIAL_RAT_STALLS.FLAGS_MERGE_UOP"); if (pos != -1) { par = partial->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { par = partial->sum * 1.0; un = unhalt->sum * 1.0; } res = par/un; ret = printf("%1.3f", res); return(ret); } static int br_mispredict(struct counters *cpu, int pos) { struct counters *brctr; struct counters *unhalt; int ret; /* 3 - (20 * BR_MISP_RETIRED.ALL_BRANCHES)/CPU_CLK_UNHALTED.THREAD_P (thresh >= .2) */ double br, un, con, res; con = 20.0; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); brctr = find_counter(cpu, "BR_MISP_RETIRED.ALL_BRANCHES"); if (pos != -1) { br = brctr->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { br = brctr->sum * 1.0; un = unhalt->sum * 1.0; } res = (con * br)/un; ret = printf("%1.3f", res); return(ret); } static int br_mispredictib(struct counters *cpu, int pos) { struct counters *brctr; struct counters *unhalt; struct counters *clear, *clear2, *clear3; struct counters *uops; struct counters *recv; struct counters *iss; /* "pmcstat -s CPU_CLK_UNHALTED.THREAD_P -s BR_MISP_RETIRED.ALL_BRANCHES -s MACHINE_CLEARS.MEMORY_ORDERING -s MACHINE_CLEARS.SMC -s MACHINE_CLEARS.MASKMOV -s UOPS_ISSUED.ANY -s UOPS_RETIRED.RETIRE_SLOTS -s INT_MISC.RECOVERY_CYCLES -w 1",*/ int ret; /* * (BR_MISP_RETIRED.ALL_BRANCHES / * (BR_MISP_RETIRED.ALL_BRANCHES + * MACHINE_CLEAR.COUNT) * * ((UOPS_ISSUED.ANY - UOPS_RETIRED.RETIRE_SLOTS + 4 * INT_MISC.RECOVERY_CYCLES) / (4 * CPU_CLK_UNHALTED.THREAD))) * */ double br, cl, cl2, cl3, uo, re, un, con, res, is; con = 4.0; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); brctr = find_counter(cpu, "BR_MISP_RETIRED.ALL_BRANCHES"); clear = find_counter(cpu, "MACHINE_CLEARS.MEMORY_ORDERING"); clear2 = find_counter(cpu, "MACHINE_CLEARS.SMC"); clear3 = find_counter(cpu, "MACHINE_CLEARS.MASKMOV"); uops = find_counter(cpu, "UOPS_RETIRED.RETIRE_SLOTS"); iss = find_counter(cpu, "UOPS_ISSUED.ANY"); recv = find_counter(cpu, "INT_MISC.RECOVERY_CYCLES"); if (pos != -1) { br = brctr->vals[pos] * 1.0; cl = clear->vals[pos] * 1.0; cl2 = clear2->vals[pos] * 1.0; cl3 = clear3->vals[pos] * 1.0; uo = uops->vals[pos] * 1.0; re = recv->vals[pos] * 1.0; is = iss->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { br = brctr->sum * 1.0; cl = clear->sum * 1.0; cl2 = clear2->sum * 1.0; cl3 = clear3->sum * 1.0; uo = uops->sum * 1.0; re = recv->sum * 1.0; is = iss->sum * 1.0; un = unhalt->sum * 1.0; } res = (br/(br + cl + cl2 + cl3) * ((is - uo + con * re) / (con * un))); ret = printf("%1.3f", res); return(ret); } + static int +br_mispredict_broad(struct counters *cpu, int pos) +{ + struct counters *brctr; + struct counters *unhalt; + struct counters *clear; + struct counters *uops; + struct counters *uops_ret; + struct counters *recv; + int ret; + double br, cl, uo, uo_r, re, con, un, res; + + con = 4.0; + + unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); + brctr = find_counter(cpu, "BR_MISP_RETIRED.ALL_BRANCHES"); + clear = find_counter(cpu, "MACHINE_CLEARS.CYCLES"); + uops = find_counter(cpu, "UOPS_ISSUED.ANY"); + uops_ret = find_counter(cpu, "UOPS_RETIRED.RETIRE_SLOTS"); + recv = find_counter(cpu, "INT_MISC.RECOVERY_CYCLES"); + + if (pos != -1) { + un = unhalt->vals[pos] * 1.0; + br = brctr->vals[pos] * 1.0; + cl = clear->vals[pos] * 1.0; + uo = uops->vals[pos] * 1.0; + uo_r = uops_ret->vals[pos] * 1.0; + re = recv->vals[pos] * 1.0; + } else { + un = unhalt->sum * 1.0; + br = brctr->sum * 1.0; + cl = clear->sum * 1.0; + uo = uops->sum * 1.0; + uo_r = uops_ret->sum * 1.0; + re = recv->sum * 1.0; + } + res = br / (br + cl) * (uo - uo_r + con * re) / (un * con); + ret = printf("%1.3f", res); + return(ret); +} + +static int splitloadib(struct counters *cpu, int pos) { int ret; struct counters *mem; struct counters *l1d, *ldblock; struct counters *unhalt; double un, memd, res, l1, ldb; /* * ((L1D_PEND_MISS.PENDING / MEM_LOAD_UOPS_RETIRED.L1_MISS) * LD_BLOCKS.NO_SR) / CPU_CLK_UNHALTED.THREAD_P * "pmcstat -s CPU_CLK_UNHALTED.THREAD_P -s L1D_PEND_MISS.PENDING -s MEM_LOAD_UOPS_RETIRED.L1_MISS -s LD_BLOCKS.NO_SR -w 1", */ unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); mem = find_counter(cpu, "MEM_LOAD_UOPS_RETIRED.L1_MISS"); l1d = find_counter(cpu, "L1D_PEND_MISS.PENDING"); ldblock = find_counter(cpu, "LD_BLOCKS.NO_SR"); if (pos != -1) { memd = mem->vals[pos] * 1.0; l1 = l1d->vals[pos] * 1.0; ldb = ldblock->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { memd = mem->sum * 1.0; l1 = l1d->sum * 1.0; ldb = ldblock->sum * 1.0; un = unhalt->sum * 1.0; } res = ((l1 / memd) * ldb)/un; ret = printf("%1.3f", res); return(ret); } + static int splitload(struct counters *cpu, int pos) { int ret; struct counters *mem; struct counters *unhalt; double con, un, memd, res; +/* 4 - (MEM_UOPS_RETIRED.SPLIT_LOADS * 5) / CPU_CLK_UNHALTED.THREAD_P (thresh >= .1)*/ + + con = 5.0; + unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); + mem = find_counter(cpu, "MEM_UOPS_RETIRED.SPLIT_LOADS"); + if (pos != -1) { + memd = mem->vals[pos] * 1.0; + un = unhalt->vals[pos] * 1.0; + } else { + memd = mem->sum * 1.0; + un = unhalt->sum * 1.0; + } + res = (memd * con)/un; + ret = printf("%1.3f", res); + return(ret); +} + + +static int +splitload_sb(struct counters *cpu, int pos) +{ + int ret; + struct counters *mem; + struct counters *unhalt; + double con, un, memd, res; /* 4 - (MEM_UOP_RETIRED.SPLIT_LOADS * 5) / CPU_CLK_UNHALTED.THREAD_P (thresh >= .1)*/ con = 5.0; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); mem = find_counter(cpu, "MEM_UOP_RETIRED.SPLIT_LOADS"); if (pos != -1) { memd = mem->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { memd = mem->sum * 1.0; un = unhalt->sum * 1.0; } res = (memd * con)/un; ret = printf("%1.3f", res); return(ret); } + static int -splitstore(struct counters *cpu, int pos) +splitstore_sb(struct counters *cpu, int pos) { /* 5 - MEM_UOP_RETIRED.SPLIT_STORES / MEM_UOP_RETIRED.ALL_STORES (thresh > 0.01) */ int ret; struct counters *mem_split; struct counters *mem_stores; double memsplit, memstore, res; mem_split = find_counter(cpu, "MEM_UOP_RETIRED.SPLIT_STORES"); mem_stores = find_counter(cpu, "MEM_UOP_RETIRED.ALL_STORES"); if (pos != -1) { memsplit = mem_split->vals[pos] * 1.0; memstore = mem_stores->vals[pos] * 1.0; } else { memsplit = mem_split->sum * 1.0; memstore = mem_stores->sum * 1.0; } res = memsplit/memstore; ret = printf("%1.3f", res); return(ret); } + static int +splitstore(struct counters *cpu, int pos) +{ + /* 5 - MEM_UOPS_RETIRED.SPLIT_STORES / MEM_UOPS_RETIRED.ALL_STORES (thresh > 0.01) */ + int ret; + struct counters *mem_split; + struct counters *mem_stores; + double memsplit, memstore, res; + mem_split = find_counter(cpu, "MEM_UOPS_RETIRED.SPLIT_STORES"); + mem_stores = find_counter(cpu, "MEM_UOPS_RETIRED.ALL_STORES"); + if (pos != -1) { + memsplit = mem_split->vals[pos] * 1.0; + memstore = mem_stores->vals[pos] * 1.0; + } else { + memsplit = mem_split->sum * 1.0; + memstore = mem_stores->sum * 1.0; + } + res = memsplit/memstore; + ret = printf("%1.3f", res); + return(ret); +} + + +static int contested(struct counters *cpu, int pos) { /* 6 - (MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HITM * 60) / CPU_CLK_UNHALTED.THREAD_P (thresh >.05) */ int ret; struct counters *mem; struct counters *unhalt; double con, un, memd, res; con = 60.0; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); mem = find_counter(cpu, "MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HITM"); if (pos != -1) { memd = mem->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { memd = mem->sum * 1.0; un = unhalt->sum * 1.0; } res = (memd * con)/un; ret = printf("%1.3f", res); return(ret); } static int contested_has(struct counters *cpu, int pos) { /* 6 - (MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HITM * 84) / CPU_CLK_UNHALTED.THREAD_P (thresh >.05) */ int ret; struct counters *mem; struct counters *unhalt; double con, un, memd, res; con = 84.0; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); mem = find_counter(cpu, "MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HITM"); if (pos != -1) { memd = mem->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { memd = mem->sum * 1.0; un = unhalt->sum * 1.0; } res = (memd * con)/un; ret = printf("%1.3f", res); return(ret); } +static int +contestedbroad(struct counters *cpu, int pos) +{ + /* 6 - (MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HITM * 84) / CPU_CLK_UNHALTED.THREAD_P (thresh >.05) */ + int ret; + struct counters *mem; + struct counters *mem2; + struct counters *unhalt; + double con, un, memd, memtoo, res; + con = 84.0; + unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); + mem = find_counter(cpu, "MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HITM"); + mem2 = find_counter(cpu,"MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_MISS"); + + if (pos != -1) { + memd = mem->vals[pos] * 1.0; + memtoo = mem2->vals[pos] * 1.0; + un = unhalt->vals[pos] * 1.0; + } else { + memd = mem->sum * 1.0; + memtoo = mem2->sum * 1.0; + un = unhalt->sum * 1.0; + } + res = ((memd * con) + memtoo)/un; + ret = printf("%1.3f", res); + return(ret); +} + + static int blockstoreforward(struct counters *cpu, int pos) { /* 7 - (LD_BLOCKS_STORE_FORWARD * 13) / CPU_CLK_UNHALTED.THREAD_P (thresh >= .05)*/ int ret; struct counters *ldb; struct counters *unhalt; double con, un, ld, res; con = 13.0; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); ldb = find_counter(cpu, "LD_BLOCKS_STORE_FORWARD"); if (pos != -1) { ld = ldb->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { ld = ldb->sum * 1.0; un = unhalt->sum * 1.0; } res = (ld * con)/un; ret = printf("%1.3f", res); return(ret); } static int cache2(struct counters *cpu, int pos) { /* ** Suspect *** * 8 - ((MEM_LOAD_RETIRED.L3_HIT * 26) + (MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HIT * 43) + * (MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HITM * 60)) / CPU_CLK_UNHALTED.THREAD_P (thresh >.2) */ int ret; struct counters *mem1, *mem2, *mem3; struct counters *unhalt; double con1, con2, con3, un, me_1, me_2, me_3, res; con1 = 26.0; con2 = 43.0; con3 = 60.0; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); /* Call for MEM_LOAD_RETIRED.L3_HIT possibly MEM_LOAD_UOPS_RETIRED.LLC_HIT ?*/ mem1 = find_counter(cpu, "MEM_LOAD_UOPS_RETIRED.LLC_HIT"); mem2 = find_counter(cpu, "MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HIT"); mem3 = find_counter(cpu, "MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HITM"); if (pos != -1) { me_1 = mem1->vals[pos] * 1.0; me_2 = mem2->vals[pos] * 1.0; me_3 = mem3->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { me_1 = mem1->sum * 1.0; me_2 = mem2->sum * 1.0; me_3 = mem3->sum * 1.0; un = unhalt->sum * 1.0; } res = ((me_1 * con1) + (me_2 * con2) + (me_3 * con3))/un; ret = printf("%1.3f", res); return(ret); } static int datasharing(struct counters *cpu, int pos) { /* * (MEM_LOAD_UOPS_L3_HIT_RETIRED.XSNP_HIT * 43)/ CPU_CLK_UNHALTED.THREAD_P (thresh >.2) */ int ret; struct counters *mem; struct counters *unhalt; double con, res, me, un; con = 43.0; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); mem = find_counter(cpu, "MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HIT"); if (pos != -1) { me = mem->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { me = mem->sum * 1.0; un = unhalt->sum * 1.0; } res = (me * con)/un; ret = printf("%1.3f", res); return(ret); } static int datasharing_has(struct counters *cpu, int pos) { /* * (MEM_LOAD_UOPS_L3_HIT_RETIRED.XSNP_HIT * 43)/ CPU_CLK_UNHALTED.THREAD_P (thresh >.2) */ int ret; struct counters *mem; struct counters *unhalt; double con, res, me, un; con = 72.0; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); mem = find_counter(cpu, "MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HIT"); if (pos != -1) { me = mem->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { me = mem->sum * 1.0; un = unhalt->sum * 1.0; } res = (me * con)/un; ret = printf("%1.3f", res); return(ret); } static int cache2ib(struct counters *cpu, int pos) { /* * (29 * MEM_LOAD_UOPS_RETIRED.LLC_HIT / CPU_CLK_UNHALTED.THREAD_P (thresh >.2) */ int ret; struct counters *mem; struct counters *unhalt; double con, un, me, res; con = 29.0; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); mem = find_counter(cpu, "MEM_LOAD_UOPS_RETIRED.LLC_HIT"); if (pos != -1) { me = mem->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { me = mem->sum * 1.0; un = unhalt->sum * 1.0; } res = (con * me)/un; ret = printf("%1.3f", res); return(ret); } static int cache2has(struct counters *cpu, int pos) { /* * Examine ((MEM_LOAD_UOPS_RETIRED.LLC_HIT * 36) + \ * (MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HIT * 72) + * (MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HITM * 84)) * / CPU_CLK_UNHALTED.THREAD_P */ int ret; struct counters *mem1, *mem2, *mem3; struct counters *unhalt; double con1, con2, con3, un, me1, me2, me3, res; con1 = 36.0; con2 = 72.0; con3 = 84.0; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); mem1 = find_counter(cpu, "MEM_LOAD_UOPS_RETIRED.LLC_HIT"); mem2 = find_counter(cpu, "MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HIT"); mem3 = find_counter(cpu, "MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HITM"); if (pos != -1) { me1 = mem1->vals[pos] * 1.0; me2 = mem2->vals[pos] * 1.0; me3 = mem3->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { me1 = mem1->sum * 1.0; me2 = mem2->sum * 1.0; me3 = mem3->sum * 1.0; un = unhalt->sum * 1.0; } res = ((me1 * con1) + (me2 * con2) + (me3 * con3))/un; ret = printf("%1.3f", res); return(ret); } + static int +cache2broad(struct counters *cpu, int pos) +{ + /* + * (29 * MEM_LOAD_UOPS_RETIRED.LLC_HIT / CPU_CLK_UNHALTED.THREAD_P (thresh >.2) + */ + int ret; + struct counters *mem; + struct counters *unhalt; + double con, un, me, res; + + con = 36.0; + unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); + mem = find_counter(cpu, "MEM_LOAD_UOPS_RETIRED.L3_HIT"); + if (pos != -1) { + me = mem->vals[pos] * 1.0; + un = unhalt->vals[pos] * 1.0; + } else { + me = mem->sum * 1.0; + un = unhalt->sum * 1.0; + } + res = (con * me)/un; + ret = printf("%1.3f", res); + return(ret); +} + + +static int cache1(struct counters *cpu, int pos) { /* 9 - (MEM_LOAD_UOPS_MISC_RETIRED.LLC_MISS * 180) / CPU_CLK_UNHALTED.THREAD_P (thresh >= .2) */ int ret; struct counters *mem; struct counters *unhalt; double con, un, me, res; con = 180.0; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); mem = find_counter(cpu, "MEM_LOAD_UOPS_MISC_RETIRED.LLC_MISS"); if (pos != -1) { me = mem->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { me = mem->sum * 1.0; un = unhalt->sum * 1.0; } res = (me * con)/un; ret = printf("%1.3f", res); return(ret); } static int cache1ib(struct counters *cpu, int pos) { /* 9 - (MEM_LOAD_UOPS_L3_MISS_RETIRED.LCOAL_DRAM * 180) / CPU_CLK_UNHALTED.THREAD_P (thresh >= .2) */ int ret; struct counters *mem; struct counters *unhalt; double con, un, me, res; con = 180.0; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); mem = find_counter(cpu, "MEM_LOAD_UOPS_LLC_MISS_RETIRED.LOCAL_DRAM"); if (pos != -1) { me = mem->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { me = mem->sum * 1.0; un = unhalt->sum * 1.0; } res = (me * con)/un; ret = printf("%1.3f", res); return(ret); } static int +cache1broad(struct counters *cpu, int pos) +{ + /* 9 - (MEM_LOAD_UOPS_L3_MISS_RETIRED.LCOAL_DRAM * 180) / CPU_CLK_UNHALTED.THREAD_P (thresh >= .2) */ + int ret; + struct counters *mem; + struct counters *unhalt; + double con, un, me, res; + + con = 180.0; + unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); + mem = find_counter(cpu, "MEM_LOAD_UOPS_RETIRED.L3_MISS"); + if (pos != -1) { + me = mem->vals[pos] * 1.0; + un = unhalt->vals[pos] * 1.0; + } else { + me = mem->sum * 1.0; + un = unhalt->sum * 1.0; + } + res = (me * con)/un; + ret = printf("%1.3f", res); + return(ret); +} + + +static int dtlb_missload(struct counters *cpu, int pos) { /* 10 - ((DTLB_LOAD_MISSES.STLB_HIT * 7) + DTLB_LOAD_MISSES.WALK_DURATION) / CPU_CLK_UNHALTED.THREAD_P (t >=.1) */ int ret; struct counters *dtlb_m, *dtlb_d; struct counters *unhalt; double con, un, d1, d2, res; con = 7.0; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); dtlb_m = find_counter(cpu, "DTLB_LOAD_MISSES.STLB_HIT"); dtlb_d = find_counter(cpu, "DTLB_LOAD_MISSES.WALK_DURATION"); if (pos != -1) { d1 = dtlb_m->vals[pos] * 1.0; d2 = dtlb_d->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { d1 = dtlb_m->sum * 1.0; d2 = dtlb_d->sum * 1.0; un = unhalt->sum * 1.0; } res = ((d1 * con) + d2)/un; ret = printf("%1.3f", res); return(ret); } static int dtlb_missstore(struct counters *cpu, int pos) { /* * ((DTLB_STORE_MISSES.STLB_HIT * 7) + DTLB_STORE_MISSES.WALK_DURATION) / * CPU_CLK_UNHALTED.THREAD_P (t >= .1) */ int ret; struct counters *dtsb_m, *dtsb_d; struct counters *unhalt; double con, un, d1, d2, res; con = 7.0; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); dtsb_m = find_counter(cpu, "DTLB_STORE_MISSES.STLB_HIT"); dtsb_d = find_counter(cpu, "DTLB_STORE_MISSES.WALK_DURATION"); if (pos != -1) { d1 = dtsb_m->vals[pos] * 1.0; d2 = dtsb_d->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { d1 = dtsb_m->sum * 1.0; d2 = dtsb_d->sum * 1.0; un = unhalt->sum * 1.0; } res = ((d1 * con) + d2)/un; ret = printf("%1.3f", res); return(ret); } static int itlb_miss(struct counters *cpu, int pos) { /* ITLB_MISSES.WALK_DURATION / CPU_CLK_UNTHREAD_P IB */ int ret; struct counters *itlb; struct counters *unhalt; double un, d1, res; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); itlb = find_counter(cpu, "ITLB_MISSES.WALK_DURATION"); if (pos != -1) { d1 = itlb->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { d1 = itlb->sum * 1.0; un = unhalt->sum * 1.0; } res = d1/un; ret = printf("%1.3f", res); return(ret); } + static int +itlb_miss_broad(struct counters *cpu, int pos) +{ + /* (7 * ITLB_MISSES.STLB_HIT_4K + ITLB_MISSES.WALK_DURATION) / CPU_CLK_UNTHREAD_P */ + int ret; + struct counters *itlb; + struct counters *unhalt; + struct counters *four_k; + double un, d1, res, k; + + unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); + itlb = find_counter(cpu, "ITLB_MISSES.WALK_DURATION"); + four_k = find_counter(cpu, "ITLB_MISSES.STLB_HIT_4K"); + if (pos != -1) { + d1 = itlb->vals[pos] * 1.0; + un = unhalt->vals[pos] * 1.0; + k = four_k->vals[pos] * 1.0; + } else { + d1 = itlb->sum * 1.0; + un = unhalt->sum * 1.0; + k = four_k->sum * 1.0; + } + res = (7.0 * k + d1)/un; + ret = printf("%1.3f", res); + return(ret); +} + + +static int icache_miss(struct counters *cpu, int pos) { /* (ICACHE.IFETCH_STALL - ITLB_MISSES.WALK_DURATION) / CPU_CLK_UNHALTED.THREAD_P IB */ int ret; struct counters *itlb, *icache; struct counters *unhalt; double un, d1, ic, res; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); itlb = find_counter(cpu, "ITLB_MISSES.WALK_DURATION"); icache = find_counter(cpu, "ICACHE.IFETCH_STALL"); if (pos != -1) { d1 = itlb->vals[pos] * 1.0; ic = icache->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { d1 = itlb->sum * 1.0; ic = icache->sum * 1.0; un = unhalt->sum * 1.0; } res = (ic-d1)/un; ret = printf("%1.3f", res); return(ret); } static int icache_miss_has(struct counters *cpu, int pos) { /* (36 * ICACHE.MISSES) / CPU_CLK_UNHALTED.THREAD_P */ int ret; struct counters *icache; struct counters *unhalt; double un, con, ic, res; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); icache = find_counter(cpu, "ICACHE.MISSES"); con = 36.0; if (pos != -1) { ic = icache->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { ic = icache->sum * 1.0; un = unhalt->sum * 1.0; } res = (con * ic)/un; ret = printf("%1.3f", res); return(ret); } static int lcp_stall(struct counters *cpu, int pos) { /* ILD_STALL.LCP/CPU_CLK_UNHALTED.THREAD_P IB */ int ret; struct counters *ild; struct counters *unhalt; double un, d1, res; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); ild = find_counter(cpu, "ILD_STALL.LCP"); if (pos != -1) { d1 = ild->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { d1 = ild->sum * 1.0; un = unhalt->sum * 1.0; } res = d1/un; ret = printf("%1.3f", res); return(ret); } static int frontendstall(struct counters *cpu, int pos) { /* 12 - IDQ_UOPS_NOT_DELIVERED.CORE / (CPU_CLK_UNHALTED.THREAD_P * 4) (thresh >= .15) */ int ret; struct counters *idq; struct counters *unhalt; double con, un, id, res; con = 4.0; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); idq = find_counter(cpu, "IDQ_UOPS_NOT_DELIVERED.CORE"); if (pos != -1) { id = idq->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { id = idq->sum * 1.0; un = unhalt->sum * 1.0; } res = id/(un * con); ret = printf("%1.3f", res); return(ret); } static int clears(struct counters *cpu, int pos) { /* 13 - ((MACHINE_CLEARS.MEMORY_ORDERING + MACHINE_CLEARS.SMC + MACHINE_CLEARS.MASKMOV ) * 100 ) * / CPU_CLK_UNHALTED.THREAD_P (thresh >= .02)*/ int ret; struct counters *clr1, *clr2, *clr3; struct counters *unhalt; double con, un, cl1, cl2, cl3, res; con = 100.0; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); clr1 = find_counter(cpu, "MACHINE_CLEARS.MEMORY_ORDERING"); clr2 = find_counter(cpu, "MACHINE_CLEARS.SMC"); clr3 = find_counter(cpu, "MACHINE_CLEARS.MASKMOV"); if (pos != -1) { cl1 = clr1->vals[pos] * 1.0; cl2 = clr2->vals[pos] * 1.0; cl3 = clr3->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { cl1 = clr1->sum * 1.0; cl2 = clr2->sum * 1.0; cl3 = clr3->sum * 1.0; un = unhalt->sum * 1.0; } res = ((cl1 + cl2 + cl3) * con)/un; ret = printf("%1.3f", res); return(ret); } + + static int +clears_broad(struct counters *cpu, int pos) +{ + int ret; + struct counters *clr1, *clr2, *clr3, *cyc; + struct counters *unhalt; + double con, un, cl1, cl2, cl3, cy, res; + + con = 100.0; + unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); + clr1 = find_counter(cpu, "MACHINE_CLEARS.MEMORY_ORDERING"); + clr2 = find_counter(cpu, "MACHINE_CLEARS.SMC"); + clr3 = find_counter(cpu, "MACHINE_CLEARS.MASKMOV"); + cyc = find_counter(cpu, "MACHINE_CLEARS.CYCLES"); + if (pos != -1) { + cl1 = clr1->vals[pos] * 1.0; + cl2 = clr2->vals[pos] * 1.0; + cl3 = clr3->vals[pos] * 1.0; + cy = cyc->vals[pos] * 1.0; + un = unhalt->vals[pos] * 1.0; + } else { + cl1 = clr1->sum * 1.0; + cl2 = clr2->sum * 1.0; + cl3 = clr3->sum * 1.0; + cy = cyc->sum * 1.0; + un = unhalt->sum * 1.0; + } + /* Formula not listed but extrapulated to add the cy ?? */ + res = ((cl1 + cl2 + cl3 + cy) * con)/un; + ret = printf("%1.3f", res); + return(ret); +} + + + + + +static int microassist(struct counters *cpu, int pos) { /* 14 - IDQ.MS_CYCLES / CPU_CLK_UNHALTED.THREAD_P (thresh > .05) */ int ret; struct counters *idq; struct counters *unhalt; double un, id, res, con; con = 4.0; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); idq = find_counter(cpu, "IDQ.MS_UOPS"); if (pos != -1) { id = idq->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { id = idq->sum * 1.0; un = unhalt->sum * 1.0; } res = id/(un * con); ret = printf("%1.3f", res); return(ret); } static int +microassist_broad(struct counters *cpu, int pos) +{ + int ret; + struct counters *idq; + struct counters *unhalt; + struct counters *uopiss; + struct counters *uopret; + double un, id, res, con, uoi, uor; + + con = 4.0; + unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); + idq = find_counter(cpu, "IDQ.MS_UOPS"); + uopiss = find_counter(cpu, "UOPS_ISSUED.ANY"); + uopret = find_counter(cpu, "UOPS_RETIRED.RETIRE_SLOTS"); + if (pos != -1) { + id = idq->vals[pos] * 1.0; + un = unhalt->vals[pos] * 1.0; + uoi = uopiss->vals[pos] * 1.0; + uor = uopret->vals[pos] * 1.0; + } else { + id = idq->sum * 1.0; + un = unhalt->sum * 1.0; + uoi = uopiss->sum * 1.0; + uor = uopret->sum * 1.0; + } + res = (uor/uoi) * (id/(un * con)); + ret = printf("%1.3f", res); + return(ret); +} + + +static int aliasing(struct counters *cpu, int pos) { /* 15 - (LD_BLOCKS_PARTIAL.ADDRESS_ALIAS * 5) / CPU_CLK_UNHALTED.THREAD_P (thresh > .1) */ int ret; struct counters *ld; struct counters *unhalt; double un, lds, con, res; con = 5.0; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); ld = find_counter(cpu, "LD_BLOCKS_PARTIAL.ADDRESS_ALIAS"); if (pos != -1) { lds = ld->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { lds = ld->sum * 1.0; un = unhalt->sum * 1.0; } res = (lds * con)/un; ret = printf("%1.3f", res); return(ret); } static int +aliasing_broad(struct counters *cpu, int pos) +{ + /* 15 - (LD_BLOCKS_PARTIAL.ADDRESS_ALIAS * 5) / CPU_CLK_UNHALTED.THREAD_P (thresh > .1) */ + int ret; + struct counters *ld; + struct counters *unhalt; + double un, lds, con, res; + + con = 7.0; + unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); + ld = find_counter(cpu, "LD_BLOCKS_PARTIAL.ADDRESS_ALIAS"); + if (pos != -1) { + lds = ld->vals[pos] * 1.0; + un = unhalt->vals[pos] * 1.0; + } else { + lds = ld->sum * 1.0; + un = unhalt->sum * 1.0; + } + res = (lds * con)/un; + ret = printf("%1.3f", res); + return(ret); +} + + +static int fpassists(struct counters *cpu, int pos) { /* 16 - FP_ASSIST.ANY/INST_RETIRED.ANY_P */ int ret; struct counters *fp; struct counters *inst; double un, fpd, res; inst = find_counter(cpu, "INST_RETIRED.ANY_P"); fp = find_counter(cpu, "FP_ASSIST.ANY"); if (pos != -1) { fpd = fp->vals[pos] * 1.0; un = inst->vals[pos] * 1.0; } else { fpd = fp->sum * 1.0; un = inst->sum * 1.0; } res = fpd/un; ret = printf("%1.3f", res); return(ret); } static int otherassistavx(struct counters *cpu, int pos) { /* 17 - (OTHER_ASSISTS.AVX_TO_SSE * 75)/CPU_CLK_UNHALTED.THREAD_P thresh .1*/ int ret; struct counters *oth; struct counters *unhalt; double un, ot, con, res; con = 75.0; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); oth = find_counter(cpu, "OTHER_ASSISTS.AVX_TO_SSE"); if (pos != -1) { ot = oth->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { ot = oth->sum * 1.0; un = unhalt->sum * 1.0; } res = (ot * con)/un; ret = printf("%1.3f", res); return(ret); } static int otherassistsse(struct counters *cpu, int pos) { int ret; struct counters *oth; struct counters *unhalt; double un, ot, con, res; /* 18 (OTHER_ASSISTS.SSE_TO_AVX * 75)/CPU_CLK_UNHALTED.THREAD_P thresh .1*/ con = 75.0; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); oth = find_counter(cpu, "OTHER_ASSISTS.SSE_TO_AVX"); if (pos != -1) { ot = oth->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { ot = oth->sum * 1.0; un = unhalt->sum * 1.0; } res = (ot * con)/un; ret = printf("%1.3f", res); return(ret); } static int efficiency1(struct counters *cpu, int pos) { int ret; struct counters *uops; struct counters *unhalt; double un, ot, con, res; /* 19 (UOPS_RETIRED.RETIRE_SLOTS/(4*CPU_CLK_UNHALTED.THREAD_P) look if thresh < .9*/ con = 4.0; unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); uops = find_counter(cpu, "UOPS_RETIRED.RETIRE_SLOTS"); if (pos != -1) { ot = uops->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { ot = uops->sum * 1.0; un = unhalt->sum * 1.0; } res = ot/(con * un); ret = printf("%1.3f", res); return(ret); } static int efficiency2(struct counters *cpu, int pos) { int ret; struct counters *uops; struct counters *unhalt; double un, ot, res; /* 20 - CPU_CLK_UNHALTED.THREAD_P/INST_RETIRED.ANY_P good if > 1. (comp factor)*/ unhalt = find_counter(cpu, "CPU_CLK_UNHALTED.THREAD_P"); uops = find_counter(cpu, "INST_RETIRED.ANY_P"); if (pos != -1) { ot = uops->vals[pos] * 1.0; un = unhalt->vals[pos] * 1.0; } else { ot = uops->sum * 1.0; un = unhalt->sum * 1.0; } res = un/ot; ret = printf("%1.3f", res); return(ret); } #define SANDY_BRIDGE_COUNT 20 static struct cpu_entry sandy_bridge[SANDY_BRIDGE_COUNT] = { /*01*/ { "allocstall1", "thresh > .05", "pmcstat -s CPU_CLK_UNHALTED.THREAD_P -s PARTIAL_RAT_STALLS.SLOW_LEA_WINDOW -w 1", - allocstall1 }, -/*02*/ { "allocstall2", "thresh > .05", - "pmcstat -s CPU_CLK_UNHALTED.THREAD_P -s PARTIAL_RAT_STALLS.FLAGS_MERGE_UOP_CYCLES -w 1", - allocstall2 }, + allocstall1, 2 }, +/* -- not defined for SB right (partial-rat_stalls) 02*/ + { "allocstall2", "thresh > .05", + "pmcstat -s CPU_CLK_UNHALTED.THREAD_P -s PARTIAL_RAT_STALLS.FLAGS_MERGE_UOP -w 1", + allocstall2, 2 }, /*03*/ { "br_miss", "thresh >= .2", "pmcstat -s CPU_CLK_UNHALTED.THREAD_P -s BR_MISP_RETIRED.ALL_BRANCHES -w 1", - br_mispredict }, + br_mispredict, 2 }, /*04*/ { "splitload", "thresh >= .1", "pmcstat -s CPU_CLK_UNHALTED.THREAD_P -s MEM_UOP_RETIRED.SPLIT_LOADS -w 1", - splitload }, -/*05*/ { "splitstore", "thresh >= .01", + splitload_sb, 2 }, +/* 05*/ { "splitstore", "thresh >= .01", "pmcstat -s MEM_UOP_RETIRED.SPLIT_STORES -s MEM_UOP_RETIRED.ALL_STORES -w 1", - splitstore }, + splitstore_sb, 2 }, /*06*/ { "contested", "thresh >= .05", "pmcstat -s MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HITM -s CPU_CLK_UNHALTED.THREAD_P -w 1", - contested }, + contested, 2 }, /*07*/ { "blockstorefwd", "thresh >= .05", "pmcstat -s LD_BLOCKS_STORE_FORWARD -s CPU_CLK_UNHALTED.THREAD_P -w 1", - blockstoreforward }, + blockstoreforward, 2 }, /*08*/ { "cache2", "thresh >= .2", "pmcstat -s MEM_LOAD_UOPS_RETIRED.LLC_HIT -s MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HIT -s MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HITM -s CPU_CLK_UNHALTED.THREAD_P -w 1", - cache2 }, + cache2, 4 }, /*09*/ { "cache1", "thresh >= .2", "pmcstat -s MEM_LOAD_UOPS_MISC_RETIRED.LLC_MISS -s CPU_CLK_UNHALTED.THREAD_P -w 1", - cache1 }, + cache1, 2 }, /*10*/ { "dtlbmissload", "thresh >= .1", "pmcstat -s DTLB_LOAD_MISSES.STLB_HIT -s DTLB_LOAD_MISSES.WALK_DURATION -s CPU_CLK_UNHALTED.THREAD_P -w 1", - dtlb_missload }, + dtlb_missload, 3 }, /*11*/ { "dtlbmissstore", "thresh >= .05", "pmcstat -s DTLB_STORE_MISSES.STLB_HIT -s DTLB_STORE_MISSES.WALK_DURATION -s CPU_CLK_UNHALTED.THREAD_P -w 1", - dtlb_missstore }, + dtlb_missstore, 3 }, /*12*/ { "frontendstall", "thresh >= .15", "pmcstat -s IDQ_UOPS_NOT_DELIVERED.CORE -s CPU_CLK_UNHALTED.THREAD_P -w 1", - frontendstall }, + frontendstall, 2 }, /*13*/ { "clears", "thresh >= .02", "pmcstat -s MACHINE_CLEARS.MEMORY_ORDERING -s MACHINE_CLEARS.SMC -s MACHINE_CLEARS.MASKMOV -s CPU_CLK_UNHALTED.THREAD_P -w 1", - clears }, + clears, 4 }, /*14*/ { "microassist", "thresh >= .05", "pmcstat -s IDQ.MS_UOPS,cmask=1 -s CPU_CLK_UNHALTED.THREAD_P -w 1", - microassist }, + microassist, 2 }, /*15*/ { "aliasing_4k", "thresh >= .1", "pmcstat -s LD_BLOCKS_PARTIAL.ADDRESS_ALIAS -s CPU_CLK_UNHALTED.THREAD_P -w 1", - aliasing }, + aliasing, 2 }, /*16*/ { "fpassist", "look for a excessive value", "pmcstat -s FP_ASSIST.ANY -s INST_RETIRED.ANY_P -w 1", - fpassists }, + fpassists, 2 }, /*17*/ { "otherassistavx", "look for a excessive value", "pmcstat -s OTHER_ASSISTS.AVX_TO_SSE -s CPU_CLK_UNHALTED.THREAD_P -w 1", - otherassistavx }, + otherassistavx, 2}, /*18*/ { "otherassistsse", "look for a excessive value", "pmcstat -s OTHER_ASSISTS.SSE_TO_AVX -s CPU_CLK_UNHALTED.THREAD_P -w 1", - otherassistsse }, + otherassistsse, 2 }, /*19*/ { "eff1", "thresh < .9", "pmcstat -s UOPS_RETIRED.RETIRE_SLOTS -s CPU_CLK_UNHALTED.THREAD_P -w 1", - efficiency1 }, + efficiency1, 2 }, /*20*/ { "eff2", "thresh > 1.0", "pmcstat -s INST_RETIRED.ANY_P -s CPU_CLK_UNHALTED.THREAD_P -w 1", - efficiency2 }, + efficiency2, 2 }, }; #define IVY_BRIDGE_COUNT 21 static struct cpu_entry ivy_bridge[IVY_BRIDGE_COUNT] = { /*1*/ { "eff1", "thresh < .75", "pmcstat -s UOPS_RETIRED.RETIRE_SLOTS -s CPU_CLK_UNHALTED.THREAD_P -w 1", - efficiency1 }, + efficiency1, 2 }, /*2*/ { "eff2", "thresh > 1.0", "pmcstat -s INST_RETIRED.ANY_P -s CPU_CLK_UNHALTED.THREAD_P -w 1", - efficiency2 }, + efficiency2, 2 }, /*3*/ { "itlbmiss", "thresh > .05", "pmcstat -s ITLB_MISSES.WALK_DURATION -s CPU_CLK_UNHALTED.THREAD_P -w 1", - itlb_miss }, + itlb_miss, 2 }, /*4*/ { "icachemiss", "thresh > .05", "pmcstat -s ICACHE.IFETCH_STALL -s ITLB_MISSES.WALK_DURATION -s CPU_CLK_UNHALTED.THREAD_P -w 1", - icache_miss }, + icache_miss, 3 }, /*5*/ { "lcpstall", "thresh > .05", "pmcstat -s ILD_STALL.LCP -s CPU_CLK_UNHALTED.THREAD_P -w 1", - lcp_stall }, + lcp_stall, 2 }, /*6*/ { "cache1", "thresh >= .2", "pmcstat -s MEM_LOAD_UOPS_LLC_MISS_RETIRED.LOCAL_DRAM -s CPU_CLK_UNHALTED.THREAD_P -w 1", - cache1ib }, + cache1ib, 2 }, /*7*/ { "cache2", "thresh >= .2", "pmcstat -s MEM_LOAD_UOPS_RETIRED.LLC_HIT -s CPU_CLK_UNHALTED.THREAD_P -w 1", - cache2ib }, + cache2ib, 2 }, /*8*/ { "contested", "thresh >= .05", "pmcstat -s MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HITM -s CPU_CLK_UNHALTED.THREAD_P -w 1", - contested }, + contested, 2 }, /*9*/ { "datashare", "thresh >= .05", "pmcstat -s MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HIT -s CPU_CLK_UNHALTED.THREAD_P -w 1", - datasharing }, + datasharing, 2 }, /*10*/ { "blockstorefwd", "thresh >= .05", "pmcstat -s LD_BLOCKS_STORE_FORWARD -s CPU_CLK_UNHALTED.THREAD_P -w 1", - blockstoreforward }, + blockstoreforward, 2 }, /*11*/ { "splitload", "thresh >= .1", "pmcstat -s CPU_CLK_UNHALTED.THREAD_P -s L1D_PEND_MISS.PENDING -s MEM_LOAD_UOPS_RETIRED.L1_MISS -s LD_BLOCKS.NO_SR -w 1", - splitloadib }, + splitloadib, 4 }, /*12*/ { "splitstore", "thresh >= .01", - "pmcstat -s MEM_UOP_RETIRED.SPLIT_STORES -s MEM_UOP_RETIRED.ALL_STORES -w 1", - splitstore }, + "pmcstat -s MEM_UOPS_RETIRED.SPLIT_STORES -s MEM_UOPS_RETIRED.ALL_STORES -w 1", + splitstore, 2 }, /*13*/ { "aliasing_4k", "thresh >= .1", "pmcstat -s LD_BLOCKS_PARTIAL.ADDRESS_ALIAS -s CPU_CLK_UNHALTED.THREAD_P -w 1", - aliasing }, + aliasing, 2 }, /*14*/ { "dtlbmissload", "thresh >= .1", "pmcstat -s DTLB_LOAD_MISSES.STLB_HIT -s DTLB_LOAD_MISSES.WALK_DURATION -s CPU_CLK_UNHALTED.THREAD_P -w 1", - dtlb_missload }, + dtlb_missload , 3}, /*15*/ { "dtlbmissstore", "thresh >= .05", "pmcstat -s DTLB_STORE_MISSES.STLB_HIT -s DTLB_STORE_MISSES.WALK_DURATION -s CPU_CLK_UNHALTED.THREAD_P -w 1", - dtlb_missstore }, + dtlb_missstore, 3 }, /*16*/ { "br_miss", "thresh >= .2", "pmcstat -s CPU_CLK_UNHALTED.THREAD_P -s BR_MISP_RETIRED.ALL_BRANCHES -s MACHINE_CLEARS.MEMORY_ORDERING -s MACHINE_CLEARS.SMC -s MACHINE_CLEARS.MASKMOV -s UOPS_ISSUED.ANY -s UOPS_RETIRED.RETIRE_SLOTS -s INT_MISC.RECOVERY_CYCLES -w 1", - br_mispredictib }, + br_mispredictib, 8 }, /*17*/ { "clears", "thresh >= .02", "pmcstat -s MACHINE_CLEARS.MEMORY_ORDERING -s MACHINE_CLEARS.SMC -s MACHINE_CLEARS.MASKMOV -s CPU_CLK_UNHALTED.THREAD_P -w 1", - clears }, + clears, 4 }, /*18*/ { "microassist", "thresh >= .05", "pmcstat -s IDQ.MS_UOPS,cmask=1 -s CPU_CLK_UNHALTED.THREAD_P -w 1", - microassist }, + microassist, 2 }, /*19*/ { "fpassist", "look for a excessive value", "pmcstat -s FP_ASSIST.ANY -s INST_RETIRED.ANY_P -w 1", - fpassists }, + fpassists, 2 }, /*20*/ { "otherassistavx", "look for a excessive value", "pmcstat -s OTHER_ASSISTS.AVX_TO_SSE -s CPU_CLK_UNHALTED.THREAD_P -w 1", - otherassistavx }, + otherassistavx , 2}, /*21*/ { "otherassistsse", "look for a excessive value", "pmcstat -s OTHER_ASSISTS.SSE_TO_AVX -s CPU_CLK_UNHALTED.THREAD_P -w 1", - otherassistsse }, + otherassistsse, 2 }, }; #define HASWELL_COUNT 20 static struct cpu_entry haswell[HASWELL_COUNT] = { /*1*/ { "eff1", "thresh < .75", "pmcstat -s UOPS_RETIRED.RETIRE_SLOTS -s CPU_CLK_UNHALTED.THREAD_P -w 1", - efficiency1 }, + efficiency1, 2 }, /*2*/ { "eff2", "thresh > 1.0", "pmcstat -s INST_RETIRED.ANY_P -s CPU_CLK_UNHALTED.THREAD_P -w 1", - efficiency2 }, + efficiency2, 2 }, /*3*/ { "itlbmiss", "thresh > .05", "pmcstat -s ITLB_MISSES.WALK_DURATION -s CPU_CLK_UNHALTED.THREAD_P -w 1", - itlb_miss }, + itlb_miss, 2 }, /*4*/ { "icachemiss", "thresh > .05", - "pmcstat -s ICACHE.MISSES --s CPU_CLK_UNHALTED.THREAD_P -w 1", - icache_miss_has }, + "pmcstat -s ICACHE.MISSES -s CPU_CLK_UNHALTED.THREAD_P -w 1", + icache_miss_has, 2 }, /*5*/ { "lcpstall", "thresh > .05", "pmcstat -s ILD_STALL.LCP -s CPU_CLK_UNHALTED.THREAD_P -w 1", - lcp_stall }, + lcp_stall, 2 }, /*6*/ { "cache1", "thresh >= .2", "pmcstat -s MEM_LOAD_UOPS_LLC_MISS_RETIRED.LOCAL_DRAM -s CPU_CLK_UNHALTED.THREAD_P -w 1", - cache1ib }, + cache1ib, 2 }, /*7*/ { "cache2", "thresh >= .2", "pmcstat -s MEM_LOAD_UOPS_RETIRED.LLC_HIT -s MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HIT -s MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HITM -s CPU_CLK_UNHALTED.THREAD_P -w 1", - cache2has }, + cache2has, 4 }, /*8*/ { "contested", "thresh >= .05", "pmcstat -s MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HITM -s CPU_CLK_UNHALTED.THREAD_P -w 1", - contested_has }, + contested_has, 2 }, /*9*/ { "datashare", "thresh >= .05", "pmcstat -s MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HIT -s CPU_CLK_UNHALTED.THREAD_P -w 1", - datasharing_has }, + datasharing_has, 2 }, /*10*/ { "blockstorefwd", "thresh >= .05", "pmcstat -s LD_BLOCKS_STORE_FORWARD -s CPU_CLK_UNHALTED.THREAD_P -w 1", - blockstoreforward }, + blockstoreforward, 2 }, /*11*/ { "splitload", "thresh >= .1", - "pmcstat -s CPU_CLK_UNHALTED.THREAD_P -s MEM_UOP_RETIRED.SPLIT_LOADS -w 1", - splitload }, + "pmcstat -s CPU_CLK_UNHALTED.THREAD_P -s MEM_UOPS_RETIRED.SPLIT_LOADS -w 1", + splitload , 2}, /*12*/ { "splitstore", "thresh >= .01", - "pmcstat -s MEM_UOP_RETIRED.SPLIT_STORES -s MEM_UOP_RETIRED.ALL_STORES -w 1", - splitstore }, + "pmcstat -s MEM_UOPS_RETIRED.SPLIT_STORES -s MEM_UOPS_RETIRED.ALL_STORES -w 1", + splitstore, 2 }, /*13*/ { "aliasing_4k", "thresh >= .1", "pmcstat -s LD_BLOCKS_PARTIAL.ADDRESS_ALIAS -s CPU_CLK_UNHALTED.THREAD_P -w 1", - aliasing }, + aliasing, 2 }, /*14*/ { "dtlbmissload", "thresh >= .1", "pmcstat -s DTLB_LOAD_MISSES.STLB_HIT -s DTLB_LOAD_MISSES.WALK_DURATION -s CPU_CLK_UNHALTED.THREAD_P -w 1", - dtlb_missload }, + dtlb_missload, 3 }, /*15*/ { "br_miss", "thresh >= .2", "pmcstat -s CPU_CLK_UNHALTED.THREAD_P -s BR_MISP_RETIRED.ALL_BRANCHES -w 1", - br_mispredict }, + br_mispredict, 2 }, /*16*/ { "clears", "thresh >= .02", "pmcstat -s MACHINE_CLEARS.MEMORY_ORDERING -s MACHINE_CLEARS.SMC -s MACHINE_CLEARS.MASKMOV -s CPU_CLK_UNHALTED.THREAD_P -w 1", - clears }, + clears, 4 }, /*17*/ { "microassist", "thresh >= .05", "pmcstat -s IDQ.MS_UOPS,cmask=1 -s CPU_CLK_UNHALTED.THREAD_P -w 1", - microassist }, + microassist, 2 }, /*18*/ { "fpassist", "look for a excessive value", "pmcstat -s FP_ASSIST.ANY -s INST_RETIRED.ANY_P -w 1", - fpassists }, + fpassists, 2 }, /*19*/ { "otherassistavx", "look for a excessive value", "pmcstat -s OTHER_ASSISTS.AVX_TO_SSE -s CPU_CLK_UNHALTED.THREAD_P -w 1", - otherassistavx }, + otherassistavx, 2 }, /*20*/ { "otherassistsse", "look for a excessive value", "pmcstat -s OTHER_ASSISTS.SSE_TO_AVX -s CPU_CLK_UNHALTED.THREAD_P -w 1", - otherassistsse }, + otherassistsse, 2 }, }; static void +explain_name_broad(const char *name) +{ + const char *mythresh; + if (strcmp(name, "eff1") == 0) { + printf("Examine (UOPS_RETIRED.RETIRE_SLOTS)/(4 *CPU_CLK_UNHALTED.THREAD_P)\n"); + mythresh = "thresh < .75"; + } else if (strcmp(name, "eff2") == 0) { + printf("Examine CPU_CLK_UNHALTED.THREAD_P/INST_RETIRED.ANY_P\n"); + mythresh = "thresh > 1.0"; + } else if (strcmp(name, "itlbmiss") == 0) { + printf("Examine (7 * ITLB_MISSES_STLB_HIT_4K + ITLB_MISSES.WALK_DURATION)/ CPU_CLK_UNHALTED.THREAD_P\n"); + mythresh = "thresh > .05"; + } else if (strcmp(name, "icachemiss") == 0) { + printf("Examine ( 36.0 * ICACHE.MISSES)/ CPU_CLK_UNHALTED.THREAD_P ??? may not be right \n"); + mythresh = "thresh > .05"; + } else if (strcmp(name, "lcpstall") == 0) { + printf("Examine ILD_STALL.LCP/CPU_CLK_UNHALTED.THREAD_P\n"); + mythresh = "thresh > .05"; + } else if (strcmp(name, "cache1") == 0) { + printf("Examine (MEM_LOAD_UOPS_LLC_MISS_RETIRED.LOCAL_DRAM * 180) / CPU_CLK_UNHALTED.THREAD_P\n"); + mythresh = "thresh >= .1"; + } else if (strcmp(name, "cache2") == 0) { + printf("Examine (36.0 * MEM_LOAD_UOPS_RETIRED.L3_HIT / CPU_CLK_UNHALTED.THREAD_P)\n"); + mythresh = "thresh >= .2"; + } else if (strcmp(name, "contested") == 0) { + printf("Examine ((MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HITM * 84) + MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_MISS)/ CPU_CLK_UNHALTED.THREAD_P\n"); + mythresh = "thresh >= .05"; + } else if (strcmp(name, "datashare") == 0) { + printf("Examine (MEM_LOAD_UOPS_L3_HIT_RETIRED.XSNP_HIT * 72)/CPU_CLK_UNHALTED.THREAD_P\n"); + mythresh = "thresh > .05"; + } else if (strcmp(name, "blockstorefwd") == 0) { + printf("Examine (LD_BLOCKS_STORE_FORWARD * 13) / CPU_CLK_UNHALTED.THREAD_P\n"); + mythresh = "thresh >= .05"; + } else if (strcmp(name, "aliasing_4k") == 0) { + printf("Examine (LD_BLOCKS_PARTIAL.ADDRESS_ALIAS * 7) / CPU_CLK_UNHALTED.THREAD_P\n"); + mythresh = "thresh >= .1"; + } else if (strcmp(name, "dtlbmissload") == 0) { + printf("Examine (((DTLB_LOAD_MISSES.STLB_HIT * 7) + DTLB_LOAD_MISSES.WALK_DURATION)\n"); + printf(" / CPU_CLK_UNHALTED.THREAD_P)\n"); + mythresh = "thresh >= .1"; + + } else if (strcmp(name, "br_miss") == 0) { + printf("Examine BR_MISP_RETIRED.ALL_BRANCHS_PS / (BR_MISP_RETIED.ALL_BRANCHES_PS + MACHINE_CLEARS.COUNT) *\n"); + printf(" (UOPS_ISSUEDF.ANY - UOPS_RETIRED.RETIRE_SLOTS + 4 * INT_MISC.RECOVERY_CYCLES) /\n"); + printf("CPU_CLK_UNHALTED.THREAD * 4)\n"); + mythresh = "thresh >= .2"; + } else if (strcmp(name, "clears") == 0) { + printf("Examine ((MACHINE_CLEARS.MEMORY_ORDERING + \n"); + printf(" MACHINE_CLEARS.SMC + \n"); + printf(" MACHINE_CLEARS.MASKMOV ) * 100 ) / CPU_CLK_UNHALTED.THREAD_P\n"); + mythresh = "thresh >= .02"; + } else if (strcmp(name, "fpassist") == 0) { + printf("Examine FP_ASSIST.ANY/INST_RETIRED.ANY_P\n"); + mythresh = "look for a excessive value"; + } else if (strcmp(name, "otherassistavx") == 0) { + printf("Examine (OTHER_ASSISTS.AVX_TO_SSE * 75)/CPU_CLK_UNHALTED.THREAD_P\n"); + mythresh = "look for a excessive value"; + } else if (strcmp(name, "microassist") == 0) { + printf("Examine (UOPS_RETIRED.RETIRE_SLOTS/UOPS_ISSUED.ANY) * (IDQ.MS_CYCLES / (4 * CPU_CLK_UNHALTED.THREAD_P)\n"); + printf("***We use IDQ.MS_UOPS,cmask=1 to get cycles\n"); + mythresh = "thresh >= .05"; + } else { + printf("Unknown name:%s\n", name); + mythresh = "unknown entry"; + } + printf("If the value printed is %s we may have the ability to improve performance\n", mythresh); +} + + +#define BROADWELL_COUNT 17 +static struct cpu_entry broadwell[BROADWELL_COUNT] = { +/*1*/ { "eff1", "thresh < .75", + "pmcstat -s UOPS_RETIRED.RETIRE_SLOTS -s CPU_CLK_UNHALTED.THREAD_P -w 1", + efficiency1, 2 }, +/*2*/ { "eff2", "thresh > 1.0", + "pmcstat -s INST_RETIRED.ANY_P -s CPU_CLK_UNHALTED.THREAD_P -w 1", + efficiency2, 2 }, +/*3*/ { "itlbmiss", "thresh > .05", + "pmcstat -s ITLB_MISSES.WALK_DURATION -s CPU_CLK_UNHALTED.THREAD_P -s ITLB_MISSES.STLB_HIT_4K -w 1", + itlb_miss_broad, 3 }, +/*4*/ { "icachemiss", "thresh > .05", + "pmcstat -s ICACHE.MISSES -s CPU_CLK_UNHALTED.THREAD_P -w 1", + icache_miss_has, 2 }, +/*5*/ { "lcpstall", "thresh > .05", + "pmcstat -s ILD_STALL.LCP -s CPU_CLK_UNHALTED.THREAD_P -w 1", + lcp_stall, 2 }, +/*6*/ { "cache1", "thresh >= .1", + "pmcstat -s MEM_LOAD_UOPS_RETIRED.L3_MISS -s CPU_CLK_UNHALTED.THREAD_P -w 1", + cache1broad, 2 }, +/*7*/ { "cache2", "thresh >= .2", + "pmcstat -s MEM_LOAD_UOPS_RETIRED.L3_HIT -s CPU_CLK_UNHALTED.THREAD_P -w 1", + cache2broad, 2 }, +/*8*/ { "contested", "thresh >= .05", + "pmcstat -s MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HITM -s CPU_CLK_UNHALTED.THREAD_P -s MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_MISS -w 1", + contestedbroad, 2 }, +/*9*/ { "datashare", "thresh >= .05", + "pmcstat -s MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HIT -s CPU_CLK_UNHALTED.THREAD_P -w 1", + datasharing_has, 2 }, +/*10*/ { "blockstorefwd", "thresh >= .05", + "pmcstat -s LD_BLOCKS_STORE_FORWARD -s CPU_CLK_UNHALTED.THREAD_P -w 1", + blockstoreforward, 2 }, +/*11*/ { "aliasing_4k", "thresh >= .1", + "pmcstat -s LD_BLOCKS_PARTIAL.ADDRESS_ALIAS -s CPU_CLK_UNHALTED.THREAD_P -w 1", + aliasing_broad, 2 }, +/*12*/ { "dtlbmissload", "thresh >= .1", + "pmcstat -s DTLB_LOAD_MISSES.STLB_HIT_4K -s DTLB_LOAD_MISSES.WALK_DURATION -s CPU_CLK_UNHALTED.THREAD_P -w 1", + dtlb_missload, 3 }, +/*13*/ { "br_miss", "thresh >= .2", + "pmcstat -s CPU_CLK_UNHALTED.THREAD_P -s BR_MISP_RETIRED.ALL_BRANCHES -s MACHINE_CLEARS.CYCLES -s UOPS_ISSUED.ANY -s UOPS_RETIRED.RETIRE_SLOTS -s INT_MISC.RECOVERY_CYCLES -w 1", + br_mispredict_broad, 7 }, +/*14*/ { "clears", "thresh >= .02", + "pmcstat -s MACHINE_CLEARS.CYCLES -s MACHINE_CLEARS.MEMORY_ORDERING -s MACHINE_CLEARS.SMC -s MACHINE_CLEARS.MASKMOV -s CPU_CLK_UNHALTED.THREAD_P -w 1", + clears_broad, 5 }, +/*15*/ { "fpassist", "look for a excessive value", + "pmcstat -s FP_ASSIST.ANY -s INST_RETIRED.ANY_P -w 1", + fpassists, 2 }, +/*16*/ { "otherassistavx", "look for a excessive value", + "pmcstat -s OTHER_ASSISTS.AVX_TO_SSE -s CPU_CLK_UNHALTED.THREAD_P -w 1", + otherassistavx, 2 }, +/*17*/ { "microassist", "thresh >= .2", + "pmcstat -s IDQ.MS_UOPS,cmask=1 -s CPU_CLK_UNHALTED.THREAD_P -s UOPS_ISSUED.ANY -s UOPS_RETIRED.RETIRE_SLOTS -w 1", + microassist_broad, 4 }, +}; + + +static void set_sandybridge(void) { strcpy(the_cpu.cputype, "SandyBridge PMC"); the_cpu.number = SANDY_BRIDGE_COUNT; the_cpu.ents = sandy_bridge; the_cpu.explain = explain_name_sb; } static void set_ivybridge(void) { strcpy(the_cpu.cputype, "IvyBridge PMC"); the_cpu.number = IVY_BRIDGE_COUNT; the_cpu.ents = ivy_bridge; the_cpu.explain = explain_name_ib; } static void set_haswell(void) { strcpy(the_cpu.cputype, "HASWELL PMC"); the_cpu.number = HASWELL_COUNT; the_cpu.ents = haswell; the_cpu.explain = explain_name_has; } + static void -set_expression(char *name) +set_broadwell(void) { + strcpy(the_cpu.cputype, "HASWELL PMC"); + the_cpu.number = BROADWELL_COUNT; + the_cpu.ents = broadwell; + the_cpu.explain = explain_name_broad; +} + + +static int +set_expression(const char *name) +{ int found = 0, i; for(i=0 ; i< the_cpu.number; i++) { if (strcmp(name, the_cpu.ents[i].name) == 0) { found = 1; expression = the_cpu.ents[i].func; command = the_cpu.ents[i].command; threshold = the_cpu.ents[i].thresh; + if (the_cpu.ents[i].counters_required > max_pmc_counters) { + printf("Test %s requires that the CPU have %d counters and this CPU has only %d\n", + the_cpu.ents[i].name, + the_cpu.ents[i].counters_required, max_pmc_counters); + printf("Sorry this test can not be run\n"); + if (run_all == 0) { + exit(-1); + } else { + return(-1); + } + } break; } } if (!found) { printf("For CPU type %s we have no expression:%s\n", the_cpu.cputype, name); exit(-1); } + return(0); } static int validate_expression(char *name) { int i, found; found = 0; for(i=0 ; i< the_cpu.number; i++) { if (strcmp(name, the_cpu.ents[i].name) == 0) { found = 1; break; } } if (!found) { return(-1); } return (0); } static void do_expression(struct counters *cpu, int pos) { if (expression == NULL) return; (*expression)(cpu, pos); } static void process_header(int idx, char *p) { struct counters *up; int i, len, nlen; /* * Given header element idx, at p in * form 's/NN/nameof' * process the entry to pull out the name and * the CPU number. */ if (strncmp(p, "s/", 2)) { printf("Check -- invalid header no s/ in %s\n", p); return; } up = &cnts[idx]; up->cpu = strtol(&p[2], NULL, 10); len = strlen(p); for (i=2; icounter_name, &p[(i+1)]); } else { strncpy(up->counter_name, &p[(i+1)], (MAX_NLEN-1)); } } } } static void build_counters_from_header(FILE *io) { char buffer[8192], *p; int i, len, cnt; size_t mlen; /* We have a new start, lets * setup our headers and cpus. */ if (fgets(buffer, sizeof(buffer), io) == NULL) { printf("First line can't be read from file err:%d\n", errno); return; } /* * Ok output is an array of counters. Once * we start to read the values in we must * put them in there slot to match there CPU and * counter being updated. We create a mass array * of the counters, filling in the CPU and * counter name. */ /* How many do we get? */ len = strlen(buffer); for (i=0, cnt=0; inext_cpu) { /* Already laced in */ continue; } lace_cpu = cpat->cpu; if (lace_cpu >= MAX_CPU) { printf("CPU %d to big\n", lace_cpu); continue; } if (glob_cpu[lace_cpu] == NULL) { glob_cpu[lace_cpu] = cpat; } else { /* Already processed this cpu */ continue; } /* Ok look forward for cpu->cpu and link in */ for(j=(i+1); jnext_cpu) { continue; } if (at->cpu == lace_cpu) { /* Found one */ cpat->next_cpu = at; cpat = at; } } } } static void process_file(char *filename) { FILE *io; int i; int line_at, not_done; pid_t pid_of_command=0; if (filename == NULL) { io = my_popen(command, "r", &pid_of_command); - if (io == NULL) { - printf("Can't popen the command %s\n", command); - return; - } } else { io = fopen(filename, "r"); if (io == NULL) { printf("Can't process file %s err:%d\n", filename, errno); return; } } build_counters_from_header(io); if (cnts == NULL) { /* Nothing we can do */ printf("Nothing to do -- no counters built\n"); - if (filename) { - fclose(io); - } else { - my_pclose(io, pid_of_command); + if (io) { + fclose(io); } return; } lace_cpus_together(); print_header(); if (verbose) { for (i=0; i= max_to_collect) { not_done = 0; } if (filename == NULL) { int cnt; /* For the ones we dynamically open we print now */ for(i=0, cnt=0; i> 12) | ((eax & 0xF0) >> 4)); printf("CPU model is 0x%x id:0x%lx\n", model, eax); switch (eax & 0xF00) { case 0x500: /* Pentium family processors */ printf("Intel Pentium P5\n"); goto not_supported; break; case 0x600: /* Pentium Pro, Celeron, Pentium II & III */ switch (model) { case 0x1: printf("Intel Pentium P6\n"); goto not_supported; break; case 0x3: case 0x5: printf("Intel PII\n"); goto not_supported; break; case 0x6: case 0x16: printf("Intel CL\n"); goto not_supported; break; case 0x7: case 0x8: case 0xA: case 0xB: printf("Intel PIII\n"); goto not_supported; break; case 0x9: case 0xD: printf("Intel PM\n"); goto not_supported; break; case 0xE: printf("Intel CORE\n"); goto not_supported; break; case 0xF: printf("Intel CORE2\n"); goto not_supported; break; case 0x17: printf("Intel CORE2EXTREME\n"); goto not_supported; break; case 0x1C: /* Per Intel document 320047-002. */ printf("Intel ATOM\n"); goto not_supported; break; case 0x1A: case 0x1E: /* * Per Intel document 253669-032 9/2009, * pages A-2 and A-57 */ case 0x1F: /* * Per Intel document 253669-032 9/2009, * pages A-2 and A-57 */ printf("Intel COREI7\n"); goto not_supported; break; case 0x2E: printf("Intel NEHALEM\n"); goto not_supported; break; case 0x25: /* Per Intel document 253669-033US 12/2009. */ case 0x2C: /* Per Intel document 253669-033US 12/2009. */ printf("Intel WESTMERE\n"); goto not_supported; break; case 0x2F: /* Westmere-EX, seen in wild */ printf("Intel WESTMERE\n"); goto not_supported; break; case 0x2A: /* Per Intel document 253669-039US 05/2011. */ printf("Intel SANDYBRIDGE\n"); set_sandybridge(); break; case 0x2D: /* Per Intel document 253669-044US 08/2012. */ printf("Intel SANDYBRIDGE_XEON\n"); set_sandybridge(); break; case 0x3A: /* Per Intel document 253669-043US 05/2012. */ printf("Intel IVYBRIDGE\n"); set_ivybridge(); break; case 0x3E: /* Per Intel document 325462-045US 01/2013. */ printf("Intel IVYBRIDGE_XEON\n"); set_ivybridge(); break; case 0x3F: /* Per Intel document 325462-045US 09/2014. */ printf("Intel HASWELL (Xeon)\n"); set_haswell(); break; case 0x3C: /* Per Intel document 325462-045US 01/2013. */ case 0x45: case 0x46: printf("Intel HASWELL\n"); set_haswell(); break; + + case 0x4e: + case 0x5e: + printf("Intel SKY-LAKE\n"); + goto not_supported; + break; + case 0x3D: + case 0x47: + printf("Intel BROADWELL\n"); + set_broadwell(); + break; + case 0x4f: + case 0x56: + printf("Intel BROADWEL (Xeon)\n"); + set_broadwell(); + break; + case 0x4D: /* Per Intel document 330061-001 01/2014. */ printf("Intel ATOM_SILVERMONT\n"); goto not_supported; break; default: printf("Intel model 0x%x is not known -- sorry\n", model); goto not_supported; break; } break; case 0xF00: /* P4 */ printf("Intel unknown model %d\n", model); goto not_supported; break; } + do_cpuid(0xa, 0, reg); + max_pmc_counters = (reg[3] & 0x0000000f) + 1; + printf("We have %d PMC counters to work with\n", max_pmc_counters); /* Ok lets load the list of all known PMC's */ io = my_popen("/usr/sbin/pmccontrol -L", "r", &pid_of_command); if (valid_pmcs == NULL) { /* Likely */ pmc_allocated_cnt = PMC_INITIAL_ALLOC; sz = sizeof(char *) * pmc_allocated_cnt; valid_pmcs = malloc(sz); if (valid_pmcs == NULL) { printf("No memory allocation fails at startup?\n"); exit(-1); } memset(valid_pmcs, 0, sz); } while (fgets(linebuf, sizeof(linebuf), io) != NULL) { if (linebuf[0] != '\t') { /* sometimes headers ;-) */ continue; } len = strlen(linebuf); if (linebuf[(len-1)] == '\n') { /* Likely */ linebuf[(len-1)] = 0; } str = &linebuf[1]; len = strlen(str) + 1; valid_pmcs[valid_pmc_cnt] = malloc(len); if (valid_pmcs[valid_pmc_cnt] == NULL) { printf("No memory2 allocation fails at startup?\n"); exit(-1); } memset(valid_pmcs[valid_pmc_cnt], 0, len); strcpy(valid_pmcs[valid_pmc_cnt], str); valid_pmc_cnt++; if (valid_pmc_cnt >= pmc_allocated_cnt) { /* Got to expand -- unlikely */ char **more; sz = sizeof(char *) * (pmc_allocated_cnt * 2); more = malloc(sz); if (more == NULL) { printf("No memory3 allocation fails at startup?\n"); exit(-1); } memset(more, 0, sz); memcpy(more, valid_pmcs, sz); pmc_allocated_cnt *= 2; free(valid_pmcs); valid_pmcs = more; } } my_pclose(io, pid_of_command); return; not_supported: printf("Not supported\n"); exit(-1); } static void explain_all(void) { int i; printf("For CPU's of type %s the following expressions are available:\n",the_cpu.cputype); printf("-------------------------------------------------------------\n"); for(i=0; itype == TYPE_VALUE_PMC) { cnt_pmc++; } at = at->next; } if (cnt_pmc == 0) { printf("No PMC's in your expression -- nothing to do!!\n"); exit(0); } mal = cnt_pmc * sizeof(char *); vars = malloc(mal); if (vars == NULL) { printf("No memory\n"); exit(-1); } memset(vars, 0, mal); at = exp; while (at) { if (at->type == TYPE_VALUE_PMC) { if(add_it_to(vars, alloced_pmcs, at->name)) { alloced_pmcs++; } } at = at->next; } /* Now we have a unique list in vars so create our command */ mal = 23; /* "/usr/sbin/pmcstat -w 1" + \0 */ for(i=0; itype == TYPE_VALUE_PMC) { var = find_counter(cpu, at->name); if (var == NULL) { printf("%s:Can't find counter %s?\n", __FUNCTION__, at->name); exit(-1); } if (pos != -1) { at->value = var->vals[pos] * 1.0; } else { at->value = var->sum * 1.0; } } at = at->next; } res = run_expr(master_exp, 1, NULL); ret = printf("%1.3f", res); return(ret); } static void set_manual_exp(struct expression *exp) { expression = user_expr; command = build_command_for_exp(exp); threshold = "User defined threshold"; } static void run_tests(void) { int i, lenout; printf("Running tests on %d PMC's this may take some time\n", valid_pmc_cnt); printf("------------------------------------------------------------------------\n"); for(i=0; i MAX_COUNTER_SLOTS) { /* You can't collect more than max in array */ max_to_collect = MAX_COUNTER_SLOTS; } break; case 'v': verbose++; break; case 'h': help_only = 1; break; case 'i': filename = optarg; break; case '?': default: use: printf("Use %s [ -i inputfile -v -m max_to_collect -e expr -E -h -? -H]\n", argv[0]); printf("-i inputfile -- use source as inputfile not stdin (if stdin collect)\n"); printf("-v -- verbose dump debug type things -- you don't want this\n"); printf("-m N -- maximum to collect is N measurments\n"); printf("-e expr-name -- Do expression expr-name\n"); printf("-E 'your expression' -- Do your expression\n"); printf("-h -- Don't do the expression I put in -e xxx just explain what it does and exit\n"); printf("-H -- Don't run anything, just explain all canned expressions\n"); printf("-T -- Test all PMC's defined by this processor\n"); + printf("-A -- Run all canned tests\n"); return(0); break; }; } - if ((name == NULL) && (filename == NULL) && (test_mode == 0) && (master_exp == NULL)) { + if ((run_all == 0) && (name == NULL) && (filename == NULL) && + (test_mode == 0) && (master_exp == NULL)) { printf("Without setting an expression we cannot dynamically gather information\n"); printf("you must supply a filename (and you probably want verbosity)\n"); goto use; } + if (run_all && max_to_collect > 10) { + max_to_collect = 3; + } if (test_mode) { run_tests(); return(0); } printf("*********************************\n"); - if (master_exp == NULL) { + if ((master_exp == NULL) && name) { (*the_cpu.explain)(name); - } else { + } else if (master_exp) { printf("Examine your expression "); print_exp(master_exp); printf("User defined threshold\n"); } if (help_only) { return(0); } + if (run_all) { + more: + name = the_cpu.ents[test_at].name; + printf("***Test %s (threshold %s)****\n", name, the_cpu.ents[test_at].thresh); + test_at++; + if (set_expression(name) == -1) { + if (test_at >= the_cpu.number) { + goto done; + } else + goto more; + } + + } process_file(filename); if (verbose >= 2) { for (i=0; i 1) { + for(i=0, cnt=0; i